AI-readable sites
Crawl4AI — Public Page Extraction
Inspect how a self-hosted crawler converts public page content into Markdown, with optional filtering and link references for agent or retrieval workflows.
Provider: Crawl4AI project. Editorial listing by Gateway; not an owner claim.
Visit official destination ↗- Best suited to
- Spot-checking important public content with an owner-controlled extraction workflow
- Owner next step
- Use a small set of public pages you control. Compare raw and filtered extraction with visible source content, respect crawl rules and rate limits, and keep private URLs and credentials out of tests.
- Access
- Public documentation; Python/browser runtime for self-hosted use or separate hosted account/key
- Cost
- Free. The open-source library is free to run yourself; browser resources, hosting and optional model services are separate. Hosted Cloud access has its own account, credit and rate limits. Review license and attribution terms before redistribution. Extracted output is not proof that every assistant reads or cites a page the same way.
- Interfaces
- Website · HTTP · Markdown
- Source evidence
- Official public source · checked 2026-10-27. This is a description review, not an execution or compatibility test.
- Traffic and availability
- Not monitored. Bot visits, citations, uptime and successful delivery have not been measured by this directory.
Listing content is informational. Inclusion does not authorize installation, changes to your site, or payment. Check current provider terms and access policies.