Crawl & indexing
Cloudflare — Content Signals and Robots Policy
Understand separate content-use preferences for search indexing, real-time AI input and model training, alongside Cloudflare's managed robots directives.
Provider: Cloudflare. Editorial listing by Gateway; not an owner claim.
Visit official destination ↗- Best suited to
- Making deliberate search, AI answer and training choices without blanket access changes
- Owner next step
- Review the search and agent access you want before enabling managed directives. Check the generated rules and the separate enforcement settings so desirable discovery is not blocked accidentally.
- Access
- Public guidance; Cloudflare account, zone and appropriate permissions for managed settings
- Cost
- Free. Free public documentation; managed robots.txt is documented on all Cloudflare plans, with an account and zone required for configuration. Preferences and voluntary robots rules are not authentication or guaranteed crawler compliance.
- Interfaces
- Website
- Source evidence
- Official public source · checked 2026-10-20. This is a description review, not an execution or compatibility test.
- Traffic and availability
- Not monitored. Bot visits, citations, uptime and successful delivery have not been measured by this directory.
Listing content is informational. Inclusion does not authorize installation, changes to your site, or payment. Check current provider terms and access policies.