Crawl & indexing
Screaming Frog — Log File Analyser
Inspect your own server logs to find crawled URLs, response errors and search or AI bot activity. Complements a site crawl with actual request evidence.
Provider: Screaming Frog. Editorial listing by Gateway; not an owner claim.
Visit official destination ↗- Best suited to
- Investigating bot traffic using actual server requests
- Owner next step
- Start with a small log sample, review important URLs and error responses, and compare bot requests with separate citation and referral measurements.
- Access
- Desktop application and access to your server logs
- Cost
- Free plan. Free version analyzes up to 1,000 log events in one project. Larger datasets and multiple projects require a paid licence; this is separate from SEO Spider.
- Interfaces
- Website
- Source evidence
- Official public source · checked 2026-10-18. This is a description review, not an execution or compatibility test.
- Traffic and availability
- Not monitored. Bot visits, citations, uptime and successful delivery have not been measured by this directory.
Listing content is informational. Inclusion does not authorize installation, changes to your site, or payment. Check current provider terms and access policies.