How do I interpret fluctuations in crawl stats?
Short answer: Fluctuations are normal; the meaning lies in the direction and the accompanying signals. A one-day spike is usually noise; a decline lasting weeks, or a picture that degrades together with response time, deserves investigation.
A reading key
- Total requests falling while average response time rises: the server has slowed down; Google adjusts crawl rate to server health.
- A drop following a 5xx spike: back-off behavior; fix the server first and crawling recovers on its own.
- A jump in discovery crawling: a new URL space has opened; check for parameter or filter combination explosions.
- A growing 404 share: broken internal links or deleted content; update the links and the sitemap.
- Host status warnings: check robots.txt availability and DNS outages.
Never interpret fluctuations in isolation: read them next to your deploy calendar and log data, and connect persistent trends to the indexing report. If indexed page counts fall along with crawling, the issue moves from the watch list to the action list.