Crawl Budget

The set of URLs Google can and wants to crawl on a site, determined by the server's capacity to handle requests and Google's assessment of a site's crawl-worthy content.

Crawl budget is Google's internal allocation of crawling resources for a given website, governed by two components: crawl capacity limit (the maximum request rate a server can handle without degradation) and crawl demand (Google's assessment of how much content is worth fetching). It primarily affects large sites with millions of pages or frequent content updates, and is influenced by server speed, content freshness, URL cleanliness, and site popularity.

Also known as

crawl budget, Google crawl budget, crawl rate limit, crawl allocation