What is crawl budget?
Definition
Crawl budget is the set of URLs Google can and wants to crawl on a site. For most small and medium business sites, it is not worth worrying about.
Google defines crawl budget as "the set of URLs that Google can and wants to crawl", made up of a crawl capacity limit, based on how much crawling a server can handle, and crawl demand, based on how popular and how fresh a site's pages are. Its guide to managing crawl budget is written for large sites: those with a million or more unique pages that change weekly, sites with ten thousand or more pages that change daily, or sites with many URLs stuck as discovered but not indexed.
For a typical business site with a few dozen or a few hundred pages, crawl budget is rarely the problem. If important pages are not being indexed, the usual causes are quality, duplication, or missing internal links, not a shortage of crawling.
Why does it matter to your business?
Worrying about crawl budget on a small site is a common distraction. Knowing when it does not apply lets you spend the effort where it counts.
Related terms
Sources, verified September 11, 2026
Want to know how your site measures up?
The free audit checks whether Google and AI answer engines can find, understand, and quote your pages.