Googlebot

Google's web crawler that discovers and fetches pages for indexing, using a headless Chromium instance to render JavaScript content in a separate processing queue.

Googlebot is Google's automated web crawling system responsible for discovering, fetching, and processing web pages for inclusion in Google's search index. It operates in two main phases: an initial crawl that fetches raw HTML, and a rendering phase that executes JavaScript using a headless Chromium browser (the Web Rendering Service). These phases run in separate queues, meaning JavaScript-dependent content experiences a delay between being discovered and being fully indexed. Googlebot follows robots.txt directives, respects crawl rate limits, and evaluates pages based on the rendered snapshot of JavaScript execution rather than the live interactive page.

Also known as