Google Search Crawl Rate Doc Adds Retry-After HTTP & More
Optimixed.com
SERoundtable
TLDR
Google Search Crawl Rate Doc Adds Retry-After HTTP & More
Optimixed Overview
Optimixed's original report on this story.
Google has just pushed some updates to the reduce the Google crawl rate documentation. Most of the changes are just organizational to the content on the page but under the "Urgently reduce crawler traffic (for emergencies)" section, Google added new information on how to use Retry-After HTTP header.
Was this useful?