Many people assume that when they see Googlebot increasing the crawl rate on a site, that it is the sign of an algorithmic issue being lifted – such as Panda or other part of the algo that is impacting the site’s rankings negatively. After all, there must be a reason for Google to be recrawling a site at a furious pace. But Google has said on many occasions that it is not a signal that rankings are going to be changed.
John Mueller addressed this again in yesterday’s hangout and confirmed that increased crawl rate isn’t related to any specific algo issue.
Usually the crawling is a bit separated from the quality side, from the ranking side, in the sense that we see crawling and index more as a technical detail that we have to do ahead of time and the ranking is just based kind of on what we have indexed. Once it’s indexed, we can figure out what to do with the ranking. But the crawling side wouldn’t neccessarily be changed dramatically if one of the ranking algorithms were updated.
He was asked if it could be related to changes in a site, which does make sense. If Google sees many pages have changed, there is often a spike in crawling, such as after a site redesign or URL structure. Many site owners have seen this happen on their site when major changes have been made.
Yeah, I mean that can happen, but it’s not specific to the Panda update in a case like that. What could happen is we notice a bigger change on the website and we think it could be helpful to kind of have these changes reflected in our search results faster, so we’ll go off and crawl it a little bit faster. So that’s like the changes you mention, the HTTPS, that’s an obvious one, or if you move domain, then that’s something where we see a few URLs and they are all redirecting to a new domain, we think oh well we can do this a little bit faster, we will double check more, or double check them a little bit faster than we otherwise would so we can reflect this change in the search results a little bit faster.
So if you are attempting to recover from some kind of algorithmic penalty or a manual action, an increase in crawl rate doesn’t mean it is being lifted or about to be lifted. But it can mean that Google is recognizing that there is some large change on the site, so it may crawl through the site faster than it otherwise would. And the results of that could be a change in rankings. But an increase in crawl rate doesn’t necessarily signal a change in rankings, just that Google is seeing changes.
Jennifer Slegg
Latest posts by Jennifer Slegg (see all)
- 2022 Update for Google Quality Rater Guidelines – Big YMYL Updates - August 1, 2022
- Google Quality Rater Guidelines: The Low Quality 2021 Update - October 19, 2021
- Rethinking Affiliate Sites With Google’s Product Review Update - April 23, 2021
- New Google Quality Rater Guidelines, Update Adds Emphasis on Needs Met - October 16, 2020
- Google Updates Experiment Statistics for Quality Raters - October 6, 2020