Many SEOs have often speculated that there is an increase in crawling on sites due to algorithmic reasons specific to a site. But did Penguin specifically cause an increase in crawling due to the new real time aspect of it, and the fact it had to refresh a ton of pages?
Google has repeatedly confirmed that an increase in crawling isn’t a sign of a Google update, manual action or being impacted by Panda.
John Mueller confirmed that they didn’t increase Googlebot crawling as a result of real time Penguin.
I wouldn’t say that. I mean, obviously we always work on crawling faster. So I am pretty sure we’d be crawling faster this week than we were a month ago. But specific to this algorithm, I don’t think that would affect the crawling of the pages.
He was asked if the increased need for refreshing all the pages made Google change crawling for Penguin.
I think we have to do that anyway. There’s always so much more happening on the web from one day to the next that keeping up is hard and making sure that we’re really on top of things and not just chasing the tail is hard too. So that’s kind of something our engineers always have to fight with.
So any increase on Penguin impacted sites when it comes to crawling isn’t Penguin specific, just Googlebot doing its normal trips around the web.
Latest posts by Jennifer Slegg (see all)
- Google Updates Quality Rater Guidelines: Reputation for News Sites; Video Content Updates; Quality for Information Sites - September 13, 2019
- Google Makes Major Changes to NoFollow, Adds Sponsored & UGC Tags - September 10, 2019
- Google Updates Quality Rater Guidelines Targeting E-A-T, Page Quality & Interstitials - May 17, 2019
- Google Local Service Ads Display Pricing Estimates for Specific Locations - August 31, 2018
- Google Testing “Relevant History” Section in Mobile Search Results - August 31, 2018