Many webmasters block Googlebot from crawling both CSS and JavaScript for ranking purposes to try and combat things such as the page layout algorithm. But Google has been advising webmasters not block Googlebot from crawling either, and it has now officially been added to Google’s technical guidelines that blocking CSS and JavaScript so can hurt a website’s rankings.
For optimal rendering and indexing, our new guideline specifies that you should allow Googlebot access to the JavaScript, CSS, and image files that your pages use. This provides you optimal rendering and indexing for your site. Disallowing crawling of Javascript or CSS files in your site’s robots.txt directly harms how well our algorithms render and index your content and can result in suboptimal rankings.
From the technical guidelines, the following section has been added:
To help Google fully understand your site’s contents, allow all of your site’s assets, such as CSS and JavaScript files, to be crawled. The Google indexing system renders webpages using the HTML of a page as well as its assets such as images, CSS, and Javascript files. To see the page assets that Googlebot cannot crawl and to debug directives in your robots.txt file, use the Fetch as Google and the robots.txt Tester tools in Webmaster Tools
When Pierre Far made the announcement on Google+ today, he also mentioned that webmasters should double check their mobile sites to ensure they aren’t blocking CSS and JavaScript too.
There is no timeline given for when this change will go into effect, but since it has been added to the Google Webmaster Guidelines now, webmasters should give Googlebot access to both CSS and JavaScript immediately so they don’t risk any negative impact in the Google search results.
Update July 2015: Google has begun sending out messages in Google Search Console about blocked resources.
Jennifer Slegg
Latest posts by Jennifer Slegg (see all)
- Rethinking Affiliate Sites With Google’s Product Review Update - April 23, 2021
- New Google Quality Rater Guidelines, Update Adds Emphasis on Needs Met - October 16, 2020
- Google Updates Experiment Statistics for Quality Raters - October 6, 2020
- Analyzing “How Google Search Works” Changes from Google - July 8, 2020
- Google Quality Rater Guidelines Update: New Introduction, Rater Bias & Political Affiliations - December 6, 2019
[…] has been making it known that they must be unblocked as part of the mobile friendly algo, they did also make the changes to the Google technical guidelines advising that it can hurt all rankings last […]