• About Us
  • Contributors
  • Guides
  • Speaking Engagements
  • Write for The SEM Post
  • Submit a tip or contact us!
  • Newsletters

The SEM Post

Latest News About SEO, SEM, PPC & Search Engines

  • Google
  • SEO
  • Mobile
  • Local
  • Bing
  • Pay Per Click
  • Facebook
  • Twitter
  • State of the Industry
You are here: Home / Google / How Many URLs & Duplicates Google Knows About

How Many URLs & Duplicates Google Knows About

November 17, 2015 at 4:20 am PST By Jennifer Slegg

  • Facebook
  • Twitter
  • Google+
  • Pinterest
  • LinkedIn
  • Email
  • WhatsApp
  • Evernote
  • SMS

google urls duplicatesDuring the State of Search keynote yesterday in Dallas, Texas, Gary Illyes shared a pretty incredible stat – Google knows about 120 trillion URLs, and of those URLs, 60% of them are duplicate.

Now keep in mind Illyes does not say these are all indexed and ranking – merely that 60% of the URLs Google knows about are duplicate.  But this is still an incredible number – more than half of them are duplicate!

We see Google regularly kick duplicates out of the search results – completely if it is a case of hardcore spam – or at least to not show them until someone clicks on the link at the bottom of the search results to reveal duplicates.  There are also duplicates that get removed via a DMCA filed with Google, to remove stolen content from the search results.

Google doesn’t tend to release crawl stats very often, so it is nice to hear that there are 140 trillion URLs that Google knows about, as well as the fact 60% of those known URLs are duplicate.

Google knows about 120 trillion URLs, and 60% of them are duplicates o.O @methode #StateofSearch

— Jennifer Slegg (@jenstar) November 16, 2015

  • Facebook
  • Twitter
  • Google+
  • Pinterest
  • LinkedIn
  • Email
  • WhatsApp
  • Evernote
  • SMS
The following two tabs change content below.
  • Bio
  • Latest Posts
My Twitter profileMy Facebook profileMy Google+ profileMy LinkedIn profile

Jennifer Slegg

Founder & Editor at The SEM Post
Jennifer Slegg is a longtime speaker and expert in search engine marketing, working in the industry for almost 20 years. When she isn't sitting at her desk writing and working, she can be found grabbing a latte at her local Starbucks or planning her next trip to Disneyland. She regularly speaks at Pubcon, SMX, State of Search, Brighton SEO and more, and has been presenting at conferences for over a decade.
My Twitter profileMy Facebook profileMy Google+ profileMy LinkedIn profile

Latest posts by Jennifer Slegg (see all)

  • 2022 Update for Google Quality Rater Guidelines – Big YMYL Updates - August 1, 2022
  • Google Quality Rater Guidelines: The Low Quality 2021 Update - October 19, 2021
  • Rethinking Affiliate Sites With Google’s Product Review Update - April 23, 2021
  • New Google Quality Rater Guidelines, Update Adds Emphasis on Needs Met - October 16, 2020
  • Google Updates Experiment Statistics for Quality Raters - October 6, 2020

Filed Under: Google, SEO

Sign up for our newsletter


Comments

  1. Reuben Yau says

    November 17, 2015 at 11:43 am

    I wonder if they also classify unintentional duplicates, like URLs with utm params, as part of that 60%.

Founder & Editor

Jennifer Slegg (2052)

Sign up for our daily news recap & weekly newsletter.


Follow us online

  • Facebook
  • Google+
  • Linkedin
  • Pinterest
  • Twitter

Latest News

2022 Update for Google Quality Rater Guidelines – Big YMYL Updates

We finally have the first Google Quality Rater Guidelines update of 2022, and like usual, it is … [Read More...]

Recent Posts

  • 2022 Update for Google Quality Rater Guidelines – Big YMYL Updates
  • Google Quality Rater Guidelines: The Low Quality 2021 Update
  • Rethinking Affiliate Sites With Google’s Product Review Update
  • New Google Quality Rater Guidelines, Update Adds Emphasis on Needs Met
  • Google Updates Experiment Statistics for Quality Raters
  • Analyzing “How Google Search Works” Changes from Google
  • Google Quality Rater Guidelines Update: New Introduction, Rater Bias & Political Affiliations
  • Google Updates Quality Rater Guidelines: Reputation for News Sites; Video Content Updates; Quality for Information Sites
  • Google Makes Major Changes to NoFollow, Adds Sponsored & UGC Tags
  • Google Updates Quality Rater Guidelines Targeting E-A-T, Page Quality & Interstitials

Categories

  • Affiliate Marketing
  • Amazon
  • Apple
  • Bing
  • Branding
  • Browsers
  • Chrome
  • Content Marketing
  • Design
  • Domains
  • DuckDuckGo
  • Email
  • Facebook
  • Firefox
  • Foursquare
  • Google
    • Analytics
    • Google RankBrain
    • Quality Rater's Guidelines
  • History of Search
  • Industry Spotlight
  • Instagram
  • Internet Explorer
  • Links
  • Local
  • Mobile
  • Native Advertising
  • Other Search Engines
  • Pay Per Click
  • Pinterest
  • Publishers
  • Security
  • SEO
  • Snapchat
  • Social Media
  • State of the Industry
  • The SEM Post
  • Tools
  • Twitter
  • Uncategorized
  • User Experience
  • Video Marketing
  • Week in Review
  • Whitepapers
  • Wordpress
  • Yahoo
  • Yelp
  • YouTube
October 2023
MTWTFSS
« Aug  
 1
2345678
9101112131415
16171819202122
23242526272829
3031 

Meta

  • Log in
  • Entries RSS
  • Comments RSS
  • WordPress.org

Copyright © 2023 · News Pro Theme On Genesis Framework · WordPress · Log in