The Internet's Cloaked Web, Explained

Web Pages That Exceed Google and Yahoo

December, 2009

Many untrained users have the mistaken expectation that they can locate anything Internet with Google or Yahoo. These two "Primary" search tools are indeed popular catalogs with amazing databases, but no, they do not index everything on the Internet.

Here are three statistics for you, taken from Google.com, Yahoo.com, Cyberatlas, and MIT:

1) Google, commonly hailed as the best search database today, uses electronic spiders to read and catalog thousands of web pages each day. Google currently indexes 4.3 billion total web pages in its database.

2) Yahoo, a database powered by human editors and collaborative people effort, works much slower than Google robots to catalogue web pages. Yahoo currently indexes 1 million web pages.

3) As of December 2008, there are an estimated 72 billion publically-available pages on the World Wide Web, and another estimated 6 billion in private intranet pages. The exact numbers are impossible to know, but the scale of these estimates is fair.

Assuming some overlap, then Google and Yahoo collectively catalog about 5.8 billion web pages and user postings, which is approximately 25% of the public World Wide Web.

That means 75% of the Web's available content is not searchable via Google or Yahoo.

This, my friends, is where the gigantic "Secondary Web" begins...

Tell me more...