1. Home
  2. Computing & Technology
  3. Internet for Beginners

The Invisible or "Cloaked" Web

(Pages Stored Within Web Databases)

By Paul Gil, About.com

The "Invisible Web" (aka "Deep Web" or "Cloaked Web") is content that is closed-off to search engines. To be specific: the Invisible Web is comprised of 220+ billion web pages that are not stored as static web pages. Instead, the Invisible Web is made of on-demand database content...pages which exist only as reports of changing data. As of August 2007, robot spiders are not advanced enough to read these private databases. Only a human reader can see these "invisible pages" by directly visiting these sites and making direct database requests.

Technical terminology:

  • "Spider": an artifical intelligence program, or robot, that is sent out weekly to scour the public Internet and read millions of static web pages. The spider reports back to its mother database with its results, and those results get collated into search engine catalogs for public use.

  • "Database-Driven Web Content": web pages that exist only temporarily, and are generated only when readers request answers from a large database. These temporary web pages are dynamic, usually cannot be bookmarked, and commonly have extremely long URL addresses.

  • Examples of databased web content: Today's job postings in Honolulu, apartments available in Singapore, today's weather report for Dublin, flights available to Istanbul, stock quotes for the NYSE, houses for sale in Winnipeg, reviews on the movie "Bourne Ultimatum", leather jackets for sale on eBay, hard drives for sale at Best Buy, your current savings account balance.

  • These temporary web pages, once displayed to the reader, cease to exist moments later. Minute-by-minute, these databased web pages are re-created to reflect updated information on the database.


"OK. So I think I understand this now. 'Invisible Web Pages' are really 'Dynamic Web Pages'. That's when a database builds me a temporary page to answer my database question! Neat! So how do I find these thousands of 'Invisible Web' databases?"

Next...

Explore Internet for Beginners

More from About.com

  1. Home
  2. Computing & Technology
  3. Internet for Beginners

©2008 About.com, a part of The New York Times Company.

All rights reserved.