CAMEO | Ask A Librarian |  Site Search | Home

About Internet Directories and Search Engines

What's the difference between a search engine and a subject directory? Simply put, search engines are automated; directories are compiled by people.

Search engines send out spiders, also called worms, crawlers, or robots (‘bots) that traverse the web, read web sites and index all the pages they come across. Each search engine has its own formula for indexing pages. Search engines decide the amount of weight that will be placed on various factors that influence results. Some (i.e. Google, Direct Hit) rank link popularity to be the most important criterion; others rank meta tags. For more information on differences between the major search engines, see Search Engines Features for Searchers.

Subject directories, or catalogs, are completely different. Directories rely on the judgement and expertise of the people compiling them. Often, directories have stringent guidelines that sites must meet before being added. Therefore they have smaller, but cleaner, indexes.

Hybrid search engines, the newest generation, combine a directory and a search engine. The top 10 search engines/directories today are hybrid. Yahoo!, for example, is a directory, which uses results from Google (a search engine) for its secondary results. See the Search Engine Alliances Chart for an inside look into parnerships formed by directories and search engines.

Meta-Search Engines
Meta-search tools allow users to send queries with one central interface to multiple Web search engines and or Web directories simultaneously. Most engines integrate search results and weed out duplicates. Most meta-search engines default to major search engines. For more information on meta-search engines see Multiple Search Engines, which describes popular meta-search tools.

Invisible Web
The so-called "invisible Web" or "deep Web" consists, in large part, of information in databases embedded within web pages and not linked in a way that would allow traditional search engines to index them. There are other hard-to-reach Web files including Adobe PDF files, some Web sites with frames, and/or Flash files, and much more. To learn more about the Invisible Web see http://www.searchenginewatch.internet.com/sereport/00/08-deepweb.html

Evaluating Results
Once you've got your search results, how do you "separate the wheat from the chaff?" Try our tutorial: Evaluating Information on the Web.

Search Engines