Google, the number one search engine, has its own problems, many caused by a cottage industry of so-called optimization firms that promise to boost rankings of customers' Web sites. Their efforts can reduce the quality of Google's search results.
Some techniques are legitimate. Marketleap and Position Technologies, for example, help site owners analyze content and better describe products so search engines can more easily identify and rank relevant content. These measures help explain why product searches mostly yield links to vendors rather than to reviews or news.
But some ratings-boosting techniques are controversial.
A practice called cloaking occurs when a page appears one way to a search engine's indexing technology but looks quite different to people who click on the link to it.
Cloaking can have perfectly legitimate uses. One example: National Public Radio creates cloaked pages with rough transcripts of its radio programs. People who click on search engine links to these pages get to the archived audio files of the broadcasts and don't see the transcripts. PC World similarly uses cloaking for articles in its database.
But the desktop advertising company WhenU ran afoul of Google in May over cloaked pages that Google believed had misled people who searched on "WhenU." The cloaked pages, which included media reports critical of WhenU's practices, placed high in Google's results, but when users clicked on those links, they saw positive reports only. Google banned WhenU from its index for violating its policies, a ban that was still in place at press time.
In an e-mail response to PC World's queries, WhenU said a search optimization firm created the pages without WhenU's knowledge, and that the company "instructed the outside firm to reverse their actions" as soon as it learned about the cloaked pages.
Another way companies try to change results is by using multiple domains to sell the same products. This increases the odds of being found by search engines and being listed multiple times.
Google tries to excise manipulated results, but it's easy to find pages that violate its policies. Google, in a quiet period before its initial public offering, did not comment.
It's easy to see why Google and Yahoo wield such power: They serve more than 90 percent of results at the top 25 search engines, says ComScore QSearch. And Jupiter Research projects that U.S. search engine ad sales will reach $2.1 billion in 2004, up from $1.6 billion last year.
But observers say consolidation of the search market hurts users. "Fewer voices in search mean fewer options for consumers," says Danny Sullivan of the Search Engine Watch online newsletter.
Web searching is clearly evolving. Microsoft's initiatives should heat up the competition (see "The Future of Search"), and new search tools are on the horizon. With any luck, the search engines that win this race will be the ones with the most relevant results.