Netscape's Volunteer Army Indexes Web

It's true, there is strength in numbers. Just ask the folks at Netscape.

Once synonymous with the Navigator browser, America Online's Netscape Communications is gaining a reputation as a search leader. Through its Open Directory Project, Netscape harnesses the power of thousands of Internet volunteers to index the Web.

At last count, an army of 18,000 Netscape "editors" has indexed more than 1.2 million Web sites. That's enough to ruffle the feathers of the granddaddy of human-powered searches, Yahoo. It could also bode ill for other competitors say experts, who see Netscape as a formidable challenger in search technology.

The Human Touch

Much of the Open Directory's success, including Yahoo, is traced to the frustration felt by the many Web surfers who can't find what they seek. Both the Open Directory and Yahoo assume humans, not machines, are better suited to finding and indexing relevant Web sites.

However, unlike Yahoo, About.com, and LookSmart, who pay squads of editors to lead searchers, Netscape depends on the efforts of loyal volunteers. Netscape, in a seemingly selfless act of socialism, lets everyday Net surfers build its directory. In turn, anybody can stop by DMOZ.com and download the Open Directory index free of charge, as long as they give Netscape credit.

That small price tag interests big portals like AltaVista, Lycos, HotBot, Netscape sibling AOL.com, and others that have cut their staffing costs by using the Open Directory.

Leveraging the Power of Many

It works like this. Instead of expecting staff motivated by salary to organize a section on, say, automobiles, Netscape tries to inspire auto enthusiasts to organize car site directories. The idea is that motivated specialists can find a better car site than a mercenary surfer who is charged with being an expert on many subjects.

At the same time, Netscape solves one of the biggest challenges facing search firms today: scaling to meet the enormous expansion of the Web. Ideally, the number of Open Directory editors will keep pace with the Web's growth. This means Netscape will always have enough "staff" to add sites to the directory.

Yahoo has an estimated 1.2 million sites in its index and 150 staff editors. But it draws criticism for being slow to list submitted sites in its index.

Volunteer Surfers

Netscape's Nintendo listing is typical of its directory. Three editors have ranked what they consider the most useful Web pages.

The "Forbidden Nintendo Information Repository" tops the list. The editors describe the site as having "lots of technical info, little-known facts, history, and oddities." Down the list is Nintendo's official home page along with scores of other sites.

Scott Schmucker, a 17-year-old high school student from Ohio who helps maintain the Nintendo list, says he surfs the Internet four hours a day and is a Nintendo fanatic. "Believe me, there are a lot of bad Nintendo sites out there. But when I find a good one, I'll get it on the index."

Schmucker says he doesn't mind working for free because he cares about pointing people to quality Nintendo sites. He says he's "proud" to be a part of the Open Directory.

"Sure, there are a lot of other search sites out there, but there is only one that is created by real Net users," Schmucker says.