• PC World
• »Computers

Everyone has an online shopping horror story. Orders that disappear. Products that never arrive. With thousands of e-merchants, how do you find the ones you can trust?

A growing number of companies aim to help, rating Web stores by using criteria such as features, selection, and service. They deliver the ratings online, for free. But keep your eyes open. Among the major merchant raters, we found key differences.

Sites such as BizRate and Reseller Ratings rely on consumers to rate online stores. Others, like Shop Online 123, employ analysts to examine sites. Gomez Advisors and RatingWonders do both. The sites share some drawbacks. Most cover only a fraction of the Web's stores. Many favor positive reviews. And some sites do a poor job of preventing ballot box stuffing.

Think Positive

The oldest and best-known rating site, BizRate, publishes ratings on more than 1000 Web merchants it chooses to review, from car rental agencies to clothing stores. (About four out of five merchants have asked to be reviewed, the company says.) BizRate derives most of its ratings by surveying people who've bought at a merchant's site.

Consumers who buy at a BizRate affiliate see a pop-up survey asking them to rate site navigation, selection, prices, and shipping options. Later, consumers receive an e-mail asking whether the product arrived and how satisfied they were with the shopping process.

But this approach has one big problem, says Steve Cook, a senior vice president for Greenfield Online, a Web-based research firm in Wilton, Connecticut: It can't capture the opinions of people who gave up before they reached the checkout counter.

"The people who were frustrated by the site, who never found what they wanted--they never have the chance to be intercepted [by the survey]," explains Cook. "That [omission] gives you a little more of a positive spin on the sites."

In fact, you'll have a hard time finding anything but positive spins on BizRate. In a spot check of the site at press time in early May, we found no overall rating lower than three stars out of five. (BizRate CEO Chuck Davis did point out some examples of lower scores in individual categories, such as on-time delivery.)

About 87 percent of stores selling desktop PCs received BizRate ratings of four stars or higher. In home audio, 88 percent of stores scored that high; among music vendors, 96 percent rated four stars.

Davis says that some merchants pay BizRate a commission for referring customers to their sites. But he says the firm's ratings are "purely objective." The site makes most of its money by selling research data to online merchants. Because e-tailers must agree to incorporate BizRate's survey into their shopping carts, he explains, only the best merchants apply. "If you are a good site, you want to be on BizRate. If you are not a good site, we're going to reveal that pretty quickly," he says.

Some big sites like Amazon.com and Dell don't participate in BizRate's point-of-sale surveys. To rate those vendors, BizRate polls a panel of consumers who've done BizRate surveys on other e-merchants. Here, too, negative ratings are almost nonexistent.

BizRate is not the only bearer of good news. Shop Online 123 takes a different approach, but its results are only slightly more critical. Instead of surveying consumers, the site's team of surfers buys products from selected vendors, returns them, and rates the sites on service, selection, and the like.

But the lowest rating any site can receive is Fair. About 250 e-tailers are currently reviewed on the site; according to editor Dave Kurns, nearly three times that number have been evaluated.

"What we've done is only give you the good stuff, " says Jackie Leo, vice president of interactive media for Meredith Corporation, which owns Shop Online 123. "We don't want to waste your time with sites we think aren't valuable."

Leo says that while the site receives commissions for some customer referrals, those deals are struck only after reviews have been written, to avoid influencing the reviewers. Shop Online 123 makes most of its money from print advertising--it's also an annual publication sent to subscribers of other Meredith magazines such as Better Homes and Gardens.

The problem? If a store isn't on the list, you don't know whether it was deemed bad or hasn't been rated. Among sites all rated positively, it's hard to choose.

Also, while the Web enables raters to garner feedback from thousands of consumers, it also can make it easy for smaller e-tailers to alter the scores.

Reseller Ratings, which focuses on computer hardware and software vendors, allows you to rate any e-merchant--even some that don't exist. Simply register, choose the merchant name from the site's list or submit the name of a new merchant, and answer seven questions. Scores are averaged, and the store is rated between 0 and 7.

If you register again using a different name and e-mail address, you can apparently vote as often as you like. We were able to review a fictional company three separate times. (The site removed this data after we notified it.) And in fairness, it's tough to weed out all fake submissions from clever spoofers.