Online User Reviews: Can They Be Trusted?
Screening Suspect Reviews
Still, most big sites have automatic and manual ways of screening out iffy reviews. Yelp's algorithms, for example, are on the lookout for suspicious patterns--such as a person who creates five new accounts and posts a positive review of the same restaurant from each account. Suspect reviews can be automatically suppressed, the reviewer's IP address blocked, and so on. Yelp caused a ruckus this summer when it yanked a number of reviews. At the time, Yelp said it was disturbed about positive reviews that some business owners had swapped with other owners.
Epinions has a panel of 10 to 15 readers in the topic area that screens reviews before they're published. "These core members will pelt the reviewer with questions and ask them to flesh out the review," says Alisa Weiner, Epinions' vice president of online comparison shopping. "They have a very high bar for what they consider a useful review."
The real secret weapon many sites rely on? Community policing. Namely, dedicated (dare we say obsessed?) readers who whomp on any review that's suspicious. "It's hard to game our core base of reviewers--they're very protective," says Jeremy Stoppelman, Yelp's CEO and founder. But he admits policing can sometimes turn vigilante.
If a dubious review appears--say, a glowing review for a known lousy restaurant--Yelpers will sometimes pile on with one-star reviews. "And of course," sighs Stoppelman, "we have to wade in and undo the mess."
Beyond community policing, what about real policing? "Fake reader reviews would violate section 255.5 of the FTC guidelines on the use of endorsements and testimonials in advertising," says Frank Dorman of the Federal Trade Commission. Such deceptive practices could conceivably land the "reviewer" in hot water. But none of the sites we talked to, from Amazon to Yelp, have ever pursued such legal action. Self-policing and filftering technology, in their view, are more than enough.
How to Help Yourself
What's a consumer to do? Here's some advice from online shopping pros, flinty-eyed reviewers, and others in the know.
Beware of illogical positivism: If a review is excessively positive (especially compared to others) and has contains only one very minor quibble--usually at the end--for balance, there is a strong possibilty that it's bogus. "You can usually tell a shill by their complete lack of balance and fact, and unusual enthusiasm for something very minor," says Nick Merritt, Editor in Chief of the UK-based TechRadar.
Consider the source: For example, on Dell's site, customers give the Dell Vostro 1310 notebook 4.1 stars out of 5. But on CNet, readers give it 3 out of 5. Says Barbara Kasser bluntly, "Don't look at reviews on a vendor's site--there's a good chance they're planted."
Weigh the balance: Look for reviews that provide both positive and negative feedback and include salient details, says one longtime professional tech reviewer. "That way you know the reviewer has actually seen and used the product."
Check for consensus: Check multiple sites to see whether there's any consensus. "You've got to look at four different sources at least," says computer curmudgeon Steve Bass. "That'll give you a broader sense of how good or bad the product is." Adds online business author Frank Fiore, "I like sites, like Amazon, that give you an aggregate score and show a history of the reviewer. That way you can get a handle on whether they're plants or have a personal agenda."
Check to see who's talking: Look for sites that identify reviewers in some fashion. "I take only 'real name' reviews on Amazon seriously" says Kasser. McConnell likes how eBay requires users to disclose a lot of verifiable information, from full physical address to e-mail to phone number. Many review sites barely ask for your e-mail address.
Caveats aside, reader reviews have their place. Even WebWatch's Brendler admits that an expert review isn't always enough. "The passionate reader review can be a plus...it can tell you how the leather seats in that car feel."
Better yet, says Brendler, reader reviews encourage "frank, open dialogue between consumers and companies." To keep that dialogue open, he says, it's in a company's best interest not to plant phony reviews.
Robert Luhn is the former editor in chief of Computer Currents magazine.