User Reviews: How to Use Them, And How Not To

Throw Out the Extremes

Jeff Keller reviews digital cameras at DCResource.com. When he goes through user reviews on a retail Website, he uses a strategy that most people are probably familiar with. "I typically throw out the really positive and really negative reviews," he says, "And try to find a general theme. The really positive reviews often look like they were written by the manufacturer, while the really negative reviews can be due to bad luck or maybe that person has an ax to grind. If most people said that the air purifier is noisy, it probably is."

The best way to protect yourself from misleading assessments is to rely primarily on professional sources that have staked their reputations on providing unbiased reviews--yes, like PCWorld and Consumer Reports. (Full disclosure: I blog for Consumerist, which, like Consumer Reports, is owned by Consumers Union.) It's also a good idea to look at forums and specialty sites (sites that focus on specific tech products such as cameras). The wider you spread your net and the farther you move away from relying exclusively on user reviews, the more accurate the picture you develop of a product's worth will be.

Avoid the Shills

"The one thing you can't trust about reviews at online retailers is you can't know if someone is just trying to move more units," says Chambers. Most user reviews on most Websites are probably authentic, but fake reviews do exist--and given the level of anonymity that the Internet permits, posting a phony review could hardly be easier. Unscrupulous bloggers might receive money, free products, or simply a boost in status for praising a company or product. Tech user Joe Smith could easily be a PR flack, or a stay-at-home parent earning some extra bucks as a reviewer-for-hire, or a desperate marketing manager determined to improve sales.

There's enough opportunity for misbehavior, in fact, that an informal marketplace has sprung up in recent years. "Reputation management" firms might offer to burnish a manufacturer's image with a little strategic fine-tuning of user review pages. Last August, the website MobileCrunch posted internal sales communication from a game publisher PR firm called Reverb Communications, in which the firm promises to frontload new releases on the Apple App Store with reviews provided by "in-house writers" and "written from the angle of each [target] age group including key words that resonate with each audience."

Shills have been discovered in a variety of retail channels. In January of 2009, blogger Arlen Parsa reported that he'd found job listings for fake reviewers on Amazon's Mechanical Turk; they turned out to have been placed by a business development rep at Belkin who was trying to improve the Amazon rating for a router. That same month, blogger Bruce Goldsteinberg reported that he'd discovered Carbonite employees pretending to be customers on the company's Amazon pages as far back as 2006.

The ongoing attempts at gaming user reviews has led to inevitable crack-downs, too. Last July, the New York Attorney General fined cosmetic surgery company Lifestyle Lift $300,000 for fabricating online reviews. (One internal email told employees to "devote the day to doing more postings on the Web [posing] as a satisfied client.") That same month, the travel review website TripAdvisor, which is owned by Expedia, was briefly in the news after reports that customers had found nearly 100 warnings posted throughout the site by the company, saying that specific property reviews were untrustworthy. By the end of 2009, the FTC had revised and re-issued guidelines instructing all bloggers to disclose any relevant business relationships to their readers, in part to combat fake personas created to provide testimonials for health drinks and diet products.

How to Spot a Shill

Spotting a shill review is more of an art than a science, but here are some red flags to look for:

  • Unduly specific use case, user narrative, or consumer segment: If the review paints too complete a picture of the reviewer's precise demographic, the odds are good that you're reading an impostor's work. Most real people don't bother (or even want) to explain their careers or provide a typical day-in-the-life scenario when they're writing about whether a camcorder works. If you can picture the reviewer too specifically, treat the review with caution.
  • Example: "I travel a lot on business, so I need a laptop that's small, durable and gets out of the way so I can do my work. The Atlas 4oo has an extra-long battery life that lets me make good use of all the down-time I spend in airports. It also looks sleek, which means a lot to me when I'm trying to impress a new client."

  • Emphasis on feature lists and marketing bullet points: Some amateur reviewers innocently add bulleted lists of features to their reviews; after all, tech companies have trained us to look at products in terms of a marketing-friendly set of bullet points. But marketers try to hammer these points in impostor reviews, too. And even if the review is authentic, you don't want to evaluate the product on the basis of someone else’s features list--and especially not the one printed on the side of the product box.
  • The single, mild criticism: The sneakiest impostor reviews will praise a product, highlight its best qualities, and coo over how much they like it. Then they'll dirty it up with a minor criticism--something they wish the product did that it doesn't. This is the old "exception that proves the rule" gambit; after all, if a quibble is the only thing that a fair and balanced reviewer can find to put in the "con" column, the product must be pretty great. When you see such a review, though, check to see whether the criticism refers to a crucial feature that's missing, or whether it's just a throwaway bit of fluff that most consumers will disregard. If it's the latter, the probable reason it's there is to prove the reviewer's objectivity--a type of proof that impostors are unusually keen to provide.
  • The deja review: Buried in a pile of reviews or scattered in review collections across multiple retailers are three or four that sound eerily similar, as if the authors were--surprise!--the same person. These sham reviews are easy enough to spot--for the same reason that student essays purloined from CliffsNotes are--but only if you read enough reviews to catch them.

Reviews Aren't Everything

Finally, remember that even the best-reviewed product may break, and at that point you'll hope that you bought it from a manufacturer who takes after-the-sale customer service seriously. I've read every sort of complaint about manufacturers, from the ridiculous to the infuriating. My advice: When it comes to user-complaint Websites, don't put too much stock in any one complaint, but try to look for themes--which manufacturer doesn't provide trained technicians, which one delivers poor warranty services, or the like.

Subscribe to the The Advisor Newsletter

Comments