Algorithms That Rule the Web

Google helps us think, Facebook finds us friends, and Pandora plays our own personalized soundtrack. It's hard to say whether the computer algorithms that these services use to anticipate our needs and wants are turning us into puppets or geniuses. But algorithms have a huge impact on our tastes, buying habits, and decisions about our digital lives.

Back in the 20th century--the primordial age of algorithms--life was simpler and harder at the same time. We never knew what else we might want to buy at Amazon; we didn't know what the most "important" news stories of the day were; and before the Netflix movie recommendation engine, we had no mechanized assistance in determining which DVD to rent next.

When we're looking for something online, Google's algorithm frees us from having to sort and search through multitudes of only not-very-relevant results. On the other hand, algorithms might trap us in a world where advertisers and government agencies couple behavioral data with computer formulas to predict and manipulate what we do or buy next.

The technological trend toward ever-more-sophisticated algorithms isn't limited to situations where consumers seek information or products. Private companies and government agencies are also harnessing the power of algorithms to boost their efficiency in dealing with inventory control and their effectiveness in monitoring behavior and predicting what a cybercriminal's next move might be.

For algorithm nerds, the Internet is a Candyland of data to model and predict behavior with. Tracking IP addresses across the Net, knowing what websites people visit and when they visit them, counting banner ad clicks, and harvesting data from social networks--all are much easier than following someone around with a clipboard all day.

Here is a look at some of the algorithms that rule the Web--and those who use them.

Google Search

Many people credit Google's search algorithm as the source of the company's $193 billion market capitalization and tight grip on the search engine market. As Steven Levy pointed out in a 2010 article on Google "[Google] is still the only company whose name is synonymous with the verb search."

Soon we might start using the verb google instead of think: "Let me google it over before I make up my mind." New research suggests that Google's algorithm could be changing the way we think. Columbia University researcher Betsy Sparrow says that search engines like Google are altering human thought patterns, causing people to remember less on their own and to rely instead on their ability to find the answer on the Internet.

The News Algorithm

Google constantly updates its news algorithm and uses it to power such popular services as Google News. If you're curious about what the day's top story is, you don't have to consult the editors at the New York Times; instead you can see what Google's algorithm considers the top story of the hour at Google News or Yahoo News.

Google News bases its assessment of what constitutes important news on a long list of article attributes including keywords, originality, freshness, quality, expertise of source. Jim Barnett, who writes about journalism for Harvard's Nieman Journalism Lab, wonders whether investigative and explanatory journalism will be steamrolled by a news algorithm that consistently favors the freshest, most popular content.

Barnett sums up his views in a Nieman Journalism Lab article: "When we don't know what we want, sometimes what we really need is to figure it out for ourselves."

The Social Algorithm

Facebook's social algorithm can help you find old high-school friend and past coworkers, of course. But it does more than find friends and determine whose Facebook updates appear in your Facebook Top News Feed. The algorithm is called EdgeRank, shown below in an image from the TechCrunch website. EdgeRank uses a combination of such factors as your affinity with someone, the type of message (Like, Comment, or Tag), and when the post was made.

Facebook recently upped its algorithm ante by tying it to facial recognition software to analyze every photo you upload to the service--including the one from last weekend's beach party. Overall, Facebook's 750 million users have uploaded some 20 billion photos. When you upload your photo to Facebook, the service uses facial recognition software coupled with your immediate and extended social circles to identify who is in the picture; then it asks you whether you want to tag (identify) the people in the image. It's your choice to tag or not, but that fact hasn't quelled privacy activists' concern over the feature.

Algorithm Booms and Busts for Business

Many companies have developed business models around displaying ads on pages of low-quality content customized to rank high in Google News or in Google's main search results. The effectiveness of these so-called content farms in exploiting Google's ranking algorithms caused Google in February to adjust its search algorithm to reduce the standing of low-quality sites within its search results.

Recently the New York Times exposed JC Penney's efforts to inflate its Google page rank by creating thousands of third-party links and sites dedicated to boosting the company's visibility in Google search results. JC Penney denied any direct knowledge of the shenanigans, but Google penalized it by reducing the company's prominence within its search results.

Smaller businesses may succeed or fail depending on the whims of Google's search algorithm. In 2006, California-based KinderStart.com sued Google in federal court, claiming that it sustained significant financial harm when Google changed its algorithm and subsequently ranked the site low in its search results. Ultimately, KinderStart.com lost its court battle, as have other companies that have filed similar suits against Google.

Next: Love, Shopping, and Music