Web News Wranglers

Illustration: P.J. Loughran
Technology advances quickly, but information grows at an even faster clip. The torrents of blog posts and news feeds on today's Internet hold way too much data to keep up with if you just browse the Web normally. Fortunately, help is here in the form of sites that filter the news for you with ever-increasing efficiency, and improved news readers that let you subscribe to news feeds and sort through them like e-mail.

For example, sites such as Digg rely on Web 2.0 techniques to turn users into editors. Other approaches include sites that mine the linking structures of top blogs to provide a front-page-news view of the Internet's conversations, and personalized news-recommendation engines that monitor your reading habits to bring you more-focused news.

What is the best news-management approach for you? We'll examine a number of sites, Web apps, and programs that can help you sort through the best news and commentary on the Net.

First, though, a little background: Almost all of these tools depend on RSS (for Really Simple Syndication) feeds--specially formatted XML files that sites use to quickly publish and exchange bare-bones information about new articles, blog posts, or other updates.

They're often identified on a Web site by a small, orange 'XML' button. (You can right-click the button or link, and then copy the link location.) Once you add that feed into a news reader, the software will periodically check it for new stories. Good news readers let you read through sites in half the time that you would take using a browser. Plus, if you're getting spam on one feed, you can just unsubscribe.

Of course, while feed readers can help you plow through stories quickly, they create a new temptation: Once you realize you've read your news in half the time, it's easy to subscribe to more and more feeds until you're spending as much time reading news as you did before.

Filtering the Feeds

That's where today's crop of news filtering sites come in. By taking into account the structure of Internet conversations, your own reading habits, and the browsing habits of like-minded readers, a good news filter can help you zero in on the news you care about. And while a lot of these tools are still in their infancy, many are plenty good enough to merit addition to your list of critical news sites.

Techmeme is among the best of these services. The site (formerly tech.memeorandum) bills itself as "Page A1" of the tech blogosphere and displays the most talked-about stories of the day. Techmeme repeatedly crawls the feeds of the top tech news sites and blogs, and identifies the most-linked-to stories. It then points you to discussions of those stories, as well as to nested related items.

Simply put, it's a condensed guide to the latest technology news on the Net--and incidentally, a fantastic place to find blogs whose feeds you might want to subscribe to. Gabe Rivera, the site's sole proprietor, harnesses the same algorithms to power an ever-growing list of news filters devoted to politics Memeorandum.com, baseball Ballbug.com, and celebrity gossip WeSmirch.com, and he intends to continue adding new topics.

A similar Web service called TailRank tries to create a top-level "newspaper" tailored specifically to your interests, based on the feeds you tell it to monitor. It then counts the number of sites that link to stories in your feeds and presents the articles in order of popularity. TailRank also produces a personalized version of Amazon.com's "people who like item X also like item Y" feature, based on a comparison of your feeds to those of others.

While TailRank doesn't have quite the sophistication of Techmeme in weeding out duplicates and making sure that related links are actually related, it does possess one mysterious, somewhat creepy, yet very useful feature: At the click of a button, TailRank can check your browser's history, cache, or cookies and figure out which blogs and news sites you've recently visited so you can add them to your list of feeds.

Link Popularity: Techmeme vs. TailRank

Techmeme (left) and TailRank (above) both use blogs and links to help pick out important stories. Techmeme's overall polish gives it a slight edge over the more tweakable approach of the up-and-coming TailRank.
Techmeme (left) and TailRank (above) both use blogs and links to help pick out important stories. Techmeme's overall polish gives it a slight edge over the more tweakable approach of the up-and-coming TailRank.
Techmeme showcases the top tech stories of the day, which the service identifies by monitoring A-list blogs and news sources. It provides a conversational view, rich with links to online discussions. The site skews heavily toward Web 2.0 news, and it often links to the same group of top-echelon blogs. While Techmeme could use some customization options, the site presents a handy overview of the tech-blog world.

Like Techmeme, TailRank relies on the conversational model of the Web, but it lets you create your own news filter that looks at the most-linked-to items in the feeds you tell it to monitor. It's rougher around the edges than Techmeme and prone to duplication, but TailRank captures a richer array of voices. Eventually, TailRank's personalization options may pull it ahead.

Edge: Techmeme

Watching Your Every Read

Three other sites take a more subtle approach to tuning news to your preferences by watching the stories you read. Google News may be the best known of these. After you log in with your Google ID, the site will monitor your news habits and Web searches, and tweak the articles it displays on its algorithmically generated Google News page. The personalized changes are slow to come and aren't explicitly marked, but the site does a remarkable job of highlighting the latest, most important news stories without human editors. Click the 'Standard News' link to see the difference between personalized and default pages.

Findory uses a similar tactic but allows you to add a list of sites you subscribe to. In surprisingly little time the site replaces its default, very scattershot array of stories with a lineup tailored to your interests. Findory gets better as you continue to use it, adding little symbols next to recommended stories. Clicking one of those icons takes you to a page that shows which story you previously read prompted the site to suggest the new article. Unfortunately, currently there's no way to let the site know which stories don't interest you.

Spotback makes it easy to call out which topics you want to read more or less about.
Spotback makes it easy to call out which topics you want to read more or less about.
You can do that, however, with Spotback, a site that debuted in May and offers several ways for you to adjust which stories you see. A slider bar lets you indicate how much you like each story on a scale of -5 to +5. Also, drop-down 'Less' and 'More' menus let you block a news source, category, or subcategory, or request more stories from it.

Spotback's excellent user interface puts a yellow background behind the stories you've read, and when you rate a story you get instant feedback: The site uses Ajax (a popular Web programming technique that employs behind-the-scenes browser requests to make Web sites feel more like desktop apps) to magically slot in a related story directly below the one you've just rated. Regrettably, Spotback falters on the matter of timeliness. The 'Computers and Internet' section sometimes features six- and seven-day-old stories--an eon in online news time.

Wisdom of Crowds

Reader voting drives the immense popularity of social news sites, which eschew algorithms in favor of the collective intelligence of its readers. The year-old Digg has used the technique to become a more popular tech news site than the venerable Slashdot.

Digg's central technology is disarmingly simple: Users submit links, and then other users comment and vote up or down on them. In theory, the most interesting content will rise to the front page. In reality, stories that make the front page can range from trivialities such as an item on a light gun for Nintendo's Wii gaming system to serious political news.

Digg plans to extend the model so that it eventually covers other kinds of news. Its success has spawned some interesting clone sites. Fantacular is a fairly straightforward clone that, like Digg, focuses solely on technology news. Gather.com, a social blogging site, has an eclectic list of top-rated posts, with entries that include everything from poems to an introduction to a person's cat.

CrispyNews and Reddit Reddit may represent the future of community news filters. CrispyNews enables users to create their own mini-community filters that cover narrow topics, such as American Idol or UCLA. Reddit, a Digg-like startup focusing on what's new and popular on the Web, complements its main pages with a recommendation service that learns your preferences and highlights stories that users with similar reading habits have voted for. Eventually, Reddit hopes to analyze the content of the stories themselves, as well as relevant metadata and keywords.

Interesting news bubbles up to the front page of the uber-popular Digg, but Reddit allows for more personalization.
Interesting news bubbles up to the front page of the uber-popular Digg, but Reddit allows for more personalization.

Reader Voting: Digg vs. Reddit

Thanks to Digg, the Web's most frequented news-ranking site, we now know: Geeks like gaming gossip, incendiary technology policy stories, and NASA photos. Diggers vote early and often, and can get breaking news to the front page surprisingly quickly. Unfortunately the comment threads are less informative than Slashdot's, and Digg's indefatigable promotion of the latest Apple rumor shows that sometimes democracy produces uninteresting results.

Reddit is beginning to catch up to Digg in popularity. Voters at this social news collaboration site generally prefer essays, advice for startups, and nongaming tech news. The comment threads are more interesting than Digg's, but Reddit's main advantage is its recommendation engine: Though rudimentary at present, its ever-improving filter will eventually turn Reddit into a combination of a social news system and a personalized recommender.

Edge: Reddit

Take Control of Your News

Here's the great thing about all news filtering sites: Once you're accustomed to using them, you can easily capture their output as an RSS feed and drop it into a stand-alone news reader to exercise even more control over the news you receive.

In fact, that's the most powerful way to customize your news intake. Start by picking an online or downloadable RSS reader as a base of operations, and use it to create a nice, stable list of your favorite feeds. You'll have plenty of choices, including customizable home pages (see "Configurable Home Pages: Netvibes vs. My Yahoo"), Web-based news readers, dedicated software that gives you granular control over each feed and story, and even built-in RSS support in the e-mail clients and Web browsers you may already be using.

Trying out different readers may be easier than you think. Once you compile a list of feeds in one reader, it should let you export those favorites as a text file with the extension .opml. If you're careful to override Windows' sometimes-annoying tendency to save these files in .txt or .html form, you can try your list of feeds in almost any other reader by importing that OPML file.

Radio Userland helped jump-start RSS in that format's early days, and it still warrants a look today. Though it requires a download, Radio Userland works as an online reader, listing all feed items in one column, in reverse chronological order. This format, which has come to be called the "river of news" look, is akin to the inbox interface of most e-mail apps.

Plenty of other programs, including Google Reader, have played off this elegantly simple approach. In many ways, Google Reader is the news reader equivalent of Gmail. It completely forgoes folders, runs incredibly quickly, and is full of sweet Ajax-driven programming, such as a scrolling list of stories to the right of the main display. But as is the case with Gmail, Google Reader's customization options are limited. For example, you can sort stories only by date or by Google's relevance algorithm.

News reader beginners should test the waters with Bloglines, a rich online-only service that Ask.com purchased in 2005. Signup requires a valid e-mail address, but the site offers convenient starter packages of feeds named for the kind of person you are (for example, Bookworm, Conservative Politico, Parental Unit), and a two-pane reader displaying easily configurable folders on the left and news items shown in full on the right.

Bloglines' best feature is an unlimited number of specialized e-mail addresses that you can use to sign up for mailing lists so you can read them alongside an RSS feed. Items can easily be e-mailed, saved, or blogged. The site isn't terribly fast and needs more display options, but Bloglines rounds out its offerings with a mobile version, a really great blog recommender based on your existing subscriptions, and a downloadable news notifier.

Rojo works much like Bloglines, but adds some community filtering to help sort items.
Rojo works much like Bloglines, but adds some community filtering to help sort items.
A competing online reader called Rojo features similar preset subscriber categories and accepts OPML files, permitting you to upload your own subscriptions. Rojo defaults to a two-pane "river of news" view of relevant stories, as determined by the number of users who give "mojo" to an item by clicking a voting button. Other helpful options include time- and category-based views.

Rojo recently added a Today tab that enables you to see which stories are popular with all users, though its value is limited because few Rojo users vote. A browser "bookmarklet" lets you quickly subscribe to sites as you surf the Web. However, while Rojo is a solid online news reader, it doesn't let you re-sort items into different folders, and it doesn't have a notifier.

News Alogorithms: Findory vs. Google News

Both Google News (left) and Findory (above) focus on personalization by tracking which stories you click on and using that information to collect new stories likely to interest you into a customized news destination.
Both Google News (left) and Findory (above) focus on personalization by tracking which stories you click on and using that information to collect new stories likely to interest you into a customized news destination.
The algorithmically edited Google News is ultrafast and comprehensive, with options to tailor your news sources by established categories or by keyword searches. Its recommendation engine seems less intelligent and transparent than Findory's. Also, you have no way to vote a story down and no easy way to know which articles are recommendations (as opposed to standard Google News selections). But Google can use both your logged-in Web searches and your news reading habits to train its filters, so theoretically it has a lot of--if not too much--information about your preferences.

Findory's recommendation system works rather nicely, but it counts your click on a story as a recommendation before you get to read it--a more primitive approach than Digg's, for example. Still, Findory does tell you which stories are recommended and why. In addition, Findory includes many more blogs as sources than Google News does, and it lets you import or add feeds that you want it to monitor closely. Its lack of news photos, standing keyword searches, and the ability to move categories around the screen should be easy for Findory's developers to fix.

Edge: Findory

Online Readers Go Ajax

A capable online news reader, Bloglines offers helpful feed collections for new users.
A capable online news reader, Bloglines offers helpful feed collections for new users.
Though both Bloglines and Rojo are useful, other sites offer more sophistication by heavily using popular Web programming techniques like Ajax.

Alesti, for instance, uses Ajax to create an Outlook-style three-pane online reader that smoothly guides you through folders of feeds. It's an impressive effort, but it needs polish: For example, clicking your browser's back button during a reading session takes you out of the site, rather than back to the last item you read. Alesti also lacks some basic features like the ability to e-mail an item.

Bloxor is an Ajax-driven, open-source online reader. It works only with Gecko-based browsers such as Firefox and Mozilla and seems geared toward the young hacker crowd (in fact, the site's official name is "t3h Blox0r"). The basic three-pane reader differentiates itself from others by effortlessly transforming into a three-column reader and, more notably, by loading items straight from the originating Web sites rather than just loading the article text in the XML feed. While this approach isn't always optimal, reading items in their normal environment is refreshing. The site lacks some basic features such as e-mail and blogging add-ons, but if you want the speed of a news reader while still seeing sites in their native HTML, this is a great choice.

Gritwire's SpeedFeed Reader chose Flash to power its three-pane reader, which makes for an exceptionally pretty interface. Unfortunately, it's short on configuration options, and in one instance the reader choked while importing an RSS 1.0 feed. It also suffers from the limitations of Flash, so an inadvertent back click takes you out of the program and loses your location in your feeds.

Get Your News Everywhere

If you would prefer a more portable approach to news reading, give NewsGator Online a try. The NewsGator site features a built-in news reader, and it can work in conjunction with either NewsGator's Outlook plug-in or a stand-alone client such as FeedDemon (which NewsGator acquired last fall). It has solid e-mail and bookmarking functions, as well as a clean "river of news" or folder-based look. The NewsGator site doesn't try to look like a client app, and unfortunately it's very slow to change from feed to feed. Unlike many online readers, it allows you to create folders and move items between them, and it syncs your changes to its client software.

Nonsubscribers receive one free e-mail account (premium users get five) for subscribing to or creating a public RSS feed for a mailing list. NewsGator users can translate one Web-wide keyword search into a feed (premium users get 20). The service also has a mobile client, POP3 access, and solid tools for bloggers and for users who want to sync podcasts and videocasts with a PC or mobile device.

Despite the service's rich feature set, the site's sluggish performance and lack of a search engine make it unwieldy; I'd recommend it only as an addition to NewsGator's serviceable Outlook plug-in or the outstanding FeedDemon (for more on this client, see the next section).

Serious Software

For basic RSS reading, you may not need much more than your trusty Web browser (see Browser Support: Built-In RSS-IE 7 Beta 2 vs. Firefox (With Sage) vs. Opera). For serious news reading, however, neither browsers nor online services are as fast or as configurable as downloadable clients.

Even a beta release like SharpReader is far more efficient at quickly navigating lots of feeds than the best online reader. Although it can be a memory hog, SharpReader has excellent notification windows and threaded category support, and it can show which feed items are linked to other feed items--a good indicator of a story's importance. The software also does a fine job of identifying the location of RSS feeds when you type in the URL of a site's home page.

Awasu is yet another step up. Its free version consists of a sleek but feature-rich three-pane news reader with a full-on browser in the third pane, including tabbed browsing that makes jumping back to a previous feed simple. Awasu also allows you to search your feeds quickly, and among its plethora of customization options are plug-ins that enable you to establish standing Google searches, save multimedia files to a designated directory, and subscribe to Yahoo groups. Awasu's only real drawbacks are its lack of clear keyboard commands, its tendency to consume lots of system resources, and its overly strict handling of RSS feeds, which causes it to choke on feeds that other readers handle. The paid version ($29) removes limits on the number of plug-ins and feeds, and lets users subscribe to password-protected feeds.

NewsGator's FeedDemon 2.0 costs $30, but it's worth every penny. It is remarkably fast, doesn't hog memory, and combines multiple intuitive layout options with a bevy of options to suit your reading style.

The send-to tool permits you to easily post an item to social bookmarking site Del.icio.us, copy it to the clipboard, or e-mail or blog about the item. The second pane supports multiple tabs, which can load feeds or display a selected item in a browser view. You can then add an item to your IE favorites or browse to a new site. The program's only noticeable flaws are its unchangeable alphabetical feed sort and its rigidly icon-based method of opening an item in a new tab.

But that's just nitpicking. FeedDemon's fantastic design and generous feature set, paired with NewsGator Online's ability to sync multiple computers and mobile readers, makes FeedDemon the best all-around news-reading application.

News as Mail

A number of programs, such as NewsGator Inbox 2.6, integrate RSS feeds with your e\0x2011mail. The NewsGator Inbox installation process is simple and informative, with a handy option to add preselected categories of feeds. Importing more than a few feeds is an intensely long and memory-hogging process, however.

Another interesting application, Omea Reader, is part of an ultra-ambitious productivity suite whose all-encompassing approach you will either love or hate. Though it can function as a free-standing reader, it integrates with the $49 Omea Pro, which aims to be an e-mail client, an RSS reader, a browser, a calendar, an instant messager, and a desktop search application.

Omea Pro has some great features, such as a tab that lets you read your feeds by byline, multiple workspaces to segregate projects, and an automated e-mail contact manager. It takes some time to get used to, but Omea is an impressive effort, and some people may find that it's just the organizational aid they need.

Though news-management software overall is still in its infancy, such tools continue to improve. While you're waiting for the perfect approach, your best bet is to mix and match. Place the RSS feeds from social sites like Digg into a news recommender like Tailrank or Findory, and then grab the personalized feed and throw the results at a news reader you like; you'll soon discover the sweet spot where you get most of the news you want without spending all day trying to keep up.

Configurable Home Pages: Netvibes vs. My Yahoo

You can add plenty of interesting modules to a My Yahoo page (left) but the Ajax underpinnings of Netvibes (above) make tweaking your home page much easier.
You can add plenty of interesting modules to a My Yahoo page (left) but the Ajax underpinnings of Netvibes (above) make tweaking your home page much easier.
Want a customized starting point for your news reading? Netvibes lets you drag and drop modules to build a flexible home page that combines RSS feeds, Web mail, weather reports, and Flickr photo streams. No log-in is required, and Netvibes will remember the setup for future visits from the same computer.

My Yahoo is also an excellent home page with simple ways to add, move, and remove preconfigured modules, but it gives no way to add non-Yahoo Web mail, and its banner ads are distracting. While its default news photos are nice, adding photos from Flickr (a Yahoo property) to My Yahoo is more difficult than doing so with Netvibes.

Edge: Netvibes

Browser Support: Built-In RSS--IE 7 Beta 2 vs. Firefox (With Sage) vs. Opera

Feeds are integrated with your favorites in the IE 7 beta (above). The Sage plug-in for Firefox (left) also organizes feeds in a sidebar, while Opera's built-in RSS support (below) feels more like an e-mail client.
Feeds are integrated with your favorites in the IE 7 beta (above). The Sage plug-in for Firefox (left) also organizes feeds in a sidebar, while Opera's built-in RSS support (below) feels more like an e-mail client.
Among Web browsers, Opera has long been ahead of the curve in supporting RSS feeds natively: It easily adds new feeds and notifies you of new items. But Opera's horizontally divided two-pane display looks a bit bland and lacks inline image support.

Firefox's Sage plug-in is a little rough around the edges. Adding a Feedburner subscription requires cutting and pasting a URL, for example, and while Sage can treat a favorites folder as if it were a set of feeds, that works only with one folder. But once your feeds are setup, pressing <Alt>-S displays the sidebar and an attractive two-column view showing each story and images in its own box.

It's been a long time coming, but the native support in IE 7 Beta 2 works quite well. Feeds are integrated with favorites, and though the feed-by-feed, single column look may not be the most efficient, it's more eye pleasing than Sage's or Opera's. You can search each feed and reorder stories by date, title, or author. If its feed detection improves, IE 7 could win over newsies and newbies alike.

Edge: Internet Explorer 7 Beta 2

Top RSS Readers (chart)

Ryan Singel is a freelance writer based in San Francisco.

Subscribe to the Daily Downloads Newsletter

Comments