Search Sites Try to Give Instant Answers

Figuring out what keywords to use for a Web search has always been a challenge. And once you get your results, you have to guess which link will have the answer to your question. Wouldn't it be nice just to type in a regular question and get a direct answer?

Three search engines promise to serve up direct answers to so-called natural-language queries: natural-language veteran AskJeeves; GuruNet's Answers.com, a relative newcomer to Web search; and MSN Search, which recently beefed up its search results with information culled from Microsoft's Encarta encyclopedia. In our sample searches, however, the three services did much better at answering our general-information questions than they did at replying to technical, topical, or geo-specific questions.

We expected the answers to our questions to appear at the top of each site's initial results page, without our having to follow a link. We gave the sites some credit when the correct answer was on the first page in the search results (one click away), or was within the top ten links returned (two or more clicks away).

We posed dozens of questions and reduced our results to a representative 15 (see chart). AskJeeves produced four on-the-money answers; MSN Search and Answers.com answered three of our questions spot-on. (We included Google as a control; its results were almost identical to those of Answers.com.)

Of course, Web information is constantly changing. Our latest results varied considerably from those we got when we asked the same questions days earlier.

All three sites were on target with the question "What was John Dillinger famous for?" (though we had to click the top link at Answers.com), and they also aced "Who was the 17th president of the United States?" Yet none of the three could immediately answer a question about the speed of USB 2.0 versus ethernet.

Extra words--ironically, the natural language--often got in the way of good results. For example, entering the keywords "high tide Santa Monica California" returned better results than "When is high tide on October 15, 2005, in Santa Monica, California?"