The personal computer has endured for more than 30 years. We understand it. It’s familiar. But digital assistants—the new breed of smartphone data butlers designed to make our lives simpler—have yet to climb out of their cribs.
This holds true for Cortana, the digital assistant built into Windows Phone 8.1, which Microsoft released as a developer preview on Monday. It’s packed with intuition, but it’s still a beta product, and it isn’t a miracle worker by any means.
The perfect digital assistant would anticpate our every need, retrieving answers from search engines, connecting us with our social networks, and tapping into maps and calendars to guide our travel and appointments. But no single digital assistant has yet figured out how to connect all these dots in an effective, intuitive way.
As we compare the three big names in digital assistants—Cortana, Apple’s Siri and Google Now—we expect these services to evolve gradually from helpful librarians into full-fledged concierges. But we’re not there yet. The assistants do an admirable job in responding to queries, but they haven’t yet mastered the ability to tell us what we need even before we ask.
Just the facts, ma’am
To test the three services, we quizzed each assistant on a variety of general facts and calculations. Then we asked them to perform a number of common smartphone operations—nearly 40 in all. We scored each assistant on the accuracy of its results and the elegance of its operation.
We awarded two points if an assistant completed a task by providing simple search results, and three points if it responded orally or displayed a “card,” which would add an image and further information to the oral response. We also awarded a bonus half-point if an assistant completed a task especially well.
We wanted to penalize outright failure heavily, which is why our scale starts at two points. If one of the assistants couldn’t provide any helpful response, it received a zero.
So how did our three contenders perform? Check out our list of questions and tasks for the full rundown (clicking the following link will download a read-only spreadsheet file):