The personal computer has endured for more than 30 years. We understand it. It’s familiar. But digital assistants—the new breed of smartphone data butlers designed to make our lives simpler—have yet to climb out of their cribs.
This holds true for Cortana, the digital assistant built into Windows Phone 8.1, which Microsoft released as a developer preview on Monday. It’s packed with intuition, but it’s still a beta product, and it isn’t a miracle worker by any means.
The perfect digital assistant would anticpate our every need, retrieving answers from search engines, connecting us with our social networks, and tapping into maps and calendars to guide our travel and appointments. But no single digital assistant has yet figured out how to connect all these dots in an effective, intuitive way.
Florence Ion Cortana includes a feature that ties reminders to proximity. But how well does it work in practice?
As we compare the three big names in digital assistants—Cortana, Apple’s Siri and Google Now—we expect these services to evolve gradually from helpful librarians into full-fledged concierges. But we’re not there yet. The assistants do an admirable job in responding to queries, but they haven’t yet mastered the ability to tell us what we need even before we ask.
Just the facts, ma’am
To test the three services, we quizzed each assistant on a variety of general facts and calculations. Then we asked them to perform a number of common smartphone operations—nearly 40 in all. We scored each assistant on the accuracy of its results and the elegance of its operation.
We awarded two points if an assistant completed a task by providing simple search results, and three points if it responded orally or displayed a “card,” which would add an image and further information to the oral response. We also awarded a bonus half-point if an assistant completed a task especially well.
We wanted to penalize outright failure heavily, which is why our scale starts at two points. If one of the assistants couldn’t provide any helpful response, it received a zero.
Mark Hachman Cortana presents visually robust results for map information, but no oral response.
So how did our three contenders perform? Check out our list of questions and tasks for the full rundown (clicking the following link will download a read-only spreadsheet file):
Here’s the short story: Google Now scored 101 points, Siri scored 97 points, and Cortana came in last with a score of 87. We really couldn’t factor in the proactive powers of each assistant—and this is exactly why our numerical scores are only part of the story.
A key value of a digital assistant is how well it anticipates what you want before you want it. When it works, it’s magic. The first time Google Now pinged me with an unprompted message telling me my flight was delayed, it blew my mind.
Without a substantial body of emails, searches, and other contextual information to pull from, over time, it’s difficult for any digital assistant to provide proactive assistance. This factor alone inhibited our ability to fully cross-compare Cortana, Google Now and Siri. And keep in mind that although these digital assistants were built with “natural language” in mind, certain phrases aren’t recognized: “tweet hello” and “tweet on Twitter hello” weren’t recognized by Google Now. “Update Twitter hello” is, but do users recognize that?
At this point, however, we can definitively state that at this point Siri lacks the proactive, “I’ll help you before you even ask” intuitive powers of Google Now and Cortana. This puts it at the bottom of the digital-assistant heap, despite its silver-medal showing in our task testing.
The general rule of thumb is this: The more data you feed a digital assistant, the better it works. Unfortunately, it’s almost impossible to reconcile this reality with privacy concerns. But that’s the future of mobile computing.
Now let’s look at each service in detail.
Cortana: an impressive debut
The beta Cortana appears to be taking aim at Google Now, as the two assistants’ capabilities overlap quite a bit. Still, it’s great to see Microsoft striking off on its own.
Mark Hachman Cortana can find highly-rated restaurants near you.
Cortana can be accessed by tapping on its Live Tile or holding down the Search button. Unlike the other two assistants, it will ask you your name and how to pronounce it. (You also have the option of adding something silly, such as “Master,” for an “I Dream of Jeannie” effect.)
At startup, Cortana will ask permission to access your email, People, and contacts, as well as your location and even your Facebook account. If you’d like to receive the full Cortana experience, allow this. Via Bing, Cortana also asks for an incredible amount of detail about you and your Facebook friends. Be warned.
Next, Cortana will ask you some personal questions, such as what you like to do in the evening and what types of information interest you most. These are mostly harmless and are used to set your initial interests—topics that Cortana will research for you.
The results appear when you launch the Cortana app: Below Cortana’s spinning circle icon, you’ll see a “daily glance” of news headlines that have been culled from your stated interests. The top news snippet will usually appear in the Cortana Live Tile. I didn’t find this especially compelling, but I’m willing to assume that it will improve over time.
Mark Hachman A list of “interests” Cortana will monitor.
You can add interests whenever you’d like, although there’s a limited number to choose from: Daily Routine (traffic and the “daily glance” of news headlines); News (spanning all sorts of topics); Eat + Drink (meal suggestions); Travel (a trip planner); and Weather.
Expect Cortana also to figure out where you live and work—i.e., the locations you frequent during the workweek and on weekends. Unfortunately, Cortana didn’t tell me any information about travel times when I set a meeting.
Keying Cortana to the search button was a smart move—it’s the easiest way to access it. Certain queries will generate an attractive “card” response, including an oral response, an image, and some additional explanation. It’s also worth noting that you can trigger Cortana without first bypassing the lock screen. Searching via Cortana can pull up contacts, music, and documents stored on the phone itself—but, oddly, can’t seem to search your OneDrive account.
Although Cortana handled everything we threw at it pretty well, there were some inexplicable omissions: Asking it to email a contact failed during my hands-on, and it still didn’t work at the time this story posted.
Neither does asking Cortana for the details of an airline flight (bad news, as that’s one of Google Now’s strengths) or telling it to take a picture. The latter generates a page of search results, which is just laughable.
Mark Hachman Cortana usually delivers you a few news snippets when opening the app.
Finally, ordering Cortana to call “Round Table Pizza” won’t work if the number isn’t saved to your contacts. Both Siri and Google Now will simply search for the number in the background and call the closest location. Cortana is a beta product, and it shows in quirks like these.
You can tell Cortana to listen to and identify a song being played. It will even match that song to a track in your collection or in the Xbox Music store. There’s also a setting to enable “vision searches” with the camera, but I couldn’t get it to work.
Cortana includes a few nifty features that aren’t available in Google Now, as far as I know: Quiet Hours, and the related Inner Circle of friends. (Apple’s iOS6 added these capabilities, called Do Not Disturb.)
As the name implies, you can set a time (after 11 pm, for example) when Cortana will intercept a call and prevent you from being disturbed. It’s convenient for bedtime, or even during critical business meetings. But Cortana will let a privileged Inner Circle of friends break through the cone of silence. You can also tell Cortana to allow repeated calls through, indicating an emergency.
Here’s another handy feature: If you receive a text saying, for example: “Let’s meet at AT&T Park at 7 to go see the game,” Cortana will highlight part of the text. Clicking it allows you to schedule it automatically on your calendar.
Siri doesn’t suck, but she’s limited
At this point, the notion that Apple’s Siri simply can’t cut it is ingrained into the national zeitgeist. The Simpsons parodied it. A disgruntled customer filed suit because of it. And when Siri launched, maybe there was a case to be made. But iOS 7 reportedly improved Siri, and our tests showed that it holds its own.
Mark Hachman Siri handles basic queries with aplomb.
In terms of supplying basic facts, Siri shines, combining accurate responses with cards that offer deeper information dives, especially when the assistant taps online math engine Wolfram Alpha for its answers.
When you ask more complicated questions, Siri often defaults to a list of search results. We didn’t ding any of the assistants more than a point if the answer was readily apparent from the list of responses. But when more complicated, interpretive questions were asked—“How long does it take light to travel to the Earth from the Sun?”—Siri failed to provide any sort of coherent result.
Perhaps it was too much to ask for Siri to play a song we didn’t own in iTunes, but its ability to handle common tasks also lacked sophistication. For example, Siri lacks geofencing capabilities, so asking it to remind me to buy eggs and milk at the grocery store set up a generic reminder, rather than keying it to a specific location, as other services do.
On the other hand, Macworld goes into several things Siri does do that other services don’t: Utter “Read me my email,” and Siri will comply. Say “Turn on Bluetooth,” and Siri dives into the settings menu. Ask her “What’s going on,” and she lists trending topics on Twitter.
Siri seems to be the only assistant that can reliably post to Twitter and Facebook through voice commands alone.Siri’s greatest weakness is that she does what you tell her to do, and little more. Siri’s good at pinging you with reminders you set, but she won’t step in to remind you that your flight is late. In fact, if you don’t say “Tell me about Southwest flight 212,” the list of search results it generates simply includes the Southwest webpage. On the other hand, iOS apps can step in. YouTube, for example, pinged me about a new video I might be interested in when I rebooted the iPhone. And Siri can post to Twitter and Facebook.
If you’re unimpressed with Siri, you can explore alternative digital assistants for iOS: Cue and EasilyDo, among others. You can also install Google Search, which hides the third—and best—digital assistant, Google Now. Yes, if you do think Siri sucks, you can actually download a superior digital assistant, Google Now, directly to your iPhone.
Google Now: Still the best
Google Now launched with Android 4.1 in July of 2012, and the software has only improved with age. Google has broadened its reach, from tracking packages to providing travel reminders. Google doesn’t thrust itself into your face. If anything, you sometimes need to dig a bit to find all it has to offer.
Mark Hachman Google Now throws up a collection of cards fr you to browse through.
And that’s Google’s strength: Google Now virtually invented proactive notifications. In general, what you need is pretty close to your fingertips, when you actually need it.
Google’s weakness? It’s the seemingly infinite variety of Android phone manufacturers, their skins and customizations, and the many permutations of Google’s operating system and apps. On an older Samsung Galaxy Nexus, it’s relatively easy to find Google Now: swipe up from the bottom of the screen. But on a Samsung Note 3, Samsung actually hides Google Now behind its inferior S Voice controls, robbing the digital assistant of much of its power.
What we think of as Google Now actually consists of two separate services: Voice Search, and Google Now proper. Voice Search is the command center. By tapping the microphone in the Google search widget (or by simply saying “OK Google” on supported phones), you can command your Android device to call a friend, navigate to a location, ask a question, set a reminder, play music, or perform a Web search.
In general, Voice Search works independently, so if you want to ask Google what the capital of North Dakota is, you can. Enabling Google Now, however, opens up a much broader set of options.
Mark Hachman Google Now also handles basic searches quite well.
To do so, swipe up in the lock screen or from the bottom of your screen. On the Note 3, for example, you can tap once on the home screen, then tap the Google button at the bottom. In any event, if you haven’t used Google Now before, you’ll be asked to define cards Google Now should suggest. There are cards for traffic, weather, public transit, and much more. (CiteWorld goes into a lengthier explanation of the Google Now setup process.)
Google uses a more organic process than Microsoft appears to. For example, if you once searched for a local Thai restaurant, you may occasionally see a card pop up with the time it will take to get there. Search for the score of the Miami Marlins game, as I did to test it, and Google thinks you’ll want to monitor the Marlins in the future. Google Now can be a bit overeager.
Once Google Now is configured, you can swipe down through the cards. On each, there’s an options menu to the upper right: Sports scores can be hidden, for example to “avoid spoilers.” You can also filter out updates from a particular website, so cards about that website quit clogging up your Google Now stream.
Mark Hachman A request to play a certain song yields options in Google Play Music (your own cloud storage locker) and YouTube.
At the bottom of Google Now’s main interface, a menu option lets you poke through the Settings menu. There’s also a “magic wand” to help you fine-tune your cards, and the finger icon will list your current reminders.
Overall, Google Now does an excellent job of answering questions, almost always directly answering them with a card. I would swear, however, that when Google previously fielded a command like “Remind me when I get to the grocery store to pick up milk,” it would lock on to the nearest grocery store. But not any more. For now, Cortana gets the nod on this trick.
Google also does a great job of combing through your email and location, suggesting places to go and setting reminders of upcoming events. One of Google Now’s claims to fame is mining your email and your calendar for your next appointment, determining when it is, checking traffic, and calculating when you need to leave. That’s integration.
I would appreciate seeing a bit more information about my current location, though, possibly culling some of Google’s Field Trip app knowledge into a better contextual awareness.
Google Now also now ties into your desktop PC. If you don’t notice your phone’s vibration in your pocket, a notice will pop up in the corner of your Chrome browser.
This isn’t the final word on the subject; all three services will continue to evolve, and improve. And all three have unique talents, providing intriguing directions for differentiation. For now, Google Now sets the bar for what a digital assistant should be, and based on our tests, it’s still the superior service. Microsoft’s Cortana makes a valiant effort, but it needs to shore up some of its weaknesses, such as Facebook and Twitter integration. And quite frankly, if Apple built in some predictive capabilities into Siri, Siri could retake the lead.