How do the big three digital assitants compare?
The rollout of Windows 10 brought with it the release of Microsoft's natural-language digital assistant, Cortana. There's been long enough since then, we would hope, for it's bugs to have been worked out, so how does it compare to the competition? We tested Cortana, Siri and OK Google - all three major digital assistants - with a spread of questions to see what happened. It's not completely scientific, as Cortana was tested on a desktop PC and Siri/OK Google on their respective iPhone apps, but we do think this should give some insight into how each one behaves.
Question 1: "What Are The Lyrics To Auld Lang Syne?”
A fairly straightforward question, and - we suspect - a common enough request at the end of most years that you'd hope for some kind of intelligent reply.
Cortana
Cortana simply opened my default browser with a pre-programmed search, which isn't particularly revolutionary. That said, it did use my default search engine (Google) rather than push me to Bing, and the first result on the Google Page did contain the answer.
Siri
In sharp contrast to Cortana, Siri displayed the lyrics I was looking without ever leaving the Siri app., thanks to its integration with the knowledge engine Wolfram Alpha.
OK Google
No surprises here. It simply Googled the question and the results were the same as Cortana came up with.
WINNER: Siri
Question 2: "How Do I Get To New York?”
A deliberately difficult route-finding question. In this case, I'd obviously need to take a plane (or possibly a boat, if I had the time), so how useful will a digital assistant be in helping me find one?
Cortana
Cortana was honest in telling me 'I don't know if you can get their via transit', but that's not much use. It did provide an illustrative Bing map of the State of New York and informed me that I was '3392.8 Mi' away, but very little else.
Siri
Siri replied 'Getting directions to New York, NY', before loading Apple Maps and telling me there were no walking instructions available. Switching to public transport made no difference. So, no help at all.
OK Google
Again, OK Google just googled that question. In this case, the first result was broadly useful in helping me find out, with the first result being travel information from a tourism website and links to flight websites following up. It wasn't what I was hoping for (which was some kind of route/example trip itinerary) but it's the closest to being useful.
WINNER: OK Google
Question 3: "Is It Raining Outside?”
This is a simple question that shouldn't be very difficult to answer based on the information the devices have (i.e. my location and the weather forecast). So let's see how they did...
Cortana
A response of 'I can't get that for you, but here's the forecast', and bringing up the weather for my current location, was a little vague. It's not clear why it can't tell me whether it's raining, when it knows where I am and what the precipitation is, but it may as well have said 'figure it out yourself, lazy.'
Siri
Responded with 'No, I don't think it's raining right now' before showing me the weather forecast for my current location. Bang on.
OK Google
Showed me the weather for my location, but provided no no audio feedback.
WINNER: Siri
Question 4: "Is My Local Supermarket Open?”
This one is more complicated as it requires the assistant to find the nearest supermarket, then check its opening hours. A relatively tough process.
Cortana
Launched Google with the search query 'Is my local supermarket open?' already entered. However, all of the results that search produced were links to a news story about Morrisons store archived from September 2015. That's pretty pathetic, frankly.
Siri
Siri presented me with a list of my nearby supermarkets. Then it asked me to confirm if I wanted to hear details for the closest one. I confirmed, vocally, after which it gave me the opening hours for the supermarket itself. Could barely have been smoother.
OK Google
Identical results to Cortana. No attempt at figuring out the context of my query.
WINNER: Siri. Again.
Question 5: "Do I Have Any Appointments Coming Up?”
This is the kind of example question they always use in adverts for digital assistants. So the question is: do I?
Cortana
The response was pretty weak, truth be told - but there's a good reason for this. I don't use Windows' calendar - all my data's in Google's calendar. Cortana didn't attempt to look there, though, insisting my schedule was empty and failing to suggest any way to link to other sources, which would've been useful.
Siri
Siri, by contrast, displayed and then read out a week's worth of appointments, which it had previously helped me sync from Google. Although it did also suggest I had an appointment for Burns' Night on the 25th - when that's actually a public holiday, which has been synced from a shared calendar.
OK Google
Showed me a few things from my Google Calendar and cut out the public holidays. It didn't read them out, but in this case that's probably better - Siri took a while to get through the list and I couldn't see all the appointments at once. Obviously, it was able to get them directly from my Google Calendar.
WINNER: OK Google
Question 6: "Open the Pod Bay Doors”
Not technically a question, but it’s important for your digital assistant to have a good sense of humour. How did these three fare with the classic quote from 2001: A Space Odyssey?
Cortana
Twice, Cortana replied with different (and completely apt) quotes from the movie. That's what you want to hear.
Siri
By comparison, Siri gave me back a selection of withering, ironic, world-weary responses designed to remind me that my joke was unoriginal. I'm not sure I want my digital assistant to have a higher status than me. Then again, that's exactly what you'd expect from an Apple employee.
OK Google
Was not interested in playing at all, although it did give me a YouTube link to the pod bay doors sequence from the movie - so in that sense it was helpful. Helpful isn't what I was looking for this time around, however. Hilariously, the second link was a list of fun things you can ask... Siri.
WINNER: Cortana
This wasn't a strong showing for Microsoft's assistant, all things considered. This far into its life, one would hope that the bugs had been worked out of Cortana so it can compete convincingly with its direct rivals. As it was, Siri was a far more consistent performer, while OK Google only really poked its nose out in front when it was integrating with Google services over Apple and Microsoft's alternatives.
Of course, if you want a movie quote answered with the correct level of pithy response, Cortana's the one to go for. Otherwise you might be better off sticking with whatever alternative you've got - at least for the moment.