Mobile

How accurate is Siri, really?

Apple's Siri
Credit: Apple
2.27Kviews

About 62%, according to research by the US investment bank Piper Jaffray, which gave the application a D grade for comprehension.

Apple's Siri
Credit: Apple

The research, by Piper Jaffray’s Gene Munster, asked Siri a series of 800 common questions in both lab settings and on a busy street.  As you’d expect Siri did best in the quiet lab setting, where it successfully resolved 68% of the testers’ questions.  Out on the street, however, Siri was slightly less successful, only returning correct responses 62% of the time.

Munster described using Siri as “playing a lottery” when it came to some results.  It appears that one of Siri’s big problems is understanding the quirks in how we speak.  Apple Insider reports that when asked “Where is Elvis buried?” Siri did a search for a person called “Elvis Buried” and when asked “When did the movie Cinderella come out?” Siri consulted Yelp to find the latest cinema listings.

Siri has been dogged with reports of inaccuracies, with some users in the States suing Apple because they feel the system’s capabilities were overstated in the ads.  Some users also reported that the system has great difficulty understanding accents and speech impediments.

So how does Siri stack up against Google?  Munster says that at the moment there is no comparison (well…at least when using Google text search).  He goes on to say that Siri may be able to compete with Google within two years.

But that won’t mean that Siri will have the market to itself by 2014.  Reports suggest that Google’s impressive Project Glass (Look at what it can do now) will be released in two years.  And last week Google announced the release of Google Now, the company’s Android 4.1-specific predictive-search system.

Google Now is an interesting take on Siri-like intelligence.  Google says that the system can learn your daily habits and provide you with information before you even ask for it, which is impressive and creepy in equal measure.

“It tells you today’s weather before you start your day, how much traffic to expect before you leave for work, when the next train will arrive as you’re standing on the platform, or your favorite team’s score while they’re playing. And the best part? All of this happens automatically. Cards appear throughout the day at the moment you need them. “

So in 2014 will we be asking Siri or using Google Now and Project Glass to navigate our day?  Who knows, but things are going to get interesting…and we’ll need more pockets.

Overall, Munster gives Siri a B for comprehension and a D for accuracy – his final report card advises Apple to be “more competitive” if Siri is going to become a standard way of interacting with smart devices.

Judging from what Google has in the pipeline, Apple will have no choice.

4 Comments

  1. I agree — Google Now seems like it’s really awesome and useful at first glance, but man, I could see that getting really creepy really fast. I don’t like a phone knowing my habits better than I do. I’m interested in seeing how Siri improves and expands in the upcoming years, but I also think the people suing Apple are a little bit silly — it IS still in Beta, after all. 

    1.  @annedreshfield Suing Apple is a bit extreme, I’d imaging it takes time to learn voice patterns and accents.  But the privacy concerns with Siri are maybe even worse than Google Now. Siri is apparently storing users’ voiceprints http://bit.ly/N7gXT6 and there are also questions over what sort of information e-readers are collecting http://bit.ly/N3CUSG
       
       

  2. I agree — Google Now seems like it’s really awesome and useful at first glance, but man, I could see that getting really creepy really fast. I don’t like a phone knowing my habits better than I do. I’m interested in seeing how Siri improves and expands in the upcoming years, but I also think the people suing Apple are a little bit silly — it IS still in Beta, after all. 

    1.  @annedreshfield Suing Apple is a bit extreme, I’d imaging it takes time to learn voice patterns and accents.  But the privacy concerns with Siri are maybe even worse than Google Now. Siri is apparently storing users’ voiceprints http://bit.ly/N7gXT6 and there are also questions over what sort of information e-readers are collecting http://bit.ly/N3CUSG
       
       

Leave a Response

Piers Dillon Scott
Piers Dillon-Scott is co-editor of The Sociable and writes about stuff he finds. He likes technology, media, and using the Oxford comma (because it just makes sense).