What’s holding Siri back

I’ll start with a disclaimer: I don’t have an iPhone, and I’ve never even used Siri. But with a spate of negative articles about her in recent weeks – following the release of a formidable new competitor, Google Voice Search – I’ve read enough to add my two cents.

Siri has two main components: using voice recognition technology to understand your question, and then finding some way to answer it. The recent critical articles have focused largely on the voice recognition half of the package, with titles ranging from Google’s Data Advantage Over Apple’s Siri to With Apple’s Siri, a Romance Gone Sour to Siri, Why Aren’t You Smarter? [published in the August 2012 Scientific American and – so far as I can tell – not yet available online].

To give you an idea of the shift in tone, that last article was written by New York Times tech writer David Pogue, who once penned columns with titles like New iPhone Conceals Sheer Magic and Siri is one funny lady.

But I was surprised to note that none of the articles I came across focused on Siri’s ability to actually answer your questions, or even mentioned one of Siri’s biggest shortcomings: it relies on Wolfram Alpha. (More like, Wolfram Beta.)

Now, Wolfram Alpha can certainly do cool things: ask it to roll a random number, count days until a given holiday, or visualize colors based on hexadecimal notation, and you’ll probably be happy with the result.

But ask it to find you real information, and you’re probably in trouble.

I was unaware of Siri’s reliance on Wolfram Alpha until May, when it came to light that if asked, “What is the best Smartphone ever?”, she would answer with Nokia Lumia 900 – a Windows Phone. This article from PC World pinpoints what went wrong:

Siri uses Wolfram Alpha, which answers questions based on sets of data maintained by the company’s researchers.

And therein lies the problem: Siri is a web-based service – your voice is captured, sent to Apple’s servers, processed, and an interpretation sent back to your phone – but once Siri understands what you asked, she might as well be Encarta with a calculator: in other words, she will consult Wolfram Alpha.

Wolfram Alpha, of course, has nothing to do with the internet. When the service first opened in 2009, I typed in random words just to see how Wolfram would handle it:

You don’t need to be a Mariners fan – or even a baseball fan – to recognize that this result is worse than useless: by the time I retrieved this data, the 2009 season was already over.  That said, you might need to be a fan to notice something wrong in the standings: the Anaheim Angels (I’m too lazy to look up when they switched to the Los Angeles Angels of Anaheim) – who actually won the American League West in 2008 – are absent.

Other search engines – that actually access the internet – have no such trouble. Here’s Google:

And here’s Bing:

Because Wolfram Alpha does not use the internet, it instead relies on an internal database to produce search results. But that approach has serious shortcomings. When I emailed Wolfram about the Mariners back in November, 2009, they responded as follows:

Thank you for your suggestion regarding Wolfram|Alpha. Unfortunately it is not clear to us what dataset you would like to see added to the website. Please elaborate by providing source links to the specific information that you are interested in seeing. When replying to this email, you can leave the subject line as is, with the [W|A #######] intact.

Best wishes,

The Wolfram|Alpha Team

I thought about responding with a link to Fangraphs, but decided I’d rather just stick with the competition.

Because the baseball data accessed by Wolfram Alpha appeared to have not been updated since 2008, I thought it could at least serve a fun time capsule to a time when Seattle still had a team in the NBA. No dice:

And the lack of information is not just because the Sonics no longer exist. Do this for any team in the NBA, and you get the same result: the topic of ‘Basketball’ is under investigation. Wolfram Alpha doesn’t know anything about basketball because no one has told it about basketball.

None of this would be a problem of Wolfram Alpha was just a cool site you could go to any time you wanted to convert text to Morse Code. But it’s not: to mix metaphors, the service is baked into the heart of Siri.

And even if you visit the site – http://www.wolframalpha.com/ – on a web browser, and entirely of your own volition, you’re still liable to be misled; it urges users to “Enter what you want to calculate or know about:”

Wolfram Alpha does exactly one of the bolded functions well. Trying to use it for the other is asking for trouble. On that note, does anyone know what happens when you ask Siri for trouble?

Siri might not be so useful, but at least she’s still good for the entertainment.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s