Post navigation

New Poll: Apple has “Oversold Siri” says 46% of Americans

Following our recent post on the dangers of overselling the capabilities of voice recognition software (see here), we commissioned a poll of over 2,000 Americans to see whether they had used Siri in iOS7, what they thought of Siri’s capabilities, and whether they had been “oversold” by Apple.

The results were a little shocking. Once we had pruned out those people who did not use Siri on iOS7 (being the most up-to-date version and now officially out of beta), we were able to ask two different groups of people to rate Siri, and to say whether they thought Siri had been oversold. Both groups came back with almost identical results.

The first group, asked if they felt that Apple had “oversold the voice recognition capabilities of Siri” came back with 46% agreeing, an amazingly high result when you consider this is a flagship application.

The second question looked more at how people rated accuracy. This question was asked of a different sample group, and they came back with a 44% “Hit or Miss” or worse score:

All told, not great news for Apple, but probably worse news for the voice recognition industry. We sell it like it’s magic, but as I’ve read elsewhere, it’s still an emerging technology that’s been emerging for 30 years. The sooner we are honest with people about what can really be achieved, the more chance we have of adoption. Back in the early days of OCR, we only got 50% accuracy, but that was 100% more than people had the day before. Perhaps we should reset people’s expectations as to what continuous speech applications can manage out of the box and in the wild.

We need to pick the applications carefully first where there can be real benefit (like audio data mining and indexing), and as the technology matures and we can get close to 100% accuracy in a variety of situation, allow the technology back into the mainstream. But that is going to require a quantum shift from how we do speech recognition now.

About Nigel

Nigel spend the first 15 years of his working life as a lawyer, qualifying at Withers, moving to Slaughters, and ending up as the EMEA General Counsel at BEA.
He left to start investing in innovative software companies, one of which later merged with Chase. Nigel is passionate about innovation, and gets very frustrated when someone tells him that something is "technically" impossible.

While I agree with the general theme of the post, I find the first survey question somewhat leading. By planting a negative concept like “overselling”, consumers may have answered positively at a higher rate than if the question was poised in a different fashion. If the question was “Has Apple impressed you with Siri’s voice recognition capabilities” or “Has Siri exceeded the expectations that Apple has set” with a scale ranging from strongly agree to strongly disagree, and extracting the bottom box results, we might have come across different findings. Just a thought.

It doesn’t appear to be mentioned, so could you clarify something for me? With the poll at the bottom – “Have you used Siri in iOS7?” – were you able to differentiate users with iOS7 who had Siri with users who did not (since iOS7 is available for iPhones without Siri)? And if so, did you throw out answers from that latter subgroup?

I ask because a CNet article is using that specific poll question in a column about overall Siri usage (rather than customer satisfaction), and since this study was mainly focused only on iPhone users who definitely use Siri, I’m worried they’re drawing wrong conclusions from your findings.

The poll was directed at US internet users in general. We asked a screening question that was specifically to ensure that the people responding had used Siri in iOS7. This was to ensure that people were on the first “non-beta” release.

The 85% figure is the percentage of consumers in general who have not used Siri in iOS7, not the number of people who have access to Siri in iOS7 who have not used it, so the CNET interpretation of the results does seem to be wrong.

For us the headline figure was that for those people who had used the bang up-to-date version of Siri, 46% felt it had been oversold, which even allowing for the potential +/-8% skew, is a really large figure.

Let’s be fair, SIRI is a powerful tool and when all is working and connected it is what the VUI industry has been waiting for. However, the “magic” depends on the web and when the connection is not there or weak, so will the performance. That is why the best combination is one of embedded solutions plus cloud-based. Embedded speech technology solutions add to the existing power of cloud-based by ensuring always available and reliable recognition and wake-up functionality.