Siri Gains in Loup Ventures’s Annual Smart Speaker IQ Test

Related to John Giannandrea’s leadership of Apple’s AI and ML teams:

We recently tested four smart speakers by asking Alexa, Siri, Google Assistant, and Cortana 800 questions each. Google Assistant was able to answer 88% of them correctly vs. Siri at 75%, Alexa at 73%, and Cortana at 63%. Last year, Google Assistant was able to answer 81% correctly vs. Siri (Feb-18) at 52%, Alexa at 64%, and Cortana at 56%. […]

Over a 12-month period, Google Home improved by 7 percentage points, Echo by 9 points, Siri (9-month) by 22 points, and Cortana by 7 points in terms of questions answered correctly.

And:

It’s also important to note that HomePod’s underperformance in many areas is due to the fact that Siri’s ability is limited on HomePod as compared to your iPhone. Many Information and Commerce questions are met with, “I can’t get the answer to that on HomePod.” This is partially due to Apple’s apparent positioning of HomePod not as a “smart speaker,” but as a home speaker you can interact with using your voice with Siri onboard.

There’s no “apparent” about it — that’s exactly how Apple has positioned HomePod. So part of the gap between HomePod and Google Home is in questions Siri can answer on other devices.

Apple’s biggest problem with Siri will soon be convincing people that it has improved enough to be worth trying again. Siri faces a “boy who cried wolf” problem — except instead of lying, people stopped trying it because it sucked.

★ Thursday, 20 December 2018