Seven Replies to the Viral Apple Reasoning Paper

Simon Willison, regarding the various rebuttals to “The Illusion of Thinking” research paper (which I linked to here) from Apple’s machine learning team:

I thought this paper got way more attention than it warranted — the title “The Illusion of Thinking” captured the attention of the “LLMs are over-hyped junk” crowd. I saw enough well-reasoned rebuttals that I didn’t feel it worth digging into.

And now, notable LLM skeptic Gary Marcus has saved me some time by aggregating the best of those rebuttals together in one place! [...]

And therein lies my disagreement. I’m not interested in whether or not LLMs are the “road to AGI”. I continue to care only about whether they have useful applications today, once you’ve understood their limitations. [...] They’re already useful to me today, whether or not they can reliably solve the Tower of Hanoi or River Crossing puzzles.

Count me in with Willison. I think it’s interesting what constitutes “reasoning”, but when it comes to these systems, I’m mostly just interested in whether they’re useful or not, and if so, how.

See also: Victor Martinez’s rebuttal to the most-cited rebuttal.

★ Tuesday, 17 June 2025