‘Apple Needs to Get Out of the Way With AI’

Gus Mueller:

A week or so ago I was grousing to some friends that Apple needs to open up things on the Mac so other LLMs can step in where Siri is failing. In theory we (developers) could do this today, but I would love to see a blessed system where Apple provided APIs to other LLM providers.

Are there security concerns? Yes, of course there are, there always will be. But I would like the choice.

The crux of the issue in my mind is this: Apple has a lot of good ideas, but they don’t have a monopoly on them. I would like some other folks to come in and try their ideas out. I would like things to advance at the pace of the industry, and not Apple’s. Maybe with a blessed system in place, Apple could watch and see how people use LLMs and other generative models (instead of giving us Genmoji that look like something Fisher-Price would make). And maybe open up the existing Apple-only models to developers. There are locally installed image processing models that I would love to take advantage of in my apps.

The analogy I used, talking with Jason Snell during my guest stint on Upgrade last week, was to the heyday of desktop publishing. The Mac was the platform for graphic design because it was the best platform for using design apps. Fonts worked better and looked better on the Mac. Printing worked better from Macs. Peripherals worked better. The apps themselves looked better on the Mac than they did on Windows. The Mac had taste and designers (hopefully) have taste. Graphic designers could understand how their machines worked, and maintain them themselves, in a way they couldn’t with PCs.

But Apple didn’t make any of the actual apps. Companies like Adobe and Macromedia and Aldus did. Independent small developers made niche extensions for use inside apps like Photoshop, FreeHand, and QuarkXPress. When a new app came along like InDesign — which quickly ate Quark’s lunch — the Mac remained the dominant platform to use.

Making a great platform where other developers can innovate is one of Apple’s core strengths. Apple got even better at it once Mac OS X hit its stride in the 2000s — the Cocoa APIs really did empower outside developers to make world-class apps providing experiences that couldn’t be matched on other platforms like Windows or Linux. Then it happened again, with a much bigger audience, with iOS. What desktop publishing was to the Mac in the 1990s, social media was to the iPhone in the 2010s. Apple didn’t make the apps — they made the best platform to use those apps.

Apple should be laser focused on doing this for AI now. Where I quibble with Mueller is that I don’t want Apple to get out of the way. I want Apple to pave the roads to create the way. Apple doesn’t have to make the cars (literally) — just pave the best roads. Make the Mac the best platform for outside developers to create innovative AI systems and experiences. Make iOS the best consumer device to use AI apps from any outside developer. Work on APIs and frameworks for the AI age. No company has ever been better than Apple at designing and delivering those sort of APIs. Lean into that. It’s as useful, relevant, and profitable an institutional strength (and set of values) today as ever.

In a follow-up post, Mueller shows he’s thinking like I’m thinking:

But off the top of my head, here’s one idea that I think could really help and reap benefits for both Apple and developers.

Build a semantic index (SI), and allow apps to access it via permissions given similar to what we do for Address Book or Photos.

Maybe even make the permissions to the SI a bit more fine-grained than you normally would for other personal databases. Historical GPS locations? Scraping contents of the screen over time? Indexed contents of document folder(s)? Make these options for what goes into the SI.

And of course, the same would be true for building the SI. As a user, I’d love to be able to say “sure, capture what’s on the screen and scrape the text out of that, but nope - you better not track where I’ve been over time”.

HealthKit already works a lot like what Mueller is suggesting here (for, say, “SemanticKit”). With explicit user permission — that can be revoked at any time — third party apps can both read from and write to your Health data. Apple does a lot of that itself, both through Apple Watch and from the various activity-related things an iPhone can track, but third-party apps and devices are welcome participants, in a private, easily-understood way.

★ Wednesday, 19 March 2025