Linked List: April 22, 2024

Monday, 22 April 2024

‘The Apple Jonathan: A Very 1980s Concept Computer That Never Shipped’ ★

Stephen Hackett, writing at 512 Pixels:

The backbone of the system would need to accept modules from Apple and other companies, letting users build what they needed in terms of functionality, as D’Agostino writes:

(Fitch) designed a simple hardware “backbone” carrying basic operations and I/O on which the user could add a series of “book” modules, carrying hardware for running Apple II, Mac, UNIX and DOS software, plus other modules with disk drives or networking capabilities.

This meant that every user could have their own unique Jonathan setup, pulling together various software platforms, storage devices, and hardware capabilities into their own personalized system. Imagining what would have been required for all this to work together gives me a headache. In addition to the shared backbone interface, there would need to be software written to make an almost-endless number of configurations work smoothly for the most demanding of users. It was all very ambitions, but perhaps a little too far-fetched.

I’d go further than “never shipped” and describe this is a concept that never could have shipped. It was a pipe dream. The concepts sure did look cool though.

Meta to Start Licensing Quest’s Horizon OS to Third-Party OEMs ★

Alex Heath, The Verge:

Meta has started licensing the operating system for its Quest headset to other hardware makers, starting with Lenovo and Asus. It’s also making a limited-run, gaming-focused Quest with Xbox.

On the theme of opening up, Meta is also pushing for more ways to discover alternative app stores. It’s making its experimental App Lab store more prominent and even inviting Google to bring the Play Store to its operating system, which is now called Horizon OS. In a blog post, Meta additionally said that it’s working on a spatial framework for developers to more easily port their mobile apps to Horizon OS. [...]

Zuckerberg has been clear that he wants his company to be a more open platform than Apple’s. Here, he’s firmly positioning Meta’s Horizon OS as the Android alternative to Apple’s Vision Pro. Given how Android was more of a reaction to the iPhone, an analogy he’d probably prefer is how Microsoft built the early PC market by licensing Windows.

It definitely seems more like Windows than Android — there’s no word that Horizon OS is going open source. But we have an answer regarding what Zuckerberg meant when he positioned the Quest platform — now the Horizon OS platform, I suppose — as the “open” alternative to Apple’s VisionOS.

‘LLM in a Flash: Efficient Large Language Model Inference With Limited Memory’ (PDF) ★

Re: my previous item on LLMs being RAM-hungry while iPhones are relatively low on RAM, this certainly isn’t news to Apple. Back in December, a team of eight researchers from Apple published this paper, which states in its abstract:

This paper tackles the challenge of efficiently running LLMs that exceed the available DRAM capacity by storing the model parameters in flash memory, but bringing them on demand to DRAM. Our method involves constructing an inference cost model that takes into account the characteristics of flash memory, guiding us to optimize in two critical areas: reducing the volume of data transferred from flash and reading data in larger, more contiguous chunks. Within this hardware-informed framework, we introduce two principal techniques. First, “windowing” strategically reduces data transfer by reusing previously activated neurons, and second, “row-column bundling”, tailored to the sequential data access strengths of flash memory, increases the size of data chunks read from flash memory. These methods collectively enable running models up to twice the size of the available DRAM, with a 4-5× and 20-25× increase in inference speed compared to naive loading approaches in CPU and GPU, respectively. Our integration of sparsity awareness, context-adaptive loading, and a hardware-oriented design paves the way for effective inference of LLMs on devices with limited memory.

On-Device AI Craves RAM ★

Ron Amadeo, writing for Ars Technica on a purported leak of a trio of Pixel 9 phones:

Rozetked says (through translation) that the phone is “similar in size to the iPhone 15 Pro.” It runs a Tensor G4 SoC, of course, and — here’s a noteworthy spec — has a whopping 16GB of RAM according to the bootloader screen. The Pixel 8 Pro tops out at 12GB.

Anything could change between prototype and product, especially for RAM, which is usually scaled up and down in various phone tiers. A jump in RAM is something we were expecting though. As part of Google’s new AI-focused era, it wants generative AI models turned on 24/7 for some use cases. Google said as much in a recent in-house podcast, pointing to some features like a new version of Smart Reply built right into the keyboard, which “requires the models to be RAM-resident” — in other words, loaded all the time. Google’s desire to keep generative AI models in memory means less RAM for your operating system to actually do operating system things, and one solution to that is to just add more RAM. So how much RAM is enough? At one point Google said the smaller Pixel 8’s 8GB of RAM was too much of a “hardware limitation” for this approach. Google PR also recently told us the company still hasn’t enabled generative AI smart reply on Pixel 8 Pro by default with its 12GB of RAM, so expect these RAM numbers to start shooting up.

That last link is to a story positing that Google’s Gemini Nano runs on the Pixel 8 Pro but not the regular Pixel because the Pro has more RAM (12 vs. 8 GB).

Comparing iPhone RAM to Android RAM has never been apples-to-apples (same goes for battery capacity), but still, it’s hard not to wonder whether Apple’s on-device AI plans are hamstrung by the relatively stingy amounts of RAM on iPhones. Here’s a list from 9to5Mac showing the RAM in each iPhone going back to the original (which had just 128 MB!). iOS 17 supports models dating back to 2018’s iPhone XS and XR (4 and 3 GB of RAM, respectively). If iOS 18 drops those models, the new baseline will be the iPhones 11 and 11 Pro, which all sport 4 GB. The most RAM on any iPhone to date is 8 GB in the 15 Pro models, but 8 GB is what Google deemed insufficient for the Pixel 8 to run Gemini Nano.

Might some iOS 18 on-device AI features be limited to newer models with more RAM?

Update: Follow-up post regarding base Mac memory.