Private

Monday, 22 December 2008

There has been much discussion lately regarding the use of private iPhone APIs. First with Google Mobile’s publicly acknowledged use of a private API to access the proximity sensor, then again with Landon Fuller’s report of having an app rejected from the App Store for appearing to use the private Cover Flow framework, when in fact Fuller created his own Cover Flow implementation from scratch because the iPhone’s system implementation has no public API.

A software platform provides a number of functions that software running in the environment can call. A public API (Application Programming Interface) is a list of these functions that are documented and supported for use by third parties writing software for the platform. The complete list of functions (or methods, classes, libraries, frameworks, or whatever specific technical terminology applies) available on the platform is usually, if not always, a superset of the functions published in the public APIs. Functions that are not in the public APIs are called private APIs or SPIs (System Programming Interface).

On the iPhone OS, like Mac OS X, APIs are divided into frameworks. Public frameworks are those with a public API for third party developers. Private frameworks are those with no public API whatsoever. Even the public frameworks, however, sometimes contain some private API calls.

There is no real technical barrier, at least in Cocoa and Cocoa Touch, that prevents third party software from calling private APIs. The public/private distinction is a social barrier, not a technical one. The difference, from a developer’s point of view, is that public APIs are more than just a list of what works now; they constitute a promise, a commitment, from the platform provider of what will continue to work in the future.

A private API call is subject to change or vanish in the future. There is some reason why a private API is private. Could be that it is here to stay, that the platform vendor simply hasn’t gotten around to documenting it yet. But it could just as easily be a stopgap that the vendor intends to completely replace. And when the platform vendor in question is an opaque entity such as Apple, you just don’t know.

With the iPhone, there’s the additional issue of the App Store license agreement, which could not be more explicit regarding the use of private APIs:

3.3.1 Applications may only use Published APIs in the manner prescribed by Apple and must not use or call any unpublished or private APIs.

App Store guidelines aside, my take on the use of private APIs is not absolute. If you, as a developer, want to use private APIs for non-essential aspects of an app, and take care to check for their existence before actually calling them, and write code that fails gracefully if they don’t, then maybe it’s OK.

But if your app depends on private APIs, or you don’t test for their existence before calling them, you risk having a mess on your hands (and your users’ hands) in the future, when a system update appears where the private APIs on which you’re relying are changed or removed.

Google Mobile is an example that may well be doing this right — or at least doing something that can’t go wrong. The private API which I publicized their use of is for the iPhone’s proximity sensor, which they’re using to enable the “just lift the phone to your ear, wait for the beep, and speak your query” feature. They’ve implemented their own code to handle the proximityStateChanged message; if proximityStateChanged goes away in a future OS update, Google’s code to handle it simply won’t get called. The lift-to-talk feature will no longer work, but the app won’t crash. Google Mobile does not depend on the lift-to-talk feature — there’s a button you can tap to initiate a voice prompt manually. The point being that private APIs should be handled with extreme caution — they’re the programming equivalent of explosive material.

What has interested me all along regarding Google Mobile’s use of the private proximity sensor API isn’t the technical aspect, but the social one: the question of whether Apple has a double standard in place regarding private API usage by Google versus typical iPhone developers.

The Theory That Some Laws Are Meant to Be Broken

After I published my initial piece on Google Mobile’s use of private APIs, Erica Sadun published a piece at Ars Technica’s Infinite Loop investigating the issue. Sadun has been a prominent iPhone developer going back to the jailbreak era, and is the author of the recently-released iPhone Developer’s Cookbook published by Addison Wesley.

Sadun, in her Infinite Loop piece, espoused the theory that “there are two very different ways that developers use non-standard calls” (where by “non-standard” she meant “unpublished or private”). These are:

Linking to private frameworks. This is the bad, the evil, the Sauron of App Store. Apple offers two sets of frameworks, or libraries that contain linkable routines for developers. There are Public Frameworks, which are fine and dandy to link to, and Private Frameworks, which are not. […]

Using unpublished APIs. If linking to private frameworks is unacceptable, unpublished APIs play the role of a minor jaywalking. You might get a ticket, a citation, and a talking to, but you’re not going to jail forever. The App Store is absolutely littered with unpublished APIs. I’ve learned to spot them pretty well. When you see applications doing things that you know they can do but that they’re not entirely supposed to be doing, you can lay odds that the developers have gently called unpublished but public routines.

This distinction was novel to me. And, upon further consideration, I deem it nonsense. The public iPhone APIs are those which are documented by Apple in the iPhone SDK. Everything else is private, whether it is a method in a public framework or a method in a private framework. They’re both just as much undocumented, just as much subject to change, and just as much in violation of the iPhone SDK license agreement.

A few days later Sadun wrote another piece for Infinite Loop, “Dumping the iPhone 2.2 Frameworks”, with instructions showing how to extract Objective-C header files listing many of the private APIs in the iPhone SDK. She published these header files on her personal web site.

Her header-dumping post does contain a cursory warning that the use of these APIs violates the SDK agreement, and that software that uses such APIs is subject to break under future iPhone OS releases, but there’s an implicit “Go ahead and use them in your own apps if you want to and it’ll probably work out just fine” attitude that pervades. Why else publish such a piece if not to encourage the use of private APIs? And why not a single word encouraging developers to file Radar bugs requesting that desired private methods be made public in the future?

Boom

This general disregard for the division between the public and the private carries over in Sadun’s book. The book contains an entire chapter on how to use the iPhone’s private Cover Flow framework. (The use of private frameworks, of course, being in Sadun’s own words, “the bad, the evil, the Sauron of App Store”.) The chapter introduction contains far more encouragement than warning:

Although Cover Flow is not officially included in the iPhone SDK, it offers one of the most beautiful features of the iPhone experience. With Cover Flow, you can offer users a gorgeously intense visual selection that puts standard scrolling lists to shame. This chapter introduces Cover Flow and shows how you can use it in your application. Boom!

It is one thing for me to lecture in the abstract regarding the dangers of using private APIs — to warn against the fact that the underlying frameworks in the iPhone OS are undergoing significant changes between releases, and that apps which make use of private APIs really are more likely to crash when running on future OS releases.

It is another thing to be able to point to a specific example where this has occurred. An example like, say, Safari Bookbag, the iPhone client for the Safari Books Online service from O’Reilly.

When iPhone OS 2.2 shipped, Safari Bookbag began crashing on launch. iPhone developer Landon Fuller investigated, and found that the source of the crash was a call to a method in the Cover Flow framework that Apple removed between iPhone OS 2.1 and 2.2:

It’s clear from the disassembly that Bookbag is using the private UICoverFlowLayer API. When Apple released iPhone OS 2.2, the -[UICoverFlowLayer initWithFrame:numberOfCovers:] method was removed, and Safari Bookbag started crashing.

It ends up that -[UICoverFlowLayer initWithFrame:numberOfCovers:] is exactly the method which Erica Sadun’s Cover Flow chapter in The iPhone Developer’s Cookbook begins with.

Fuller, as previously mentioned, is the author of Peeps, an iPhone application that provides a Cover Flow style interface for browsing your address book. After a 33-day wait in the App Store submission queue, Peeps was initially rejected by Apple on the grounds that it was using the private Cover Flow API. What made the rejection noteworthy is that Peeps was not using the private framework. Fuller had done the right thing: he wrote his own Cover Flow implementation from scratch, without the use of any private APIs:

The last thing I would do is deliver time-bomb code to a paying customer. Private API can be broken or removed at any time by the vendor, and relying on it is unfair to your customers — they rarely have any idea that the application they just purchased may not work next week, or next month.

So Fuller took the extra time to do the right thing and had his work rejected anyway because whoever it was at Apple who reviewed the submission saw “Cover Flow” and assumed, incorrectly, that it was using the private Cover Flow framework. (Fuller’s story has a happy ending: Peeps was accepted into the App Store over the weekend.)

Given the situation with Safari Bookbag, and presumably, any other application which uses the private Cover Flow framework as prescribed in The iPhone Developer’s Cookbook, this wasn’t necessarily an unreasonable assumption on the part of the App Store reviewer who initially rejected Peeps. Incorrect, unfair, and frustrating, yes — but not altogether unreasonable. Bookbag was not the only iPhone app that was caught in this framework change. Here’s a comment on Erica Sadun’s weblog from a developer who used Sadun’s Cover Flow code for an application named Yoga Trainer Pro. The version of Yoga Trainer Pro currently in the store crashes on iPhone OS 2.2 (see the customer reviews), and the updated version submitted by the developer has been rejected by Apple because it continues to use the private Cover Flow framework.

The more widespread the use of private iPhone APIs becomes, the more likely it is that the iPhone will become the sort of platform where users resist installing OS updates, on the grounds that previous OS updates “broke” third-party applications they had installed.

But so while Erica Sadun’s judgment regarding the use of private APIs may be suspect, she is no hypocrite — Sadun herself is credited as the lead developer of Safari Bookbag. Boom, indeed.