Apple’s Mysterious Fisheye Projection

Mike Swanson:

If you’ve read my first post about Spatial Video, the second about Encoding Spatial Video, or if you’ve used my command-line tool, you may recall a mention of Apple’s mysterious “fisheye” projection format. Mysterious because they’ve documented a CMProjectionType.fisheye enumeration with no elaboration, they stream their immersive Apple TV+ videos in this format, yet they’ve provided no method to produce or playback third-party content using this projection type.

Additionally, the format is undocumented, they haven’t responded to an open question on the Apple Discussion Forums asking for more detail, and they didn’t cover it in their WWDC23 sessions. As someone who has experience in this area — and a relentless curiosity — I’ve spent time digging-in to Apple’s fisheye projection format, and this post shares what I’ve learned.

Fascinating deep dive.

★ Monday, 15 April 2024