The Talk Show Live From WWDC 2024

Friday, 14 June 2024

Recorded in front of a live (and lively) audience at The California Theatre in San Jose Tuesday evening, special guests John Giannandrea, Craig Federighi, and Greg Joswiak join me to discuss Apple’s announcements at WWDC 2024.

3D video with spatial audio: Exclusively in Sandwich Vision’s Theater app on Vision Pro, available on the App Store. Just launch Theater and tap the “Watch Live Event” button. Tip: For this show, try choosing a seat in the front row center, not the middle of the theater.

Presenting sponsors:

  • iMazing 3 — The all-new version of the world’s best iPhone manager, for Mac and Windows. Save 20 percent.

  • Flexibits Premium — Fantastical. Scheduling. Cardhop. All your devices. Save 20 percent, both for new and existing subscribers.

  • Flighty — The first-class flight tracker for those on the move. Unmatched notifications and Live Activities. Delay predictions. Power through clarity.

As ever, I implore you to watch on the biggest screen you can (real, or virtual). We once again shot and mastered the video in 4K, and it looks and sounds terrific. All credit and thanks for that go to my friends at Sandwich, who are nothing short of a joy to work with.

The livestream of 3D video with spatial audio went almost perfectly, and the feedback from viewers who joined the stream has been unanimously positive. My sincere thanks and gratitude to SpatialGen for their remarkable work on that.

Not just shooting this event in 3D, but also streaming it live, was entirely the initiative of my dear friend, Mr. Sandwich himself, Adam Lisagor. I was asked a few times this week whether it was Apple who wanted to stream this live in 3D for viewing on Vision Pro. Nope. It was Adam who pitched me on the idea, only about eight weeks ago. I was like, “Well, sure, that sounds awesome, but how in the world would we do that? What camera could we use to shoot with? How would we stream it? What app would people be able to view it in? It’s a great idea but none of this seems possible.” Adam was like “I think we can do it.”

And, son of a bitch, they did it.

Once I started talking with Apple about arranging for guests, we did let them know our plans to shoot and livestream in 3D for viewing in Theater on Vision Pro — an app that, at the time, was in early beta. Maybe even alpha. Apple’s reaction echoed my own: sounds great but how?

The how is a long story and I get zero credit for any of it, but it involves a custom camera rig with two Lumix BGH1 cameras, each with Olympus 17mm PRO prime ƒ/1.2 lenses (35mm equivalent field of view with the BGH1’s Micro Four Thirds sensor). The lenses were about 3 inches apart — as close as possible — and microphones were placed throughout the theater to capture spatial audio.

The results exceeded my expectations, and I think everyone else’s as well.

To be clear, we shot the show two entirely different ways. The standard multi-camera cut on YouTube, embedded above, was shot exactly as we’ve done in previous years. That video turned out great too. The 3D version in the Theater app was shot from a single point of view, roughly simulating the perspective from a front-row center seat in front of the stage. It looks cool and sounds even better. If you have a Vision Pro or access to one, I highly encourage you to check it out.

We’re describing it as “3D video with spatial audio”, not “spatial video”, because that’s a more precise description of the effect. True spatial video would be more immersive — “Look left to see Joz, look right to see Gruber” — and the effect we achieved wasn’t quite like that. But what we did get is immersive, and very compelling. To the best of my knowledge, this was the first event livestreamed in 3D for viewing in VisionOS, and while my role was simply as the host of the show, that’s pretty damn cool. I’m just amazed at what Lisagor and his team at Sandwich (and SpatialGen) were able to pull off in a matter of weeks.

Reactions From Social Media

Matt Birchler: “I was only able to watch the first 40 minutes or so live, but it was really good, and really compelling to watch in immersive video. Can’t wait for the rest.”

Kevin Pfefferle: “Wow, The Talk Show Live exceeded all expectations. So many candid and insightful answers straight from Apple execs (plus a fair share of generic no-comment answers with a wink and a smile). Bravo! 👏🏻”

Kalani Helekunihi: “I watched the Talk Show Live on Vision Pro tonight. It was a really great experience, and felt close to sitting in the front row. Only real issue was bitrate / bandwidth related. If that can be solved, I imagine I’d prefer this to the real thing.”

Dave Marquand: “Really enjoyed the spatial livestream of The Talk Show Live. Taking the front row center seat in the Theater app definitely did feel a lot like sitting in row 1 at a stage performance. Oddly, the “most 3D” thing about the show was the glint off of your watch reflecting off of the floor of the stage in the foreground.”

Kalani Helekunihi, again: “Yep, there was even a moment where the glint caught the right eye camera. Was one of the most “accidentally real” experiences I’ve had with a headset, ever. Those sort of things just can’t be experienced with a TV screen. It all felt like sitting there, and looking into the stage.”

Michael Edlund: “I’ve never been lucky enough to attend @gruber’s The Talk Show live in person, and I was blown away by the 3D #live stream for #VisionPro that was literally like sitting in the first row at the edge of the stage.”

Kendall Gelner: “The 3D live stream was amazing! I just had one small hang in the first few seconds but went back in and it worked great the entire time, was just like sitting front row only in the most comfortable seat and I could easily refill a drink any time. TYSM!”

Nick Bodmer: “Watched The Talk Show Live on my Vision Pro last night — what an incredible experience! 🌟 Streaming it in spatial vision made me feel like I was right there in the front row. Sure, there were a few frame rate hiccups, but the immersive experience more than made up for it. Kudos to SpatialGen, Sandwich, and @gruber for pulling off a fantastic live event! 👏”