Transcript of The Talk Show Episode 188

The Announcer (Paul Kafasis): Hello, fellow nerds!

[applause]

Good evening, and welcome to San Jose’s historic California Theater. Tonight, we are very pleased to present The Talk Show Live!

[woos, cheers, clapping]

Introductions and Sponsors

Before we begin, please take a moment to set your Apple Watches to Theater mode…

[scattered laughter]

… silence your iPhones, and turn off any other electronic devices.

And now, please join me in welcoming Mr. John Gruber!

[huge cheers and applause as Gruber takes the stage]

Gruber: Hello, welcome. I am John Gruber.

[scattered laughter]

We have a great show. We have two first-time guests on the Talk Show.

[excited “oohs”]

But before I get to them, I have some very, very good people to thank. They are the sponsors who made all this possible. First sponsor: great company. MacStadium.

[woos and clapping]

MacStadium provides enterprise-class hosting for Macs. Get your build server out of your office closet or wherever you have it hidden, and put it in a world-class data center. MacStadium can securely handle it for you.

Thousands of companies all over the world trust MacStadium. They have multiple ISO 27000 data centers, including a brand-new one right here in Silicon Valley. (I think it’s in Sunnyvale.) I don’t know what ISO 27000 certification is for a data center…

[laughter]

… but apparently it is an enormous pain in the ass and is very important.

[more laughter]

And I think it means that it’s like really good.

All-size companies can host with them, though! You can be like “I don’t know what ISO 27000 is, I just want to put a Mac mini in there and have builds go to it.” You can do that. They’re set up to scale from like a single iOS developer who just wants like one Mac mini colocated to a company that really needs like a full cloud infrastructure.

All of their clouds start with a 30-day free trial. I love this. I think this is probably why they’re a successful company. 30-day free trial, fully production ready — it’s not like you get like a play server for the free trial, and then you have a real server — you get a 30-day free trial on a production server, and then when you’re like, “Okay, this is good,” they flip the switch, you start paying, and it’s already there. Couldn’t be easier.

So if you’re ready for your own Mac private cloud, visit Macstadium.com/df (for Daring Fireball)](http://Macstadium.com/df).

Second sponsor, another great company. Instabug. Instabug provides comprehensive bug reporting — I wonder why they’re advertising at WWDC. I don’t know.

[scattered laughs]

Comprehensive bug reporting and in-app feedback in an SDK for mobile apps. Tens of thousands of companies like Lyft, eBay, T-Mobile, and more rely on Instabug. With one line of code you can integrate their SDK in your app — right now.

And then once you do, what happens is your beta testers, when they’re using a beta version of your app with Instabug, if they want to report a bug they just shake the phone, and they get this great interface where it takes a screenshot. It gives you, the developer, all the information: the analytics, it gives you steps to reproduce — like, how they got to where they are when taking the bug — and then, these bug reports, that’s [only] part of what Instabug offers.

The other part is a great developer dashboard where these reports come in, and you can see the statistics and everything you want. They have a great trial system which I went through; I mean, I don’t have an app anymore, but I went through it just to see how it worked, and it’s fantastic. The developer dashboard thing, you just get these notifications. It couldn’t be easier. Your whole team can share it.

And they have integrations with all these great companies like Slack, Jira, [and] Github. So, you can try it for free, it’s one line of code. Here’s the URL: try.instabug.com/df then they’ll know you came from the show.

And, last but not least, they’re offering $150 in credit to anybody who uses the code “df18”. DF18, and you’ll get $50 pre-loaded on your account when your trial is over.

Last but not least is a fourth-time sponsor of this show: It’s a little company up in Redmond called “Microsoft”.

[huge laughter, clapping]

So any developer — this is what they believe — Microsoft believes any developer should be able to build, deploy, and scale without worrying about managing your services or the underlying infrastructure of your cloud and stuff like that.

Whether you’re writing Objective-C or Swift, they’ve got what you need. They’ve got the SDKs, they hook up —

And lots of applause for Microsoft. I think you guys know this, but like the old days where Microsoft had their own sort of developer ecosystem and everything was like, Microsoft-land… I mean, they still have Windows, they still have all of that, but they’ve really gone and embraced just about — if you develop anything, if you write code, Microsoft wants to be your friend.

I mean, I don’t know if you guys heard, they bought — speaking of Github, who we just mentioned — they just bought Github, which is probably… I mean, how many people here use Github?

[loud applause]

But they, really. They have your back. They know iOS developers, they know Mac developers, and they really have all the things you can do. Here’s some of the cool things you can do in the Microsoft cloud: You can get your server, you can build your app in the cloud, you can test on real devices that they have hooked up in their infrastructure, distribute your betas, monitor your apps with crash reports and analytics…

For game developers, they have a complete back-end platform for iOS games, with analytics, player management, live ops — that’s another one, I don’t know what “live ops” are but I guess if you’re a game developer that means something.

[chuckles]

Here’s where you go to find out more. Really, this is great stuff. If you’re an iOS developer, you really oughta check out what they have to offer. Here’s their website: AKA.ms/iosandazure. I guess they got the .ms for the Microsoft. That’s pretty cool.

So my thanks to Microsoft, my thanks to Instabug, my thanks to MacStadium. Last but not least, hopefully you guys partook (or are partaking) in the open bar.

[woos and clapping]

And that open bar was provided by Setapp.

[applause]

Setapp gives users access to a curated selection of high-quality Mac apps from trusted developers, and it’s just $9.99 per month. So check out Setapp — I think it’s setapp.com, I didn’t write down the URL.

[laughter]

You’ll find it. Google it! Just — just search the web for Setapp and you’ll find it.

[more laughter]

So my thanks to all of them, my thanks to you for being here. I think it’s going to be a very interesting show.

Without any further ado, I’d like to introduce my guests for the evening. First is Mike Rockwell, who is the VP of AR and VR engineering at Apple, and… Greg Jozwiak [Apple’s vice president of iOS, iPad, and iPhone Product Marketing].

[huge cheers and applause]

Joz and Rockwell get comfy

Joz: Where you sitting?

Rockwell: I’m sitting [gestures to the chair next to Gruber].

Joz: Mike told us to run out here.

Rockwell: Yeah, it was the Craig Federighi [Apple’s senior vice president of Software Engineering] school of entrances.

Gruber: We’ve known each other for a while [gesturing to Joz], we’ve [gesturing to Rockwell] just met — [turning back to Joz] I don’t think I’ve ever called you Greg before.

Joz: No, only my wife calls me Greg.

Gruber: Everybody else… [chuckles]

But — [turning back to Rockwell] you are, of the three of us, probably the lesser-known entity up here. So…

Joz: But the smartest guy.

Gruber: [gesturing to Joz] Well, definitely not over on this end.

[Rockwell laughs]

A little bit of your background, and how you got to where you are. So before Apple, you were the executive VP of research and development at Dolby?

Rockwell: Yup.

Gruber: And then, prior to that… you were the CTO at Avid?

Rockwell: Yep, that’s right.

[scattered claps and whistles]

Somebody remembers.

[Joz and audience chuckle]

Gruber: But it’s sort of a mix between… a lot of audio in your background.

Rockwell: Yeah. Audio, and video, and 3D. So at Avid, we did all three. But I started in audio, way back when. So I’m actually an Apple nerd from very old days. First computer was an Apple II, and then had an original Mac, and the first…

The reason I got into software in the first place was I was studying physics and music — so I was a musician — and I was incredibly frustrated by the applications that were out there. So I started to pivot into engineering and write software for doing audio. I went to a company called Digidesign, which built a product called ProTools…

[small cheers]

… And while I was there, I did the — one of the very first host-based audio mixing and editing systems. So we wrote it, the mixing in PowerPC assembly language; this was a long time ago…

[Joz laughs]

And it was a lot of fun. But yeah, it’s been an interesting path to get here.

ARKit 2 and USDZ

Gruber: So there were a bunch of AR-related announcements at the keynote yesterday, all of them very interesting. I thought it was kind of cool in a nerdy way — like, you know you’re at a developer conference when one of the major tentpole features is… [chuckles] a new file format?

[both Joz and Rockwell laugh]

Right?

Joz: With a very sexy name.

[crowd laughs]

Rockwell: Yeah, yeah, yeah, our crack marketing team helped us with that…

Gruber: But WWDC is a little different than other keynotes —

Joz: Yeah.

Gruber: Because come September, there’s probably — there might be an event in September, you might know — I don’t know. Sometimes in September there’s events…

[Joz shrugs as the crowd laughs]

Joz: I don’t keep track of that stuff.

[more laughter]

Gruber: And every once in awhile, there will be a geeky section where it goes into like, how the pixels are working on the new sensor, on the camera, or whatever. But it’s still fundamentally about something that everybody cares about — which is having a good camera.

Joz: Sure.

Gruber: But a file format? You’re at a developer conference.

Rockwell: Yeah. [chuckles]

Gruber: But this seems like it’s a really big deal.

Rockwell: It is a big deal. Yeah. So y’know AR’s been bouncing around universities for over a decade, and when we brought out ARKit, we made it available to an incredibly large audience of developers, and a larger audience of consumers.

One of the things that has been challenging is that in the 3D world, there are a ton of different file formats, and compatibility is not great. And there really wasn’t a format that was optimized for delivering AR experiences. So we wanted to create something that was really powerful, that was very efficient, that was easy to use, and that would get broad support.

So we were trying to start something which would be, like, the PDF of AR, which is really what we think USDZ can become.

Gruber: Right. So… am I right — somebody told me, and it could have been misinformation. People lie to me all the time. But the Z in USDZ is for .zip? It’s like, USD is “universal…”

Rockwell: Scene description, yes.

Gruber: And Z is for… zipping it up, for…

Rockwell: Yeah. Basically what it is: Inside of a USDZ file, there’s mesh data and information, and then there are a bunch of textures that are in there for the meshes. And we have textures that allow you to do physically based rendering. So it gives you the ability to do very realistic-looking 3D objects as well as animation. So for those of you who have seen the demos, you’ve seen that they look fantastic. Like the koi fish you saw [in Gruber’s demo]…

Gruber: Right.

Rockwell: It’s just spectacular animation with really realistic-looking objects. So we wanted to have something that had that power, but was also going to be incredibly simple.

Gruber: I thought that Adobe and Pixar made for an interesting… so it’s Apple, Adobe, and Pixar who sort of got together to do this. It’s not like a giant sprawling consortium…

Rockwell: Well, but USDZ… USD is an open file format. So Pixar originated it, but it is an open file format. So you can go and you can get the source and the spec is completely open — this is not something that’s closed. And USDZ is the same thing.

We worked closely with Pixar, and then Adobe also came working with us earlier on, but we’ve also connected with all of the other large vendors for 3D tools. And because it’s going to be, by far, the most broadly-supported format, what they’re telling us is they’re going to provide native support for it in their tools.

Gruber: So what else is in a USD file?

M; Well, so, as I said, it’s really the animation meshes, the textures for those meshes, and then the textures can include things like all of the textures you would need for doing physically-based rendering.

That might include, like, ambient occlusion, where you’ve got areas where the object is occluding itself. Or being able to have, effectively, the bump map, or the normal map so that you’re able to render without requiring incredibly-complex geometry; you’re able to create the illusion of incredibly complex geometry and stuff that looks very accurate to the scene.

And then we take that and in the Quick Look viewer, we render it with very high fidelity. We use this new capability of ARKIt 2 called environment texturing, where it looks at what the camera’s seeing and it’s building a texture reflection map in real time. And then it uses that to reflect on the object that you put in the scene.

Gruber: Right, because there was the… I think it was in the State of the Union, where the demo showed like a silver…

Rockwell: A silver globe.

Gruber: Right.

Rockwell: Yeah.

Gruber: And the camera is pointing down on it — 

Rockwell: Yeah.

Gruber: But the lighting is from above, so it’s out of frame.

Rockwell: Yes. There’s two parts to environment texuring. One is that it takes what it sees and creates the texture. But of course that leaves something that has lots of holes in it. So it has to fill it in with something, otherwise it would look really odd as the reflection.

And so what we did is we actually went out and captured something like 10,000 different spaces. We trained a neural network to look at what is in a scene. And then… uh, hallucinate… the rest of the, uh — 

[chuckles from the audience]

yeah — the rest of the reflection map so that it’s plausible. So it’s not perfect but what happens is that the reflections are generally a little more diffused, it’s not a perfect image, and for most folks it just looks natural and it looks real.

Gruber: It was uncanny once it was explained in the demo, and it’s like, “Oh my god, that stuff is all out of frame,” and it’s — there’s something reflecting there that…

Rockwell: It just makes a huge difference, you know. We’ve been doing everything possible to make it incredibly easy for folks to make AR content that looks great.

Gruber: Right.

Joz: Well, that render quality on ARKit, too, is amazing. I mean, one of our big challenges with the demo we did with Lego was to make sure we had a good enough establishing shot from the camera of what the physical model was, because once the augmented reality pieces came in — the digital pieces came in — it was impossible on screen to tell what was different!

Gruber: Yeah.

Joz: Which was real and which was augmented. It was impossible. So we had to really establish that.

Gruber: I literally — I was like right in the middle of the keynote. Center stage, sort of back. My question was that! And I’m like standing up to see… “Is there a real Lego house on that table?”

Rockwell: Right.

Joz: Yeah!

Gruber: And I was like, “Yeah!” And like “Oh!” Because I couldn’t see it from back here. And then I went to the hands-on area and I got to play with it and I still couldn’t… [laughs]

Joz: It’s amazing, isn’t it?

Gruber: Right.

Rockwell: Yeah, it’s really fun.

Joz: It’s amazing, considering that we just started shipping [ARKit] this past fall and this is our third major release, and it’s gotten so good.

Gruber: Yeah. The Lego demo — I don’t know if everybody, how many people here are at the conference and got to play with it, but it’s technically amazing. It’s not really a game that you would want to play, like… it is a demo.

I can see how you would turn it into a game, but it’s sort of like…

Rockwell: Well, when we engaged with the Lego team, and brought them in and showed them ARKit 2 and they got very excited about what the possibility was, they had this idea for this “play experience,” so that you could go in, and you could have a couple of kids, and they could play together, and it would start from building a physical Lego model — but they wanted to be able to recognize that model.

So that was one of the other things that we did in ARKit 2 was added the ability to detect objects that you’ve trained on before. They built one path through it for the demo — the show — but what they’re going to do, fully, when they release it is that it won’t be just one path, y’know? It’ll have lots of different experiences…

Gruber: Right.

Rockwell: Your friend could bring over their model, and those could be combined into the same scene, and yeah. It’s just going to be tons of fun.

Gruber: Yeah. The other thing I found when I was doing it myself with the Lego one in particular — okay, so I’m holding an iPad.

Rockwell: Mmhmm.

Gruber: And I’m like, “Oh, yeah, this is cool.” And then, it would be like “I’ve gotta move the fire truck to put the fire out,” and I would want to get into detail. And I’d pinch the screen... and nothing would happen. And I’d be like… “Oh, yeah.” [laughs]

Rockwell: Exactly.

[big laughs from the crowd]

Voice from the crowd: Do it with your feet, John!

J, laughing: Yeah, move with your feet, exactly.

Gruber: The Apple fellow who was there to guide me through the demo said [in a low voice] “Everybody does that.”

[laughter]

Joz: Well, old habits die hard.

Gruber: He’s like, “You get used to — you get used to it.’

Rockwell: You do! And actually, it becomes quite natural to do that. And for certain kinds of things, it’s actually much more natural. When you think about manipulating 3D, having a sense of space — and particularly things that are at a real-world scale — it’s really valuable to be able to move there, because you get much more of a sense of what the object is like in the real world.

On AR persistence

Gruber: So, my next question is about AR persistence.

Rockwell: Yes.

Gruber: Uh… explain this to me.

[laughter]

M, laughing: It’s when AR is trying really, really hard — no.

[huge laughter, claps]

Rockwell: [to the audience] Thank you. [to Gruber] So we had to make up names for this stuff, so that’s sort of what we came up with. [side glance at Joz] Our marketing folks helped us with that.

[more laughter]

So, the basic idea is that you want to be able to have an AR experience and come back to it later and have it be the same place, and share it with other people, and maybe have multi-user experiences.

So we use a technique where we map the environment, and we create, essentially, a set of points that allow us to understand that environment, and you’re able to save it. And then, you can use that map in a number of different ways. One would be to reload it on the same device, so you could come back.So let’s say that you had created an AR pinboard in your home, or you had a game that you were playing and you wanted to come back to it.

But you can also share it with other people. And those can be asynchronous, so they can come see it later — you could leave a gift for somebody that’s an AR gift, and they could find it, y’know, say on the kitchen table. Or they can be simultaneous, where you share it in real-time. You don’t have to go up to the cloud to do that, you can share it peer-to-peer, and that allows for multiple people to see the same coordinate system.

Gruber: But it’s… in terms of sharing, it’s something that you do, like — I can share with you, but it wouldn’t be something that I would leave here and everybody who has the app…

Rockwell: Well, you could! So you could in fact do that. Somebody who had an installation, or they had something where they wanted to create an experience in an app where when you went there, [it] would localize to that particular position, and give everybody the same experience.

Could be a museum, could be… so you could create installed experiences. Could be in retail. Could be in a museum. Could be in other applications. The idea is that it allows you to do — to make AR experiences that transcend one session.

Gruber: Alright. See, now this is great. Because I’ve got a million dollar app idea.

[laughter]

Rockwell: All right!

Gruber: Tell me if this is possible.

Rockwell: [mock surprise] Oh, you’re going to tell all these folks your million dollar app idea?

Gruber: I’m gonna tell —

Rockwell: Oookay.

Gruber: They’re — 

Rockwell: Get your pens ready.

Gruber: [to the audience] Just… keep it secret.

[laughter]

Gruber: It’s… when the App Store first came out, there were like, a lot of fart apps that came out.

[huge laughter and clapping]

And now we’re in the AR world!

Rockwell: So when you sit on this chair, you want to…

Gruber: No, my app is called “Leave a Turd.”

[laughter]

Rockwell: [incredulously] Leave a Turd!

Gruber: Free download! You can go to like, a coffee shop, and there will be one — 

Rockwell: And you pay for the different shapes of the…?

Gruber: Oh there you go! So you’re thinking like me.

[huge laughter from the audience]

Rockwell: In-game purchase.

[clapping]

Gruber: There’s one default turd that everybody gets for free, and everybody can see all of the turds.

Rockwell: Oh, yeah. [laughing] Technology at its finest.

Gruber: I could buy like a Yankees turd, or a Red Sox turd would be even better! [laughter from the audience]

And I could leave a Red Sox turd.

Joz: Bill and everybody on the App Store thank you right now.

Rockwell: [laughing] Yeah, right now.

[crowd laughter and clapping]

Gruber: I tell you, we get the MLB guys involved in this, and I tell you, it’s a lot of money. [all laugh]

But you could, in theory, build something like that.

Rockwell: Yeeeeah, you could.

[laughter]

If you wanted to.

Joz: Emphasis on you could.

Gruber: Do you do freelance work?

[huge laughter]

Are you busy? [laughs]

Rockwell: Not at all.

Gruber: All right. But there is a privacy implication to this, too, though. But if you wanted to, you can make an app like you said — like a family pinboard or something.

Rockwell: Sure.

Gruber: Like that. And it would be invitation-only, like I’m only going to invite my family.

Rockwell: Yeah. So the interesting thing about these maps is that they don’t contain any RGB data. So they’re actually far less revealing than a photo would be of that particular environment.

So we do, just as apps have to have access to the Camera for ARKit, you would need to do them for the maps. But you can’t — you couldn’t reconstruct an image. You could get a little bit of the geometry of the scene. But they’re fairly sparse. So there’s not a lot of risk of privacy around that.

But even with that, we do — it’s not something we’re really worried about.

Gruber: Alright. So I joke about fart apps. But there was an argument — and we talked about this briefly yesterday — but there was an argument when the App Store first came out of people who were dismissive of, well, “It’s just so small,” and previous phones… There were apps for other phone platforms. 

In hindsight, nobody even really remembers. The App Store opens, and developers had a couple of months, they didn’t know what the phone was good for. And they built some silly apps.

Rockwell: Sure.

Gruber: But there was — y’know, I think that people in this room, not just us, but I think everybody here was excited and knew that this was going to be a serious thing.

Rockwell: Right.

Gruber: But I think that with AR, I see some of that reaction from the skeptical crowd, which is: “What is this? Is this for games?”

Rockwell: Right.

Gruber: And it is for games!

Joz: Sure!

Gruber: Games, y’know, are big business.

Joz: Business, it’s true. It drives me nuts a little bit, because it shows a lack of vision. Right? Because to your point, we got [this pushback on the App Store] in 2008 — we got this even in 2009, it’s like “What are these apps?”

Gruber: Right.

Joz: “What a waste of time.” And it’s — we knew we had something big. This is the same thing with AR. This is a big deal.

Gruber: Yeah.

Joz: This is really a big deal. And we’re already doing some really cool stuff. We’re working with some developers, we know some really cool stuff and we could give you some examples of some cool things that we can talk about, certainly. There’s ones we can’t, but you just need a teeny bit of vision here to see this is… this is a big deal.

Rockwell: Yeah, well, you have to remember that ARKit has been out for nine months.

[Gruber laughs]

Right? So it’s always interesting to me to see folks saying “Oh, it hasn’t taken off yet.” It’s been three months! As the folks in this room know, it takes a little more time than that to make great apps.

Gruber: Right.

Rockwell: There’ve already been some fantastic apps made on ARKit. So, y’know, we just had an event in March around education. And there was one really fantastic app where they built a virtual frog dissection so that we weren’t killing all those poor frogs when we’re in junior high anymore, and in fact, it really worked, it was fantastic.

Kids loved it, and it revealed so much more than you would get, actually, in the direct dissection.

Joz: And now you can bring a shared experience to that, right? So the whole class…

Rockwell: Now we can all dissect frogs together.

[laughter]

Joz: How cool is that?

Rockwell: But you think about that, actually, with a shared experience — with teachers, you can now prepare a lesson and it can be shared. And the teacher can be teaching you about biology or mathematics or history, and it’s something that everybody can have their perspective on and it can be something that’s a group experience.

So you look at that, you look at in retail, that was one of the early applications that folks have been doing work with. Whether it’s what Amazon is doing, or what IKEA’s doing, or there’s a whole range of folks where they don’t have the, let’s say, the real estate to put all — to have all the inventory that they would want. And now they’re able to show that to folks.

So you put it across every one of these, and we’re just seeing the beginning of it. But it is really going to be quite something.

Gruber: And there was an example where somebody was — there’s a company that you guys are working with that’s using it in an industrial setting where — 

Rockwell: Oh, yeah, absolutely. So with the same object detection that Lego’s using to recognize the little town there, there’s a big industrial company that is building maintenance applications for large machines. So you can recognize the machine, you can be telling the maintenance personnel where the thing is that needs to be replaced, taking you through a procedure to do that, and…

Joz: Having X-ray vision.

Gruber: Right.

Rockwell: Yeah. X-ray vision into what’s there.

Gruber: You could see things…

Rockwell: You could actually see and go “Oh, okay, in order to do this particular maintenance operation, I need to remove this, and replace this, and take them literally through it step by step — something that might be in, y’know, 10,000 pages of a printed manual.

And you know, on a flat page, you can’t tell where everything is.

Gruber: Right, like right there on screen, it could pop up and say “I’ve identified this part, it’s part #DL44…”

Rockwell: Exactly, and the number of mistakes that are made when people have to do this [mimes flipping through pages] and go back to manuals is really is really high. Actually, NASA found that in the space station, that an astronaut would go back four or five times to reverify that they had the right operation.

Gruber: Fascinating. So that’s… like a real psychological cognitive benefit to having…

Rockwell: Yeah. It really reduces your cognitive load.

Gruber: Right.

On AR and machine learning

Gruber: So that leads me to my next topic, which is… and something that I’m just lost on. It’s that clearly, there’s a very significant intersection between AR and — maybe it’s ‘cos I’m a visual person, I can get what AR is — 

Rockwell: Sure.

Gruber: I’m looking at reality, and it’s augmented.

Rockwell: Yeah.

[laughter]

Gruber: Where that intersects with machine learning.

Rockwell: Right, right. So machine learning is really about teaching machines to classify things, generally today, it’s one of the primary things; and [also to] understand from and be able to divide things up in large data sets.

AR uses machine learning quite a lot. So ARKit uses machine learning in that — the environment texturing that we talked about. It uses it in detecting planes and extending those out. It uses it in being able to fundamentally understand the environment in which it’s operating.

But machine learning is also a separate thing that you can use in addition to ARKit. So with Create ML and Core ML, we’ve provided a framework that allows you to take what’s coming from the camera — so, in an ARKit session, the camera’s also passed through to the application. And you can take those frames and you could create your own machine-learned classifiers that might allow you to be specific to your app, recognize a particular object in a scene, or recognize a particular scenario.

They’re completely complimentary technologies. AR uses machine learning, ARKit does significantly, all the way through it. But it’s something you can use to even augment ARKit.

On wearables

Gruber: Okay, here’s where I’m going to try and get Joz in a little trouble here.

[laughter]

Rockwell: Okay.

Joz: Oh, great, thank you.

[more laughter]

Rockwell: All right. I get to watch.

Gruber: You guys have had a long-stated explanation when people ask about “Hey, how come MacBooks don’t have touch screens?” Or “How come the iMac doesn’t have a touch screen?”

Which is that: reaching out like his [mimes putting arms straight forward] for a significant amount of time and poking at a screen is actually ergonomically uncomfortable.

Joz: Correct.

Gruber: “We think that the natural position when you’re on a desktop is with your hands on a flat surface, and we think that when you’re on a flat screen, it’s like this — “ [mimes using an iPad] I’m on board with that. I think people, probably a lot of people agree.

But if this is bad for a touchscreen Mac… how is this [mimes holding the iPad and waving it around] good for AR?

[Laughter. Rockwell looks downward. Huge claps]

Rockwell: I thought he was going to get you in trouble!

[clapping continues]

Joz: But it’s different, right? I mean this is not… [mimes Gruber’s frantic iPad waving] this. I’m using this as my viewfinder, which is not very different than holding it in general.

Gruber: Well, have you guys investigated any other form factors…?

[laughter]

Joz: One of the things that’s really cool about what we’ve done with ARKit is hundreds and hundreds of millions of devices…

[Gruber chuckles]

… that you have today in the form of iPhones and iPads. [They] work with AR. That’s pretty freakin’ cool.

Gruber: That’s very cool!

[clapping]

I can think of some other things, though, too…

Joz: We don’t have to sell you on what’s going to happen some… [mumbles] number of years from now.

[laughter]

Gruber: Alright.

Rockwell: Yeah. [laughs]

On Apple & VR

Gruber: In a similar vein… here, let me go back to my introduction cards.

Joz: It’s like you’re dealing poker here.

Gruber: I am!

[Joz laughs]

Your title [gestures to Rockwell] is VP of AR/VR, engineering at Apple. [all laugh]

Joz: [at Rockwell] Explain that one!

Gruber: It’s right on the title!

Rockwell: Oh, man, we got change that title. [laughter]

Gruber: [through laughter] Where’s VRKit?!

Joz: The VR is — it’s AR/VR, but the VR is silent.

[laughter]

Rockwell: I think I’ll go with that answer.

[laughter]

Rockwell: Well, actually, just — all joking aside. So we are providing framework support and a lot of work with other folks who are building VR headsets. We’ve been doing a lot of work in the OS, our Metal team has been doing a ton of work to improve the performance of Metal, we added the eGPU support, so there’s a lot of stuff that we’re doing there that is really to enable higher-performance graphics, which can drive VR [on the Mac].

Joz: You remember that Star Wars demo [with the iMac Pro in 2017]?

Gruber: Yeah.

Rockwell: Yeah, so, we’re doing tons of work in that area. And it’s a lot with partners.

Gruber: No, there were a lot of cool demos at the iMac Pro event. “Here we plug in an extra eGPU, now look at how the…”

Rockwell: Yeah. If you came to, if you went to the State of the Union and you saw the performance that we’re getting with the eGPUs — it’s really pretty much scaling linearly for tons of applications. The work the metal team did is astounding at how efficiently we’re able to use those eGPUs.

It’s actually amazing for doing — if you’re doing content creation for AR, so you might be using an Autodesk app or something from the Foundry, having the ability to plug in an eGPU that is extremely powerful to your MacBook Pro is fantastic because it gives you the portability and also the ability to have performance that’s close to an iMac pro — for graphics at least.

Gruber: And the last part about AR/VR — really near and dear to my heart and it’s why I’m so excited that Apple is so fully on board with this — because I think one of the things Apple has always been is historically unique. I would just say unique in the history of computing. It’s that when a new interface or new paradigm comes along, it’s Apple that gets it right first, right?

The Mac wasn’t the first graphical user interface. There were other things. The mouse was invented in 1968. I mean, and there were other windowing systems. The Mac was the first one that was good. Right? And there were touch screens before the iPhone came out.

Rockwell: Sure.

Gruber: And they got all sorts of interactions wrong. The iPhone got it right. “Oh, it’s all direct manipulation.” And AR/VR, it’s like — I’m telling you this, but… even from my perspective, it’s mind-boggling how many human factors problems left to serve.

Rockwell: There are.

Gruber: And the one that made me think about it was that March [event] demo that you guys had. It was an HTC headset and it was, y’know, really interesting. But there was one app where when you’re in the VR world and you look down, you don’t have any legs.

[scattered laughs]

Gruber: And it was weird!

Rockwell: It is!

Gruber: And there was another demo, and again — this is me, and you [gestures to Joz] know this because you’ve seen me at these meetings — I’ll go, I just, I’m not on script. I was supposed to be playing music in this demo, and instead I’m like looking at my feet.

[laughter]

And then there was another demo, where you looked down and there were legs and feet. And then I would move my feet, and they didn’t move. They were just… and I was like, “this is really… I don’t like it either way!” But there’s like a — there must be, what, a hundred thousand little things like that that you guys must be thinking about.

Joz: Well, the beauty of AR is that you don’t have that problem.

Gruber: Right.

Joz: Your legs still move.

[laughter]

Rockwell: There are a lot of tough technical challenges for sure in this space.

Gruber: Right.

Rockwell: Y’know, we’ve been working really hard to solve them in a way so that at each step, we make those really hard things easy for developers and for users.

And so we’re — you see that with the pace that we’ve been on with ARKit 1, and 1.5, and 2 — we’re taking those really hard problems and we’re making them easy. And for us, it’s really important. There’s a lot of stuff that we have in my team [that] we could just throw out there, but doing that would confuse developers, make your lives a lot harder. It would confuse users. And so it’s important to take the time to really, really get them right before you make them.

Gruber: Right. And the new Measure app in iOS 12 is a great example of that. It’s a lot of fun! 

M: Yeah. You talk about machine learning — there’s a lot of machine learning in that measure app to really try to make something that seems like it should be easy, but from a technical standpoint, to do it right is actually quite a bit hard.

Gruber: Alright, moving on. Let’s talk about some of the other stuff from the keynote yesterday.

Joz: Yep.

Gruber: I thought one of the most telling things was that when iOS 12 was introduced and Craig came out, the whole first segment of his thing was about doubling down on performance. And the specific models of phones that he talked about were the oldest ones — the iPhone 5s and iPhone 6 was for like the benchmarks, and we’ve got all these great results on our other devices too, but we’re really focused on this.

And reading between the lines, it’s like — I can’t help but think that part of the emphasis on that, both in putting the engineering behind it but also making it a big part of the marketing message, is to counter this popular notion that major iOS updates, when you put them on your older phone, makes the phone slower, deliberately, because Apple wants you to go to the Apple Store and fix it by buying a new iPhone.

Joz: Which is about the craziest thinking in the world, right? We’re going to give you a shitty experience so that you go buy our new product.

Gruber: Right! [laughs]

[woos and claps]

Joz: But to your point, there’s been so much that people forgot about how great software updates are. And that was the first part of what Craig was reminding you. First of all, it’s that we’ve got a 95% customer satisfaction with iOS 11. It’s great. Right? And we have delivered, through the years, amazing features. From the App Store to iMessage to… y’know, you saw the whole list [on stage at the keynote], I don’t want to repeat it, but software updates are super important.

Now, what Craig talked about is a few things [about iOS 12]. One is making everything faster and more responsive. And you gotta remember, we’re supporting devices that were introduced in 2013. Devices that are more recently introduced, iPhone Xs, are a lot faster than those just by the nature of how fast our chips have gotten. We’ve got the fastest chips in the business. Our chips this year, y’know, or last year, are faster than theirs this year.

Gruber: Well, what about this year?

[laughter]

Joz: [waves Gruber off] So, we want to remind…

[more laughter]

Joz: … of how great that is! And so what we wanted to also do is to pay some special attention to the fact that some of these older devices [are] under load — that was a big part of that, right? It’s like they test out of the labs just fine, but you realize that some people are heavier users than others, right? They’re using more things in the background. They’ve got more things loaded on the screen.

It’s those folks, you know, that have experienced more of a slow down, if you will. And Craig was making a point of showing, saying “Look, we did a lot of engineering and a lot of testing.” He talked about the stress rack to show that we can double the performance for those people with iOS 12, and iOS 12 supports the same set of devices that iOS 11 did. Again, going all the way back to 2013, introduced. All the way back to the 5s. And it’s going to be a really good update for those people.

Rockwell: And actually, I’ll just add a little bit around ARKit: it would be much easier for us to have ARKit only support, like, iPhones X and newer, right? Because they are so much faster. But we wanted to have a platform that would have hundreds of millions of devices that could support AR from the initial release. So I can tell you that my team, we have a lot of effort put into optimizing for devices that go back multiple years, and I know Craig’s team does as well. I’ve been in those meetings, and there is — it is so far from the truth that anybody would think that [we were intentionally sabotaging old devices].

Joz: If we only wanted you to buy new hardware, we’d only have updates for like, 6% of our users, not 81%.

[Huge laughter and clapping]

Gruber: Well, the other crazy part of that theory, that Apple wants your phone to be slower so that you get a new phone, there’s — I agree with you. No one would ever accuse a car maker of that. Like, if you bought Brand X of automobile and it had a 5-year warranty, and 5 years and six months after you bought it, it falls apart, nobody would go “Well, I’m going to go buy another one from the same company,” you know?

Joz: Exactly!

Gruber: Casey Liss isn’t buying another BMW!

[laughter, whoops]

Gruber: But the flip side of it is, I know Apple engineers who work at Apple; I think there’s some in the audience right now.

[woos]

I have never met anybody — and really, WWDC attendees in general — like, the whole point of being an Apple developer and writing for these platforms is that you care. You care about getting the UI right. You care about having an app that like, is smaller to download.

And imagine Joz going into an engineer’s office and saying “Okay, here’s your job: When iOS 12 comes out…”

[Joz laughs]

Joz: Yeah, they would run me out of town.

Rockwell: You would not live the rest of that day.

Gruber: Privacy. That was another aspect of the whole message. You guys, though, have been on it for years. This isn’t like a new…

Joz: Way before it was popular.

[laughter]

Gruber: It’s very true though! The message has been very consistent.

[clapping]

So there were two parts of it that really stood out to me. The one was — the privacy aspects — one was the new reciprocal sharing in Photos. So if I share some photos through iCloud Photo Sharing to you, it only goes to you, and then you get the option of also sharing back to me, and it does some…

Joz: It does it through Messages.

Gruber: Right, but it tries to be smart, and guesses. “Well, it looks like these two guys were in the California Theater together at the same time.”

Joz: All on-device, yep.

Gruber: All on-device.

Joz: All on-device.

Gruber: And will suggest to me the goofy stuff we were doing backstage, or whatever. Pictures we got.

[laughter]

Gruber: It goes unsaid that the sharing with some of your competitors is not all on-device.

Joz: [laughing] No, that’s a differentiator for us. I mean, we’re big for doing things on device. We’ve got chips that, as we talked about, are super-fast. We have software teams that know how to do amazing things without having to resort to the cloud doing the heavy lifting. We’ve got devices that are capable of doing incredible amounts of heavy lifting.

Gruber: And then the other one, and it really is near and dear to my heart, because y’know, I make my living on the web, and selling sponsorships and advertising and stuff, and I’ve studiously stayed away from anything that involves tracking and user profiles and stuff like that. Whereas some other sites on the web haven’t.

And I thought that the new feature that was introduced in Safari — where Craig pointed out, “Hey, they’ve got these buttons where you, y’know, click this button and it’ll take you to Twitter or take you to LinkedIn, or whatever,” that all of these buttons, all this Javascript track you and creates these digital fingerprints. And to have a web browser built-in, without any kind of extension or something like that, is going to allow you to block that.

Joz: Yeah. Yeah.

[huge cheers and applause]

I mean, we still believe in the ad model and for a website to be able to do the ads, but it’s the cross-site tracking that’s a problem. Especially because people don’t know! People don’t know that’s why that comment field is there. It’s there so that they can see you’ve been there, and then you went there, and then you went there, so they build a profile on you.

And again, we allow that to happen if the user says “Allow.”

Gruber: Right.

Joz: It’s now just trying to make it so the user actually has a role in deciding what happens —

Rockwell: It’s their choice.

Joz: — with their information.

Gruber: I think the thing that… so tell me, typical people, like 99% of people out there have no idea how it’s happening.

Joz: Exactly.

Gruber: But they know it’s wrong, they know it’s wrong when they searched for whatever brand of sneakers, and then for the next 7 days everywhere they go on the web, no matter… totally different topic, totally different website, there’s an ad for the thing they were just searching for. They just know that that’s weird.

Joz: You know, these data companies end up building profiles and having your web browsing history. Which I don’t think most people in here are probably comfortable with.

Gruber: Group FaceTime. [whistles]

Finally. Right?

But pretty cool! Because people have been clamoring for it for a… I don’t know if you know this, but for awhile. [laughs]

Joz: I’ve heard that.

[laughter]

Joz: There’s like, a group of people.

Gruber: And it’s pretty cool, though, that you went from, well, you can FaceTime between two people, and now you can FaceTime with up to 32 people.

Joz: Yeah.

Gruber: Like… that’s wow. Explain to me how there’s not… it’s not all cross-talk and…

Joz: Well, that’s what we’re trying to show in the demo, right? That it’s smart enough to — we don’t want 32 people mismatched! You’re going up on the screen at one time. It’s a roster down below; the people who are active go on the bigger image; you get bigger as you speak. Right? So it senses that. People on the roster, if they begin to speak, they go and replace somebody up top. So it’s a nice system. It’s actually kind of well done.

Rockwell: I mean, if you have 32 people in a meeting, you’re not all trying to talk — well, maybe you are [laughter] — but most of the time, you’re not all trying to jump in and talk.

Joz: No, those are marketing meetings.

[laughter]

Rockwell: Engineering… [shakes head] So you might have two or three people who are interacting, and other folks are watching, or it could be somebody presenting to a larger group. So there are those natural turn-taking things that happen with people.

Gruber: Right. And here’s a question I know the answer to, but I think it’s worth bringing up: It’s still end-to-end encrypted?

Joz: Absolutely.

Gruber: Right. So even with 32 people in a chat, everything going up and back is end-to-end encrypted and the only people who have access to the audio and video are the 32 participants.

Joz: That’s right.

Gruber: That’s fantastic.

[cheers, whistles, and clapping from the audience]

Gruber: So do you guys use it internally? Like, do you trust group FaceTime enough that you guys, y’know…

Joz: Sure.

Gruber: Yeah?

Rockwell: Yeah. Absolutely.

Joz: It is end-to-end encrypted.

Gruber: Well, I remember when iMessage first came out, or when it first came to the Mac I think, because it was iOS only when it first debuted —

Joz: Correct, correct.

Gruber: And when it came to the Mac, I remember talking to someone at Apple, but they were like “You know, we use this internally all the time.”

Joz: Sure.

Gruber: And I… y’know, I don’t know, I think you guys know, Apple tends to be a little secretive? And…

Joz: I can neither confirm nor deny.

[laughter]

Gruber: But you guys trusted and used it, though, you know. You use it for your own communication.

Joz: All the time, absolutely.

Gruber: All right. Another big — I’m going to call these attention-related features of iOS 12 — the Do Not Disturb extensions or improvements.

Joz: I’m using it right now, by the way.

Gruber: [laughs] You’re on the beta? Right?

Joz: Of course!

Gruber: [to Rockwell] Are you?

Rockwell: [quietly] Yes.

Gruber: Wow.

Joz: Would you be disappointed [if we weren’t]?

Gruber: I would be a little bit.

Joz: Alright. [chuckles] Alright. Literally. It integrated with my calendar so I pulled up Control Center and said “Do not disturb until I’m done with this event.”

Gruber: Okay, let’s do a quick [audience] poll: How many people in the audience have installed iOS 12 on their main iPhone?

[large cheers]

Gruber: This is our crowd. [laughs]

Joz: It’s a pretty solid developer release!

Gruber: Yeah. Yeah. It actually does seem to — I don’t have it on my main phone yet, but it does seem to be pretty good. So there’s Do Not Disturb improvements, where you can set it… I’m going to forget some of them. You can…

Joz: you can set geofence (so until I leave this location), you can set whatever amount of time and say don’t bother me for an hour, you can say — it’s integrated into the calendar — so you can say while I’m in this event. Of course, you can still toggle the whole thing on and off. And you have now a new Bedtime one as well.

Gruber: Alright. The group notifications…

Joz: Yup. And the ability to instantly tune the notifications as well.

Gruber: Right. Which I think is really interesting and like, we were talking yesterday about how sometimes you’ll sign up for a news site or news app or something, and you’ll realize you’re getting, they’re telling you about things I don’t care about. Why are you sending me a notification for y’know, “There’s a new flavor of Crystal Pepsi,” or something like that. That’s…

[laughter]

But it never — one extra annoying notification never quite seems like it’s enough to justify unlocking the phone, going into Settings, finding the panel in the settings where notifications are, scrolling down that list to the app, and then adjusting it. It never seems — it’s like “Ahhhh. I’ll do it next week.”

Joz: We’ve all been there.

Gruber: If you could just poke at the notification and just say “No more of these.”

Joz: Or deliver it quietly, which is what I’ve chosen on a bunch of those. I still want to see ‘em when I go to Notification Center, I just don’t need them to buzz my wrist or buzz my pocket.

Gruber: Right. So the quietly feature specifically is the difference between the notifications on your Lock screen and Notification Center, right?

Joz: Correct.

Gruber: I think that there’s been some confusion over why the Lock screen is not exactly the same as Notification Center? And I think this…

Joz: It’s the fact that it buzzes you, right?

Gruber: And then the last part is the reminders of how much you’re using X, Y, and Z, and the weekly reports that you can get about it, and the app limits that you can set. And for whatever reason that seems to have popped up in just the world at large, y’know? I mean, even outside the Apple universe as a just sort of… I don’t really want to call it mindfulness, just sort of self-awareness of how much time you’re spending on phones.

Is it a coincidence that it’s coming out this year, or is this something that you guys have been working on?

Joz: Well, for one, y’know, we’ve been working on stuff like this since 2008 when the App Store came out, and we’re continually adding to it, including Do Not Disturb, and Do Not Disturb while Driving, by the way — which is a great feature. AAA is very happy with us on that one.

[scattered laughter]

They really are! So this specific set of features, which is pretty comprehensive, the team’s been working on them for over a year, right? This is not, like, a reaction to something happening in the last few months. This is something that’s been worked on for a long time.

And it is comprehensive, but also at the same time, when you go out and look at these things — Phil loves to say, “Hey, we have over a billion customers, and there’s like a billion different opinions on how to do this.” But what we knew had to be the right basis for this is the information.

Gruber: Right.

Joz: Right? Letting people know how much they’re using the different apps, different categories of apps. How many notifications you’re getting. Where those notifications are coming from, which ties into that notification tuning, right? It’s one of those ones where I say “Ah, these guys send me a lot of notifications,” and the next time it came up I just set them to deliver quietly.

Even how much you pick up the device, right?

Gruber: Right. That’s the one I’m afraid to find out.

[laughter]

Joz: It’s really really interesting to see! And I think for 95% of people, that’s it. They’re just going to want to see this information. Somebody told me the other day, it’s kind of like calorie count: If you count calories, you’re probably more likely to not consume too many.

Gruber: Right.

Joz: And that’s how this is. You give people the information, you think it helps them understand “Okay, am I playing too many games, I’m on instagram too much, whatever it is, right?” And they can help balance that. For people who are like the “Hey, stop me before I kill again.” Y’know.

[laughter]

That’s where they can put a control on for it. They can put an allowance on that says hey, “Remind me when I hit my limit that I wanted to impose upon myself.”

And then there’s the whole kid aspect, which is, remember, this first part’s just for everybody. Right? It’s for everybody. And then there’s the kid aspect, which is the same thing. The parents can see how are the kids actually using their devices. Because [the parents] don’t know how they’re using their own devices, let alone the kids’. Now for the first time they can see, and they can have a conversation about it.

And again, I think for 95% of parents that’s going to be where’s it’s going to go. And then again, where there is an issue, they have the ability to put a control in, put an allowance, and the kid even has the ability to say “Hey, look, I finished my homework, give me some more time!” Right?

So we built that in. It’s really cool. Obviously we’re going to keep working on this stuff, but we’re pretty happy with where it’s at.

[applause]

Gruber: A big applause line in the keynote — not quite as big as dark mode, but [chuckles] maybe second? The announcement that CarPlay is expanding to allow third-party navigation apps.

[huge cheers]

And I think part of the applause was based on just that this is a feature people wanted, but I also think that part of it was that a lot of people might have assumed that was not going to happen. That Apple Maps is the map system for CarPlay and that’s it because that’s what companies do. They promote their own service.

Joz: Yeah, so CarPlay isn’t about trying to just lock in our stuff. We’ve had this on our list for awhile. As you know, we’ve had a lot of things on our list for months. It takes time to get to them. And so we’ve been talking to the folks at Waze and all that for a long time. There’s no doubt there’s a bunch of people who want to use that, and we wanted to give it to them.

And CarPlay is awesome! CarPlay, if you’ve read the news, it’s all over the place. It’s in millions of cars now. So it’s getting pervasive.

Gruber: Was it — has it been frustrating for you guys? ‘Cos a lot of this stuff, you guys do; famously, Apple’s thing is “We do the whole kit, and that lets us do…”

Y’know, the AR story is all about: “Well, we know the camera, we know the GPU, we’ve actually worked with the GPU team to get the GPUs that we need to do this.” But then with something like CarPlay, you can’t do it all yourself ‘cos…

Joz: Well, ARKit we can’t do it ourselves, right? We’re building ARKit so that you guys can go create this amazing stuff. We didn’t go do a bunch of first-party apps.

Gruber: Well, but also — 

Joz: I mean, we did a game, which was pretty cool, and a sample code, right?

Gruber: Right. Well, I’m just guessing that maybe the [CarPlay] rollout of a lot of car companies to integrate CarPlay across their line maybe didn’t happen as fast as you’d hoped that it did.

Joz: Well, car guys move at a slightly different pace. They take…

[laughter]

Gruber: [innocently] Maybe you guys can build your own car.

[Rockwell laughs; audience joins]

Gruber: Put it on the list!

[huge laughter]

Gruber: Alright.

C: I can’t wait to see the next card.

Gruber: Related notes! Here’s a question I’ve been dying to ask. So, in relation to… Apple’s at its best when it’s a platform, and the platform on iOS, in the same way that you can now use Waze instead of Apple Maps in CarPlay, you’ve long been able to — thanks to the App Store, you could take Apple Mail out of your Dock on your phone and put the Gmail app in or y’know, there’s a bunch of great email apps in the App Store. Where you can take Safari out and put Firefox in your Dock.

But the one thing that iOS doesn’t have that the Mac has is a way to say “Make my default email client some other app.” Or “Make my default browser another app.” Is there…

Joz: Yeah, there’s always a tension there as to… again, we want to offer a very, very integrated experience. You know that. And sometimes that’s easier said than done to do what you’re asking, right? To replace those apps and then find out that the experience breaks.

And so, remember, we’ve got over a billion customers. They expect an easy-to-use device. They expect an experience that just works. And so we’re working hard to offer a very integrated experience.

Gruber: So… that’s not in iOS 12.

[Joz laughs]

Joz: No.

Gruber: Alright.

[laughter]

Joz: Next card.

[huge laughter]

Gruber: There was a joker last year up in the balcony — I hope he’s not here again — who shouted out at one point as we were getting to this point in the show where we’re wrapping up, he just shouted out “When is Siri going to get good?”

And it was like, “What? Nobody’s here to listen to you.”

[laughter]

Voice from the balcony: It was “When was Siri going to get better?”

Joz: He’s still here.

Gruber: Oh, god. We gotta find this guy’s name. He doesn’t get a ticket next year.

[more laughter]

Joz: I think it was Panzarino, wasn’t it?

[even more laughter]

Gruber: But I… the Siri announcements yesterday sound great. And a lot of it is… okay, for the most part up until now Siri has been — here’s a list of things that we, Apple, have made Siri be able to do. And with the Siri shortcuts, and being able to assign your own…

Y’know, the Tile thing [from the demo] with “I lost my keys,” if you’re always losing your keys, say “I lost my keys” and it beeps your Tile or whatever.

Joz: Any app now can integrate.

Gruber: Alright. It seems like a huge step forward, and sort of a delivery on the promise of Siri — that it would be a very personal technology.

[huge applause]

Joz: Absolutely. I mean, y’know, people sometimes lose sight of the fact that Siri has over 500 million active users. I mean, it’s far and away the most popular personal assistant. And we talked about it — 10 billion requests every month. People are using it a ton.

And certainly, again, there’s things we want to do to make the experience even better. This was a big one. How do you have an app integration? And how do you do it in a way, not just this… you know, we’ve had the main integration with SiriKit, but how to do it in a way that’s not just a ton of things that people aren’t going to use — that are prescriptive, and so it’s like: “How do you have it so developers can figure out what are the meaningful things in our apps to allow to be assigned, and a customer to assign that and give it their phrase, and we think it’s going to be a pretty good feature.

Gruber: Alright. Last segment: macOS. Near and dear to my heart. My favorite platform.

[cheers]

macOS Mojave. And I know you guys love all of your children equally.

[Joz laughs]

I think that’s what Phil told me one time. [laughs] macOS Mojave. How come the Mac gets — the last few years, the Mac gets a name. macOS gets a name and iOS just gets a number? What? What’s… I don’t understand that.

Joz: Wow. [laughs]

[more laughter]

Rockwell: [leans in] Why is that, Joz?

Joz: [laughing] Do you not like numbers?

Gruber: I find that now that the numbers are getting big, I’m losing track of them.

Joz: All right: After 11 comes 12…

[huge laughter]

Joz: Do I have to keep going?

[more laughter]

Gruber: Um. [laughs] The features that stuck out to me. The Finder got a lot of love. And that’s…

Joz: Desktop and Finder.

Gruber: Yeah. Well, I kind of…

Joz: They’re related, but a little bit different.

Gruber: Yeah. The Finder integrates — the thing on the right now, I forget what it’s called. It’s like the new Get Info panel but it’s integrated right in a window and has all of this metadata, and the action buttons at the bottom where you can customize them with scripts, and Automator actions and stuff like that.

If there was like a betting game on what technologies were going to get mentioned in the keynote, I would have lost a lot of money betting against Automator.

[laughter]

But I love it! I love it. It’s just, often the automation stuff doesn’t get a lot of love in the keynote. And I thought that was great.

Joz: Well, and Siri Shortcuts is its own form of automation as well.

Gruber: Right. I thought the new screenshot features were super cool.

Joz: They’re awesome.

Gruber: They’re super cool. And almost like — I went from, it was one of those features where I went from “These are super cool, all these people are going to be amazed at these things that you could do if you knew the magic six-finger-then-hit-space incantation. And now it’s all just obvious! It’s like “Oh my god, why didn’t you guys think of that before?”

Joz: It was on the list.

[laughter]

Gruber: [laughing] It’s on the list!

Rockwell: Near and dear.

Joz: Do you like dark mode?

Gruber: Dark mode… is next on the list.

[applause]

The biggest applause line in the keynote, by far. Right? And we were talking backstage that pro users have been…

Rockwell: Yeah, we’ve had dark mode for a decade. All of our apps are dark mode. So it’s fantastic, actually. And for folks who spend a lot of time, y’know, staring at their screen in dark rooms… like developers…

[laughter]

Rockwell: It’s really great to have dark mode.

Joz: That was part of the inspiration for the [marketing] line we used, which was “Inspired by pros but designed for everyone.” It was inspired by pros. Pros want their content to pop, they want everything else to recede. Yet for the rest of us — and trust me, I’m no pro — it’s really cool.

Right? And I’m running dark mode just ‘cos it’s cool.

Gruber: Right. And then, my favorite feature — my single favorite announcement in the whole keynote: Favicons in browser tabs.

[cheers]

Joz: I knew you were going to say that. I believe you said it was the only reason people were using competitive browsers other than Safari.

[laughter]

Gruber: This is true! And it was a real eye-opener to me. I started writing about it, and it’s often funny to me what I’ll write about or mention in a podcast that gets my email like.. whooooa whoa, what happened? Did the counter in Mail get reset? I just got like 1000 emails.

The favicon thing was tremendous! And I just kept writing back to people, and they were like, “Yeah, it’s the only reason I use, y’know, Chrome.” I know you won’t say [the name of a competing browser] but…

[Joz laughs]

Joz: I’ve adopted the Craig pronunciation of “FA-vik-on.”

Rockwell: Yeah!

[laughter]

Gruber: So we need another poll! Is it “Fave-icon” or “Fav-icon”? We’ll do a — 

Joz: Favicon.

Rockwell: Favicon.

Joz: Right.

[huge claps from the audience, who agrees]

Gruber: Favicon?! I don’t buy it. But I pronounce everything wrong, so… It probably is favicon.

But that was coo.. And that’s in iOS, too! It was part of the Mac segment of the keynote, but that’s now an option in iOS.

Joz: Right, right.

Gruber: Well, that’s… I love that.

[laughter; Joz chuckles]

Gruber: The Mac App Store.

[cheers]

It more or less got what the iOS App Store got last year.

Joz: Yeah. We put the focus last year on the iOS App Store, and it turned out pretty well. You saw our numbers, I mean. The amount of people reading the Today tab is blowaway. We’re getting 500 million users a week, which is… just think about that! 500 million users are going to the App Store every week.

Rockwell: You can’t count that high.

Joz: [laughs] So this year — he said that, not me! — this year, it’s putting the attention on the Mac App Store, and I think it’s a really nice redesign.

Gruber: Well, one of the things my friend Khoi Vinh wrote a blog post about a week or two ago just pointing out how many custom illustrations are being — clearly being commissioned by the App Store teams to illustrate the articles that they’re doing.

And y’know, “maybe you don’t think about it” (him saying to his readers) “Maybe you don’t think about it, but most places would just use clip art.”

Joz: Yeah.

Gruber: It’s… being an illustrator has gotten hard, and the print industry, like a lot of sites that previously commissioned a lot of artwork don’t have the money for it anymore. It’s crazy, it’s mind-boggling, but remember a couple of years ago, the Chicago Sun-Times fired all of their staff photographers…

Joz: Yeah.

Gruber: Just gave their reporters, y’know, “Use your phone.”

Joz: I think they said iPhone, but…

[laughter]

Gruber: Yeah. But. There’s a difference between reporters — even with a great camera phone like an iPhone — 

Joz: Right.

[laughter]

Gruber: — Is still not a professional photographer in the same way that a professional photographer, given a great word processor, is not a reporter.

Joz: Sure.

Gruber: It’s just crazy. But the App Store, it really is [pro illustration], and it’s a typical Apple way where it, there’s not like bylines and people aren’t getting credit, it’s just the App Store is publishing this. There’s fresh editorial content with illustrations and photographs and movies and animations —

Joz: Serious editorial team that works on this. They do a great job. And that’s why people are coming back and reading that Today tab every time. But at the same time, we didn’t want to just take the Mac App Store and say, “It’s a clone of the iOS store;” it wouldn’t make sense. We needed a great Mac App Store. A Discover tab instead of a Today tab, right? It’s a different experience, designed to be for a Mac user.

Gruber: Well, and the other thing, too, and I thought it was a really interesting announcement; [the slide where] companies that were fully supporting the Mac App Store. And it was Adobe with the Creative Cloud, which is Adobe’s main pro tool subscription thing. I mean, it’s a big part of Adobe; it’s not like a little side project. This is a big deal.

Microsoft Office 365. Probably the first thing people think of when they think of what apps do you use from Microsoft. Um, big companies.

And then the next two, I just loved the, y’know… “Here’s two of our biggest app partners.” And then BareBones Software and…

[cheers and clapping]

And Panic with Coda!

Joz: And they got pretty good applause at the developer conference.

Gruber: But that was one of my all-time favorite slides in WWDC history. Adobe, Microsoft, BareBones — where I used to work! — and Panic. And like, what a great set of four apps.

But part of the backstory with including BareBones and Panic is, at least in the developer community, people know that BBEdit was in the App Store, and then it wasn’t. And Coda was in the App Store, and then it wasn’t. And it wasn’t angry or mean-spirited, or acrimonious, it was running into limits with sandboxing (to make a long story short)…

Joz: Mmhmm.

Gruber: That were incompatible with what the serious pro apps wanted to be able to do with the computer. And the backstory behind it is this: It’s not just that the Mac App Store has gotten a visual refresh — it’s that people at Apple have spent a lot of time recently talking to Mac developers and saying, “How do we make the Mac App Store and sandboxing work for you? All we want to do is protect users’ data, you guys do too, right?”

Joz: Right.

Gruber: It’s not just a visual refresh. It’s a lot of different stuff…

Joz: Absolutely. And we’re glad to have these guys back, by the way.

Gruber: Yeah. Well, it’s really great to have you. Last card!

Joz: Uh-oh, the infamous last card!

Rockwell: Alright.

Gruber: UIKit on the Mac.

Joz: Yup.

[clapping]

Gruber: An unusual announcement. Apple doesn’t often announce things a year in advance? [laughs] But I kind of get why you did it because you were eating your own cooking by using this to make… at least four of the apps that are shipping on Mojave. Let me see if I can remember them.

Stocks, News, Home…

Joz: [whispers] Voice Memos. Voice Memos.

Gruber: Voice Recording — Voice Memos.

[Joz laughs]

Gruber: Which I, y’know. I think is the way to do it. And I think, not to get mean, but there have been times in the past where there have been APIs where I talk to my developer friends, and like maybe a couple years ago, Core Data and iCloud was sort of a pain point, and it turned out, like, Apple stuff wasn’t really using the same stuff, y’know.

When Apple uses the same stuff, and then they’re like “We’ve polished it to the point where we can now share it with you,” it seems like they’re better APIs.

Joz: Yeah, there’s there’s no doubt that this is eating our own dog food. And we do want to get the APIs right, because as you know, if you change them later, things break, and once you create APIs, you’re wed to them. It’s a very long commitment around these APIs, we wanted to get it right. And so we’re going to try and get this one right. Because if we do, it’s going to be a big deal for bringing software over to the Mac.

Gruber: So how come this system doesn’t have a name?

[laughter]

Joz: It was called “Sneak Peek”!

[laughter]

Gruber: And here’s where I ran into trouble, because I linked to an article about it — Lauren Goode at Wired had an interview with Craig and got… really not much detail. [laughs]

Joz: Way to go, Craig.

Gruber: In my system, when I post to Daring Fireball, I tag articles. I didn’t know what to tag it with, ‘cos I don’t know what to call it!

J & M: Sneak peek…

[laughter]

Gruber: So Craig emphasized on stage, vehemently, that this is not a replacement for AppKit, this is a new thing. We have, y’know, unified these underlying layers between the systems, and it’s a new thing, and we, y’know…

Joz: Yes. It’s bringing over key frameworks to enable this.

Gruber: But there’s a lot of people out there — it doesn’t matter what you say, they think that this is, that AppKit is going away.

Joz: Did you not see the letters that were about 65 feet tall?

[laughter]

Gruber: But it’s like with any conspiracy theory, though. The more dispositive the evidence is, they’re like, “Well that’s, of course they’re going to say that!”

Joz: Next year we’ll get a bigger screen with 85 foot letters.

Gruber: it occurs to me, and let’s see if you agree. I don’t think from what I’ve seen so far, I don’t think that this is a replacement for AppKit, and that’s not meaning AppKit’s going away. It’s staying. But even for most Mac developers, I think most Mac apps as we know them are still better served in AppKit. But that there’s certain apps that just weren’t getting written in the first place.

Joz: It’s just like there’s WebKit apps that are there today, there’s Metal apps. There’s lots of things that are designed to go around different places. And there’s going to be Mac applications that are going to use all the traditional APIs that they have.

But there’s a lot of apps in iOS. As Craig says, there’s millions of them. And not all of them are going to be great Mac apps. But there’s going to be a lot of them that could be great Mac apps. If we do our job right, it shouldn’t be a ton of work for that to happen. All of our development’s done on the Mac to start with, so it’s an opportunity staring developers in the face, to say, “Hopefully not a whole lot of work [if] you have an app that’s appropriate to the Mac,” open up additional revenue for very little work, nice ROI. They win, our customers win, everybody’s happy.

Gruber: Yeah, so. Electron is one of those things where you can take a web app and shove like a whole web browser into each and every application, and every single thing it opens up, opens up a new instance of it, and it’s…

Joz: Yeah. I’d rather have the iOS experience than…

Gruber: Right. So if there’s a company that has a good iOS app, and they have a web app, it seems like there’s a lot of them, in the past, that when they were like, “Well, let’s get something on the desktop on the Mac,” they bundle up the web app and squeeze it into a thing.

If they’re not going to run a proper Mac app, like in AppKit and go the full way, taking the iOS app and Mac-ifying it is way better.

Joz: And remember, at the end of the day, you’re getting a Mac app.

Gruber: Yeah.

Joz: Those are Mac apps. They’re not iOS apps that are somehow emulated or run in some crazy mode, they end up being Mac apps.

Gruber: Right. File, Edit…

Joz: Exactly.

Rockwell: Everything.

Gruber: Everything’s up there in the menu bar. I think it’s really great. I think it’s pretty cool, and I think people will — 

Joz: Yeah, we’re excited for it.

Gruber: Yeah. That is about it for me. Do you guys have anything else you wanted to talk about?

I have some thank yous to give: I thank you both for coming. So let me thank you guys first. Mike and Joz, thank you for coming.

[huge applause]

I want to thank the staff and crew here at the California Theater. These people are very nice and they did a terrific, terrific professional job, really just a great place to hold an event. I cannot thank them enough.

Joz: I remember when we did an iPod event here many many years ago, we had U2 play here.

Gruber: Yeah, yeah.

Joz: I think it was 2004.

Gruber: I don’t remember. I do remember the event, and the reason — I remember thinking “Wow, this place is beautiful.” And then, remember the hands-on area? It was…

Joz: Yeah, absolutely. A little crunched.

Gruber: It was like the size of a van.

[laughs]

Joz: Yeah.

Gruber: And had very low ceilings.

Joz: Yeah. That wasn’t… I think that was the only event we had done here.

[more laughter]

Gruber: I want to thank Tito. That’s the ticketing service that everybody here used to get their tickets. I honestly don’t know what I would do without it. It makes something that — I don’t know what I would do. I would probably just not let people come into the show.

[laughter]

But I think my friend Paul Campbell who runs Tito is here, I thank you.

[clapping]

If you ever hold any kind of event and you need to sell tickets, I’m telling you, go to Tito, it’s so great.

I want to thank our sponsors: Instabug, MacStadium, and Microsoft. Seriously, we would not have this event without some sponsors. I have a couple of friends here who have been helping out all day: Marco Arment is up there running the live audio stream. Jake Schumacher, who you guys know from the app documentary, has been shooting the video for these live shows of mine.

I’ve told this story before. But he came in to do an interview with me for this documentary he’s making, and was in town for WWDC, and it’s like three in the afternoon, and I said: “Well, alright, I’ll do it, but… hey, why don’t you shoot my show?”

[laughter]

And now he shoots the show. And it gets better and better every year. The show looks so great. And he set up live streaming over YouTube, so I don’t know how many people are watching it on that, but there’s possibly more people than are even in this room who are watching, so all thanks to Jake.

[clapping]

My friend Caleb Sexton edits the show, the audio every week, he is here to make sure we sound good and that the show actually gets recorded. [laughs] So I thank Caleb.

[clapping]

Paul Kafasis has been the announcer, I think, also as long as I’ve been doing the [live] show, and he does a bang-up job announcing. I don’t know if you’ve seen him this week. He has a little bit of an ankle injury, he’s sort of got like, a peg leg thing? Easily could have punted on WWDC this year, but he came. And he won’t tell me this, but I can see it in his eyes that he came just to do the announcing for this show.

[laughter and clapping]

Last but not least, I need to thank my wife, Amy. I really don’t know how I would do this show without her. She handles about 300 issues that all pop up between 5:45 and 7:00, and she’ll come by and she’ll start to explain it to me, and she’ll look at me, and I’m there going through my cards, like, y’know…

And she’ll start to explain, and then go — “Ah… I got it.” And then she just goes and takes care of it. Seriously, so many little goofy things with seating and stuff go wrong. I really don’t know what I would do without her. So my thanks to Amy.

[huge claps and laughter]

And my sincerest thanks to all of you. Everybody in this room, everybody watching on the video stream. It is an enormous privilege to have so many people come here, and to be so excited to see this show every year. It still boggles my mind. And every time we have this event, the events staff always says “These are the nicest people.”

Except for that one guy [points in the balcony].

[huge laughter]

But other than him! Everybody always says, “My god, these people are so nice and orderly. They don’t leave trash behind.”

[laughs]

I thank you for coming. Hopefully, I will see you next year. And that’s it!

[lots of clapping]