By John Gruber
World’s most powerful 130W USB-C battery pack — get up to 50% off on Kickstarter.
Early this week I noticed that I wasn’t able to use the Instant Hotspot feature with my iPhone XS. That’s the feature where you can leave the cellular hotspot turned off in Settings, but enable it on-the-fly from a Mac when you connect via the Wi-Fi menu. These “Personal Hotspots” show up at the top of the list of available Wi-Fi networks, in their own special section of the menu. My Wi-Fi menu no longer listed my iPhone, only my iPad. If I went into the iPhone’s Settings app and enabled the Personal Hotspot manually — i.e. turned it on and left it on — my iPhone’s hotspot was listed as a regular Wi-Fi network on my Mac, and when I connected, it worked just fine. So the hotspot worked, but the magic Instant Hotspot feature wasn’t working.
I tried rebooting the Mac and iPhone, of course. No dice. I reset network settings on the phone. No dice. I then noticed that my iPhone’s name (Settings → General → About → Name) had been changed to “iPhone”. Not even “John’s iPhone”, which is the default when you set up a new iPhone. Just plain “iPhone”. I changed it back to my custom name. Rebooted the phone again. Still no Instant Hotspot. And then eventually the device name was changed back to “iPhone” again. Weird, right? This was all on the release version of iOS 12.0.1, by the way.
I had a trip to New York coming up, and wanted to fix this. I did some searching on the web and eventually stumbled on a thread that suggested signing out of iCloud and then signing back in. This makes some sense, because all of these Continuity features go through iCloud. So I did that on the iPhone, and, long story short, that seemed to fix the issue. After one more reboot of the phone, Instant Hotspot was working perfectly.
A side effect of signing out of and back into iCloud is that it seemed to reset my iPhone’s photo library sync state. It didn’t delete my photos, but once I was signed back in to iCloud, the Photos app was trying to re-upload my entire library (over 28,000 photos and 1,100 videos) back to iCloud. I don’t think it was actually uploading them — I think that’s just the word Photos uses to indicate what it’s doing — but rather checking each of the photos on the phone against each of the photos in my iCloud library.
It got through most of them fairly quickly, but the last 4,500 or so were effectively stuck. This process was proceeding really slowly. Profoundly slowly. I kept the phone plugged in last night and checked every hour, and it was only processing about 15 or 16 items per hour. I let it run overnight and it only moved from 4,183 remaining items to just over 4,000.
Effectively, I think what happens is that when you turn off iCloud Photo Library, it leaves all the photos and videos on your phone in your local library. When you turn iCloud Photo Library back on, it has no idea which of the items in your local iPhone library are duplicates of items in your iCloud library, and so it has to check them one by one. Whatever algorithm it’s using for this is slow as molasses.
Adam Engst wrote about a similar problem on the Mac earlier this year:
I was seeing some strange problems on my 27-inch iMac running macOS 10.13.3 High Sierra. Messages wasn’t getting or sending messages, Wi-Fi calling wasn’t working, and after upgrading to 10.13.3, I was unable to enable auto-unlock with my Apple Watch. To solve these problems, I turned iCloud off and back on. Despite the iCloud preference pane throwing an ominous error, the problems did indeed disappear.
However, there’s a nasty side effect of turning iCloud off and back on: iCloud Photo Library needs to re-upload all your photos. It does this in order to compare the library’s contents to the synchronization “truth” at iCloud. Fair enough, except that this process can take days, depending on the size of your Photos library and the speed of your Internet connection. Bad Apple! We don’t see that sort of poor performance with Dropbox or Google Drive, and this behavior is both unnecessary and driving people away from iCloud Photo Library.
That’s pretty much exactly what I was seeing on my iPhone.
What surprised me about this isn’t just that it’s so dreadfully slow, but that iCloud Photo Library has gotten amazingly good in the last few years. It’s not just very reliable, but very fast. I took a lot of photos using three different iPhones (my old iPhone X, and my review unit iPhones XS and XS Max) while writing my XS review last month. And I worked on the review on two different Macs. Every photo and video I took on every iPhone synced to all the other devices in a matter of seconds every single time. iCloud Photo Library made the whole process ridiculously easy.
Wiping and restoring my entire iPhone seemed like overkill when the only issue I was having was photo syncing. So my next idea was to delete all the photos from my phone and start over from scratch with iCloud Photo Library.
So here’s what I did, and it seems to have worked. First, I eyeballed all the recent photos and videos I’d shot using my iPhone and double-checked that they had all already been synced to iCloud. They were — I could see all my recent shots on my other devices.
Next, I disabled iCloud Photo Library on my iPhone again. You do that by going into the Apple ID section of Settings (where your name is shown at the very top of the root level) → iCloud → Photos and turned off everything. When it asked if I wanted to download a copy of the photos and videos from my iCloud library I declined.
Next, I wanted to delete every single photo and video from my iPhone. To my knowledge there is no easy way to do this on the iPhone itself. (There are a lot of tasks like this that are easy on the Mac thanks to Edit → Select All that are painfully tedious on iOS. Update: Here’s a clever way to use iOS 12’s Shortcuts app to delete all photos and videos from your Library.) I connected the iPhone to my Mac with a Lightning cable and used Image Capture to delete all photos and videos from my phone. Image Capture just treats the iPhone like a regular camera. Image Capture crashed three times during this process (I’m still running MacOS High Sierra 10.13.6, for what it’s worth), but after the fourth run the iPhone had no photos or videos left.
Then I re-enabled iCloud Photo Library on the phone, and about 20 minutes later, everything was back to normal. My iPhone reported exactly the same number of photos and videos in my library as on all my other devices. Most of those items are still just placeholders, even as I write this, but they’re filling in steadily — which is exactly how iCloud Photo Library works when you start syncing a large library to a new device.
So if you temporarily turn off iCloud Photo Library and turn it back on, it might be easier to just delete all your photos from your iPhone first, and let them all sync back from iCloud. ★
Bloomberg Businessweek today published an absolutely incredible story alleging that Chinese intelligence compromised thousands of data center servers by infiltrating the supply chain to insert hard-to-detect rogue chips on motherboards from a company named Supermicro. The entire report, by Jordan Robertson and Michael Riley, is worth reading in full.
Bloomberg alleges that Apple and Amazon were both among the companies that installed the compromised hardware. Apple and Amazon both vehemently deny the report. Someone is either wrong or lying. This cannot all be true.
From Bloomberg’s report, regarding Amazon:
Nested on the servers’ motherboards, the testers found a tiny microchip, not much bigger than a grain of rice, that wasn’t part of the boards’ original design. Amazon reported the discovery to U.S. authorities, sending a shudder through the intelligence community. Elemental’s servers could be found in Department of Defense data centers, the CIA’s drone operations, and the onboard networks of Navy warships. And Elemental was just one of hundreds of Supermicro customers.
Apple was an important Supermicro customer and had planned to order more than 30,000 of its servers in two years for a new global network of data centers. Three senior insiders at Apple say that in the summer of 2015, it, too, found malicious chips on Supermicro motherboards. Apple severed ties with Supermicro the following year, for what it described as unrelated reasons.
And regarding both companies’ denials:
The companies’ denials are countered by six current and former senior national security officials, who — in conversations that began during the Obama administration and continued under the Trump administration — detailed the discovery of the chips and the government’s investigation. One of those officials and two people inside AWS provided extensive information on how the attack played out at Elemental and Amazon; the official and one of the insiders also described Amazon’s cooperation with the government investigation. In addition to the three Apple insiders, four of the six U.S. officials confirmed that Apple was a victim.
The companies’ denials are seemingly unequivocal, however. Apple’s statement to Bloomberg:
Over the course of the past year, Bloomberg has contacted us multiple times with claims, sometimes vague and sometimes elaborate, of an alleged security incident at Apple. Each time, we have conducted rigorous internal investigations based on their inquiries and each time we have found absolutely no evidence to support any of them. We have repeatedly and consistently offered factual responses, on the record, refuting virtually every aspect of Bloomberg’s story relating to Apple.
On this we can be very clear: Apple has never found malicious chips, “hardware manipulations” or vulnerabilities purposely planted in any server. Apple never had any contact with the FBI or any other agency about such an incident. We are not aware of any investigation by the FBI, nor are our contacts in law enforcement.
That statement is credited only to “Apple”, so presumably it was written by Apple PR. Amazon issued a similar statement to Bloomberg, but later published a full response, signed by Steve Schmidt, the company’s chief information security officer. Schmidt is adamant and clear:
There are so many inaccuracies in this article as it relates to Amazon that they’re hard to count. We will name only a few of them here. First, when Amazon was considering acquiring Elemental, we did a lot of due diligence with our own security team, and also commissioned a single external security company to do a security assessment for us as well. That report did not identify any issues with modified chips or hardware. As is typical with most of these audits, it offered some recommended areas to remediate, and we fixed all critical issues before the acquisition closed. This was the sole external security report commissioned. Bloomberg has admittedly never seen our commissioned security report nor any other (and refused to share any details of any purported other report with us).
The article also claims that after learning of hardware modifications and malicious chips in Elemental servers, we conducted a network-wide audit of SuperMicro motherboards and discovered the malicious chips in a Beijing data center. This claim is similarly untrue. The first and most obvious reason is that we never found modified hardware or malicious chips in Elemental servers. Aside from that, we never found modified hardware or malicious chips in servers in any of our data centers.
I see no way around it: either Bloomberg’s report is significantly wrong, at least as pertains to Amazon and Apple, or Apple and Amazon have issued blatantly false denials. You can, perhaps, chalk up Apple’s denial to it being written by Apple PR. I don’t think this would happen, but hypothetically this issue could be deemed so sensitive — either within the company or as a national security issue — that the people at Apple with knowledge of the situation lied to Apple PR. But in my experience, Apple PR does not lie. Do they spin the truth in ways that favor the company? Of course. That’s their job. But they don’t lie, because they understand that one of Apple’s key assets is its credibility. They’d say nothing before they’d lie.
Schmidt signing his name to Amazon’s response is more telling. Presumably no one at Amazon would be more familiar with the details of such a breach than Schmidt.
One way or the other, there is more to come on this story, and the credibility of either Bloomberg, or Apple and Amazon, is going to take a significant hit. Currently those are the two most valuable publicly-traded companies in the world.
A few other notable tidbits. From Bloomberg’s report:
One government official says China’s goal was long-term access to high-value corporate secrets and sensitive government networks. No consumer data is known to have been stolen.
And then this from Amazon’s response:
Because Elemental appliances are not designed to be exposed to the public internet, our customers are protected against the vulnerability by default.
I do not understand how, if these servers are not exposed to the public internet, they could “phone home” to Chinese servers outside the data centers.
Technical details aside, the whole central thesis of the story rings true — China cannot be trusted as a state actor, but the entire technology industry is dependent upon the Chinese supply chain. It is completely credible that the managers of Chinese factories are susceptible to bribes and threats of “inspections” that would shut down their plants. From the Bloomberg report:
Over the decades, the security of the supply chain became an article of faith despite repeated warnings by Western officials. A belief formed that China was unlikely to jeopardize its position as workshop to the world by letting its spies meddle in its factories. That left the decision about where to build commercial systems resting largely on where capacity was greatest and cheapest. “You end up with a classic Satan’s bargain”, one former U.S. official says. “You can have less supply than you want and guarantee it’s secure, or you can have the supply you need, but there will be risk. Every organization has accepted the second proposition.”
Lastly, whatever the veracity of the report, Bloomberg deserves kudos for this sentence:
Two of Elemental’s biggest early clients were the Mormon church, which used the technology to beam sermons to congregations around the world, and the adult film industry, which did not.
This week, DF has seemed incredibly slow for some people, sometimes. Here’s a Twitter search for tweets to me with the word “slow” this week. This was killing me, because I pride myself on Daring Fireball being a fast-loading website, and because this was a pretty big week content-wise.
It was not my server, and had nothing to do with higher levels of traffic from my iPhone XS and Series 4 Apple Watch reviews. When DF itself is slow — which happens rarely but does happen — you almost always see DF’s trademark #4a525a slate gray background first, then the elements of the page slowly fill in. If you experienced slowness this week, you probably just saw a white background in your browser tab, and then all of a sudden the whole thing filled in. This sometimes took 30-60 seconds. Long story short, it was taking that long for the initial request to even get to my server; once it did, everything after that was as fast as usual. I’m still not sure what exactly was causing this, but I’ve worked around it by having Cloudflare act as an HTTP/S proxy for daringfireball.net. If any of you continue to see slow page loads, let me know.
Second, there was an issue where requests using the “www” prefix over HTTPS were triggering an SSL certificate warning from Safari that the site might not be legitimate. Only when using the “www” prefix, and only over HTTPS, not HTTP, because it was an SSL problem. I hadn’t changed anything on my end, but the latest version of Safari has tightened its SSL security. (Which is good.) I’ve never made use of a “www” prefix at Daring Fireball — I hate that prefix. But you have to support it, because so many people type it out of habit. So I’ve always redirected requests for “www.daringfireball.net” to “daringfireball.net”. The problem is, because I don’t actually use the “www” domain, I never properly supported it in my SSL certificate. Until a few weeks ago, Safari would just let you get redirected; now it doesn’t.
I solved this problem using Cloudflare too. In fact both problems were fixed the same way — by clicking one button in Cloudflare’s list of my DNS entries to allow Cloudflare to proxy HTTP/S requests. I switched to Cloudflare a year or so ago for managing my DNS, and I couldn’t be happier. It’s like magic. DNS can seem like voodoo (at least to me) but Cloudflare makes it easy. I don’t even pay them — I’m just using their free basic account level. This is not a sponsorship or ad — I just had to thank them for their wonderful service.
Anyway, I want to apologize to any of you affected by either of these issues. ★