Study Comparing Android to iPhone Web Browsing Speed Flawed

Jim Dalrymple:

One of the biggest surprises the Blaze team found was that “despite significant JavaScript performance gains in the latest Apple iOS 4.3 release and Google Android 2.3 releases, these improvement made no measurable improvement on the actual page load times of the sites tested.”

There is a good reason for this. According to Blaze’s own documentation the “measurement itself was done using the custom apps which use the platform’s embedded browser. This means WebView (based on Chrome) for Android, and UIWebView (based on Safari) for iPhone.”

Right. The problem with Blaze’s entire study is that they didn’t test what they claimed to be testing. They used custom apps for iOS and Android, but claim the results show that Android’s browser is faster than iOS’s Mobile Safari. Instead, their results show that Android’s WebView control is faster than iOS’s UIWebView control. Mobile Safari is not just a thin wrapper around the system’s UIWebView control — it has its own caching system, its renderer uses asynchronous multithreading (UIWebView does not), and, as of iOS 4.3, Mobile Safari uses its own much faster JavaScript engine (“Nitro”).

That’s not to say it isn’t interesting that Android’s WebView for apps is faster than iOS’s UIWebView for apps, but it just isn’t true that these results are indicative of anything regarding Mobile Safari’s performance. It’s easy to see that Mobile Safari is faster than UIWebView — just run something like the SunSpider benchmark twice, once in Mobile Safari and once in any app from the App Store with a web content view. On my iPhone 4, Mobile Safari runs SunSpider almost three times as fast as an app using UIWebView.

These Blaze guys are either incompetent or dishonest attention seekers, given that they claim this, in an update to their report:

Some wonder whether the new Nitro JavaScript engine was used in our measurements. We’re still investigating this issue, as the report was completed before it was made known. So far we’ve seen indications in both directions, so we can’t say for sure it’s being applied.

That said, the results from measuring Android show that JavaScript only accounts for a small percentage of the total load time, about 15% on average. This implies that even if Nitro is not in use, it likely can only slightly narrow the gap. We’ll follow up with any additional info.

Because the thing is, Nitro isn’t the only difference between Mobile Safari’s rendering and UIWebView’s rendering. Mobile Safari has better caching and asynchronous multithreading, too.

★ Thursday, 17 March 2011