From a statement Apple sent to TechCrunch:
We appreciate the opportunity to work with Consumer Reports over
the holidays to understand their battery test results. We learned
that when testing battery life on Mac notebooks, Consumer Reports
uses a hidden Safari setting for developing web sites which turns
off the browser cache. This is not a setting used by customers and
does not reflect real-world usage. Their use of this developer
setting also triggered an obscure and intermittent bug reloading
icons which created inconsistent results in their lab. After we
asked Consumer Reports to run the same test using normal user
settings, they told us their MacBook Pro systems consistently
delivered the expected battery life. We have also fixed the bug
uncovered in this test.
So there’s a bug in Safari when you disable the cache (Develop: Disable Caches — and the entire Develop menu is off by default). Disabling the cache should decrease battery life in a test like CR’s. And if there’s a bug, I can see why it might dramatically decrease battery life. But that still doesn’t explain how Consumer Reports’s testing showed results ranging from 3.75 hours (poor) to 19.5 hours (seemingly too good to be true).
I still think something was/is wrong with Consumer Reports’s testing (19.5 hours?) but I don’t think it’s fair to say that disabling the caches is unfair or a flawed method. And while the preference setting is obscure, I wouldn’t call it “hidden”. To me, hidden preferences are the ones you can only enable from calls to
defaults in Terminal. You can turn the Develop menu on by clicking a visible checkbox in the “Advanced” tab of Safari’s preferences.
★ Tuesday, 10 January 2017