Tom Yager on Apple’s Intel Benchmarking

There are lies, damn lies, and benchmarks, but InfoWorld’s Tom Yager thinks Apple’s new benchmarks are especially bogus:

In short, Apple used multiprocessor benchmarks to skew the performance advantage that its Intel-based machines enjoy compared to single-core PowerPC G4 and G5. Apple used the industry-standard SPEC suite components SPECint2000 and SPECfp2000, but here’s the catch: Apple used SPECint_rate2000 and SPECfp_rate2000. Both tests spawn multiple parallel benchmark processes and are specifically intended for comparing multiprocessor systems. Single CPU, or single-core machines do positively lousy on SPEC*_rate2000 tests. That’s predictable and universally understood. […]

Apple uses SPEC*_rate2000 tests as a foundation for claims that Intel-based Macs outperform PowerPC G4 and G5 by a factor of 2 to 5. Well, yeah. A dual-core anything outperforms a single-core anything else by a factor of 2 to 5 in benchmark tests that make use of multiple threads or processes, tests crafted specifically for the purpose of stressing SMP-based systems.

★ Tuesday, 10 January 2006