GIF, H.264, and Patents

Arthur Wyatt, in a post on MetaFilter regarding the battle over HTML5 video codecs:

However Mozilla have taken a stance against incorporating H.264 into Firefox on the grounds that it is patented and has to be licensed. Arguments are now being made for and against Mozilla sticking to its ideals. John Gruber of Daring Fireball points out that Firefox already supports proprietary formats such as GIF. Um, perhaps not the best example.

A few readers have emailed me expressing a similar sentiment — more or less that GIF serves as an example showing why Mozilla should continue to refuse to support H.264.

The GIF story in a nut: the file format was introduced by CompuServe in 1987. It was relatively simple and could produce files that were decent-looking at remarkably small file sizes — file size being the most important aspect of a graphics format at a time when online access was via modems. (You can argue that every kilobyte still matters today, but in the ’80s and even ’90s, every byte counted.) Because of its relative simplicity and small file sizes, GIF was the first major image format of the web. GIF uses LZW compression, however, and LZW compression was covered by a patent held by Unisys. After the format became widespread, Unisys attempted — in various ways and with various (but not much) success — to extract licensing fees for it.

The main result of Unisys’s effort was to drive the creation and adoption of the PNG format — which not only was unencumbered by patents but was also technically superior.

The analogy some people are drawing between H.264 and GIF is that the MPEG LA — the industry consortium that controls the rights and licensing for the patent pool behind H.264 — could pull a similar bait-and-switch type trick: give H.264 a few years to become ever more popular, and then, boom, come 2016 (when the current license expires), begin requiring web publishers to pay a licensing fee for distributing H.264-encoded video.

I disagree.

I don’t know what the MPEG LA will do come 2016. Perhaps they will attempt to charge web publishers for licenses to distribute H.264 video. But if they do, web publishers will react the way they did to Unisys’s GIF threats: by switching to another format. Switching to another format would be an expensive time-consuming pain in the ass, but that’s how I think publishers will react if the MPEG LA pulls the rug out from under them.

Other than adding to the pile of evidence that software patents suck and are a drain on the industry, there isn’t that much similarity between the GIF and H.264 situations. Unisys didn’t invent or hold the rights to the GIF format, they held the rights to a compression algorithm patent that the GIF format was deemed to violate after the format was created and in widespread use. Unisys was a single entity that attempted to profit from a patent that it held. H.264’s rights and patents are held by MPEG LA, which is not a company like Unisys, but an industry consortium. MPEG LA licenses patent pools for a variety of technologies, not just H.264. It seems to me it’s in their interests to behave consistently and predictably, not capriciously and opportunistically.

Again, perhaps I’m naive. We’ll find out in a few years. But web site publishers are clearly betting on H.264 remaining free to use for freely distributed web video. I don’t think anyone is arguing that Mozilla should drop Ogg Theora. What some of us would like them to do is follow Google’s lead with Chrome by supporting both H.264 and Ogg Theora. (The same goes for Apple with Safari.)

But if anything, I’d say it’s Ogg Theora that more resembles GIF, patent-wise. Prior to Unisys’s assertion of its LZW patent, GIF was considered a free file format. (According to Wikipedia, CompuServe developed the format in the 1980s without knowledge of Unisys’s LZW patent.) The developers of Ogg Theora have released the format and implementations freely, and claim no patent rights to it. But that doesn’t mean someone else doesn’t hold patents — or won’t be granted patents — that they believe or will claim Ogg Theora violates. For one example, consider this interview by Jan Ozer with MPEG LA’s Larry Horn, where Horn states:

Horn: In addition, no one in the market should be under the misimpression that other codecs such as Theora are patent-free. Virtually all codecs are based on patented technology, and many of the essential patents may be the same as those that are essential to AVC/H.264. Therefore, users should be aware that a license and payment of applicable royalties is likely required to use these technologies developed by others, too. […]

Ozer: It sounds like you are saying that some of your patent holders own patents that are used in Ogg. Is that correct?

Horn: We believe that there are patent holders who do.

In short, using Ogg Theora is no guarantee that the MPEG LA isn’t going to pursue you for licensing fees.

This uncertain patent landscape is reportedly one reason Apple doesn’t support Ogg Theora. If some patent troll decides H.264 violates a patent, they must go to court with MPEG LA, not individual licensees. If a patent troll decides Ogg Theora violates a patent, they might sue those who are using it. I bring this up not to say browsers should not support Ogg Theora — I’m just saying Ogg is a lot closer to being in GIF’s boat than H.264 is.

I don’t like the idea that the standard format for web video isn’t a free, open standard like PNG or JPEG are for graphics. But that’s the way it is. H.264 is popular and growing, and while Mozilla’s refusal to support it directly in Firefox may temper that, it’s not going to drive web publishers to use Ogg Theora instead or even in addition to H.264. It just means Firefox users will continue to be served video using Flash — often encoded as H.264.

The idealist perspective is expressed here by Thom Holwerda at OSNews:

Now that Internet Explorer 9 has been let out its cage, we all know a great deal more about Microsoft’s position towards the video codec situation with the HTML5 video tag. Microsoft has chosen for H264, a codec it already includes in Windows by default anyway. This means that apart from Firefox and Opera, every other major browser will support H264. Some are seeing this as a reason for Mozilla to give in to their ideals and include support for H.264 as well — I say: Mozilla, stick to your ideals. The last people you should be listening to in matters like this are web developers.

This presents a sort of chicken-or-egg scenario. Should browser makers support the formats being used by web publishers? Or should web publishers use the formats supported by browser makers? (Sort of like the descriptivism/prescriptivism divide in the world of language and usage.)

I come down on the side of web publishers, but, admittedly, even though I don’t publish much video content, I’m clearly biased insofar as that I am a web publisher. But I think history shows that practical concerns, not idealism, is what drives web publishers to adopt new formats.

But if Mozilla’s position were really about idealism — tough love for the good of the web in the name of free, open file formats — then in addition to not supporting H.264, they’d drop support for plugins like Flash Player. I believe such a move would just drive Firefox users to Chrome and Safari (or even back to IE), and I suspect Mozilla knows this, too, which is why dropping plugin support isn’t being discussed. But they can’t say Firefox only supports free and open video formats while still supporting Flash.