Ben Thompson’s DeepSeek FAQ

Chinese AI lab DeepSeek made waves last week when they dropped their new “open reasoning” LLM named R1. But over the weekend the full repercussions of their achievements (plural) began to sink in across the industry. My partner-in-Dithering Ben Thompson has written an extraordinarily helpful FAQ-style explanation at Stratechery today. If you, like me, were looking at the news today and thinking “Jeebus what the hell is going on with this DeepSeek thing?”, read Thompson’s piece first. Two choice excerpts, first regarding OpenAI:

R1 is notable, however, because o1 stood alone as the only reasoning model on the market, and the clearest sign that OpenAI was the market leader.

R1 undoes the o1 mythology in a couple of important ways. First, there is the fact that it exists. OpenAI does not have some sort of special sauce that can’t be replicated. Second, R1 — like all of DeepSeek’s models — has open weights (the problem with saying “open source” is that we don’t have the data that went into creating it). This means that instead of paying OpenAI to get reasoning, you can run R1 on the server of your choice, or even locally, at dramatically lower cost.

Second, regarding DeepSeek’s use of distillation (using existing LLMs to train new smaller ones):

Here again it seems plausible that DeepSeek benefited from distillation, particularly in terms of training R1. That, though, is itself an important takeaway: we have a situation where AI models are teaching AI models, and where AI models are teaching themselves. We are watching the assembly of an AI takeoff scenario in realtime.

★ Monday, 27 January 2025