What’s Good for the Goose, AI Training Edition

Stephanie Palazzolo, writing for The Information (paywalled, alas):

Researchers at OpenAI believe that some rival AI developers are training their reasoning models by using OpenAI’s o1 reasoning models to generate training data, according to a person who has spoken to the company’s researchers about it. In short, the rivals can ask the o1 models to solve various problems and then use the models’ chain of thought — the “thought process” the models use to solve those problems — as training data, the person said.

You might be wondering how rival developers can do that. OpenAI has explicitly said it hides its reasoning models’ raw chains of thought due in part to competitive concerns.

But in answering questions, o1 models include a summarized version of the chain of thought to help the customer understand how the models arrived at the answer. Rivals can simply ask another LLM to take that summarized chain of thought and predict what the raw chain of thought might have been, the person who spoke with the researchers said.

And I’m sure these OpenAI researchers are happy to provide this training data to competitors, without having granted permission, in the same way they trained (and continue to train) their own models on publicly available web pages, without having been granted permission. Right?

★ Thursday, 5 December 2024