Masnick’s Impossibility Theorem: Content Moderation at Scale Is Impossible to Do Well

Mike Masnick, back in 2019:

I’ve argued for years that while many people like to say that content moderation is difficult, that’s misleading. Content moderation at scale is impossible to do well. Importantly, this is not an argument that we should throw up our hands and do nothing. Nor is it an argument that companies can’t do better jobs within their own content moderation efforts. But I do think there’s a huge problem in that many people — including many politicians and journalists — seem to expect that these companies not only can, but should, strive for a level of content moderation that is simply impossible to reach.

And thus, throwing humility to the wind, I’d like to propose Masnick’s Impossibility Theorem, as a sort of play on Arrow’s Impossibility Theorem. Content moderation at scale is impossible to do well. More specifically, it will always end up frustrating very large segments of the population and will always fail to accurately represent the “proper” level of moderation of anyone. While I’m not going to go through the process of formalizing the theorem, a la Arrow’s, I’ll just note a few points on why the argument I’m making is inevitably true.

★ Wednesday, 21 June 2023