Ilya Sutskever left OpenAI to found SSI and in November told Dwarkesh Patel that the industry had moved into the wrong era: scaling was over, data are finite, and pre-training would run out. This week DeepSeek provided concrete evidence supporting that view by releasing a lean model with only a fraction of the parameters of its flagship that nevertheless outperformed the larger model on tasks representative of what work agents actually do.
What DeepSeek reported
- Models and numbers cited by DeepSeek:
- V4.1 Flash: 552 billion parameters, but only 8–16 billion parameters are activated per token.
- V4-Pro: 1.6 trillion parameters, with 49 billion activated per token.
- On DeepSeek’s internal coding tests the lean model scored 74.2 versus 62.7 for the larger model.
- According to the company, as of September 14 every V4-Pro request is being rerouted to the lean model.
Why this matters
Sutskever’s point, made in November, was not simply that models must be smaller, but that the returns to raw scaling have shifted. The DeepSeek results illustrate that runtime efficiency — how many parameters are actually woken per token — can matter more for practical performance than total parameter count.
Technically, the V4-Pro’s much larger total parameter count corresponds to a higher number of activated parameters per token, while V4.1 Flash achieves better task results with a much lower activation budget. In DeepSeek’s coding benchmark this translated into a measurable advantage for the leaner model.
Implications
- For roughly five years, a competitive moat in the industry was often measured by cluster size and sheer parameter count. DeepSeek’s announcement suggests competition may increasingly favor models and systems that optimize activation efficiency and runtime resource use.
- If the pattern generalizes beyond DeepSeek’s internal tests, priorities could shift toward hardware and software optimizations that reduce active parameter usage (for example, sparsity techniques or conditional computation) rather than unabated increases in total parameters.
Closing note
DeepSeek’s published numbers do not invalidate research on large models, but they do support Ilya Sutskever’s observation that the marginal returns on scaling are changing. The September 14 routing change and the benchmark scores provide concrete examples of how practical outcomes can diverge from expectations based solely on total parameter counts.
