The Big AI Labs Are Suddenly Competing with Your Own Data
Source: Gradient Flow, Ben Lorica, 2026-07-28 · raw/rss
TL;DR
Lorica's argument is that open weights are no longer the story; what teams can do with them is. The post-training stack has filled in around reinforcement fine-tuning (where a model practices a task and learns from whether it succeeded, as opposed to supervised fine-tuning on labeled examples of good output), and more than 25 startups are now building in that space. The algorithm is rarely the product. What they are actually assembling is the unglamorous machinery: environments where models practice, graders and verifiers that decide whether the work was done correctly, data generation and curation tooling, and evaluation, deployment, and monitoring.
The demand-side explanation is the sharpest part. Teams running generic frontier models in production hit the same three walls. Prompt adjustments become whack-a-mole as each fix creates a new failure. Costs stop being ignorable at real scale. And the provider changes a model, retires it, raises the price, or starts competing in the same market as its customer. At that point renting a general-purpose model stops looking like a long-term architecture. Reported effect of specialization on well-defined tasks: 10 to 30 percentage points.
Why the third wall is the essay's real subject
The title is about the third wall, and it is the one enterprises are least prepared for. Supply-chain risk in AI has been discussed as price and deprecation risk. Lorica's framing is that the model provider is a competitor with visibility into your workload, which is a different category of risk and one that specialization on owned weights directly answers. Most enterprise AI work is narrower than what frontier labs are building toward: classification, extraction, forecasting, tool selection, execution of repeatable workflows. Those are exactly the tasks where a specialized small model wins on cost and where the strategic exposure of renting is highest.
How this connects to the wiki's research record
The technical enabler landed on the Kurate board the same week. Cross-Tokenizer On-Policy Distillation via Byte-Prefix Marginalization (07-29) removes the shared-tokenizer requirement that has quietly confined multi-teacher on-policy distillation (training a student on its own generations under dense token-level supervision from bigger teachers) to model families a single lab controls. Every multi-teacher run the wiki has logged, including Nemotron 3 Ultra (06-16) with more than ten teachers, used in-house teachers with one tokenizer. BPM lets a team distill Qwen, GLM, and MiniMax into one owned student, gaining 3.7 to 6.6 points over prior cross-tokenizer methods. Lorica describes the market pull; BPM is the mechanism that makes the specific version of it (consolidate the open ecosystem into a model you own) technically clean.
The verifier layer he describes is the wiki's distillation bottleneck seen from the buy side. The knowledge-distillation page's recurring limit across every paper on it is that reliability gating needs a verifier or an answer key, which confines results to math and code. Lorica reports 25 startups building graders and verifiers as products. If that layer commoditizes for enterprise task domains, the domain restriction that caps the research line lifts for exactly the narrow, well-specified tasks he says enterprises actually have. That is a real research-to-industry handoff, running in the unusual direction: industry building the missing component the papers keep flagging.
It also sits directly against the week's pricing news. The Information reports an IT consulting firm quoted roughly $1.5M to renew a Cursor contract that had cost about $200K for 800 licenses, for essentially the same usage, after Cursor's shift to usage-based pricing. Separately, Anthropic's Claude Code retains enterprise dominance despite surging usage-based costs. Lorica's second wall is not hypothetical; it is being enforced in contract negotiations this quarter, and it is the most concrete reason to expect the specialization thesis to be tested at scale within a year.
Gaps
The 10-to-30-point figure comes from examples described to the author, not a study, and "well-defined tasks" is doing selection work. The essay does not price the total cost of owning the post-training stack against renting, which is the calculation that actually decides this, and the 25-startup count is a supply signal rather than a demand one.