Nathan Lambert Explains Open Model Inference Market Growth

What changed

AI researcher Nathan Lambert, in a research note for his Interconnects platform, discusses the market for open model inference. He notes that current measurements of open model adoption are imprecise, relying on noisy platform metrics, marketing claims, and imperfect evaluations. Lambert uses data from open model inference startups like Baseten, Fireworks, and Together AI, along with OpenRouter's reported token volume, as bellwethers. He highlights a slide from a previous talk showing near-exponential growth in token volume for these startups, with Fireworks announcing they serve over 40 trillion tokens per day to customers' customized models. This figure represents a vertical that grew over twofold since the summer, suggesting a total well above 100 trillion tokens for their inference services in the fall.

Why it matters for builders

Lambert's work is significant because it attempts to quantify the adoption and performance of open-source AI models, a critical area for developers seeking alternatives to proprietary systems. By tracking metrics like token volume, he provides insights into the viability and growth trajectory of the open-source AI ecosystem, which directly impacts the tools and platforms builders can leverage.

Practical impact

Lambert's analysis suggests that the open model inference market is experiencing substantial growth, potentially challenging the dominance of closed models. Builders should pay attention to these trends as they inform the future development and accessibility of AI technologies. The increasing adoption of open models could lead to more diverse and customizable AI solutions.

Caveats and source limits

Lambert himself states that all measurements on open model adoption are "incredibly imprecise." The data presented relies on public disclosures from startups and is not independently verified by AI on Radar. While Lambert uses these figures to approximate market trends, they should be viewed as informed approximations rather than definitive facts. The analysis focuses on token volume as a primary metric, and other factors influencing adoption may not be fully captured.