Open Source Models

Search documents
Top LLM Providers for Enterprises
Bloomberg Technology· 2025-08-04 20:17
In the enterprise domain in particular, you guys are saying that anthropic leads in terms of market share, then open air is just behind. Just for transparency. While our audience of course, Menlow Low is on the cap table of anthropic and you have a sort of partnership with them for compute access but break down the data and how in Prop X pulled ahead in that space.Yeah, I mean, there's really three things that stood out in the report. First is, you know, Anthropic really pulled ahead in terms of enterprise ...
The Rise of Open Models in the Enterprise — Amir Haghighat, Baseten
AI Engineer· 2025-07-24 15:30
AI Adoption in Enterprises - Enterprises' adoption of AI is crucial for realizing AI's full potential and impact [2] - Enterprises initially experiment with OpenAI and Anthropic models, often deploying them on Azure or AWS for security and privacy [7] - In 2023, enterprises were "toying around" with AI, but by 2024, 40-50% had production use cases built on closed models [9][10] Challenges with Closed Models - Vendor lock-in is not a primary concern for enterprises due to the increasing number of interoperable models [12][13] - Ballooning costs, especially with agentic use cases involving potentially 50 inference calls per user action, are becoming a significant concern [20] - Enterprises are seeking differentiation at the AI level, not just at the workflow or application level, leading them to consider in-house solutions [21] Reasons for Open Source Model Adoption - Frontier models may not be the right tool for specific use cases, such as medical document extraction, where enterprises can leverage their labeled data to build better models [16][17] - Generic API-based models may not suffice for tasks requiring low latency, such as AI voices or AI phone calls [18] - Enterprises aim to reduce costs and improve unit economics by running models themselves and controlling pricing [20][21] Inference Infrastructure Challenges - Optimizing models for latency requires both model-level and infrastructure-level optimizations, such as speculative decoding techniques like Eagle 3 [23][24][25][26] - Guaranteeing high availability (four nines) for mission-critical inference requires robust infrastructure to handle hardware failures and VLM crashes [27][28] - Scaling up quickly to handle traffic bursts is challenging, with some enterprises experiencing delays of up to eight minutes to bring up a new replica of a model [29]