LMSYS Org, the nonprofit behind the open-source SGLang inference engine, has launched the SGLang Consortium, bringing eight of the biggest names in AI infrastructure together around a shared roadmap for the software that serves AI models to users. The announcement came in the opening keynote of the inaugural SGLang Summit in San Francisco on November 12.

The founding members read like a cross-section of the entire AI stack: Nvidia, AMD, Intel, Google, Meta, Cloudflare, Modal and RadixArk. It is a rare sight — direct silicon rivals and competing cloud giants agreeing to steer the same open-source project, because the software that runs AI models in production has become too important for any one company to own alone.

Why inference is now the battleground

Training a model grabs the headlines, but serving it is where the money now goes. Deloitte estimates that inference will account for roughly two-thirds of all AI compute in 2026, up from about half in 2025. SGLang sits at the centre of that shift: the engine, part of the PyTorch ecosystem, runs on hundreds of thousands of GPUs and powers some of the largest production inference deployments in the world, for companies including Meta, Microsoft, Google, Amazon, Thinking Machines Lab, Cloudflare, Reflection and Modal.

The engine is best known for RadixAttention, a technique that reuses cached computation across requests, and for fast structured generation. As models grow more agentic — chaining dozens of reasoning steps and tool calls into a single answer — squeezing more work out of every GPU matters more than ever.

What the consortium will actually do

The group gives its members a standing forum to shape SGLang's roadmap with its maintainers, turning shared production needs into joint engineering work. Five focus areas were named: support for frontier models, reasoning workloads, multimodal inference, distributed serving, and hardware optimisation.

That last item is the telling one. SGLang already runs across Nvidia GPUs, AMD Instinct accelerators, Intel Xeon processors, Google TPUs, Huawei Ascend chips and Moore Threads hardware. Each founding member will designate technical representatives who meet with the maintainers twice a year — but LMSYS keeps final authority over technical decisions and code acceptance, so the consortium guides the roadmap rather than taking over the project.

A governance move, not a product launch

No new feature shipped with the announcement, and the real test will be whether the group produces public cross-vendor benchmarks and faster, cheaper serving in code rather than press releases. Still, the signal is hard to miss: with inference costs climbing, even Nvidia — whose CUDA software has long been its moat — is now sitting at a table built to make AI serving work well on everybody's hardware.

The timing adds weight. The consortium lands days after a security flaw in SGLang was exploited at NEAR AI, a reminder that infrastructure this widely used needs shared stewardship as well as shared roadmaps. The first SGLang Summit, held November 12–13 at San Francisco's Fort Mason Center, drew more than 100 speakers across reinforcement learning, multimodal serving and hardware sessions.

By Hamza Faisal