By Hamza Faisal
A day after the U.S. government ordered its agencies to rip DeepSeek's models out of federal systems, the Chinese AI lab answered with open code. DeepSeek published DeepInfer, the inference framework it says powers its own large-scale production services, on GitHub on October 20, 2026, under the permissive Apache 2.0 license.
DeepInfer is the engine behind DeepSeek-V4.1-Flash: the software that takes a trained model and serves it to users at scale — splitting work across chips, batching requests, and deciding how fast answers come back. Serving costs are a huge part of what makes AI expensive, and DeepSeek has now handed its own production playbook to everyone.
Two chip families, one codebase
The headline detail is hardware. DeepInfer ships with first-class support for two very different chip families: Nvidia's SM100 (Blackwell) GPUs and Huawei's Ascend 950 NPUs. Few frontier AI labs have ever treated a Chinese chip as an equal to Nvidia's in a serving framework — and the timing is deliberate: DeepSeek said last week it plans to pre-train on Huawei silicon by the end of 2026.
The release stands on familiar foundations. DeepSeek's README says DeepInfer was "originally built primarily on vLLM, with additional inspiration from SGLang and TensorRT-LLM" — the open-source engines that already power much of the world's AI serving. On top of that base, DeepInfer bundles DeepSeek's own recent systems research into operational form: custom kernels such as FlashMLA, DeepGEMM and DeepEP, speculative decoding that drafts tokens ahead and verifies them in one pass, deterministic sampling, batch-invariant inference, and structured outputs constrained by regex, JSON schema or grammar.
What it is — and what it isn't
For now, this is a first release, not a complete serving stack. It runs a single model — DeepSeek-V4.1-Flash — and only in offline inference mode. DeepSeek's README is explicit about what's missing: a KV cache system, prefill/decode disaggregation, and load balancing across data-parallel replicas. In other words, you can study and run DeepInfer, but replicating DeepSeek's actual service is another job entirely.
Alongside DeepInfer, DeepSeek also open-sourced deepseek-recipe, its production token encoding and decoding component, under the MIT license. DeepSeek said it is sharing the stack "to help the community lower compute costs, reduce energy use, and make AI more affordable," and that inference code for future models will land in the same repository.
One day after the U.S. ban
The release landed the day after the U.S. Department of Homeland Security ordered federal civilian agencies to remove DeepSeek models from their systems over security concerns. DeepSeek's response is characteristically its own: give away the infrastructure instead of retreating.
The community reaction was immediate. The GitHub repository drew thousands of stars within hours of launch, and DeepSeek's announcement post passed 3,400 likes in its first hour. A popular reply summed up the irony: OpenAI, founded to be open, stayed closed — while the lab under U.S. sanctions keeps publishing.
Whether DeepInfer actually lowers serving costs for anyone else depends on hardware most teams don't own. But as a rare public look at how a frontier lab really runs its models — across both Nvidia and Huawei silicon — it is already one of the most significant open-source AI releases of the year.
By Hamza Faisal
