What happened
DeepSeek has made DeepInfer public on GitHub. The repository describes it as a “high-performance, large-scale inference framework for DeepSeek models,” and the README says it is “the high-performance inference framework that powers DeepSeek’s large-scale production services.” According to the GitHub API, the repository’s first commit is dated 20 October 2026 (UTC); the repository has two commits as of our reading on 21 October.
Key details
- License: Apache 2.0, per the README.
- Supported model: DeepSeek-V4.1-Flash; the default checkpoint is DeepSeek-V4.1-Flash-1017.
- Hardware: NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs. Prerequisites listed include Linux, Python 3.12, CUDA 13 (NVIDIA) and CANN 9.2 (Ascend).
- Features listed: optimised kernels, attention data parallelism, MoE expert parallelism with load balancing, asynchronous scheduling, batch-invariant inference, deterministic sampling and structured outputs.
- What this release leaves out: the README lists a KV-cache system, prefill/decode disaggregation and load balancing across data-parallel replicas as “required for a complete inference service (not included in this release).” It describes the release as offline inference.
- Lineage: the README says the code was “originally built primarily on vLLM,” with inspiration from SGLang and TensorRT-LLM.
- Benchmarks: the README shows charts measured on 4 SM100 GPUs for prefill and 64 for decode; the charts are images, and we did not extract numbers from them.
- Unconfirmed figures: some newsletters have circulated large daily-token and revenue numbers in connection with the release. We did not find them in the README or in DeepSeek’s news posts we read, so we do not report them.
Why it matters
Inference code is the part that determines how cheaply a model can be served, and DeepSeek is publishing a version of its own production stack rather than only model weights. Because this release omits the KV-cache system and request routing, it is a building block for offline or research use rather than a ready-made serving platform. Support for Huawei Ascend 950 alongside NVIDIA hardware is stated in the README; real-world performance on either was not independently measured here.
Disclosure: Claude, made by Anthropic, was one of the AI tools used to research and draft this article. Details come from DeepSeek’s GitHub repository, read on 21 October 2026; DeepSeek’s own announcement post on X could not be fetched. Repository dates come from the GitHub API.
Sources
- deepseek-ai/DeepInfer (GitHub repository and README) Primary source , DeepSeek
- DeepSeek API docs news: DeepSeek-V4.1-Flash-1017 Primary source , DeepSeek