Cerebras has laid out exactly how its wafer-scale chips can run one of the largest open AI models in existence, using Moonshot AI's Kimi K3 as the worked example. The answer, according to the chipmaker's technical breakdown, is 59 wafers working as a single pipeline, with each wafer holding whole layers of the roughly 3-trillion-parameter model in its on-chip memory.
The post tackles a growing problem: a single chip can no longer hold the weights of the biggest models, which need terabytes of memory for weights alone. Cerebras's approach is to spread the model across many WSE-3T wafers and keep the weights parked on the chips, so only intermediate results move between stages.
Why Kimi K3 needs 59 wafers
The arithmetic starts with the size of each layer. Kimi K3 has 93 layers, and at 4-bit weight quantization, each layer works out to about 16 gigabytes. A single WSE-3T wafer carries 44 gigabytes of on-chip SRAM, so the question becomes how many layers fit on each wafer.
That depends on the layer type. Kimi K3 mixes 69 linear-attention layers with 24 dense, full-attention layers. Dense layers need extra room for a key-value cache, which stores the keys and values for every token in every active request, so each dense layer takes a full wafer on its own. Linear-attention layers keep only a small, fixed-size state per request, so two of them fit on a single wafer. The total is therefore 35 wafers for the linear layers plus 24 for the dense layers: 59 wafers in all.
Cerebras says the arrangement can deliver 1,000 to 2,000-plus tokens per second for multi-trillion-parameter models. Cerebras also cites Kimi K3 benchmark scores of 88.3 percent on Terminal-Bench 2.1 and 91.2 percent on BrowseComp.
A pipeline, not a cluster
The wafers are connected as a pipeline. After the Cerebras compiler maps the model's layers across the wafers, tokens flow from stage to stage, with activations passed between wafers over RDMA on Ethernet in about 2 microseconds per hop. Because entire layers fit on chip, the weights never have to move; only activations cross from one wafer to the next.
Each wafer acts as one stage of an assembly line. While one stage processes a token for one request, the next stage works on a different request, and the stage before that on yet another. Stages can also handle micro-batches, small groups of requests at once, so the number of requests in flight is roughly the number of stages multiplied by the micro-batch size. The ceiling on concurrent requests is set by the key-value cache: every request in flight needs its own cache on each dense-layer wafer, and that cache must fit in the roughly 28 gigabytes left beside the weights.
What it means for giant models
The broader point of the post is that the recipe scales. To serve a model larger than 3 trillion parameters, Cerebras says, the same steps apply: calculate the memory needed for weights and request state, distribute the model across wafers, and connect them into a pipeline. Each additional wafer adds compute and on-chip memory, so models that exceed the capacity of any single chip can still be served without repeatedly shuttling weights in and out of memory.
There are caveats. The post is a deployment design rather than a measured benchmark of the full 59-wafer layout, and independent verification of the advertised speeds for that configuration has not been published. Still, as open models keep growing, it offers one of the clearest public looks yet at how wafer-scale hardware intends to keep up.
