Cerebras explains how to run 3T models on WSE-3T chips

The guide breaks down the memory calculations for 4-bit weight quantization and 8-bit KV caching. It notes that linear-attention layers do not require large KV caches, allowing two such layers to fit on one wafer, while dense layers require one wafer each to accommodate the cache. For Kimi K3, this configuration requires 59 wafers in total. The system uses RDMA over Ethernet to pass activations between wafers with roughly 2 microseconds of latency per hop.

Cerebras emphasizes that this pipeline design allows for high concurrency. Unlike traditional batch processing, requests stream through the pipeline continuously, with different stages handling different requests simultaneously. The company states that this approach enables multi-trillion parameter models to run at speeds of 1000-2000+ tokens per second. The post concludes by noting that the same methodology scales to even larger models by adding more wafers to the pipeline.