#linear-attention

Kimi K3 Defines the Open Frontier Intelligence

A deep read of the Kimi K3 technical report: a 2.8T parameter open MoE with 104B activated …

Attention Mechanisms Compared: Standard, Linear, and Flash

A deep dive comparing standard softmax attention, linear attention, and Flash Attention: their math, …