Tag: long-context

All the articles with the tag "long-context".

• Hybrid Attention and MLA: The Tradeoff

  23 May, 2026

  A side-by-side dive into Xiaomi MiMo's hybrid sliding-window/global attention and DeepSeek's Multi-head Latent Attention. The two answer the same question — how to make attention affordable at long context — with very different bets, and those bets shape everything from training infra to KV cache size.

• Inside DeepSeek's Sparse Attention: From NSA to DSA

  18 May, 2026

  A deep dive into DeepSeek's two sparse attention designs — Native Sparse Attention (NSA) and DeepSeek Sparse Attention (DSA) — covering the math, the hardware story, and why DSA in V3.2 looks so different from NSA.