Numa & Furrballs
A topology-aware cache, chapter by chapter: placement, data gravity, and the Annex.
The subject
Modern servers expose asymmetric memory: access latency depends on which domain holds the bytes. NUMA is today's widespread case (1.2–2× across two sockets), with CXL-attached memory at 3–5× and tiered architectures at 5–10×. The instruction stream never changes; only where the data rests.
Mainstream caches ignore that: Redis and Memcached never ask which socket they are on, and CacheLib answers it with static per-domain sharding. Furrballs makes topology an adaptive input to per-page placement, and this notebook walks its answers chapter by chapter, each with an interactive trace.
Chapters
Introduction
What asymmetric memory is, why mainstream caches ignore it, and how a page-granular cache turns topology into an input.
Placement
Where keys land on a 4-node machine: round-robin, thread-local, and the hash-mod strategy the whitepaper rejected.
Data gravity
Remote reads remember: the per-key cross-node counter, threshold-triggered migration, and why it waits for the eviction boundary.
Eviction
Making room in a topology-aware cache: ARC's adaptive replacement vs LRU, page-granular migration-based eviction, and the deferred locality policy.
The Annex
The cross-node announce ring: batched outbound batches, a single-consumer drain, directed remote reads, plus a design note on the Annex as an RDMA indirection layer.
Companion piece: the Swiss Table Probe Visualizer, what happens inside one of these nodes. Same FNV-1a hash throughout.