Introduction
Modern server memory is asymmetric. It sits in NUMA sockets today, with CXL-attached memory and tiered DRAM/HBM stacks next, and a core's access latency depends on which domain holds the bytes: 1.2–2× across two sockets, 3–5× for CXL-attached memory, 5–10× on tiered architectures. The compiled instructions are identical; only the memory controller knows the difference.
Most caching software ignores this. Redis and Memcached never ask which socket they are on; CacheLib answers the question once, at startup, with static per-domain sharding. Neither treats topology as something a running system can adapt to. Furrballs takes the opposite position: allocate cache pages on the memory domain closest to the threads that access them, and let placement, lookup, and eviction all take the topology as input.
The machinery sits in three layers, and this notebook walks them in order: placement (which node receives a key), data gravity (what happens when the reader is on the wrong node), and the Annex, a batched cross-node announce ring that turns remote probing into directed reads. Under all of it sits the same concurrent Swiss table covered in the probe visualizer: routing decides which node, the Swiss table decides which slot.
One abstraction carries the whole design: the page. Furrballs allocates one physical block per NUMA node at init and subdivides it into pages; every key's bytes share the page's domain affinity, and eviction moves whole pages, so locality, allocation, and reclamation all speak the same granularity.
Watch for
- · The latency ladder: same-socket 1×, cross-socket 1.2–2×, CXL 3–5×, tiers 5–10×.
- · The page is the locality unit: allocation, placement, and eviction all speak page.
- · The three chapters build on one another: placement → gravity → the Annex.
Placement
NUMA machines are asymmetric by construction: memory sits in domains, and a core reaches its own domain's memory faster than anyone else's. On the 2-socket servers that dominate today, the ratio is 1.2–2×; on CXL-attached memory it grows to 3–5×, and tiered architectures push it to 5–10×. Nothing about the instruction stream changes, only where the bytes physically rest.
That makes placement the cheapest moment to be topology-aware: a decision made before the data exists costs nothing to revisit later. Every mainstream cache ignores it. Redis and Memcached never ask which socket they are on, and CacheLib answers it with static per-domain sharding decided at startup.
Furrballs treats the page as the atomic locality unit: one physical block is allocated per NUMA node at init and subdivided into pages, so every key's bytes share one domain affinity. Placement then becomes a routing question,which node gets this key, with three strategies: round-robin (uniform), thread-local (the requesting thread's own node), and hash-mod, which is uniform and deterministic but blind to topology. The whitepaper rejects hash-mod for exactly that reason (§3.3); the demo lets you watch each strategy spread keys and draw your own conclusion.
Watch for
- · Round-robin spreads uniformly but ignores where readers sit.
- · Thread-local makes reads local, at the cost of node skew under hot threads.
- · Hash-mod is deterministic and uniform, and rejected: topology-blind (§3.3).
Data gravity
Placement sets the bill up front; remote reads collect it. A Get() that crosses a domain pays the latency ratio on every access, and nothing in a traditional cache even counts that. Systems usually answer with replication or eager migration. Each is correct and immediate, and each is expensive: the data moves the moment the access appears.
Furrballs defers the move instead. Every remote Get() bumps a per-key atomic counter (CrossNodeAccesses, relaxed ordering, because it is a hint rather than a lock). Only when the counter crosses a threshold does the key become a migration candidate, and it moves at the eviction boundary, where pages are already being reshuffled and one more memcpy costs effectively nothing (§2.1.2). Data still gravitates toward its readers; the counter decides the pace.
The walk that finds remote keys is made cheap on purpose: the reading node tries home first, then a per-node probe order where bloom filters skip the nodes that never saw the key. False positives still cost one full remote probe, which is the price of a probabilistic skip.
Watch for
- · The walk: home node first, then probeOrder with bloom-filter skips.
- · CrossNodeAccesses is a hint, not a lock; relaxed ordering is deliberate.
- · Migration happens at the eviction boundary, where movement is cheapest (§2.1.2).