Placement
NUMA machines are asymmetric by construction: memory sits in domains, and a core reaches its own domain's memory faster than anyone else's. On the 2-socket servers that dominate today, the ratio is 1.2–2×; on CXL-attached memory it grows to 3–5×, and tiered architectures push it to 5–10×. Nothing about the instruction stream changes, only where the bytes physically rest.
That makes placement the cheapest moment to be topology-aware: a decision made before the data exists costs nothing to revisit later. Every mainstream cache ignores it. Redis and Memcached never ask which socket they are on, and CacheLib answers it with static per-domain sharding decided at startup.
Furrballs treats the page as the atomic locality unit: one physical block is allocated per NUMA node at init and subdivided into pages, so every key's bytes share one domain affinity. Placement then becomes a routing question,which node gets this key, with three strategies: round-robin (uniform), thread-local (the requesting thread's own node), and hash-mod, which is uniform and deterministic but blind to topology. The whitepaper rejects hash-mod for exactly that reason (§3.3); the demo lets you watch each strategy spread keys and draw your own conclusion.
Watch for
- · Round-robin spreads uniformly but ignores where readers sit.
- · Thread-local makes reads local, at the cost of node skew under hot threads.
- · Hash-mod is deterministic and uniform, and rejected: topology-blind (§3.3).
Data gravity
Placement sets the bill up front; remote reads collect it. A Get() that crosses a domain pays the latency ratio on every access, and nothing in a traditional cache even counts that. Systems usually answer with replication or eager migration. Each is correct and immediate, and each is expensive: the data moves the moment the access appears.
Furrballs defers the move instead. Every remote Get() bumps a per-key atomic counter (CrossNodeAccesses, relaxed ordering, because it is a hint rather than a lock). Only when the counter crosses a threshold does the key become a migration candidate, and it moves at the eviction boundary, where pages are already being reshuffled and one more memcpy costs effectively nothing (§2.1.2). Data still gravitates toward its readers; the counter decides the pace.
The walk that finds remote keys is made cheap on purpose: the reading node tries home first, then a per-node probe order where bloom filters skip the nodes that never saw the key. False positives still cost one full remote probe, which is the price of a probabilistic skip.
Watch for
- · The walk: home node first, then probeOrder with bloom-filter skips.
- · CrossNodeAccesses is a hint, not a lock; relaxed ordering is deliberate.
- · Migration happens at the eviction boundary, where movement is cheapest (§2.1.2).
Eviction
Placement decides where keys arrive; eviction decides what the cache forgets when memory runs out. Furrballs' Phase 2a answer was ARC, the adaptive replacement policy that keeps recency (T1) and frequency (T2) in separate lists, with ghost lists remembering what was just evicted so the filter between the two can learn.
On the report's own Zipfian workload the adaptation pays measurably: PolicyBench puts ARC at 34.96% hits vs LRU's 24.77%, about ten points earned by the adaptive filter. The cost is mechanics: the reference lists are slow, and the flat-list rewrite recovers the throughput (§5.22 documents it; the same optimization was rediscovered here, same wall, same fix).
Inside Furrballs the policy doesn't own its keys: tracking is hash-keyed (no strings in the ARC lists, §3.13), promotions arrive through a deferred MPSC buffer (§3.14), and eviction runs through FindAndEraseByHash, the same shape as the Annex. And when a key dies, it can't simply be deleted: bump-allocated pages reclaim space by migrating cold keys and evicting whole pages.
Watch for
- · Ghost hits are the adaptation signal: watch p move as recency vs frequency demand shifts.
- · The victim rule is p-driven: T1 pays when it outgrew its target, T2 otherwise.
- · Same access stream through every policy; the hit-rate bars are the comparison.
synthetic stream above (24 ops, capacity 5); real PolicyBench run below
| PolicyBench · Zipf θ=0.99 | hit rate | throughput |
|---|---|---|
| LRU | 24.77% | 9.7M/s |
| ARC | 34.96% | 4.9M/s |
| ARC-FLATLL2 | 34.96% | 9.5M/s |
§3.28 draws the line this chapter walks up to: recency and locality are orthogonal axes, and ARC's single-axis model cannot express "this key is hot but on the wrong node." The REMARC framework attempted the reduction, collapsing recency, frequency, and locality into one score.
The attempt is kept in the report as a documented negative result: most REMARC variants land at LRU-level hit rates with as many or more evictions, because page-management fought the policy for the same keys. The salvage, running the scorer as the cache itself, matched ARC's hit rate exactly with better tails, and that variant is what the whitepaper ships.
The multi-node policy comparison itself is deferred (§5.20): the data was collected but held back, because page-level eviction smothers policy differentiation. At page granularity the differences between per-key policies smooth out, and measuring what each mechanism contributes requires a benchmark redesign. The limitation is reported as a result rather than hidden.