Day 13 results: the hot shard, and the cost of adding a server

Measured on: (machine, OS, Python version)

Output

Paste part 1, part 2, part 3 and the scoreboard here.

One sentence per prediction

  • P1 (celebrity key’s share of all traffic):
  • P2 (range partitioning, hottest shard’s share):
  • P3 (modulo resharding 8 to 9, percent of keys remapped):
  • P4 (consistent hashing 8 to 9, percent of keys remapped):

The hot shard, in my own words

Hash sharding spread the distinct keys evenly, yet the hottest shard was still well above the ideal. Why, and what happened when I pushed the shard count up to 256:

The number that matters

To add one shard, plain modulo moved __ percent of the keys and consistent hashing moved __ percent. In one line, why that is the difference between a weekend migration and a shrug:

The trap I want to remember

Consistent hashing fixed the resharding churn but did not fix the ___ . The fix for that is a different thing entirely:

Can’t explain yet