"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
Hyperloglog Lightning Talk
1. Lightning Talk: Counting with
Hyperloglog, Node.js & Redis
SIMON PRICKETT, REDIS UNIVERSITY.
TWITTER: @SIMON_PRICKETT
WEB: HTTPS://SIMONPRICKETT.DEV
2. Hyperloglog?
• An algorithm for the "count distinct" problem.
• What's the approximate size (cardinality) of a set?
• Optimizes space efficiency over absolute accuracy.
5. Counting Uniques with the Hyperloglog Algorithm
Benefits
• Similar interface to a Set
• Same performance (time complexity) as using a Set
• Much more space efficient than a Set (~ 12Kb)
• Can't retrieve items, unlike a Set
Trade-Offs
• Absolute accuracy
• Can't retrieve items, unlike a Set
• Not built into JS, need an implementation...
7. Redis Hyperloglogs
• Fast, open source
• Uses maximum 12kb
• Count up to 2^64 uniques
(18,446,744,073,709,552,000)
• Persists to Redis key/value store
• Support for most programming languages
• Count estimates have 0.81% standard error