Scaling real-time visualisations for Elections 2014
•Télécharger en tant que PPTX, PDF•
2 j'aime•1,690 vues
How Gramener scaled up the Indian Elections 2014 live website
Recommandé
Recommandé
Contenu connexe
En vedette
En vedette (15)
Similaire à Scaling real-time visualisations for Elections 2014
Similaire à Scaling real-time visualisations for Elections 2014 (20)
Plus de Gramener
Plus de Gramener (20)
Dernier
Dernier (20)
Scaling real-time visualisations for Elections 2014
- 1. S Anand, Chief Data Scientist, Gramener Scaling Real-time Visualisations for Elections 2014 @sanand0
- 4. What’s the largest number of people that stood in an election?
- 6. “We’ll cross 5 million visitors tomorrow”
- 9. Nielsen’s server ETL Candidate Votes Visualisation template 1 2 3 4 Azure Ubuntu server Singapore Gramener Visualisation server Real time nginx 1 2 3 4 SQL Server CNN Windows server Noida, India ETL rsync Candidate Votes CNN WinXP laptop Noida, India Every 10 seconds Every 10s Let’s optimize backwards
- 13. Compress content
- 15. wiki.nginx.org
- 16. h5bp.github.io
- 17. Only 1 image … but a 3MB SVG
- 18. Kraken.io
- 20. Inkscape
- 21. 2 decimal places 3 decimal places 4 decimal places ●●●●●●●●● ●●●●●●●●●● ●●●●● ●●●●●●●●●● ●●●●●●●●●● ●●●●●●●●●● ●●●●●●●●●● ●●●●●●●●●● ●●●●●●●●●● ●●● 95KB 145KB 613KB SVG Compression
- 22. Nielsen’s server ETL Candidate Votes Visualisation template 1 2 3 4 Azure Ubuntu server Singapore Gramener Visualisation server Real time nginx 1 2 3 4 SQL Server CNN Windows server Noida, India ETL rsync Candidate Votes CNN WinXP laptop Noida, India Every 10 seconds Every 10s Now, optimize the rendering
- 23. We need these filters to work instantly We cannot afford a server request for every filter change We need client-side content generation, driven by data
- 24. HTML XML Prolog Javascript Python Java How content is written Declarative Procedural
- 25. How data is used to write it Map attributes to functions Templates Binding Create HTML strings
- 26. Declarative Procedural Templates Binding Underscore knockout jQuery d3 Let’s make a bar chart with each of these Examples of representative libraries https://github.com/sanand0/fifthel-2
- 27. underscore: declare a template
- 28. jQuery: procedurally create the HTML
- 29. knockout: declaratively bind data to HTML
- 30. d3: procedurally bind data to elements and attribu
- 31. Nielsen’s server ETL Candidate Votes Visualisation template 1 2 3 4 Azure Ubuntu server Singapore Gramener Visualisation server Real time nginx 1 2 3 4 SQL Server CNN Windows server Noida, India ETL rsync Candidate Votes CNN WinXP laptop Noida, India Every 10 seconds Every 10s Finally, optimize data
- 32. 1.5 MB of data every second but some of it is static some is redundant and some misspelt or wrong
- 33. Correct mis-spellings Load just what you need (query time reduced by 70%)
- 35. Refresh only the changed data When gzipped, JSON is no larger than CSV JSON is natively parsed and more flexibleJSON? Redundancy 27KB
- 36. “We’ll cross 5 million visitors tomorrow”
- 37. 0 200 400 600 800 1000 1200 1400 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Thousands Half a million just in the first hour
- 39. 0 200 400 600 800 1000 1200 1400 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Thousands Over 1.3 million in the next!
- 40. 0 200 400 600 800 1000 1200 1400 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Thousands 10 million visits election day
- 42. Does age make a difference? Do old candidates win less often?
- 43. 1% 2% 4% 6% 9% 11% 14% 11% 16% 18% 22% 22% 33% 0% 10% 20% 30% 40% 25-30 30-35 35-40 40-45 45-50 50-55 55-60 60-65 65-70 70-75 75-80 80-85 85-90 0 500 1000 1500 2000 2500 Win % The number of winning candidates as a % of candidates in the age group Candidates The number of candidates in each age group LokSabha(2004onwards)
- 46. Name length