Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
Just Build    It! Scott Rutherford, DIBI 2012
A Quick Question
CERN
• Axomic (OpenAsset)• Caron Software• Caron Records / Publishing• Yabb / Cominded• ThatsUseful (BuddyMap, Boovu, et al.)• ...
Market leading customer feedback &support solution21 strong in San Francisco & Raleigh90,000+ organizations have usedus$2M...
Failing Up
Happen’s To The     Best• Facebook Beacon (Feature)• Google Wave (Product)• Twitter - Scaling (Technology)• Coke - New Cok...
“were not that dumb, and were not thatsmart!” - Don Keough
Shit Happens!
You Can’t Plan For   Everything!Things are going to go wrong. It’s the way we  react at these moments that defines to a   g...
Just Build    It!
ActiveScaffold      (AKA AjaxScaffold Plugin) One line of Code
ActiveScaffold      (AKA AjaxScaffold Plugin) One line of Code
ActiveScaffold      (AKA AjaxScaffold Plugin) One line of Code
ActiveScaffold      (AKA AjaxScaffold Plugin) One line of Code >100K Installs > 750K Visitors
Version 1.0• Rails 1.2.3 / HopToad / NewRelic /  CloudFront• Engine Yard • Shared MySQL database • 2 slices (and staging)•...
Building Fast• Created our own problems• Insufficient test coverage and no CI
Building Fast• Created our own problems• Insufficient test coverage and no CI • deploy production:rollback• Watch those AJA...
Building Fast• Created our own problems• Insufficient test coverage and no CI • deploy production:rollback• Watch those AJA...
Building Fast• Created our own problems• Insufficient test coverage and no CI • deploy production:rollback• Watch those AJA...
Stunning Timing
Stunning Timing           April 2009          Raised $800k
Version 2.0
Features, Features, Features   • Reporting / Administration Tools   • Multiple Languages (now crowd sourced)   • Search th...
Growing....• 15 - 20% month• 6 web boxes• 2 databases (master / slave)• Staging• Admin• Office!
Problems• Shared database started to creak• Tables growing in size - queries not scaling• Overused MySQL - queue, stats• N...
“Zee Germans”
Into The Cloud....• Moved servers to SoftLayer• Passenger and Ruby Enterprise• Signed up 4 million new users in 2 hours• 6...
Still Shaky• Redis / Resque for the queue• Redis for stats• More memcached• Make as many things background jobs as  possib...
Meltdown• 10 days of almost stability• Flew to SXSW• MySQL melted (still on 5.0)• Percona rescued us from the dreaded  MYS...
DNS can fail?• Using ZoneEdit (free)• All nameservers went down, all hell broke  loose• Contacted by DynECT over Twitter!•...
Retention / Metrics                91% :(               10 months
What now?• Ongoing monitoring - daily slice stored in  Mongo• Surveys at exit• Addition of HelpDesk & Knowledge Base• Inte...
“Zee Germans”
And back to Earth   with a bump• Softlayer had a few major outages on their  cloud infrastructure• Slices are still shared...
So where are we now? •   20 systems in production (2 cloud instances) •   Web - 6x dual cpu/quad core 12G RAM nginx/passen...
What have we  learned?
What have we  learned?   if anything.....
Performance          Realization        > 200ms stuffed
Gems
More Gems
Mouth / UGC Styles
The right tool!
Databases• Redis - Queue, Stats / Admin• Mongo - Stats / Mouth, PageViews,  Financials, Retention• MySQL - Main data store...
Libraries• jQuery• Backbone• D3• SVG• MooTools Server• SASS• Compass
Software• DSpam• Chef• Ganglia• Nagios• Capistrano• Passenger• Nginx / HAProxy
External Services• Mailgun      • Akamai• Embedly      • Akismet• Pusher App   • Dynect• Airbrake     • SoftLayer• S3     ...
Wish We’d Had
K.I.S.S      200 moving parts        to change a        lightbulb.....
Localize
Unsinkable?
Stealth mode is for..
Listen, no really...        "If I had asked people        what they wanted,        they would have said        faster hors...
Our Creations• Resque Sliders - https://github.com/  kmullin/resque-sliders• Serve - http://get-serve.com/• Mouth - https:...
Just Build    It!The best time to start is NOW!!
Scott Rutherford  @scottrutherfordscott@uservoice.com
Dibi Conference 2012
Dibi Conference 2012
Dibi Conference 2012
Dibi Conference 2012
Dibi Conference 2012
Dibi Conference 2012
Dibi Conference 2012
Dibi Conference 2012
Dibi Conference 2012
Dibi Conference 2012
Prochain SlideShare
Chargement dans…5
×

Dibi Conference 2012

My talk from the DIBI conference in Newcastle in April 2012

  • Soyez le premier à commenter

Dibi Conference 2012

  1. Just Build It! Scott Rutherford, DIBI 2012
  2. A Quick Question
  3. CERN
  4. • Axomic (OpenAsset)• Caron Software• Caron Records / Publishing• Yabb / Cominded• ThatsUseful (BuddyMap, Boovu, et al.)• RecommendBox• Njoyed• SlimTimer
  5. Market leading customer feedback &support solution21 strong in San Francisco & Raleigh90,000+ organizations have usedus$2M+ raised over 2 fundingroundsTens of millions ofregistered users182 people/sec see ourproduct
  6. Failing Up
  7. Happen’s To The Best• Facebook Beacon (Feature)• Google Wave (Product)• Twitter - Scaling (Technology)• Coke - New Coke! (Marketing)
  8. “were not that dumb, and were not thatsmart!” - Don Keough
  9. Shit Happens!
  10. You Can’t Plan For Everything!Things are going to go wrong. It’s the way we react at these moments that defines to a great degree how successful we will be. Don’t spend time planning for everyeventuality. Just be prepared to move quickly and decisively when situations arise. Don’t build failure in though!!!
  11. Just Build It!
  12. ActiveScaffold (AKA AjaxScaffold Plugin) One line of Code
  13. ActiveScaffold (AKA AjaxScaffold Plugin) One line of Code
  14. ActiveScaffold (AKA AjaxScaffold Plugin) One line of Code
  15. ActiveScaffold (AKA AjaxScaffold Plugin) One line of Code >100K Installs > 750K Visitors
  16. Version 1.0• Rails 1.2.3 / HopToad / NewRelic / CloudFront• Engine Yard • Shared MySQL database • 2 slices (and staging)• Admin section based on ActiveScaffold• No billing ability until the end of 2008 (then using Freemium OSS)
  17. Building Fast• Created our own problems• Insufficient test coverage and no CI
  18. Building Fast• Created our own problems• Insufficient test coverage and no CI • deploy production:rollback• Watch those AJAX calls
  19. Building Fast• Created our own problems• Insufficient test coverage and no CI • deploy production:rollback• Watch those AJAX calls• Be careful when testing.....
  20. Building Fast• Created our own problems• Insufficient test coverage and no CI • deploy production:rollback• Watch those AJAX calls• Be careful when testing.....
  21. Stunning Timing
  22. Stunning Timing April 2009 Raised $800k
  23. Version 2.0
  24. Features, Features, Features • Reporting / Administration Tools • Multiple Languages (now crowd sourced) • Search that worked (Sphinx) • Single Sign On • API • Billing (constant PITA)
  25. Growing....• 15 - 20% month• 6 web boxes• 2 databases (master / slave)• Staging• Admin• Office!
  26. Problems• Shared database started to creak• Tables growing in size - queries not scaling• Overused MySQL - queue, stats• Not enough caching• Dark corners - data exports, email notifications• Still insufficient test coverage and no CI• Us...... (production deploy:rollback)
  27. “Zee Germans”
  28. Into The Cloud....• Moved servers to SoftLayer• Passenger and Ruby Enterprise• Signed up 4 million new users in 2 hours• 6 slices expanded to 24• Stability resumed.......
  29. Still Shaky• Redis / Resque for the queue• Redis for stats• More memcached• Make as many things background jobs as possible• Things are under control• Or so we thought.....
  30. Meltdown• 10 days of almost stability• Flew to SXSW• MySQL melted (still on 5.0)• Percona rescued us from the dreaded MYSQL AUTO INCREMENT BUG (Moved us to 5.1)• Finally have a scalable redundant system, right?
  31. DNS can fail?• Using ZoneEdit (free)• All nameservers went down, all hell broke loose• Contacted by DynECT over Twitter!• Gave us a free account to get going again. Moved to them• Stopped thinking about DNS!
  32. Retention / Metrics 91% :( 10 months
  33. What now?• Ongoing monitoring - daily slice stored in Mongo• Surveys at exit• Addition of HelpDesk & Knowledge Base• Integrations• API, Facebook, iOS
  34. “Zee Germans”
  35. And back to Earth with a bump• Softlayer had a few major outages on their cloud infrastructure• Slices are still shared resources• You can’t get someone to just reboot the box• Lesson: cloud is great to scale with, not so good for stability
  36. So where are we now? • 20 systems in production (2 cloud instances) • Web - 6x dual cpu/quad core 12G RAM nginx/passenger, memcache • Resque - 3x single cpu/quad core 8gb RAM • Redis - Dual CPU/quad core 12g RAM • Mongo - 2x dual cpu/hex core 48G RAM 12x 15k RPM SAS drives RAID10 • MySQL - 2x dual cpu/hex core 72G RAM 24x 15k RPM SAS drives RAID10 • 97% month / month retention
  37. What have we learned?
  38. What have we learned? if anything.....
  39. Performance Realization > 200ms stuffed
  40. Gems
  41. More Gems
  42. Mouth / UGC Styles
  43. The right tool!
  44. Databases• Redis - Queue, Stats / Admin• Mongo - Stats / Mouth, PageViews, Financials, Retention• MySQL - Main data store• Sphinx - Search• Memcached - Caching
  45. Libraries• jQuery• Backbone• D3• SVG• MooTools Server• SASS• Compass
  46. Software• DSpam• Chef• Ganglia• Nagios• Capistrano• Passenger• Nginx / HAProxy
  47. External Services• Mailgun • Akamai• Embedly • Akismet• Pusher App • Dynect• Airbrake • SoftLayer• S3 • GitHub
  48. Wish We’d Had
  49. K.I.S.S 200 moving parts to change a lightbulb.....
  50. Localize
  51. Unsinkable?
  52. Stealth mode is for..
  53. Listen, no really... "If I had asked people what they wanted, they would have said faster horses." Attributed to Henry Ford
  54. Our Creations• Resque Sliders - https://github.com/ kmullin/resque-sliders• Serve - http://get-serve.com/• Mouth - https://github.com/cypriss/ mouth• UGC Styles - https://github.com/ joshuarudd/ugc-styles• ActiveScaffold - https://github.com/ activescaffold/active_scaffold
  55. Just Build It!The best time to start is NOW!!
  56. Scott Rutherford @scottrutherfordscott@uservoice.com

×