Presented at the Ruby Drink-up of Sophia Antipolis on the 8th of January 2013 by Karan Malkani.

1. Ruby Drink-up / January 2013
http://rivierarb.fr/2013/01/08/Drinkup/
Karan Malkani

2. ABOUT THESE SLIDES…
A Presentation of the Twitter / Ruby couple and why it did not last
Motivated by my own interest in Ruby, born 2 years ago when reading that
Twitter was made using Ruby (or Ruby on Rails)
An Internet-based desk research
But…
Not so much / easy available information due to the excellent Twitter
communication

3. TWITTER HISTORY WITH (A BIG) SHORTCUT
 Born in the postcasting company Odeo
 Initial idea: an SMS service within a small group / Original code name: twttr / First prototype
was used as an internal service for Odeo employees / Full version introduced publicly in July
2006
 October 2006: Jack Dorsey, Biz Stonen, and Evan Williams formed Obvious Corporation,
acquired Odeo (including twitter.com).
 Twitter, as a company, was launched in April 2007
 Today: the 2nd social network in the world after Facebook:
(Estimated Unique Monthly Visitors – January 2013)
• 1 Facebook - 750,000,000
• 2 Twitter - 250,000,000
• 3 LinkedIn - 110,000,000
• 4 Pinterest - 85,500,000
• 5 MySpace - 70,500,000
• 6 Google Plus+ - 65,000,000
Source:
How Twitter Was Born http://www.140characters.com/2009/01/30/how-twitter-was-born/
Twitter Development History in 3 minute and 53 seconds! http://www.shoutmeloud.com/twitter-development-history-in-3-
minute-and-53-seconds.html
Wikipedia: Twitter http://en.wikipedia.org/wiki/Twitter
eBizMBA (http://www.ebizmba.com/articles/social-networking-websites)

4. A FANTASTIC GROWTH – (> 100M NEW USERS IN 2010)
Source:
http://blog.twitter.com/2010/02/measuring-tweets.html
http://mashable.com/2010/12/16/twitter-stats-2010/

5. THE PROBLEM
Source:
http://www.whatisfailwhale.info/
http://en.wikipedia.org/wiki/Twitter#Outages
http://www.wired.com/underwire/2008/07/twitter-fans-tu/

6. WHY SCALING?
Number of users
Number of messages (impacted by the Retweet feature realeased end 2009)
Applications that can publish tweets / Applications that access tweets
Search Engine (Twitter acquired Summize – July 2008)
2009 : Some critical outages / Many FailWhales experiences
June 12, 2009 – In what was called a potential "Twitpocalypse", the unique numerical identifier associated with each
tweet exceeded the limit of 32-bit signedintegers (2,147,483,647 total messages).[142] While Twitter itself was not
affected, some third-party clients could no longer access recent tweets. Patcheswere quickly released, though
some iPhone applications had to wait for approval from the App Store.[143]
June 25, 2009 – Twitter ran slowly for some time after over 50,000 tweets on Michael Jackson’s death were recorded
in an hour.[144]
August 6, 2009 – Twitter and Facebook suffered from a denial-of-service attack, causing the Twitter website to go
offline for several hours.[145] It was later confirmed that the attacks were directed at one pro-Georgian user around
the anniversary of the 2008 South Ossetia War, rather than the sites themselves.[146]
September 22, 2009 – The identifier exceeded the limit for 32-bit unsigned integers (4,294,967,296 total messages)
again breaking some third-party clients.[147]
Source:
http://blog.twitter.com/2009/11/retweet-limited-rollout.html
http://techcrunch.com/2008/07/15/confirmed-twitter-acquires-summize-search-
engine/
http://en.wikipedia.org/wiki/Twitter#Outages
http://www.businessinsider.com/twitter-doesnt-have-product-geniuses-so-its-
buying-them-2010-4

7. STARTING TWITTER WITH RUBY
An excellent choice to
 Quickly prototype
 Quickly launch a first real site
 Quickly deploy an API for third-party applications
 Tease and attract developers
Improving performances with Ruby:
Example - mid 2009: Evan Weaver (Twitter)
We recently migrated Twitter from a custom Ruby 1.8.6 build to a
Ruby Enterprise Edition release candidate, courtesy of
Phusion. Our primary motivation was the integration of
Brent’s MBARI patches, which increase memory stability.
Some features of REE have no effect on our codebase, but we
definitely benefit from the MBARI patchset, the Railsbench
tunable GC, and the various leak fixes in 1.8.7p174. These
are difficult to integrate and Phusion has done a fine job.
Source:
http://blog.evanweaver.com/2009/09/24/ree/
http://www.linkedin.com/in/evanweaver/

8. RUBY ISSUES KNOWN WHEN TWITTER WAS FOUNDED
FULL COPY/PASTE OF: HOW HAS RUBY ON RAILS BEEN HOLDING UP TO THE INCREASED LOAD? (5 QUESTION INTERVIEW WITH TWITTER
DEVELOPER ALEX PAYNE – APRIL 2007)
By various metrics Twitter is the biggest Rails site on the net right All the convenience methods and syntactical sugar that makes
now. Rails such a pleasure for coders ends up being absolutely
punishing, performance-wise.
Running on Rails has forced us to deal with scaling issues - issues
that any growing site eventually contends with – far sooner than I Once you hit a certain threshold of traffic, either you need to
think we would on another framework. strip out all the costly neat stuff that Rails does for you (RJS,
ActiveRecord, ActiveSupport, etc.) or move the slow parts of
The common wisdom in the Rails community at this time is that your application out of Rails, or both.
scaling Rails is a matter of cost: just throw more CPUs at it.
It’s also worth mentioning that there shouldn’t be doubt in
The problem is that more instances of Rails (running as part of a anybody’s mind at this point that Ruby itself is slow.
Mongrel cluster, in our case) means more requests to your database.
It’s great that people are hard at work on faster implementations
At this point in time there’s no facility in Rails to talk to more than one of the language, but right now, it’s tough. If you’re looking to
database at a time. deploy a big web application and you’re language-agnostic,
The solutions to this are caching the hell out of everything and realize that the same operation in Ruby will take less time in
setting up multiple read-only slave databases, neither of which are Python.
quick fixes to implement. All of us working on Twitter are big Ruby fans, but I think it’s
So it’s not just cost, it’s time, and time is that much more precious worth being frank that this isn’t one of those relativistic language
when people can[‘t] reach your site.None of these scaling issues. Ruby is slow.
approaches are as fun and easy as developing for Rails.
Source:
http://yasulab.tumblr.com/post/10271634919/5-question-interview-with-
twitter-developer-alex-payne/
The original article on Radical Behavior is no longer accessible

9. ABANDONNING RUBY ON RAILS
April 2008 - After 2 years of high scaling problems: decision is made to abandon
RoR
Start to change the infrastructure: Middleware moved to a mixture of C and
Scala/JVM to run a messaging model, asynchronous process, 3 levels of
cache (Twitter kept Ruby for the front-end only)
April 2009 - Twitter on Scala
Spring 2010 – changing the back-end from MySQL to a real-time version of
Lucene (Apache database)
2011 – RoR front-end for search replaced with Blender, a Java server
Source:
http://techcrunch.com/2008/05/01/twitter-said-to-be-abandoning-ruby-on-rails/
http://www.artima.com/scalazine/articles/twitter_on_scala.html
http://engineering.twitter.com/2011/04/twitter-search-is-now-3x-faster_1656.html

10. THE “GEEK” SLIDE (I DON’T UNDERSTAND THIS STUFF)
> Ruby side
Mongrel
Kiji (generational garbage collector)
Starling (light-weight persistent queue server speaking MemCache
protocol)
> Java side
Modified Lucene (real-time version of the java database)
Optimized JSON fragment cache (to handle initial page load and
every call from the client)
Mustache (for the rendering – client and server side)
oEmbed (standard to handle the inline media)
Source:
http://engineering.twitter.com/2010/09/tech-behind-new-twittercom.htm
http://engineering.twitter.com/2010/10/twitters-new-search-architecture.html
http://engineering.twitter.com/2011/04/twitter-search-is-now-3x-faster_1656.html
http://www.infoq.com/news/2012/11/twitter-ruby-to-java

11. HOW TO TERMINATE THIS SLIDE SET?
Sorry - This was not so much about Ruby
Yellowpages.com did exactly the contrary in 2008, moving from Java on Rails to Ruby on Rails
The “New Twitter” is fantastic – and survived the US Election end 2012. Some impressive numbers:
327,452 tweets/minute, 31 M election-related tweets during the day, with a 15,107 tweets/second
peak)
Back to June 2009 (copy/paste from Scaling Twitter: Making Twitter 10000 Percent Faster)
Twitter started as a side project and blew up fast, going from 0 to millions of page views within
a few terrifying months. Early design decisions that worked well in the small melted under the
crush of new users chirping tweets to all their friends. Web darling Ruby on Rails was fingered
early for the scaling problems, but Blaine Cook, Twitter's lead architect, held Ruby blameless:
For us, it’s really about scaling horizontally - to that end, Rails and Ruby haven’t been
stumbling blocks, compared to any other language or framework. The performance boosts
associated with a “faster” language would give us a 10-20% improvement, but thanks to
architectural changes that Ruby and Rails happily accommodated, Twitter is 10000% faster
than it was in January.
If Ruby on Rails wasn't to blame, how did Twitter learn to scale ever higher and higher?
Source:
http://techcrunch.com/2008/05/01/twitter-said-to-be-abandoning-ruby-
on-rails/
http://highscalability.com/scaling-twitter-making-twitter-10000-percent-
faster
http://www.infoq.com/news/2012/11/twitter-ruby-to-java