SlideShare une entreprise Scribd logo
1  sur  46
UC Berkeley - CS10
Giv ing a talk at Ca l’s CS10 - The
Beauty and Joy     of Computing.
inst. eecs.berkeley.e   du/~cs10/fa10
20 Oct via Twitter for iPhone



  from UC Berkeley
  155 Donner Lab
  Berkeley, CA
   View Tweets at this place
To start... What’s a Tweet?
Follow me!




 twitter.com/raffi
 or send a SMS to 40404
 follow raffi
“Rudimentary communication
among individuals in real time
allows many to move together as
one” - @biz
One Word
April 10, 2008
  4:33:23 PM
One Word
                 April 10, 2008
                   4:33:23 PM




 And it’s over
April 11, 2008
1:27:26 PM
Haiti
January 2010




                Chile
               February 2010
MTV VMAs
              September 2010




 Emmys
August 2010
The humble beginnings
How does Twitter work?
Twitter is just A “struct”, a graph,
& a LIST
A Tweet is a simple thing




                            140 characters
What’s The data structure of a
tweet?

  {
   “id ” : 787167620,
   “text” : “Free”,
   “ date” : “2008-04 -11 13:27:26”
   “user”   : “jamesbuck”
  }*
                                      *not quite, but something like this
Where do they go?

                    Following
                    Followed by
Tweets multiply!
It’s a huge graph
Viewing Tweets
Home Timeline
                 [27886578700,
                  27856404785,
                  27727644786,
                  27679412331,
                 27594550593,
                 27583976236,
                 27580246968,
                 27576503025,
                 27545926564,
                 27545450496,
                27539864336...
Is that it? ... YES! ... sorta ...
things get more complicated at scale
And scaling real-time is hard
How many Tweets are there?
How many Tweets are there?

                             90M!
90M tweets
   per day   ≈   1000 tweets
                 per second
Lots of deliveries
Tweets multiply




                    5,162 people
                  need to see my
                           tweet
@ladygaga
mother mons†er
6.8 million followers

@justinbieber
Justin Bieber
5.7 million followers

@BarakObama
44th President of the United States
5.6 million followers

@raffi
me!
5.1 thousand followers
@ladygaga
   mother mons†er
   6.8 x 106 followers



   @raffi
   me!
   5.1 x 103 followers




differs by               10 3
@ladygaga
 mother mons†er
 6.8 x 106 followers




scaling                10 6
6000
       106

 deliveries
per minute
              =   deliveries
                  per
                  millisecond
A System is only as strong as its
weakest link
80 MB
    per sec
≈   10 ms
    per seek
10 ms
per seek   =   100 seeks
               per second
MySQL
         Can’t generate IDs fast enough
Centralized and a single point of failure




                                     snowflake
                                    Highly available and uncoordinated (10kqps)
                                    Compatible with the           ecosystem
                                    http://github.com/twitter/snowflake
How big are they?


1 tweet text        =   140
                        characters
                    ≈   200 bytes
1000 tweets
 per second       ≈       195 KB/sec
                  ≈ 11 MB/min
                  ≈ 16 GB/day
       Just tweet text!
Lots of deliveries
1

                   Digraph                           2
            Need to represent this

                                                         4
    1   2      3     4                    3
1




                             Matrix
2




                             Naïve implementation is not scalable
3
4
Photo used under Creative Commons from jurvetson




          Distributed graph database

flockdb   High rate of CRUD operations
          Complex set arithmetic queries
          http://github.com/twitter/flockdb
160M registered users




       2006         2008   2010
Using the system


6B API calls
    per day        ≈   70,000 calls
                       per second
Where do we want to be?

           Today - 160M people generate ~1000 TPS

Tomorrow - we want to support half the world and all its devices

        (right now, there are 6B people and 5B phones)
Follow me at
Questions?   twitter.com/raffi

Contenu connexe

En vedette

Securing Your Ecosystem (FOWA Las Vegas 2011)
Securing Your Ecosystem (FOWA Las Vegas 2011)Securing Your Ecosystem (FOWA Las Vegas 2011)
Securing Your Ecosystem (FOWA Las Vegas 2011)Raffi Krikorian
 
Twitter and the Real-Time Web
Twitter and the Real-Time WebTwitter and the Real-Time Web
Twitter and the Real-Time WebRaffi Krikorian
 
Developing for @twitterapi (Techcrunch Disrupt Hackathon)
Developing for @twitterapi (Techcrunch Disrupt Hackathon)Developing for @twitterapi (Techcrunch Disrupt Hackathon)
Developing for @twitterapi (Techcrunch Disrupt Hackathon)Raffi Krikorian
 
Real-time systems at Twitter (Velocity 2012)
Real-time systems at Twitter (Velocity 2012)Real-time systems at Twitter (Velocity 2012)
Real-time systems at Twitter (Velocity 2012)Raffi Krikorian
 

En vedette (6)

Securing Your Ecosystem (FOWA Las Vegas 2011)
Securing Your Ecosystem (FOWA Las Vegas 2011)Securing Your Ecosystem (FOWA Las Vegas 2011)
Securing Your Ecosystem (FOWA Las Vegas 2011)
 
#rtgeo (Where 2.0 2011)
#rtgeo (Where 2.0 2011)#rtgeo (Where 2.0 2011)
#rtgeo (Where 2.0 2011)
 
Users and Geo
Users and GeoUsers and Geo
Users and Geo
 
Twitter and the Real-Time Web
Twitter and the Real-Time WebTwitter and the Real-Time Web
Twitter and the Real-Time Web
 
Developing for @twitterapi (Techcrunch Disrupt Hackathon)
Developing for @twitterapi (Techcrunch Disrupt Hackathon)Developing for @twitterapi (Techcrunch Disrupt Hackathon)
Developing for @twitterapi (Techcrunch Disrupt Hackathon)
 
Real-time systems at Twitter (Velocity 2012)
Real-time systems at Twitter (Velocity 2012)Real-time systems at Twitter (Velocity 2012)
Real-time systems at Twitter (Velocity 2012)
 

Similaire à Twitter - Guest Lecture UC Berkeley CS10 Fall 2010

Joyent circa 2006 (Scale with Rails)
Joyent circa 2006 (Scale with Rails)Joyent circa 2006 (Scale with Rails)
Joyent circa 2006 (Scale with Rails)bcantrill
 
From incubator to exit: A brief history of Reddit, the first YCombinator success
From incubator to exit: A brief history of Reddit, the first YCombinator successFrom incubator to exit: A brief history of Reddit, the first YCombinator success
From incubator to exit: A brief history of Reddit, the first YCombinator successStartupfest
 
Evolution of The Twitter Stack
Evolution of The Twitter StackEvolution of The Twitter Stack
Evolution of The Twitter StackChris Aniszczyk
 
Storm session
Storm sessionStorm session
Storm sessionABC Talks
 
LeWeb 2011 - Analysis
LeWeb 2011 - AnalysisLeWeb 2011 - Analysis
LeWeb 2011 - AnalysisVisibli
 

Similaire à Twitter - Guest Lecture UC Berkeley CS10 Fall 2010 (9)

500Startups @ Twitter
500Startups @ Twitter500Startups @ Twitter
500Startups @ Twitter
 
Twitter Rules
Twitter RulesTwitter Rules
Twitter Rules
 
Joyent circa 2006 (Scale with Rails)
Joyent circa 2006 (Scale with Rails)Joyent circa 2006 (Scale with Rails)
Joyent circa 2006 (Scale with Rails)
 
08 multimedia delivering
08 multimedia delivering08 multimedia delivering
08 multimedia delivering
 
From incubator to exit: A brief history of Reddit, the first YCombinator success
From incubator to exit: A brief history of Reddit, the first YCombinator successFrom incubator to exit: A brief history of Reddit, the first YCombinator success
From incubator to exit: A brief history of Reddit, the first YCombinator success
 
Evolution of The Twitter Stack
Evolution of The Twitter StackEvolution of The Twitter Stack
Evolution of The Twitter Stack
 
Storm session
Storm sessionStorm session
Storm session
 
Twitter Awesomeness
Twitter AwesomenessTwitter Awesomeness
Twitter Awesomeness
 
LeWeb 2011 - Analysis
LeWeb 2011 - AnalysisLeWeb 2011 - Analysis
LeWeb 2011 - Analysis
 

Plus de Raffi Krikorian

How to use Geolocation in your webapp @ FOWA Dublin 2010
How to use Geolocation in your webapp @ FOWA Dublin 2010How to use Geolocation in your webapp @ FOWA Dublin 2010
How to use Geolocation in your webapp @ FOWA Dublin 2010Raffi Krikorian
 
Intro to developing for @twitterapi
Intro to developing for @twitterapiIntro to developing for @twitterapi
Intro to developing for @twitterapiRaffi Krikorian
 
"What's Happening" to "What's Happening Here" @ Chirp
"What's Happening" to "What's Happening Here" @ Chirp"What's Happening" to "What's Happening Here" @ Chirp
"What's Happening" to "What's Happening Here" @ ChirpRaffi Krikorian
 
Handling Real-time Geostreams
Handling Real-time GeostreamsHandling Real-time Geostreams
Handling Real-time GeostreamsRaffi Krikorian
 
Adding the "Where" to the "When"
Adding the "Where" to the "When"Adding the "Where" to the "When"
Adding the "Where" to the "When"Raffi Krikorian
 
Scala + WattzOn, sitting in a tree....
Scala + WattzOn, sitting in a tree....Scala + WattzOn, sitting in a tree....
Scala + WattzOn, sitting in a tree....Raffi Krikorian
 
WattzOn Whole Earth Simulator
WattzOn Whole Earth SimulatorWattzOn Whole Earth Simulator
WattzOn Whole Earth SimulatorRaffi Krikorian
 
Broken Hearts: How Valentine's Day causes global warming
Broken Hearts: How Valentine's Day causes global warmingBroken Hearts: How Valentine's Day causes global warming
Broken Hearts: How Valentine's Day causes global warmingRaffi Krikorian
 
WattzOn presentation @ Web 2.0 Summit
WattzOn presentation @ Web 2.0 SummitWattzOn presentation @ Web 2.0 Summit
WattzOn presentation @ Web 2.0 SummitRaffi Krikorian
 

Plus de Raffi Krikorian (14)

How to use Geolocation in your webapp @ FOWA Dublin 2010
How to use Geolocation in your webapp @ FOWA Dublin 2010How to use Geolocation in your webapp @ FOWA Dublin 2010
How to use Geolocation in your webapp @ FOWA Dublin 2010
 
Intro to developing for @twitterapi
Intro to developing for @twitterapiIntro to developing for @twitterapi
Intro to developing for @twitterapi
 
Twitter API Annotations
Twitter API AnnotationsTwitter API Annotations
Twitter API Annotations
 
"What's Happening" to "What's Happening Here" @ Chirp
"What's Happening" to "What's Happening Here" @ Chirp"What's Happening" to "What's Happening Here" @ Chirp
"What's Happening" to "What's Happening Here" @ Chirp
 
Energy / Tweet
Energy / TweetEnergy / Tweet
Energy / Tweet
 
Handling Real-time Geostreams
Handling Real-time GeostreamsHandling Real-time Geostreams
Handling Real-time Geostreams
 
Adding the "Where" to the "When"
Adding the "Where" to the "When"Adding the "Where" to the "When"
Adding the "Where" to the "When"
 
What's happening here?
What's happening here?What's happening here?
What's happening here?
 
WattzOn @ ETech 2009
WattzOn @ ETech 2009WattzOn @ ETech 2009
WattzOn @ ETech 2009
 
Scala + WattzOn, sitting in a tree....
Scala + WattzOn, sitting in a tree....Scala + WattzOn, sitting in a tree....
Scala + WattzOn, sitting in a tree....
 
WattzOn Whole Earth Simulator
WattzOn Whole Earth SimulatorWattzOn Whole Earth Simulator
WattzOn Whole Earth Simulator
 
Broken Hearts: How Valentine's Day causes global warming
Broken Hearts: How Valentine's Day causes global warmingBroken Hearts: How Valentine's Day causes global warming
Broken Hearts: How Valentine's Day causes global warming
 
WattzOn presentation @ Web 2.0 Summit
WattzOn presentation @ Web 2.0 SummitWattzOn presentation @ Web 2.0 Summit
WattzOn presentation @ Web 2.0 Summit
 
holmz @ Ignite! NYC
holmz @ Ignite! NYCholmz @ Ignite! NYC
holmz @ Ignite! NYC
 

Dernier

Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 
unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxunit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxBkGupta21
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxNavinnSomaal
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxLoriGlavin3
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brandgvaughan
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 3652toLead Limited
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersRaghuram Pandurangan
 
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfHyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfPrecisely
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteDianaGray10
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLScyllaDB
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxLoriGlavin3
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Commit University
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxLoriGlavin3
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsRizwan Syed
 

Dernier (20)

Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 
unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxunit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptx
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptx
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information Developers
 
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfHyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test Suite
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQL
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
 
DMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special EditionDMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special Edition
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL Certs
 

Twitter - Guest Lecture UC Berkeley CS10 Fall 2010