SlideShare une entreprise Scribd logo
1  sur  140
@raffi
          Raffi Krikorian


Giving a talk at @StanfordACM.
Want to know about engineering for
real-time?
1 min ago via Twitter for iPhone   Favorite   Retweet   Reply
@JordnJnkieJuice
         ✔Finally✈MF✈Famous✔



EARTHQUAKE !!!!!
1 min ago via Echofon   Favorite   Retweet   Reply
@usgs
300 miles
   to
  NYC
@remedyoakland
         Remedy


Strawberry scones and rhubarb
turnovers in the house!
14 Apr via Facebook   Favorite   Retweet   Reply
What’s a Tweet?
48 characters
                                              +1 for the bird

What’s a Tweet?
It’s a short message that's sent through




                                   140 characters max
1 tweet
sizeof(1 tweet)
sizeof(1 tweet)= 140 characters
sizeof(1 tweet)= 140 characters
sizeof(1 tweet)= 140 characters
sizeof(1 tweet)= 140 characters
sizeof(1 tweet)≈ 200 bytes
sizeof(1 tweet)≈ 200 bytes
= 140 characters
sizeof(1 tweet)
                ≈ 200 bytes
= 140 characters
sizeof(1 tweet)
                ≈ 200 bytes
@Astro_Soichi
          Soichi Noguchi



http://twitpic.com/17dfqb -




8 Mar 10 via TwitPic       Favorite   Retweet   Reply
@Astro_Soichi
          Soichi Noguchi



http://twitpic.com/17dfqb -




8 Mar 10 via TwitPic       Favorite   Retweet   Reply
About


Tweets a day
About

230,000,000
  Tweets a day
2.3E6 tweets
    day
2.3E6 tweets 2600 tweets
            ≈
    day        second
2.3E6 tweets 2600 tweets
            ≈
    day        second
2.3E6 tweets 2600 tweets
            ≈
    day        second
2.3E6 tweets 2600 tweets
            ≈
    day        second
2.3E6 tweets 2600 tweets
            ≈
    day        second

                       steady state
7,196
TPS
Tokyo
6,600 TPS
8,800 TPS
status/update
status/update
status/update
status/update




    HTTP 200 OK


{
status/update




    HTTP 200 OK


{
status/update




 HTTP 200 OK
status/update




 HTTP 200 OK
st
s
Snowflake
https://github.com/twitter/snowflake




                                       s
s
s
s
s
s



Rockdove
s
s
t.co
{
{
Storage
          {
{
status/update




    HTTP 200 OK


{
status/update




    HTTP 200 OK


{
status/update




    HTTP 200 OK


{
HTTP 200 OK


{
    …
    "text":"Tweet!"
    …
}
HTTP 200 OK


{
    …
    "text":"Tweet!"
    …
}
HTTP 200 OK


{
    …
    "text":"Tweet!"
    …
}
status/update




    HTTP 200 OK


{
    …
    "text":"Tweet!"
    …
}
status/update




    HTTP 200 OK


{
    …
    "text":"Tweet!"
    …
}
status/update




    HTTP 200 OK


{
    …
    "text":"Tweet!"
    …
}
status/update


                      Fanout




    HTTP 200 OK


{
    …
    "text":"Tweet!"
    …
}
status/update


                      Fanout




    HTTP 200 OK


{
    …
    "text":"Tweet!"
    …
}
te


     Fanout




K
te


     Fanout




K
te


     Fanout




K
te

              Flock
     Fanout




K
te


     Fanout




K
te


     Fanout




K
status/update


                      Fanout




    HTTP 200 OK


{
    …
    "text":"Tweet!"
    …
}
status/update


                      Fanout




    HTTP 200 OK


{
    …
    "text":"Tweet!"
    …
}
followed by   following
@ladygaga
mother mons†er
13.2 million followers

@justinbieber
Justin Bieber
12.4 million followers

@BarackObama
44th President of the United States
9.9 million followers

@raffi
me!
0.01 million followers
@ladygaga
mother mons†er
13.2 million followers

@raffi
me!
0.01 million followers
@ladygaga
mother mons†er
13.2 million followers

@raffi
me!
0.01 million followers
@ladygaga
       mother mons†er
       13.2 million followers

       @raffi
       me!
       0.01 million followers




10000x more followers than me
@ladygaga
mother mons†er
13.2 million followers

@raffi
me!
0.01 million followers




  scaling for            107
Has...
⇢ huge number of concurrent connections
⇢ lots of I/O
⇢ few persistent objects
Gosai Kiji

Phasianus versicolor also
known as Japanese Pheasant
is a bird of the lowlands.
Closely related to the
Common Pheasant, the cock
is distinguished by dark
green plumage on breast and
mantle. The male has an
iridescent violet neck, red
bare facial skin and purplish
green tail. The female is
smaller than male, and has a
dull brown plumage with
dark spots.
Gosai Kiji

Phasianus versicolor also
known as Japanese Pheasant
is a bird of the lowlands.
Closely related to the
Common Pheasant, the cock
is distinguished by dark
green plumage on breast and
mantle. The male has an
iridescent violet neck, red
bare facial skin and purplish
green tail. The female is
smaller than male, and has a
dull brown plumage with
dark spots.
needS...
⇢ ability to handle server workloads
needS...
⇢ ability to handle server workloads
⇢ flexibility in language
needS...
⇢ ability to handle server workloads
⇢ flexibility in language
needS...
⇢ ability to handle server workloads
⇢ flexibility in language
needS...
⇢ ability to handle server workloads
⇢ flexibility in language
needS...
⇢ ability to handle server workloads
⇢ flexibility in language
needS...
⇢ ability to handle server workloads
⇢ flexibility in language
⇢ a real concurrency model
finagle |fəˈnāgəl|

          needS...
                                       verb [ trans. ] informal
                                       obtain (something) by devious or dishonest means : Ted
                                       attended all the football games he could finagle tickets for.
                                       • [ intrans. ] act in a devious or dishonest manner : they
                                       wrangled and finagled over the fine points.
⇢ ability to handle server workloads   DERIVATIVES
⇢ flexibility in language              finagler |fəˈnāg(ə)lər| noun


⇢ a real concurrency model             ORIGIN 1920s (originally U.S.): from dialect fainaigue
                                       [cheat] ; perhaps from Old French fornier ‘deny.’
finagle |fəˈnāgəl|

          needS...
                                       verb [ trans. ] informal
                                       obtain (something) by devious or dishonest means : Ted
                                       attended all the football games he could finagle tickets for.
                                       • [ intrans. ] act in a devious or dishonest manner : they
                                       wrangled and finagled over the fine points.
⇢ ability to handle server workloads   DERIVATIVES
⇢ flexibility in language              finagler |fəˈnāg(ə)lər| noun


⇢ a real concurrency model             ORIGIN 1920s (originally U.S.): from dialect fainaigue
                                       [cheat] ; perhaps from Old French fornier ‘deny.’
finagle |fəˈnāgəl|

          needS...
                                       verb [ trans. ] informal
                                       obtain (something) by devious or dishonest means : Ted
                                       attended all the football games he could finagle tickets for.
                                       • [ intrans. ] act in a devious or dishonest manner : they
                                       wrangled and finagled over the fine points.
⇢ ability to handle server workloads   DERIVATIVES
⇢ flexibility in language              finagler |fəˈnāg(ə)lər| noun


⇢ a real concurrency model             ORIGIN 1920s (originally U.S.): from dialect fainaigue
                                       [cheat] ; perhaps from Old French fornier ‘deny.’
finagle |fəˈnāgəl|
                                       verb [ trans. ] informal

          needS...                     obtain (something) by devious or dishonest means : Ted
                                       attended all the football games he could finagle tickets for.
                                       • [ intrans. ] act in a devious or dishonest manner : they
                                       wrangled and finagled over the fine points.
⇢ ability to handle server workloads   DERIVATIVES

⇢ flexibility in language              finagler |fəˈnāg(ə)lər| noun


⇢ a real concurrency model             ORIGIN 1920s (originally U.S.): from dialect fainaigue
                                       [cheat] ; perhaps from Old French fornier ‘deny.’
’S Infrastructure trades in...
⇢ RAM & CPU
⇢ bandwidth
⇢ machine density
Switching to      doesn’t imply
that     is a mistake
is a

Real-Time Event-driven
         &
         Problem
What happens in the

    World
    happens on
Follow me at
Questions?   twitter.com/raffi

Contenu connexe

Plus de Raffi Krikorian

Twitter for CS10 @ Berkeley (Spring 2011)
Twitter for CS10 @ Berkeley (Spring 2011)Twitter for CS10 @ Berkeley (Spring 2011)
Twitter for CS10 @ Berkeley (Spring 2011)Raffi Krikorian
 
Twitter by the Numbers (Columbia University)
Twitter by the Numbers (Columbia University)Twitter by the Numbers (Columbia University)
Twitter by the Numbers (Columbia University)Raffi Krikorian
 
Twitter and the Real-Time Web
Twitter and the Real-Time WebTwitter and the Real-Time Web
Twitter and the Real-Time WebRaffi Krikorian
 
Twitter - Guest Lecture UC Berkeley CS10 Fall 2010
Twitter - Guest Lecture UC Berkeley CS10 Fall 2010Twitter - Guest Lecture UC Berkeley CS10 Fall 2010
Twitter - Guest Lecture UC Berkeley CS10 Fall 2010Raffi Krikorian
 
Developing for @twitterapi #hack4health
Developing for @twitterapi #hack4healthDeveloping for @twitterapi #hack4health
Developing for @twitterapi #hack4healthRaffi Krikorian
 
Intro to developing for @twitterapi (updated)
Intro to developing for @twitterapi (updated)Intro to developing for @twitterapi (updated)
Intro to developing for @twitterapi (updated)Raffi Krikorian
 
How to use Geolocation in your webapp @ FOWA Dublin 2010
How to use Geolocation in your webapp @ FOWA Dublin 2010How to use Geolocation in your webapp @ FOWA Dublin 2010
How to use Geolocation in your webapp @ FOWA Dublin 2010Raffi Krikorian
 
Intro to developing for @twitterapi
Intro to developing for @twitterapiIntro to developing for @twitterapi
Intro to developing for @twitterapiRaffi Krikorian
 
"What's Happening" to "What's Happening Here" @ Chirp
"What's Happening" to "What's Happening Here" @ Chirp"What's Happening" to "What's Happening Here" @ Chirp
"What's Happening" to "What's Happening Here" @ ChirpRaffi Krikorian
 
Handling Real-time Geostreams
Handling Real-time GeostreamsHandling Real-time Geostreams
Handling Real-time GeostreamsRaffi Krikorian
 
Adding the "Where" to the "When"
Adding the "Where" to the "When"Adding the "Where" to the "When"
Adding the "Where" to the "When"Raffi Krikorian
 
Scala + WattzOn, sitting in a tree....
Scala + WattzOn, sitting in a tree....Scala + WattzOn, sitting in a tree....
Scala + WattzOn, sitting in a tree....Raffi Krikorian
 
WattzOn Whole Earth Simulator
WattzOn Whole Earth SimulatorWattzOn Whole Earth Simulator
WattzOn Whole Earth SimulatorRaffi Krikorian
 
Broken Hearts: How Valentine's Day causes global warming
Broken Hearts: How Valentine's Day causes global warmingBroken Hearts: How Valentine's Day causes global warming
Broken Hearts: How Valentine's Day causes global warmingRaffi Krikorian
 

Plus de Raffi Krikorian (20)

Twitter for CS10 @ Berkeley (Spring 2011)
Twitter for CS10 @ Berkeley (Spring 2011)Twitter for CS10 @ Berkeley (Spring 2011)
Twitter for CS10 @ Berkeley (Spring 2011)
 
Twitter by the Numbers (Columbia University)
Twitter by the Numbers (Columbia University)Twitter by the Numbers (Columbia University)
Twitter by the Numbers (Columbia University)
 
Users and Geo
Users and GeoUsers and Geo
Users and Geo
 
Twitter and the Real-Time Web
Twitter and the Real-Time WebTwitter and the Real-Time Web
Twitter and the Real-Time Web
 
Twitter - Guest Lecture UC Berkeley CS10 Fall 2010
Twitter - Guest Lecture UC Berkeley CS10 Fall 2010Twitter - Guest Lecture UC Berkeley CS10 Fall 2010
Twitter - Guest Lecture UC Berkeley CS10 Fall 2010
 
Developing for @twitterapi #hack4health
Developing for @twitterapi #hack4healthDeveloping for @twitterapi #hack4health
Developing for @twitterapi #hack4health
 
Twitter by the Numbers
Twitter by the NumbersTwitter by the Numbers
Twitter by the Numbers
 
Intro to developing for @twitterapi (updated)
Intro to developing for @twitterapi (updated)Intro to developing for @twitterapi (updated)
Intro to developing for @twitterapi (updated)
 
How to use Geolocation in your webapp @ FOWA Dublin 2010
How to use Geolocation in your webapp @ FOWA Dublin 2010How to use Geolocation in your webapp @ FOWA Dublin 2010
How to use Geolocation in your webapp @ FOWA Dublin 2010
 
Intro to developing for @twitterapi
Intro to developing for @twitterapiIntro to developing for @twitterapi
Intro to developing for @twitterapi
 
Twitter API Annotations
Twitter API AnnotationsTwitter API Annotations
Twitter API Annotations
 
"What's Happening" to "What's Happening Here" @ Chirp
"What's Happening" to "What's Happening Here" @ Chirp"What's Happening" to "What's Happening Here" @ Chirp
"What's Happening" to "What's Happening Here" @ Chirp
 
Energy / Tweet
Energy / TweetEnergy / Tweet
Energy / Tweet
 
Handling Real-time Geostreams
Handling Real-time GeostreamsHandling Real-time Geostreams
Handling Real-time Geostreams
 
Adding the "Where" to the "When"
Adding the "Where" to the "When"Adding the "Where" to the "When"
Adding the "Where" to the "When"
 
What's happening here?
What's happening here?What's happening here?
What's happening here?
 
WattzOn @ ETech 2009
WattzOn @ ETech 2009WattzOn @ ETech 2009
WattzOn @ ETech 2009
 
Scala + WattzOn, sitting in a tree....
Scala + WattzOn, sitting in a tree....Scala + WattzOn, sitting in a tree....
Scala + WattzOn, sitting in a tree....
 
WattzOn Whole Earth Simulator
WattzOn Whole Earth SimulatorWattzOn Whole Earth Simulator
WattzOn Whole Earth Simulator
 
Broken Hearts: How Valentine's Day causes global warming
Broken Hearts: How Valentine's Day causes global warmingBroken Hearts: How Valentine's Day causes global warming
Broken Hearts: How Valentine's Day causes global warming
 

Dernier

"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxLoriGlavin3
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxLoriGlavin3
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfAddepto
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .Alan Dix
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningLars Bell
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteDianaGray10
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebUiPathCommunity
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxNavinnSomaal
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxLoriGlavin3
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupFlorian Wilhelm
 
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESmohitsingh558521
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLScyllaDB
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity PlanDatabarracks
 
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersRaghuram Pandurangan
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxLoriGlavin3
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxLoriGlavin3
 

Dernier (20)

"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptx
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test Suite
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio Web
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptx
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project Setup
 
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQL
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity Plan
 
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information Developers
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
 

Twitter: Engineering for Real-Time (Stanford ACM 2011)