SlideShare une entreprise Scribd logo
1  sur  50
Télécharger pour lire hors ligne
A Small Talk on Getting Big.
Scaling a Rails App & all that Jazz.
Good Dog Picture.
Medium Dog.
Bad Dog Picture.
How did this Happen?
Mongrel Picture.
Too Much Time in the Application.
 Too Much Time in the Database.
A Really Cool Trick
class ApplicationController < ActionController:Base
    include ResponseManagementAdditions
    after_filter :show_runtimes

   around_filter { |controller, action| controller.total_runtime =
   Benchmark.measure(&action).real }
end
A Really Cool Trick
module ResponseManagementAdditions
   def show_runtimes
    return if response.headers['Status'] == '304 Not Modified' || !
(response.body && response.body.respond_to?(:sub!))

   db_runtime = ((0 + (@db_rt_before_render || 0) +
(@db_rt_after_render || 0)) * 1000).truncate
   rendering_runtime = ((@rendering_runtime || 0) * 1000).truncate
   total_runtime = ((@total_runtime || 0) * 1000).truncate

    response.body.gsub!(/</body></html>$/, quot;<!-- served to you
through a copper wire by #{HOSTNAME} at
#{Time.now.to_s(:short)} in #{total_runtime} ms (d #{db_runtime} / r
#{rendering_runtime}). thank you, come again. -->
n</body></html>quot;)
   end
end
Two Ways People Design Sites.

       Over-architect.
       Under-architect.
Move Fast. Scale Quickly.
Too Much Time in the Application.
Abstract Long Running Processes
          to Daemons.
An Ugly but Amazingly Simple Queuing
              System.
class FooDaemon < TwitterDaemon::Base
 before_startup :set_fugly_dist_idx

 def process
  unprocessed_content do |c|
   increment_counter(:total)

       # Do work ...

   break unless running?
  end
 end

 ...
...

def unprocessed_content(&block)
   loop do
     content = ContentModel.pending_content(quot;substring(truncate(id,
0),-2,1) = #{@fugly_dist_idx}quot;)
     messages.each { |message| yield message }
     sleep 1 if messages.nil? || messages.empty?
   end
 end

 def set_fugly_dist_idx
  @fugly_dist_idx = ARGV.find { |v| v.match(/[0-9]/) }
  raise quot;You need to specify a dist idx between 0 and 9.quot; unless
@fugly_dist_idx
  @fugly_dist_idx = @fugly_dist_idx.to_i
 end
end
A Better Queuing System.
Starling.
Distributed Queuing.
   Transactional Playback.
            Fast.
           Simple.
Speaks Memcache's Language.
      100% Pure Ruby.
Not Open Source.
(yet ...)
Too Much Time in the Database.
The Basics. Database 101.
(I shouldn't need this slide in here.)
Index everything you will query on.
            Avoid complex joins.
Use joint indices when you must join tables.
     Avoid scanning large sets of data.
Cache.
Cache.
Cache.
CACHE!
But How?
That Sounds Hard.
Turns Out it Isn't.
Serialize, Denormalize.
class User < ActiveRecord::Base
 serialize :following_ids

 def following_ids
  # this accessor is overwritten because we want to lazily set the
  # friends_ids column, rather than running a gigantic slow migration.
  RAILS_DEFAULT_LOGGER.debug quot;loading following_idsquot;
  ids = read_attribute(:following_ids)

  if ids.nil? || !ids.kind_of?(Array)
    ids = connection.select_values(quot;SELECT DISTINCT followed_user_id
FROM followed_users WHERE user_id =
#{self.id}quot;).map(&:to_i).compact
    update_attribute(:following_ids, ids)
  end

  ids
 end

 ...
def following_ids_add(the_id)
  ids = self.following_ids.dup
  ids << the_id
  write_attribute(:following_ids, ids)
 end

 def following_ids_delete(the_id)
  ids = self.following_ids.dup
  ids.delete(the_id)
  write_attribute(:following_ids, ids)
 end

end # End Class
Oh yeah, and Cheat.
     (It's ok!)
Thing about your application.
How can you cheat and get away
           with it?
Is your data delivered in real time?
    Is your data static content?
      How do users interact?
Interestingness.
(Little things that don't deserve other space.)
It's OK to use Monit to kill processes
          if they get too big.
Ensure you can deploy frequently.
Ensure you can roll back easily.
Scale where it matters.
Some code is ugly. It's OK.
  (who needs a hug?)
Ensure your users can give feedback
              easily.
Use the Community.
Make an API.
(Scale your Developer-base.)
We run on Edge (but with Piston).
A Cool Trick. Gems in Vendor.
Rails::Initializer.run do |config|

 # Load Gems from the /vendor/gems folder first, if they exist.
 config.load_paths += Dir[quot;#{RAILS_ROOT}/vendor/gems/**quot;].map do |dir|
  File.directory?(lib = quot;#{dir}/libquot;) ? lib : dir
 end

 ...
Personal Pet Peeve.

It's 2007. Every spammer has your email address.
  Put it on your goddamn webpage so people can
      get ahold of you about interesting things.
Questions?
Britt Selvitelle

      IM & Email
anotherbritt@gmail.com

Contenu connexe

Tendances

Painless Persistence in a Disconnected World
Painless Persistence in a Disconnected WorldPainless Persistence in a Disconnected World
Painless Persistence in a Disconnected WorldChristian Melchior
 
Java script at backend nodejs
Java script at backend   nodejsJava script at backend   nodejs
Java script at backend nodejsAmit Thakkar
 
A language for the Internet: Why JavaScript and Node.js is right for Internet...
A language for the Internet: Why JavaScript and Node.js is right for Internet...A language for the Internet: Why JavaScript and Node.js is right for Internet...
A language for the Internet: Why JavaScript and Node.js is right for Internet...Tom Croucher
 
A million connections and beyond - Node.js at scale
A million connections and beyond - Node.js at scaleA million connections and beyond - Node.js at scale
A million connections and beyond - Node.js at scaleTom Croucher
 
Why Task Queues - ComoRichWeb
Why Task Queues - ComoRichWebWhy Task Queues - ComoRichWeb
Why Task Queues - ComoRichWebBryan Helmig
 
Realm of the Mobile Database: an introduction to Realm
Realm of the Mobile Database: an introduction to RealmRealm of the Mobile Database: an introduction to Realm
Realm of the Mobile Database: an introduction to RealmMartin Grider
 
node.js, javascript and the future
node.js, javascript and the futurenode.js, javascript and the future
node.js, javascript and the futureJeff Miccolis
 
#NoXML: Eliminating XML in Spring Projects - SpringOne 2GX 2015
#NoXML: Eliminating XML in Spring Projects - SpringOne 2GX 2015#NoXML: Eliminating XML in Spring Projects - SpringOne 2GX 2015
#NoXML: Eliminating XML in Spring Projects - SpringOne 2GX 2015Matt Raible
 
modern module development - Ken Barber 2012 Edinburgh Puppet Camp
modern module development - Ken Barber 2012 Edinburgh Puppet Campmodern module development - Ken Barber 2012 Edinburgh Puppet Camp
modern module development - Ken Barber 2012 Edinburgh Puppet CampPuppet
 
Building an App with jQuery and JAXER
Building an App with jQuery and JAXERBuilding an App with jQuery and JAXER
Building an App with jQuery and JAXERBrian Moschel
 
Introduction to rest.li
Introduction to rest.liIntroduction to rest.li
Introduction to rest.liJoe Betz
 
Nodejs Event Driven Concurrency for Web Applications
Nodejs Event Driven Concurrency for Web ApplicationsNodejs Event Driven Concurrency for Web Applications
Nodejs Event Driven Concurrency for Web ApplicationsGanesh Iyer
 
&lt;x> Rails Web App Security Title
&lt;x> Rails Web App Security Title&lt;x> Rails Web App Security Title
&lt;x> Rails Web App Security Title'"><x> '"><x>
 
Getting Started with Apache Cassandra by Junior Evangelist Rebecca Mills
Getting Started with Apache Cassandra by Junior Evangelist Rebecca MillsGetting Started with Apache Cassandra by Junior Evangelist Rebecca Mills
Getting Started with Apache Cassandra by Junior Evangelist Rebecca MillsDataStax Academy
 
"Full Stack frameworks or a story about how to reconcile Front (good) and Bac...
"Full Stack frameworks or a story about how to reconcile Front (good) and Bac..."Full Stack frameworks or a story about how to reconcile Front (good) and Bac...
"Full Stack frameworks or a story about how to reconcile Front (good) and Bac...Fwdays
 
Server Side Event Driven Programming
Server Side Event Driven ProgrammingServer Side Event Driven Programming
Server Side Event Driven ProgrammingKamal Hussain
 
An Introduction to Windows PowerShell
An Introduction to Windows PowerShellAn Introduction to Windows PowerShell
An Introduction to Windows PowerShellDale Lane
 
Async servers and clients in Rest.li
Async servers and clients in Rest.liAsync servers and clients in Rest.li
Async servers and clients in Rest.liKaran Parikh
 
Scaling up task processing with Celery
Scaling up task processing with CeleryScaling up task processing with Celery
Scaling up task processing with CeleryNicolas Grasset
 

Tendances (20)

Painless Persistence in a Disconnected World
Painless Persistence in a Disconnected WorldPainless Persistence in a Disconnected World
Painless Persistence in a Disconnected World
 
Java script at backend nodejs
Java script at backend   nodejsJava script at backend   nodejs
Java script at backend nodejs
 
A language for the Internet: Why JavaScript and Node.js is right for Internet...
A language for the Internet: Why JavaScript and Node.js is right for Internet...A language for the Internet: Why JavaScript and Node.js is right for Internet...
A language for the Internet: Why JavaScript and Node.js is right for Internet...
 
A million connections and beyond - Node.js at scale
A million connections and beyond - Node.js at scaleA million connections and beyond - Node.js at scale
A million connections and beyond - Node.js at scale
 
Why Task Queues - ComoRichWeb
Why Task Queues - ComoRichWebWhy Task Queues - ComoRichWeb
Why Task Queues - ComoRichWeb
 
Realm of the Mobile Database: an introduction to Realm
Realm of the Mobile Database: an introduction to RealmRealm of the Mobile Database: an introduction to Realm
Realm of the Mobile Database: an introduction to Realm
 
node.js, javascript and the future
node.js, javascript and the futurenode.js, javascript and the future
node.js, javascript and the future
 
#NoXML: Eliminating XML in Spring Projects - SpringOne 2GX 2015
#NoXML: Eliminating XML in Spring Projects - SpringOne 2GX 2015#NoXML: Eliminating XML in Spring Projects - SpringOne 2GX 2015
#NoXML: Eliminating XML in Spring Projects - SpringOne 2GX 2015
 
modern module development - Ken Barber 2012 Edinburgh Puppet Camp
modern module development - Ken Barber 2012 Edinburgh Puppet Campmodern module development - Ken Barber 2012 Edinburgh Puppet Camp
modern module development - Ken Barber 2012 Edinburgh Puppet Camp
 
Building an App with jQuery and JAXER
Building an App with jQuery and JAXERBuilding an App with jQuery and JAXER
Building an App with jQuery and JAXER
 
Introduction to rest.li
Introduction to rest.liIntroduction to rest.li
Introduction to rest.li
 
Nodejs Event Driven Concurrency for Web Applications
Nodejs Event Driven Concurrency for Web ApplicationsNodejs Event Driven Concurrency for Web Applications
Nodejs Event Driven Concurrency for Web Applications
 
&lt;x> Rails Web App Security Title
&lt;x> Rails Web App Security Title&lt;x> Rails Web App Security Title
&lt;x> Rails Web App Security Title
 
Getting Started with Apache Cassandra by Junior Evangelist Rebecca Mills
Getting Started with Apache Cassandra by Junior Evangelist Rebecca MillsGetting Started with Apache Cassandra by Junior Evangelist Rebecca Mills
Getting Started with Apache Cassandra by Junior Evangelist Rebecca Mills
 
"Full Stack frameworks or a story about how to reconcile Front (good) and Bac...
"Full Stack frameworks or a story about how to reconcile Front (good) and Bac..."Full Stack frameworks or a story about how to reconcile Front (good) and Bac...
"Full Stack frameworks or a story about how to reconcile Front (good) and Bac...
 
Server Side Event Driven Programming
Server Side Event Driven ProgrammingServer Side Event Driven Programming
Server Side Event Driven Programming
 
An Introduction to Windows PowerShell
An Introduction to Windows PowerShellAn Introduction to Windows PowerShell
An Introduction to Windows PowerShell
 
Async servers and clients in Rest.li
Async servers and clients in Rest.liAsync servers and clients in Rest.li
Async servers and clients in Rest.li
 
Scaling up task processing with Celery
Scaling up task processing with CeleryScaling up task processing with Celery
Scaling up task processing with Celery
 
AJAX Transport Layer
AJAX Transport LayerAJAX Transport Layer
AJAX Transport Layer
 

En vedette

Twitter - Architecture and Scalability lessons
Twitter - Architecture and Scalability lessonsTwitter - Architecture and Scalability lessons
Twitter - Architecture and Scalability lessonsAditya Rao
 
Improving Running Components at Twitter
Improving Running Components at TwitterImproving Running Components at Twitter
Improving Running Components at TwitterEvan Weaver
 
Evolution of The Twitter Stack
Evolution of The Twitter StackEvolution of The Twitter Stack
Evolution of The Twitter StackChris Aniszczyk
 
Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to RedisDvir Volk
 
Scalability, Availability & Stability Patterns
Scalability, Availability & Stability PatternsScalability, Availability & Stability Patterns
Scalability, Availability & Stability PatternsJonas Bonér
 

En vedette (6)

Twitter - Architecture and Scalability lessons
Twitter - Architecture and Scalability lessonsTwitter - Architecture and Scalability lessons
Twitter - Architecture and Scalability lessons
 
Improving Running Components at Twitter
Improving Running Components at TwitterImproving Running Components at Twitter
Improving Running Components at Twitter
 
OAuth
OAuthOAuth
OAuth
 
Evolution of The Twitter Stack
Evolution of The Twitter StackEvolution of The Twitter Stack
Evolution of The Twitter Stack
 
Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to Redis
 
Scalability, Availability & Stability Patterns
Scalability, Availability & Stability PatternsScalability, Availability & Stability Patterns
Scalability, Availability & Stability Patterns
 

Similaire à A Small Talk on Getting Big

[FT-7][snowmantw] How to make a new functional language and make the world be...
[FT-7][snowmantw] How to make a new functional language and make the world be...[FT-7][snowmantw] How to make a new functional language and make the world be...
[FT-7][snowmantw] How to make a new functional language and make the world be...Functional Thursday
 
The Art Of Readable Code
The Art Of Readable CodeThe Art Of Readable Code
The Art Of Readable CodeBaidu, Inc.
 
Beyond php it's not (just) about the code
Beyond php   it's not (just) about the codeBeyond php   it's not (just) about the code
Beyond php it's not (just) about the codeWim Godden
 
Beyond php - it's not (just) about the code
Beyond php - it's not (just) about the codeBeyond php - it's not (just) about the code
Beyond php - it's not (just) about the codeWim Godden
 
"Scala in Goozy", Alexey Zlobin
"Scala in Goozy", Alexey Zlobin "Scala in Goozy", Alexey Zlobin
"Scala in Goozy", Alexey Zlobin Vasil Remeniuk
 
Crossing the Bridge: Connecting Rails and your Front-end Framework
Crossing the Bridge: Connecting Rails and your Front-end FrameworkCrossing the Bridge: Connecting Rails and your Front-end Framework
Crossing the Bridge: Connecting Rails and your Front-end FrameworkDaniel Spector
 
He stopped using for/while loops, you won't believe what happened next!
He stopped using for/while loops, you won't believe what happened next!He stopped using for/while loops, you won't believe what happened next!
He stopped using for/while loops, you won't believe what happened next!François-Guillaume Ribreau
 
Beyond PHP - It's not (just) about the code
Beyond PHP - It's not (just) about the codeBeyond PHP - It's not (just) about the code
Beyond PHP - It's not (just) about the codeWim Godden
 
Random Ruby Tips - Ruby Meetup 27 Jun 2018
Random Ruby Tips - Ruby Meetup 27 Jun 2018Random Ruby Tips - Ruby Meetup 27 Jun 2018
Random Ruby Tips - Ruby Meetup 27 Jun 2018Kenneth Teh
 
Intro to-rails-webperf
Intro to-rails-webperfIntro to-rails-webperf
Intro to-rails-webperfNew Relic
 
Rails Tips and Best Practices
Rails Tips and Best PracticesRails Tips and Best Practices
Rails Tips and Best PracticesDavid Keener
 
Zero, one, two, Freddy's coming for you
Zero, one, two, Freddy's coming for youZero, one, two, Freddy's coming for you
Zero, one, two, Freddy's coming for youAndrey Karpov
 
#SPSEMEA SharePoint & jQuery - What I wish I would have known a year ago..
#SPSEMEA SharePoint & jQuery - What I wish I would have known a year ago..#SPSEMEA SharePoint & jQuery - What I wish I would have known a year ago..
#SPSEMEA SharePoint & jQuery - What I wish I would have known a year ago..Mark Rackley
 

Similaire à A Small Talk on Getting Big (20)

The Joy Of Ruby
The Joy Of RubyThe Joy Of Ruby
The Joy Of Ruby
 
[FT-7][snowmantw] How to make a new functional language and make the world be...
[FT-7][snowmantw] How to make a new functional language and make the world be...[FT-7][snowmantw] How to make a new functional language and make the world be...
[FT-7][snowmantw] How to make a new functional language and make the world be...
 
The Art Of Readable Code
The Art Of Readable CodeThe Art Of Readable Code
The Art Of Readable Code
 
Beyond php it's not (just) about the code
Beyond php   it's not (just) about the codeBeyond php   it's not (just) about the code
Beyond php it's not (just) about the code
 
Beyond php - it's not (just) about the code
Beyond php - it's not (just) about the codeBeyond php - it's not (just) about the code
Beyond php - it's not (just) about the code
 
"Scala in Goozy", Alexey Zlobin
"Scala in Goozy", Alexey Zlobin "Scala in Goozy", Alexey Zlobin
"Scala in Goozy", Alexey Zlobin
 
Rspec
RspecRspec
Rspec
 
Crossing the Bridge: Connecting Rails and your Front-end Framework
Crossing the Bridge: Connecting Rails and your Front-end FrameworkCrossing the Bridge: Connecting Rails and your Front-end Framework
Crossing the Bridge: Connecting Rails and your Front-end Framework
 
NodeJS for Beginner
NodeJS for BeginnerNodeJS for Beginner
NodeJS for Beginner
 
Migrating legacy data
Migrating legacy dataMigrating legacy data
Migrating legacy data
 
He stopped using for/while loops, you won't believe what happened next!
He stopped using for/while loops, you won't believe what happened next!He stopped using for/while loops, you won't believe what happened next!
He stopped using for/while loops, you won't believe what happened next!
 
Coding Naked 2023
Coding Naked 2023Coding Naked 2023
Coding Naked 2023
 
Beyond PHP - It's not (just) about the code
Beyond PHP - It's not (just) about the codeBeyond PHP - It's not (just) about the code
Beyond PHP - It's not (just) about the code
 
Random Ruby Tips - Ruby Meetup 27 Jun 2018
Random Ruby Tips - Ruby Meetup 27 Jun 2018Random Ruby Tips - Ruby Meetup 27 Jun 2018
Random Ruby Tips - Ruby Meetup 27 Jun 2018
 
Intro to-rails-webperf
Intro to-rails-webperfIntro to-rails-webperf
Intro to-rails-webperf
 
Rails Tips and Best Practices
Rails Tips and Best PracticesRails Tips and Best Practices
Rails Tips and Best Practices
 
Zero, one, two, Freddy's coming for you
Zero, one, two, Freddy's coming for youZero, one, two, Freddy's coming for you
Zero, one, two, Freddy's coming for you
 
#SPSEMEA SharePoint & jQuery - What I wish I would have known a year ago..
#SPSEMEA SharePoint & jQuery - What I wish I would have known a year ago..#SPSEMEA SharePoint & jQuery - What I wish I would have known a year ago..
#SPSEMEA SharePoint & jQuery - What I wish I would have known a year ago..
 
Refactoring
RefactoringRefactoring
Refactoring
 
Refactoring
RefactoringRefactoring
Refactoring
 

Dernier

Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slidespraypatel2
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...shyamraj55
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slidevu2urc
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking MenDelhi Call girls
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Servicegiselly40
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsMaria Levchenko
 
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024BookNet Canada
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Drew Madelung
 
Maximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptxMaximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptxOnBoard
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersThousandEyes
 
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure serviceWhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure servicePooja Nehwal
 
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024BookNet Canada
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationRidwan Fadjar
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsEnterprise Knowledge
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonAnna Loughnan Colquhoun
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)Gabriella Davis
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountPuma Security, LLC
 
Unblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesUnblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesSinan KOZAK
 

Dernier (20)

Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slides
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
#StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
Maximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptxMaximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptx
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
 
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure serviceWhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
 
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 Presentation
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
 
Unblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen FramesUnblocking The Main Thread Solving ANRs and Frozen Frames
Unblocking The Main Thread Solving ANRs and Frozen Frames
 

A Small Talk on Getting Big

  • 1. A Small Talk on Getting Big. Scaling a Rails App & all that Jazz.
  • 5. How did this Happen?
  • 7. Too Much Time in the Application. Too Much Time in the Database.
  • 8. A Really Cool Trick class ApplicationController < ActionController:Base include ResponseManagementAdditions after_filter :show_runtimes around_filter { |controller, action| controller.total_runtime = Benchmark.measure(&action).real } end
  • 9. A Really Cool Trick module ResponseManagementAdditions def show_runtimes return if response.headers['Status'] == '304 Not Modified' || ! (response.body && response.body.respond_to?(:sub!)) db_runtime = ((0 + (@db_rt_before_render || 0) + (@db_rt_after_render || 0)) * 1000).truncate rendering_runtime = ((@rendering_runtime || 0) * 1000).truncate total_runtime = ((@total_runtime || 0) * 1000).truncate response.body.gsub!(/</body></html>$/, quot;<!-- served to you through a copper wire by #{HOSTNAME} at #{Time.now.to_s(:short)} in #{total_runtime} ms (d #{db_runtime} / r #{rendering_runtime}). thank you, come again. --> n</body></html>quot;) end end
  • 10. Two Ways People Design Sites. Over-architect. Under-architect.
  • 11. Move Fast. Scale Quickly.
  • 12. Too Much Time in the Application.
  • 13. Abstract Long Running Processes to Daemons.
  • 14. An Ugly but Amazingly Simple Queuing System.
  • 15. class FooDaemon < TwitterDaemon::Base before_startup :set_fugly_dist_idx def process unprocessed_content do |c| increment_counter(:total) # Do work ... break unless running? end end ...
  • 16. ... def unprocessed_content(&block) loop do content = ContentModel.pending_content(quot;substring(truncate(id, 0),-2,1) = #{@fugly_dist_idx}quot;) messages.each { |message| yield message } sleep 1 if messages.nil? || messages.empty? end end def set_fugly_dist_idx @fugly_dist_idx = ARGV.find { |v| v.match(/[0-9]/) } raise quot;You need to specify a dist idx between 0 and 9.quot; unless @fugly_dist_idx @fugly_dist_idx = @fugly_dist_idx.to_i end end
  • 19. Distributed Queuing. Transactional Playback. Fast. Simple. Speaks Memcache's Language. 100% Pure Ruby.
  • 22. Too Much Time in the Database.
  • 23. The Basics. Database 101. (I shouldn't need this slide in here.)
  • 24. Index everything you will query on. Avoid complex joins. Use joint indices when you must join tables. Avoid scanning large sets of data.
  • 30. Turns Out it Isn't.
  • 32. class User < ActiveRecord::Base serialize :following_ids def following_ids # this accessor is overwritten because we want to lazily set the # friends_ids column, rather than running a gigantic slow migration. RAILS_DEFAULT_LOGGER.debug quot;loading following_idsquot; ids = read_attribute(:following_ids) if ids.nil? || !ids.kind_of?(Array) ids = connection.select_values(quot;SELECT DISTINCT followed_user_id FROM followed_users WHERE user_id = #{self.id}quot;).map(&:to_i).compact update_attribute(:following_ids, ids) end ids end ...
  • 33. def following_ids_add(the_id) ids = self.following_ids.dup ids << the_id write_attribute(:following_ids, ids) end def following_ids_delete(the_id) ids = self.following_ids.dup ids.delete(the_id) write_attribute(:following_ids, ids) end end # End Class
  • 34. Oh yeah, and Cheat. (It's ok!)
  • 35. Thing about your application. How can you cheat and get away with it?
  • 36. Is your data delivered in real time? Is your data static content? How do users interact?
  • 37. Interestingness. (Little things that don't deserve other space.)
  • 38. It's OK to use Monit to kill processes if they get too big.
  • 39. Ensure you can deploy frequently.
  • 40. Ensure you can roll back easily.
  • 41. Scale where it matters.
  • 42. Some code is ugly. It's OK. (who needs a hug?)
  • 43. Ensure your users can give feedback easily.
  • 45. Make an API. (Scale your Developer-base.)
  • 46. We run on Edge (but with Piston).
  • 47. A Cool Trick. Gems in Vendor. Rails::Initializer.run do |config| # Load Gems from the /vendor/gems folder first, if they exist. config.load_paths += Dir[quot;#{RAILS_ROOT}/vendor/gems/**quot;].map do |dir| File.directory?(lib = quot;#{dir}/libquot;) ? lib : dir end ...
  • 48. Personal Pet Peeve. It's 2007. Every spammer has your email address. Put it on your goddamn webpage so people can get ahold of you about interesting things.
  • 50. Britt Selvitelle IM & Email anotherbritt@gmail.com