Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
Scala @foursquare
     Jorge Ortiz
    @jorgeortiz85
foursquare runs on...

• Scala 2.8.0
• Lift 2.1
• MongoDB
• Twitter open source projects
• ~46k lines of Scala
foursquare server team


• 9 out of 11 had no major prior Scala experience
• Previously: Java, Python, C#
• Most exposed t...
foursquare loves Scala

• Pimped: Option, List, Map
• Type-safe equality: =?
• BackgroundOperation { ... }
• Throttle(“sec...
2.7.7 => 2.8.0
           (painless migration)

• Compile: 2.7.7 with -deprecation
• Branch
• Compile 2.8.0 without -depre...
foursquare on Lift

• REST API
• Out-of-the-box:
  • Rich Ajax/Comet support
  • ORMs for PostgresQL, MongoDB
  • Random: ...
Lift Deployment


• nginx
• HAProxy
• API machines - stateless
• WWW machines - stateful
GC issues


• Stateless API machines
  • Tens of millions of requests per day
  • Only a dozen or so servers
  • 12 GB hea...
GC issues
• GC tuning?
  • “There's no way to get around the fact that
    we're allocating faster than we can collect.”

...
GC issues

• Stateful WWW machines
  • Stateful Lift + Scala closures + Java GC
  • Lots of variables getting caught in Se...
MongoDB in Scala

• Some support built-in to Lift
• No expressive query language, so we wrote one:
 Venue where (_.mayor e...
Twitter open source


• Kestrel
• Ostrich
• GrabbyHands
• (probably more soon...)
We’re hiring
http://foursquare.com/jobs
Questions?
Scala at foursquare
Prochain SlideShare
Chargement dans…5
×

Scala at foursquare

4 355 vues

Publié le

Presentation at the New York Scala Meetup - 10/20/2010

Publié dans : Technologie, Formation
  • Soyez le premier à commenter

Scala at foursquare

  1. 1. Scala @foursquare Jorge Ortiz @jorgeortiz85
  2. 2. foursquare runs on... • Scala 2.8.0 • Lift 2.1 • MongoDB • Twitter open source projects • ~46k lines of Scala
  3. 3. foursquare server team • 9 out of 11 had no major prior Scala experience • Previously: Java, Python, C# • Most exposed to: ML, Lisp
  4. 4. foursquare loves Scala • Pimped: Option, List, Map • Type-safe equality: =? • BackgroundOperation { ... } • Throttle(“secret feature!”) { ... } • Any Java library
  5. 5. 2.7.7 => 2.8.0 (painless migration) • Compile: 2.7.7 with -deprecation • Branch • Compile 2.8.0 without -deprecation • Compile 2.8.0 with -Xmigration • Compile 2.8.0 with -Xwarninit • Switch
  6. 6. foursquare on Lift • REST API • Out-of-the-box: • Rich Ajax/Comet support • ORMs for PostgresQL, MongoDB • Random: Email, Facebook, JSON, utils...
  7. 7. Lift Deployment • nginx • HAProxy • API machines - stateless • WWW machines - stateful
  8. 8. GC issues • Stateless API machines • Tens of millions of requests per day • Only a dozen or so servers • 12 GB heap => 15sec GCs *ouch*
  9. 9. GC issues • GC tuning? • “There's no way to get around the fact that we're allocating faster than we can collect.” • Allocating ~200 MB/s • Opportunity to reduce wasteful allocations • Better load-balancing strategy • Cap # of concurrent connections
  10. 10. GC issues • Stateful WWW machines • Stateful Lift + Scala closures + Java GC • Lots of variables getting caught in Session • val venues: RequestCache[List[Venue]] = RequestCache(Venue.findAll(...))
  11. 11. MongoDB in Scala • Some support built-in to Lift • No expressive query language, so we wrote one: Venue where (_.mayor eqs userid) modify (_.tags push “I’m mayor!”) updateMulti
  12. 12. Twitter open source • Kestrel • Ostrich • GrabbyHands • (probably more soon...)
  13. 13. We’re hiring http://foursquare.com/jobs
  14. 14. Questions?

×