Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

The Why and How of Scala at Twitter

53 090 vues

Publié le

A presentation at Twitter's official developer conference, Chirp, about why we use the Scala programming language and how we build services in it. Provides a tour of a number of libraries and tools, both developed at Twitter and otherwise.

Publié dans : Technologie
  • Did You Get Dumped? Do you still want her back? If you act now, I can help you. ♥♥♥ http://goo.gl/FXTq7P
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici
  • It doesn't matter how long ago you broke up, these methods will make your Ex crawling back! See how at here. ♥♥♥ http://t.cn/R50e5nn
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici
  • Controversial method reveals inner psychology of techniques you can use to get your Ex back! See it now! ◆◆◆ http://scamcb.com/exback123/pdf
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici
  • Is Your Ex With a Woman? Don't lose your Ex boyfriend! This weird trick will get him back! ♣♣♣ http://ishbv.com/exback123/pdf
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici
  • Get Your Ex Back Today, Relationship expert Justin Sinclair, shows you how with 3 easy steps. ➤➤ http://t.cn/R50e5nn
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici

The Why and How of Scala at Twitter

  1. The Why and How of Scala at Twitter Alex Payne (@al3x) Twitter Infrastructure
  2. Who’s this guy? •Working at Twitter since 2007, before it was even a company. •Spent a couple years on the API. •Co-authored Programming Scala (O’Reilly, 2009). •Into programming languages, scaling stuff.
  3. Part I: Why Scala?
  4. So, like, Ruby or Scala? •Both! •Ruby mostly on the frontend. •Scala mostly on the backend. •Thrift and data stores to connect the two.
  5. Elevator Pitch • It’s fun. • It’s fast. • It’s runs on a great virtual machine (JVM). • It’s got an awesome community. • It can borrow from Java.
  6. No, seriously, why? • A rich static type system that gets out of your way when you don’t need it, is awesome when you do. • Flexible syntax makes it easy to write readable, maintainable code. • Traits for cross-cutting modularity. • Lots of OOP goodness.
  7. I’m not sold yet. • How about the choice between immutable and mutable variables? • Powerful functional programming: mapping, filtering, folding, currying, so much more. • Lazy values. • Pattern matching. • XML literals, and clean syntax for working with XML documents built right in!
  8. What about concurrency? • You can do Actor concurrency (message passing), kinda like Erlang. • Or, use threads. • Or, use all the java.util.concurrent tools. • Or, use other JVM concurrency frameworks (ex: Netty, Apache Mina).
  9. Basically, it’s the future.
  10. Part II: How do you guys do Scala?
  11. We build services. • Isolated components, can swap them out. • Talk to the rest of the system via Thrift (for now, maybe Avro in the future). • Independently tested for correctness, load. • Can have custom operational properties.
  12. Scala Services at Twitter • Kestrel: queuing. • Flock (and Gizzard): social graph store. • Hawkwind: people search. • Hosebird: streaming API. • more in the works all the time...
  13. Thrift Example struct Results { 1: list<i64> people 2: string suggestion 3: i32 processingTime /* milliseconds */ 4: list<i32> timings 5: i32 totalResults } service NameSearch { Results find(1: string name, 2: i32 maxResults, 3: bool wantSuggestion) Results find_with_ranking(1: string name, 2: i32 maxResults, 3: bool wantSuggestion, 4: Ranker ranking) }
  14. We build libraries. • Reusable chunks of code that we can share between projects. • We open source them. • We try to keep the “NIH” to a minimum, but we also have very particular requirements.
  15. Ostrich • Gather stats from inside your running process. • Counters, gauges, and timings. • Share stats many ways: JMX, JSON-over- HTTP, plain text Telnet-style socket, log files.
  16. Ostrich Examples Stats.time(“find-users”) { users.find(queryString) } Stats.incr(“write-errors”) Stats.makeGauge("database-lag") { dbConnection.getLagInMillis() }
  17. Configgy • Configuration files and logging. • Flexible file format offers includes, more. • Inheritance, variable substitution. • Tunable logging, Scribe support. • Subscription API: push and validate configuration changes. • Ruby support, Emacs mode.
  18. Configgy Examples hostname = “service001” port = 8080 use_admin_interface = true log { filename = “my-service.log” level = “error” roll = “hourly” }
  19. Configgy Examples Configgy.configure(“/etc/my-service.conf”) val config = Configgy.config val port = config.getInt(“port”, 8080) val hostname = config.getString(“hostname”) log.info(“service started”) log.debug(“running on port %d”, port)
  20. xrayspecs • A set of extensions to the fantastic Specs BDD test framework. • Handles testing concurrency, time, creating temporary folders. • The concurrency features have been rolled into the main distribution of Specs!
  21. xrayspecs Examples “response arrives from server” in { get(“/example”) must eventually(notBe(null)) } “time should stop” in { Time.freeze() val nowSnapshot = Time.now Thread.sleep(30) Time.now mustEqual nowSnapshot }
  22. scala-json • A cleaned-up version of the official Scala JSON codec. • Makes clever use of parser combinators – a good way to learn about them. • Heavily tested, fixes a bunch of edge cases.
  23. JSON Examples val myJsonString = """ {"key": "value", "1": 2, "3": 4, "myList": [5, 6, 7] """ Json.parse(myJsonString) Json.build(List("user_id", 666))
  24. Other Twitter Libraries • Naggati: build protocols for Apache Mina. • Smile: a memcached client that uses Actors. • Querulous: a nice database client. • Jackhammer: a load testing framework (coming soon). • probably stuff I'm leaving out...
  25. Stuff We Use • There's great open source code in the Scala community. • We try to use and contribute back to third- party projects. • Follow @implicit_ly for new Scala releases. • Also subscribe to Repopular Scala.
  26. sbt – the simple build tool • Scala’s answer to Ant and Maven. • Sets up new projects. • Maintains project configuration, build tasks, and dependencies in pure Scala. Totally open- ended. • Interactive console. • Will run tasks as soon as files in your project change – automatically compile and run tests!
  27. specs • As aforementioned: great way to do BDD testing. • Set up pre- and post-conditions for individual tests, suites, etc. • Extremely flexible and extensible. • Great, responsive maintainer. • Supports mocking (we like Mockito).
  28. The IDE Question • We've had the best luck with IntelliJ IDEA, but it's still pretty rough. • Most of us use a plain text editor like Emacs, VIM, or TextMate. • Combined with sbt, it's actually a really nice workflow!
  29. Questions! Thanks for listening!