Sean Parsons and Mark Baker from MindCandy discuss their use of Scala in services and tools. They chose Scala due to familiarity with the JVM, static types, and its large community. They used Scala to build backend services including for users, games, news and more. They also used Scala for tools like the REPL, scripts, a setup tool and Play-based websites. They found Scala good for expressing algorithms clearly and taking advantage of immutability and actors to scale. While learning Scala required learning new syntax and paradigms, they plan to continue using it for tools, scripts and services going forward.
4. Why we chose Scala
Familiarity with the JVM.
Transforming collections, Guava is still a bit of
a blunt tool.
Static types are cool.
Of the non-Java JVM languages seemed to
have the bigger community.
Commercial Support
23. Why Scala for Tools
Fewer dependencies
Easy to distribute tools
Lots of useful Java libs
Move from Python "felt good"
XML / Pattern match features
Language Scalability
24. How we used Scala - Tools (1)
REPL investigations
Quick Scripts
Setup Tool
Play-based websites
25. How we used Scala - Tools (2)
private val HudsonColorRegex = new Regex(""".*"color":"(w+)".*""")
private def jenkinsBuildStatus(build:models.AutomatedBuild):String = {
val hudsonUrl = build.url + "/api/json"
val result = scala.io.Source.fromURL(hudsonUrl).getLines().mkString
val HudsonColorRegex(color) = result
color match {
case "blue" => GOOD
case "red" => BAD
case "aborted" => BAD
case "disabled" => DISABLED
case "grey_anime" => UNKNOWN // build has not run yet...
case _ => unknownStatus("can't match jenkins build status color of: " + color)
}
}
26. How we used Scala - Tools (3)
getCachedFileSizes(path, revision)
.groupBy{ _.size / groupByBytes }
.map { case (group,files) => group -> files.size }
.toList
.sorted
.foreach {
case (group, count) => data.addRowFromValues(
humanGroupSize(group * groupByBytes,
groupByBytes),
new Integer(count))
}
31. Our Scala Experience
Training - lots of syntax to learn
New language & New paradigms
Tools maturity
Compiler errors
Still learning about maintenance
behaviour
Who we areWhat is scalaWhy we chose itHow we use itWhy you should use it\n\naim for 25 mins tops\n
http://mindcandy.com/\n\nLet’s see a show of hands... and who has actually played?Mindcandy - we make Moshi Monsters, have millions of users and digital entertainment is heart of our company\n\nSean - "Back end" developer\nMark - "Tools" developer\n \n
http://mindcandy.com/\n\nLet’s see a show of hands... and who has actually played?Mindcandy - we make Moshi Monsters, have millions of users and digital entertainment is heart of our company\n\nSean - "Back end" developer\nMark - "Tools" developer\n \n
(not needed for Scala Days)\n\nIMPORTANT HIGHLIGHTS ONLY\n\nWhat are the most relevant bits to us - 5 mins total\n\nStandard blurb is\nScala is a general purpose programming language designed to express common programming patterns in a concise, elegant, and type-safe way. It smoothly integrates features of object-oriented and functional languages, enabling Java and other programmers to be more productive. Code sizes are typically reduced by a factor of two to three when compared to an equivalent Java application.\n\nso what does that mean?\n\nit's functional AND object oriented\n\nit's type safe and uses static typing with compile-time type checks\n\nit compiles to Java Bytecode, runs on the Java Virtual Machine and can call any Java library / be called by java libraries very easily\n\nit can infer types at compile time so you don't have to write them again and again (and again)\n\nit has a REPL console\n\nfocus on productivity / not repeating boilerplate\n\n\nhttp://www.scala-lang.org/node/104 is good reference\nLanguage features of scala\nTraits and Type parameters\nthough you don't need to use (generics) often, they are convenient for library writers\n\nNO MACROS OR TEMPLATES!\n
\n
News feed for the best friends of users.\n\nWhen a best friend does something of note, it should appear in here.\n\nPolled from the client.\n
News feed for the best friends of users.\n\nWhen a best friend does something of note, it should appear in here.\n\nPolled from the client.\n
Write phase.\n\n- User 1 performs some action that has a news item associated with it.\n- News item pushed through HTTP to the news service, service returns immediately once the request has been parsed.\n- News service looks up users that have marked user 1 as a best friend from Cassandra.\n- If necessary populates Cassandra from the game database.\n- Performs a fanout of the news item to individual feeds for each user.\n
Write phase.\n\n- User 1 performs some action that has a news item associated with it.\n- News item pushed through HTTP to the news service, service returns immediately once the request has been parsed.\n- News service looks up users that have marked user 1 as a best friend from Cassandra.\n- If necessary populates Cassandra from the game database.\n- Performs a fanout of the news item to individual feeds for each user.\n
Write phase.\n\n- User 1 performs some action that has a news item associated with it.\n- News item pushed through HTTP to the news service, service returns immediately once the request has been parsed.\n- News service looks up users that have marked user 1 as a best friend from Cassandra.\n- If necessary populates Cassandra from the game database.\n- Performs a fanout of the news item to individual feeds for each user.\n
Write phase.\n\n- User 1 performs some action that has a news item associated with it.\n- News item pushed through HTTP to the news service, service returns immediately once the request has been parsed.\n- News service looks up users that have marked user 1 as a best friend from Cassandra.\n- If necessary populates Cassandra from the game database.\n- Performs a fanout of the news item to individual feeds for each user.\n
Write phase.\n\n- User 1 performs some action that has a news item associated with it.\n- News item pushed through HTTP to the news service, service returns immediately once the request has been parsed.\n- News service looks up users that have marked user 1 as a best friend from Cassandra.\n- If necessary populates Cassandra from the game database.\n- Performs a fanout of the news item to individual feeds for each user.\n
Write phase.\n\n- User 1 performs some action that has a news item associated with it.\n- News item pushed through HTTP to the news service, service returns immediately once the request has been parsed.\n- News service looks up users that have marked user 1 as a best friend from Cassandra.\n- If necessary populates Cassandra from the game database.\n- Performs a fanout of the news item to individual feeds for each user.\n
Write phase.\n\n- User 1 performs some action that has a news item associated with it.\n- News item pushed through HTTP to the news service, service returns immediately once the request has been parsed.\n- News service looks up users that have marked user 1 as a best friend from Cassandra.\n- If necessary populates Cassandra from the game database.\n- Performs a fanout of the news item to individual feeds for each user.\n
Write phase.\n\n- User 1 performs some action that has a news item associated with it.\n- News item pushed through HTTP to the news service, service returns immediately once the request has been parsed.\n- News service looks up users that have marked user 1 as a best friend from Cassandra.\n- If necessary populates Cassandra from the game database.\n- Performs a fanout of the news item to individual feeds for each user.\n
Write phase.\n\n- User 1 performs some action that has a news item associated with it.\n- News item pushed through HTTP to the news service, service returns immediately once the request has been parsed.\n- News service looks up users that have marked user 1 as a best friend from Cassandra.\n- If necessary populates Cassandra from the game database.\n- Performs a fanout of the news item to individual feeds for each user.\n
Write phase.\n\n- User 1 performs some action that has a news item associated with it.\n- News item pushed through HTTP to the news service, service returns immediately once the request has been parsed.\n- News service looks up users that have marked user 1 as a best friend from Cassandra.\n- If necessary populates Cassandra from the game database.\n- Performs a fanout of the news item to individual feeds for each user.\n
Write phase.\n\n- User 1 performs some action that has a news item associated with it.\n- News item pushed through HTTP to the news service, service returns immediately once the request has been parsed.\n- News service looks up users that have marked user 1 as a best friend from Cassandra.\n- If necessary populates Cassandra from the game database.\n- Performs a fanout of the news item to individual feeds for each user.\n
Write phase.\n\n- User 1 performs some action that has a news item associated with it.\n- News item pushed through HTTP to the news service, service returns immediately once the request has been parsed.\n- News service looks up users that have marked user 1 as a best friend from Cassandra.\n- If necessary populates Cassandra from the game database.\n- Performs a fanout of the news item to individual feeds for each user.\n
Write phase.\n\n- User 1 performs some action that has a news item associated with it.\n- News item pushed through HTTP to the news service, service returns immediately once the request has been parsed.\n- News service looks up users that have marked user 1 as a best friend from Cassandra.\n- If necessary populates Cassandra from the game database.\n- Performs a fanout of the news item to individual feeds for each user.\n
Write phase.\n\n- User 1 performs some action that has a news item associated with it.\n- News item pushed through HTTP to the news service, service returns immediately once the request has been parsed.\n- News service looks up users that have marked user 1 as a best friend from Cassandra.\n- If necessary populates Cassandra from the game database.\n- Performs a fanout of the news item to individual feeds for each user.\n
Write phase.\n\n- User 1 performs some action that has a news item associated with it.\n- News item pushed through HTTP to the news service, service returns immediately once the request has been parsed.\n- News service looks up users that have marked user 1 as a best friend from Cassandra.\n- If necessary populates Cassandra from the game database.\n- Performs a fanout of the news item to individual feeds for each user.\n
Read phase.\n\n- Front-end polls news service.\n- News service performs a read against the feed for said user.\n- User information for the users mentioned in those news items are looked up from Cassandra.\n- Users who are not populated in the cache are eliminated from the response and a message is fired to populate that user cache asynchronously.\n\nDesigned to be overly cautious and easy on the infrastructure as it was the first project at Mind Candy to use Scala/Cassandra/Akka.\n
Read phase.\n\n- Front-end polls news service.\n- News service performs a read against the feed for said user.\n- User information for the users mentioned in those news items are looked up from Cassandra.\n- Users who are not populated in the cache are eliminated from the response and a message is fired to populate that user cache asynchronously.\n\nDesigned to be overly cautious and easy on the infrastructure as it was the first project at Mind Candy to use Scala/Cassandra/Akka.\n
Read phase.\n\n- Front-end polls news service.\n- News service performs a read against the feed for said user.\n- User information for the users mentioned in those news items are looked up from Cassandra.\n- Users who are not populated in the cache are eliminated from the response and a message is fired to populate that user cache asynchronously.\n\nDesigned to be overly cautious and easy on the infrastructure as it was the first project at Mind Candy to use Scala/Cassandra/Akka.\n
Read phase.\n\n- Front-end polls news service.\n- News service performs a read against the feed for said user.\n- User information for the users mentioned in those news items are looked up from Cassandra.\n- Users who are not populated in the cache are eliminated from the response and a message is fired to populate that user cache asynchronously.\n\nDesigned to be overly cautious and easy on the infrastructure as it was the first project at Mind Candy to use Scala/Cassandra/Akka.\n
Read phase.\n\n- Front-end polls news service.\n- News service performs a read against the feed for said user.\n- User information for the users mentioned in those news items are looked up from Cassandra.\n- Users who are not populated in the cache are eliminated from the response and a message is fired to populate that user cache asynchronously.\n\nDesigned to be overly cautious and easy on the infrastructure as it was the first project at Mind Candy to use Scala/Cassandra/Akka.\n
Read phase.\n\n- Front-end polls news service.\n- News service performs a read against the feed for said user.\n- User information for the users mentioned in those news items are looked up from Cassandra.\n- Users who are not populated in the cache are eliminated from the response and a message is fired to populate that user cache asynchronously.\n\nDesigned to be overly cautious and easy on the infrastructure as it was the first project at Mind Candy to use Scala/Cassandra/Akka.\n
Read phase.\n\n- Front-end polls news service.\n- News service performs a read against the feed for said user.\n- User information for the users mentioned in those news items are looked up from Cassandra.\n- Users who are not populated in the cache are eliminated from the response and a message is fired to populate that user cache asynchronously.\n\nDesigned to be overly cautious and easy on the infrastructure as it was the first project at Mind Candy to use Scala/Cassandra/Akka.\n
Read phase.\n\n- Front-end polls news service.\n- News service performs a read against the feed for said user.\n- User information for the users mentioned in those news items are looked up from Cassandra.\n- Users who are not populated in the cache are eliminated from the response and a message is fired to populate that user cache asynchronously.\n\nDesigned to be overly cautious and easy on the infrastructure as it was the first project at Mind Candy to use Scala/Cassandra/Akka.\n
Read phase.\n\n- Front-end polls news service.\n- News service performs a read against the feed for said user.\n- User information for the users mentioned in those news items are looked up from Cassandra.\n- Users who are not populated in the cache are eliminated from the response and a message is fired to populate that user cache asynchronously.\n\nDesigned to be overly cautious and easy on the infrastructure as it was the first project at Mind Candy to use Scala/Cassandra/Akka.\n
Wanted to avoid the monolithic WAR file accompanied by Tomcat, wired up with Spring.\n\nUsed implicit constructor arguments to perform a simple form of dependency injection, which avoids circular dependencies. However be careful of SI-5618 as specifying types can cause things to be wired with nulls out of order.\n\nUnfiltered provides a clean way to do REST like services that can be easily separated to use elsewhere. Wanted to make components.\n\nAkka allowed us to build a primitive CQRS system internally to the application, creating read and write paths.\n\nIf you're not using Scalacheck, you should be. Helpful to catch the issues you don't think about. Allows me to assume that I'm stupid and that I wont think of everything.\n\nMore a general point, we can write OO/functional/actor based code where it fits most appropriately.\n
\n
From a Tools/Infrastructure POV\n\nPreviously had used Python\n\nlots of 'pure java' libraries I can use e.g. talk to Subversion so few BINARY dependencies\n\nwe already have a JVM on every machine - desktop and server\n\nMakes for very easy deployment of tools to users -- it's just a (big) .jar file\n\nFewer platform differences between Mac/Linux/Win compared to, say, Python\n\nComing from Python, code feels very familiar - terse but can understand it later!\n\nLanguage scalability - from REPL tests / scripts up to big apps - Tools/Infrastructure covers a LOT of different code sizes so having one language that works everywhere is really useful for me\n
Testing pattern matching / parsing with REPL\n\nQuick scripts - e.g. migration of DB update scripts from folder-of-folders to a flat list. -> clean when done functionally\n\nsetup tool using Swing UI\n\nPlay based websites - migrating to play 2\n\n\ntalking to RESTful web services using XML pattern matching\n\nbuilt in regexes make it cleaner\n
PLAY based site\ngood if migrating from Django I think\n\ntwitter for JSON parsing - similar to python's json parse/dump\n\npattern matching / regex parsing make life easier\n\neverything returning a value = cleaner code\n
populating a data table \nties together Java subversion lib, \njava google data visualisation lib, scala logic!\n\nI'm not saying its good Scala, but I found it quite easy to write / understand\n
Game developers especially - great migration path from dynamic scripting languages (Python, Ruby, Lua) that you may already be using.\n\n Looks like a dynamic language, scales up like a static language.\n\nFunctional programming can give very clean, well expressed algorithms so you can focus on that, not on boilerplate.\n\nStrong focus on immutability and message passing means its easier to build scalable/reliable services -- which will strongly interest Game Devs these days!\ne.g. twitter moved from Ruby -> Scala\n\nScala books/docs great!\n\n
Game developers especially - great migration path from dynamic scripting languages (Python, Ruby, Lua) that you may already be using.\n\n Looks like a dynamic language, scales up like a static language.\n\nFunctional programming can give very clean, well expressed algorithms so you can focus on that, not on boilerplate.\n\nStrong focus on immutability and message passing means its easier to build scalable/reliable services -- which will strongly interest Game Devs these days!\ne.g. twitter moved from Ruby -> Scala\n\nScala books/docs great!\n\n
large amount of syntax to learn\nfeatures better for library writers than code authors sometimes?\nBackwards compatibility - e.g. play 2 lack of Magic[T]\n\nObviously, you can write bad code in it, but no worse (I would argue) than C++ with macros/templates etc. \noperator overloading is still scary\n\n\n
large amount of syntax to learn\nfeatures better for library writers than code authors sometimes?\nBackwards compatibility - e.g. play 2 lack of Magic[T]\n\nObviously, you can write bad code in it, but no worse (I would argue) than C++ with macros/templates etc. \noperator overloading is still scary\n\n\n
large amount of syntax to learn\nfeatures better for library writers than code authors sometimes?\nBackwards compatibility - e.g. play 2 lack of Magic[T]\n\nObviously, you can write bad code in it, but no worse (I would argue) than C++ with macros/templates etc. \noperator overloading is still scary\n\n\n
large amount of syntax to learn\nfeatures better for library writers than code authors sometimes?\nBackwards compatibility - e.g. play 2 lack of Magic[T]\n\nObviously, you can write bad code in it, but no worse (I would argue) than C++ with macros/templates etc. \noperator overloading is still scary\n\n\n
currently using it in 2 production services & as standard language for Tools/Infrastructure\n(alongside Java/Python)\nWill continue to train people up in Scala and use it more\nStill learning what  best "coding standards" will be for Scala to allow people to understand each other's code\n
currently using it in 2 production services & as standard language for Tools/Infrastructure\n(alongside Java/Python)\nWill continue to train people up in Scala and use it more\nStill learning what  best "coding standards" will be for Scala to allow people to understand each other's code\n