2024: Domino Containers - The Next Step. News from the Domino Container commu...
Building financial systems in scala
1. GSA Capital Partners LLP
March 2012
Name
Chris Marshall @oxbow_lakes
Private & Confidential. Not for distribution.
GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority.
Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261
3. Background
who am i
Mathematics Degree
Working in financial software since 1999
• Smalltalk for ~6 months
• Java thereafter
• Scala since Dec 2008
JP Morgan for 6 years
• ~200,000 employees
GSA Capital for 5 ¾ years
• Quant hedge fund
• ~90 employees
Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority.
Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261 3
4. GSA
What do we do? Roughly half the company are technologists
Low-latency links & feeds
• Up to 750,000 trades /day
• 108 market events / day
Historic / Current Market Data
• Listing changes (e.g. SUNW.O becomes JAVA.O becomes ORCL.O)
• News Events
Backtesting / Execution Framework
Everything Else
• This is where I come in
• What are our positions? What is our P&L?
• Are our trades reconciled?
• Reporting (brokers, regulators, administrators)
Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority.
Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261 4
5. Financial IT
Does financial IT differ from IT elsewhere?
Fragmentation / Diversity
• Vertical business silos (Equity, fixed-income etc)
Java
• Widespread adoption in the first decade of this century
Willing to try new technologies
• Even in large organizations
• Potentially driven by M & A
Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority.
Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261 5
6. Financial IT now
Where do we find ourselves in 2012?
Dependency on Java
• The platform
• The ecosystem
Older, wiser
• 10 years’ accumulated experience
• Java no longer enough
• Frustrations
JVM alternatives
• A plethora of languages
• Dynamically and statically-typed
• Groovy, Clojure, JRuby, Jython, Scala, Kotlin, Ceylon, Gosu
Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority.
Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261 6
7. Straw Poll
scala> val fibs: Stream[Int]
= 0 #:: 1 #:: (fibs zip fibs.tail map { case (x, y) => x + y })
fibs: Stream[Int] = Stream(0, ?)
scala> (fibs take 10).toList
res12: List[Int] = List(0, 1, 1, 2, 3, 5, 8, 13, 21, 34)
val mustReport = trades filter (uncoveredShort ∨ exceedsDollarMax)
val european = {
val Europe = (_ : Market).exchange.country.region == Region.EU
trades filter (_.market ∈: Europe)
}
def run
= checks.traverse[({type λ[α] = ValidationNEL[String, α]})#λ, Person](_ andThen lfn apply p)
for {
c :: _ <- run
}
yield c.age + 1.5
8. What is Scala?
An “object-functional language”
Fully object-oriented
• No primitives or operators at language level
– Compiles down to primitives in the bytecode
• No statics
• traits (interfaces with implementation)
Terse
• Type inference
Functional
• Function types (A ⇒ B) built in to the language, including closures
• Pattern matching and ADTs
• Fully-immutable collections library
• Syntactic support for monads and functors (for-comprehensions)
• Lazy evaluation (optional)
• Everything is an expression
– That goes for if/else, try/catch etc
Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority.
Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261 8
9. What is Scala?
Scala is broad where other languages are deep
Unrestrictive
• Symbolic identifiers (including unicode – e.g. ∃, ∀, ∈)
• Syntactic sugar
– Unary methods
– Infix notation (methods and types)
– Right-associativity allowed (e.g. 1 :: Nil where :: is a method invocation against Nil)
• Uniform access principle
• Arbitrarily-scoped definitions/values
Rich
• State of the art typesystem
– Self types, higher-kinded types, dependent method types, intersection types, abstract types
– Declaration-site variance annotations (covariance/contravariance)
• Hugely powerful collections library
• Implicits - values, type conversions, definitions
• Multi-threading support (actors, parallelism, STM – via libraries)
Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority.
Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261 9
10. What is Scala?
but most importantly…
It compiles down to bytecode
• Two-way Java interop
It runs on the JVM
• Just add scala-library.jar to the classpath
It is active
• Extensive ecosystem (SBT, Lift, Play, Scalate, Akka)
• New languages features
– Macros
– String interpolation
– Reflection
• It has an active community
– Hosted on github (make a pull request!)
– Docsprees
Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority.
Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261 10
11. The flip side
The problems of Scala
Binary incompatibility
• Libraries compiled against 2.9.x will not be compatible with 2.10.x
Unwelcoming “academic” community
• FP advocacy
• Do I need to learn Haskell first?
Too Complex!
• Intersection of features
• Unrestrictive syntax options can lead to confusion
• Places too high a burden on a library designer
Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority.
Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261 11
12. traits
Why Java’s collections are useless
Java interfaces contain no implementation
• Means API designers are miserly about adding methods to them
• Java.util.Collection defines 13 methods
• scala.collection.immutable.Traversable declares >90 methods
Java interfaces are a pain to implement
• If you want to implement Collection, you must implement 13 methods
– OK, you can extend AbstractCollection and implement 2 methods
• If you want to implement TraversableLike, you must implement 1 method
The scala collection library is unbelievably awesome
• Creating new collections from existing ones
• Getting data out of your collection
• Collapsing your collection (folds)
Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority.
Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261 12
13. scala> val isEven = (_ : Int) % 2 == 0
isEven: Int => Boolean = <function1>
scala> 1 to 10 filter isEven
res3: scala.collection.immutable.IndexedSeq[Int] = Vector(2, 4, 6, 8, 10)
scala> 1 to 10 filterNot isEven
res4: scala.collection.immutable.IndexedSeq[Int] = Vector(1, 3, 5, 7, 9)
scala> 1 to 10 span (_ < 3)
res5: (scala.collection.immutable.Range, scala.collection.immutable.Range) =
(Range(1, 2),Range(3, 4, 5, 6, 7, 8, 9, 10))
scala> 1 to 50 by 5 take 3
res6: scala.collection.immutable.Range = Range(1, 6, 11)
scala> Seq("London", "Paris", "New York") map (_.length)
res7: Seq[Int] = List(6, 5, 8)
scala> Seq("London", "Paris", "New York") zip res7
res8: Seq[(java.lang.String, Int)] = List((London,6), (Paris,5), (New York,8))
scala> List(1, 2, 3, 4).foldLeft(0)(_ + _)
res9: Int = 10
14. FX rates example
case class FxRate(from: Currency, to: Currency, rate: BigDecimal) {
trait MarketEnv {
def rate: Currency ⇒ Currency ⇒ Option[FxRate]
}
15. FX rates example
case class SimpleEnv(rates: Map[(Currency, Currency), FxRate])
extends MarketEnv {
def rate = from ⇒ to ⇒ rates get (from → to)
}
16. FX rates example
type CcyPair = (Currency, Currency)
case class FxRate(from: Currency, to: Currency, rate: BigDecimal) {
17. FX rates example
case class SimpleEnv(rates: Map[CcyPair, FxRate])
extends MarketEnv {
def rate = from ⇒ to ⇒ {
def attemptDirect(c1: Currency, c2: Currency) = rates get (c1 → c2)
attemptDirect(from, to) orElse (attemptDirect(to, from) map (~_))
}
}
18. FX rates example
type CcyPair = (Currency, Currency)
case class FxRate(from: Currency, to: Currency, rate: BigDecimal) {
def unary_~ = FxRate(to, from, 1 / rate)
}
19. FX rates example
case class SimpleEnv(rates: Map[CcyPair, FxRate])
extends MarketEnv {
def rate = from ⇒ to ⇒ {
def attempt(c1: Currency, c2: Currency)
= rates get (c1 → c2) orElse (rates get (c2 → c1) map (~_))
def viaUsd = {
val Usd = Currency.getInstance("USD")
for {
f2u <- attempt(from, Usd)
u2t <- attempt(Usd, to)
} yield f2u * u2t
}
attempt(from, to) orElse viaUsd
}
}
20. FX rates example
type CcyPair = (Currency, Currency)
case class FxRate(from: Currency, to: Currency, rate: BigDecimal) {
def unary_~ = FxRate(to, from, 1 / rate)
def *(that: FxRate) = {
require(this.to == that.from)
FxRate(this.from, that.to, this.rate * that.rate)
}
}
22. FX rates example
type CcyPair = (Currency, Currency)
case class FxRate(from: Currency, to: Currency, rate: BigDecimal) {
def pair: CcyPair = from → to
def unary_~ = FxRate(to, from, 1 / rate)
def *(that: FxRate) = {
require(this.to == that.from)
FxRate(this.from, that.to, this.rate * that.rate)
}
}
23. FX rates example
type CcyPair = (Currency, Currency)
case class FxRate(from: Currency, to: Currency, rate: BigDecimal) {
lazy val pair: CcyPair = from → to
def unary_~ = FxRate(to, from, 1 / rate)
def *(that: FxRate) = {
require(this.to == that.from)
FxRate(this.from, that.to, this.rate * that.rate)
}
}
https://gist.github.com/2028575
24. Functional Programming
Not so complex up to now?
One Extra Level Of Abstraction
• “Shapes”, “Patterns”, “Structures”
• Usually refer to some (embarrassingly simple) mathematical concepts (like monoids)
Typeclasses
• Scala does not have typeclasses
• This is why you should have asked Santa for a proper type system
– Java’s type system is so poor, most developers don’t realize it has one
– Scala’s type system includes higher-kinds
– Scala’s implicit resolution mechanism allows the compiler to solve your problems for you
Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority.
Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261 24
25. Monoids
Preparation ace
Definitions
• A Set with an associative operation and an identity under the operation
• Without the identity, this is a Semigroup
Defined trivially in scala as an interface
trait Monoid[A] {
def identity: A
def mplus(a1: A, a2: A): A
}
With some implementations marked as implicit
implicit val IntMonoid = new Monoid[Int] {
def identity = 0
def mplus(i1: Int, i2: Int) = i1 + i2
}
Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority.
Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261 25
26. “Pimp my Library”
Adding methods to any type
Retrofit functionality to anything
trait Id[A] {
val value: A
def mplus(b: A)(implicit m: Monoid[A]) = m.mplus(value, b)
}
object Syntax {
implicit def mkId[A](a: A) = new Id[A] {
val value = a
}
}
scala> import Syntax._._
scala> 1 mplus 2
res0: Int = 3
Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority.
Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261 26
27. Monoids
So, huh. You can add numbers
Monoids beget monoids
• If A, B … N all monoids
– the n-tuple (A, B … N) is a monoid
• If A is a monoid
– Then Option[A] is a monoid
• If V is a monoid
– Then Map[K, V] is a monoid
• If B is a monoid
– The function A ⇒ B is a monoid
This is actually really rather powerful
• How do we prove it?
Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority.
Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261 27
28. private def monoid[A: Monoid] = implicitly[Monoid[A]]
implicit def OptionMonoid[A: Monoid] = new Monoid[Option[A]] {
def identity = None
def mplus(o1: Option[A], o2: Option[A]) = (o1, o2) match {
case (Some(a1), Some(a2)) ⇒ Some(monoid[A].mplus(a1, a2))
case (x @ Some(_), None) ⇒ x
case (None, x @ Some(_)) ⇒ x
case (None, None) ⇒ None
}
}
implicit def PairMonoid[A: Monoid, B: Monoid] = new Monoid[(A, B)] {
def identity = (monoid[A].identity, monoid[B].identity)
def mplus(a1: (A, B), a2: (A, B))
= (monoid[A].mplus(a1._1, a2._1), monoid[B].mplus(a1._2, a2._2))
}
implicit def Function1Monoid[A, B: Monoid] = new Monoid[A ⇒ B] {
def identity = a ⇒ monoid[B].identity
def mplus(a1: A ⇒ B, a2: A ⇒ B) = a ⇒ monoid[B].mplus(a1(a), a2(a))
}
30. Academic nonsense
What use is this stuff in the real world?
Example 1: P&L
• Inventory P&L
– The amount I have made or lost on the position I started the day with
• Trading P&L
– The amount I have lost or gained on the trades I have done today
trait Position {
def investment: Investment
def tradingPnL: Option[Double]
def inventoryPnL: Option[Double]
final def totalPnL = inventoryPnL → tradingPnL
}
• Now, suppose I have a bunch of positions in my trading book, keyed by investment
Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority.
Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261 30
32. scala> List(‘a, ‘b, ‘c).msum
<console>:8: error: could not find implicit value for parameter m: Monoid[Symbol]
List('a, 'b, 'c).msum
^
scala> "abc".reverse
res1: String = cba
scala> Seq('a, 'b, 'c).reverse
res2: Seq[Symbol] = List('c, 'b, 'a)
https://gist.github.com/2028584
33. Conclusions
With great power comes great responsibility
Scala as a better Java
• Tuples, Functions & Closures
• Pattern-matching as a better “switch”
• Symbolic math
• Type Inference
• Amazing collection library
Scala as a functional gateway drug
• Write more modular code
• Lower cyclomatic complexity
• Delay side effects
What I left out
• Too much stuff
Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority.
Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261 33
34. Extras
for the masochists
This presentation
• http://www.slideshare.net/oxbow_lakes/building-financial-systems-in-scala
• (tinyurl): http://slidesha.re/xGfdww
• Runnable Code: https://github.com/oxbowlakes
References
• Martin Odersky keynote: http://days2011.scala-lang.org/sites/days2011/files/01.%20Martin%20Odersky.pdf
• Scalaz: http://code.google.com/p/scalaz/
• Shapeless: https://github.com/milessabin/shapeless
Examples
• Handling exceptions via types: https://gist.github.com/970717
• Scala collections examples: https://gist.github.com/1869377
Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority.
Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261 34
35. Contact us
GSA Capital Partners LLP
investor.relations@gsacapital.com
T +44 (0)20 7959 8850
London Office
Stratton House
5 Stratton Street
London W1J 8LA
T +44 (0)20 7959 8800
F +44 (0)20 7959 8845
New York Office
1140 Ave of the Americas
9th Floor
New York NY 10036
Private & Confidential. Not for distribution. GSA Capital Partners LLP is authorised and regulated by the Financial Services Authority.
Registered in England & Wales. Stratton House, 5 Stratton Street, London, W1J 8LA. Number OC309261 35
Editor's Notes
CLICK!
This talk is going to be about the building of financial systems in Scala – I’ll give a brief overview of what I think is the “state of the industry” at the moment and why a lot of people are looking beyond Java. Then I want to take a short look at a couple of the language features of scala which should hopefully whet the appetite to a deeper-dive to what a decent type system can do for youCLICK!
OK, so this is my background1 minuteCLICK!
A couple of months back, Doug Ward, who is head of business development at GSA gave a talk here at Imperial about the company. Essentially we are a technology-driven quant hedge fund. Practically every piece of software used in the company, and this amounts to hundreds of systems, we have built ourselves. I work in the CORE technology group, which is a lot more business-focused than it sounds – my “speciality” is on the post-trade execution side of GSA. Particularly reconciliation, allocation and P&L.2 minutesCLICK!
Is financial IT different?I’ve asked around, both within the company and outside, for people’s experiences and it seems that the technological fragmentation I can see is not hugely unusual. IT people everywhere are looking for the best tool for the jobAs for Java; did it ever “win”? Perhaps it is fairer to say that it “won” the first decade of this century, which saw a huge expansion in IT.3 minutesCLICK!
This means that there are now large numbers of experienced Java devs in the sector, and many are frustrated by the shortcomings of the language. Yet a decade of familiarity with the platform and its ecosystem, makes jumping off it more difficult. What are the frustrations? - a lack of high-level abstractions (closures, function types) - amount of repetition/boilerplate at all levels (e.g. getters/setters, type annotations) - they made a mistake when they added genericsIt’s been said that code is read much, much more than it is written, so having a language that allows you to state your intent as clearly is possible is the key. Java fails at this abjectly. Hence in the last few years there have been a number of JVM languages all promising to be “a better Java” – they are simply supplying a demand in the market. What are the sorts of features these languages have? - null-safe dereferencing, or avoiding null altogether - Reducing boilerplate - more correct type-system - type inference at some level - functions / closuresAlthough the financial sector has seen an explosion in the use of scala – both at large banks and at various hedge funds, it’s growth is no means confined to the sector – large famous scala adopters are Foursquare, Zeebox, Twitter, Yammer and the Guardian6 minutesCLICK!
How many of you in this room think you know what scala is? The reason I ask that is that I think there’s a feeling () that Scala is “Java with bells and whistles”. Alternatively, some experieced Java devs (like Stephen Colebourne and Gavin King) have looked at scala and decided that it’s too complicated. 8 minutesCLICK!
Scala is statically typed with local and non-local type inference. It contains a ton of ideas from functional programming and yet is more properly OO than Java. You can refer to variables within the lexical scope of a closure and even mutate them (i.e. reassign them). There’s no restriction to final-only variables like within a Java anonymous inner classThe distribution comes with something called the REPL, that is Read-Eval-Print-Loop. It’s a lot like the sort of dynamic console you’d find in languages like Ruby or Groovy, except that in scala, it is not dynamic at all. It compiles the input and runs it. Some of the examples you’ll see here (all of which are on github), where you see “scala>” are taken from the REPL10 minsCLICK!
Scala allows developers more freedom in pretty much every respect; in terms of naming choices, style choices - it avoids special syntax in favour of re-use. For example, an exception block in scala is just a partial function – it can even be *a reference to a partial function*. The freedom and features, such as implicits allow you to design libraries which look like parts of the language. They are not – they are libraries. Even break and continue are implemented as library constructsScala blurs the distinction between fields and methods; in a Scala interface (actually: trait), you can declare an abstract method which is then implemented in some concrete class with a value.12 minsCLICK!
It allows you to re-use your Java libraries; you understand the runtime. Indeed, there doesn’t have to be a bespoke runtime at all. The language is amazingly active, major versions appearing every year with updates which are a big deal. The latest, which will be out very soon, is to add reflection, macros among a ton of other stuff. It is dynamic where Java is bloated and unresponsive – although in fairness Oracle has really been pushing things forward in the last few years.According to the Scala Dev Team, the latest version has had the most work thrown at performance (of the compiler!) and at the Eclipse IDE – they are really serious about tooling13 minsCLICK!
So; all this does not come without a price. The fact that Scala straddles the imperative and functional-programming worlds means that, at times, its mailing list can be a kind of San Andreas fault. Quite a few experienced Java developers have not only decided that scala is too complex for them but they have decided to write their own language to achieve what they view as the same shortcomings (of Java). That is a huge undertaking, so they must really, really not like scala!The general feel is that it’s “too complex”; that the rope given to developers will be more than enough to hang themselves withWell, I’m going to start delving into some actual code now, so you can decide that for yourself14 minutesCLICK!
I’d like to talk for a minute about traits and how important they are for OO development. The lack of traits (interfaces with implementation) permeates Java to a huge extent. It’s the lack of traits that hobbles the Java collections API. By allowing an interface to have implementation, whilst removing the diamond inheritance problem, scala essentially chooses the “many styles” approach. The collection library has every method you ever wanted and then some more.Let’s have a look at some of them14 minutesCLICK!
Scala has both mutable and immutable collection libraries – in practice, I pretty much exclusively use the immutable collections. The hash trie (as seen in Clojure) was actually designed by Phil Bagwell, who is a member of the EPFL team. Immutability brings amazing benefits in terms of lack of bugs
Here we go with some basic stuff. Let’s define a class to represent FxRate. Notice the lack of boilerplate here – the class and its constructor are declared once. By using the “case” keyword against the class I specify that it gets equals/hashcode/copy constructors and an extractor. This would have been about 30-40 lines of Java.Let’s also define an interface which represents a lookup of Fx rates when given two currencies. In the next few slides, we are going to implement this15 minsCLICK!
OK, so here’s a simple implementation of the FX lookup interface. It takes a simple Map from keys, which are currency pairs to values which are Fx rates.It’s nice and simple; Map’s get method returns an Option. That is, the Map may not contain the pair, in which case we will get nothing, or None back.In the next few slides, we’re going to look at improving this. Firstly; let’s define a type alias to denote the Currency/Currency tuple and then let’s also think about how, if we have a rate for, say, EURUSD, we logically have a rate for USDEUR. We just need to flip it.17 minsCLICK!
OK, that’s pretty easy: let’s see our modified lookup17 minsCLICK!
What have we done here? Well, notice how we can embed defs within defs. Notice how we can chain calls on the Option datatype:attemptDirect returns an Option[FxRate]. orElse is lazy in its parameter, which means the RHS only gets evaluated in the None case.We’ve decided to co-opt the tilde here as a unary method to indicate that the Fx rate need be flipped19 minsCLICK!
So let’s implement the flip method. It’s pretty nice – there’s no need to declare return type; no need for braces in a 1 liner. A nice and terse method. Also notice the infix arithmetic with BigDecimal.Now; what else can we do? Well, typically Fx rates are declared against USD – if I have, say, a GBPUSD rate and a EURUSD rate, then obviously I can derive a EURGBP rate. Let’s implement that20 minsCLICK!
OK, slight refactor; I’m changing theattemptDirect to attempt, which will try to get a B to A rate if an A to B rate doesn’t exist. But now I’m using Scala’s language support for monads and functors. If I can get a rate from A to USD (either directly or indirectly), and I can get a rate from USD to B, then I can yield the transitively-implied rate. Here I have decided to call this method *, so I’m going to have to implement that in a secondThe for-comprehension is syntactic sugar for calls to underlying methods flapMap and map (and filter). Any data type which exposes flatmap (monad) and map (functor) can be used in a for-comprehension.Imagine what this would look like in Java. It would be if-elses, null-testing. Either one huge method or I would have to refactor into a class-level (private) method, even though that means I am polluting my class with functionality that does not need to be there.Anyway, let’s go back an look at our transitive rate definition…23 minsCLICK!
OK, so require is a nice little method available everywhere (also assume and assert), which will throw an IllegalArgumentException should the predicate resolve to false. You can optionally add a message there as well. Again I can use math infix operators on my BigDecimals.OK; that’s nice – let’s see it at work24 minsCLICK!
I’m creating a list of Fx rates and then turning them into a map using the collections library. How little code was that? And then I can get some transitive rates.If I try and get a rate which cannot be calculated, I get None back26 minsCLICK!
OK, lastly, just to finish this off I’m going to declare a definition which will give me back the currency pair for a given rate. Actually, hold on. I might want to change this – it might get called a lot and I’m going to be constructing a tuple time and time again. What I’ll do is make it a lazy val. Use-site is unaffected by this change27 minsCLICK!
There we go. The pair is now lazily computed when it is first accessed. And we’re done. A nice little example of using some of scala’s features to knock together a terse little programWhat I want you to take away from this talk is that scala is a lot like Java. Or at least it can be. You can write scala code just like you wrote Java code – you can omit types and let the compiler infer them; you can use a closure instead of a SAM interface and you can take advantage of the amazing collection library that the language ships with. But you’re still writing imperative code. I’m going to make the assumption that everyone here has the imagination to see that this is pretty nice. But what is really stunning is the ability to do *much* more gymnastics than this. For almost a year of programming scala, I used none of this stuff – but now I use it every single day27 minsCLICK!
So I want to talk a little about the functional side of Scala. You’ve seen a little of this already; some collection examples; using orElse and map on Option. But scala’stypesystem allows us to go further; much further. Those familiar with typeclasses in haskell might be surprised to find out that the scalatypesystem and implicit mechanism allows us to implement the same thing in scala. So what is a “typeclass”. Well, it’s extremely simple – think of it as a very basic piece of behaviour – a very simple interface. Except that you don’t require the class itself to actually implement/extend it. You can retrofit it later.29 minsCLICK!
OK – trivially integers under addition and zero form a monoid. As do integers under multiplication and 1. As do Booleans under disjunction and false, or under conjunction and true30 minsCLICK!
This is a small aside on a common mechanism in scala (actually shortly to involve less boilerplate) where we can add functionality to existing types. In this example, we are going to retrofit the mplus method (we could call it anything, even a symbolic name) onto any type A for which there is a monoid available. Firstly we declare a simple wrapper type, called Id here. The mplus method simply delegates the addition to the monoid. How do we use this? CLICK!Well, we have some mechanism by which we bring an implicit conversion into scope – in this case, we define our conversion in the Syntax module – which we can then selectively import. CLICK!Now we can write 1 mplus 2So what are we using here? Well, we are using infix style (i.e. missing periods and parenthesis); we are using the ability of scala to apply implicit conversions from one type to the next if a method is not found. We are taking advantage of multiple parameter lists; of the fact that parameters can be implicitly injected by the compiler *at compile time*33 minutesCLICK!
Well, all that seemed very nice; but you are obviously left with an important question. Namely; can’t we already add numbers? Can’t we already say “or” to booleans?Well, of course that is true but the real beauty in these things comes from the fact that monoids beget monoids; if you have a monoid for Int, you have a monoid for Option[Int]Now let’s prove those assertions by actually implementing the relevant monoidsCLICK!34 mins
OK, so firstly we are going to define a little helper method that’s going to give us the monoid instance for a given type A where A is a monoid. There’s no language trickery going on here – these are just methods, pure and simple.Firstly we’ll create the option monoid because handily it gives me a chance to show off pattern-matching, another of Scala’s killer featuresCLICK!What is the identity of the option monoid? Well, it’s None. CLICK! What is the mplus operation? Well, assuming that we had two values of type A, we use the monoid for A to add them and return the result enclosed in a Some. If we have only one Some, we can return it directly; if we have None on each side then we return None. Simple – it’s easy to see that the operation is associative and that None is indeed the identity under the op.Now let’s have a look at what the monoid looks like for tuples: CLICK! What is the identity for the tuple A/B? Well, if A and B are both monoids, it’s just the pair composed of the identities of A and B. CLICK! And what about mplus? Well, we’d take both tuples and add them pairwise. Simple.CLICK! For a Function1, we require that B is a monoid. The identity requires that we send any input value to the identity of B. CLICK! But how do we add two functions? Remember that the result is going to be a function – it will send its input to the monoidal sum of evaluating the input against each of the functions37 minsCLICK!
Here are these at work in the REPL. We can add options to options, tuples to tuples and Maps to maps. Notice how, in the case of Map, that where there is a key collision, the monoid is used to add the values at that key38 minutesCLICK!
OK – so now for a use-case for this. Imagine we are writing a system to calculate P&L; we may wish to split out the P&L we have made by trading and inventory P&L (assuming we had some). Now, let’s have a look at why monoids are useful40 minutesCLICK!
So let us suppose we have a map of positions keyed by investment. What if I want to have a look at the total inventory and total trading P&L across all of the positions. I can declare values by summing across all of these. All of what? Well, I am taking each value in the map, which is a Position, mapping it to its totalPnL which is a pair of Option[Double] and then summing these pairs of Option[Double]s all together via an msum method.Now, does msum method exist on List? Well, no, it doesn’t. SO we’ll have to add it, using the extend-my-library pattern. CLICK!As before, we can some across some collection of A’s assuming that there is a monoid for A. We fold the identity into the collection and accumulate using the binary operation. We then create an implicit conversion and it all ties together.But, what if we wish to look at our P&L aggregated across different traders’ books but still separated by investment? CLICK! Well, this is trivial, because Map is a monoid.If anyone is doubting the usefulness of this kind of thing, I hope this makes it a lot clearer – this kind of code exists in our risk systems, our allocation algorithms. Last August, during the uncertainty surrounding the Greek bailout, this helped me get our head of trading an ad-hoc view of our exposure across a number of books as we were looking into our counterparty risks. It took a handful of characters to achieve what in Java would have been a doubly-nested loop with mutable collections; prone to bugs when under pressure.CLICK!44 minutes
Let’s just take a closer look and see what happens when we try and call msum on a collection of a non-monoidal type.CLICK!It *does not compile*. Here we have a method on a Traversable[Int] or a Traversable[Double] but which does not exist on a Traversable[Symbol]. That is truly amazing. The collection library, as an aside is full of fantastic stuff like this. CLICK! For example, there exists a method called reverse which can be called on any sequence. This method has a single declaration. You can call the method on a java String and it returns a String, yet on a scala Sequence it returns a sequence. 45 minutes
So what have I tried to show you here? Well, what I have *attempted* to demonstrate is that Scala can be written a lot like a slightly nicer Java. But also I hope that I’ve shown how the very simple concepts of implicit conversions and implicit parameters, taken together with a proper type system, open up amazing new possibilities that are simply unachhievable in dynamic languages and unexpressable even in statically-typed ones.I have left out quite an astonishing amount of stuff; I have not covered higher kinds, or exception handling with types. I have not covered in any real detail the collections library, I’d urge you to look at martin odersky’s keynote from the 2011 scaladays talk; I’ve not covered parallel collections, the actors library for concurrent, event-driven systems. I haven’t talked about Play and LiftCLICK!46 minutes
