A Slice of Scala

A Slice of
Scala
August 2013

Kevin O’Neill
CTO PlayUp
@kevinoneill

Kevin O’Neill - CTO PlayUp - @kevinoneill

Melbourne Scala User Group - August 2013

A little History



Engage & Entertain
Around Live Sport



Mobile Games



SMS Based



Number of Runs



Complex



Carrier Integration



Smart Phone Revolution



Move from a Content
Creator to a Content
Enabler


Top 10 in Engagement



Over 3 Million Likes



30% - 40% Active



Manually Curated



We Know there's
an Opportunity



October 2011



Technologist at Large



Version 2



Rebuild Based on
Hypermedia*



Same Feature Set



Different Skin



iOS



Android



Ruby



Version 3



Platform Extensions



Tiles



Third Party



Ticketing



PHP



Version 4



Engagement Too Deep



Streams



Graph Storage



Story

User
Region

Story
follows

about

about
about

follows

Team
Team

Story

Region
competes in

League

competes in

plays in

Team
follows

competes in

follows

about
about
about

Contest

League

User
Story
Story

follows

competes in

plays in

about

Story
Story

Team
about


Story

Story

about


Value is in the
Connections



50 Seconds



Cypher Processing
too much Data



Java plugin



5 Seconds



10 Times too Long*



Aiming Around 200ms



Time to Change



If you hear a voice
within you say 'you
cannot paint,' then by
all means paint, and
that voice will be
silenced.
– Vincent Van Gogh


Scala



Just Couldn't Face
Java



Some Experience with it
on a Large Project



The Stack



Scala 2.10.x



akka 2.1.x*



Spray M8+



Play 2.1.x*



Other Bits



Neo4j 1.9.x



Scalaz 7.0.x



Blueprints 2.4.x



Scala Test 2.0.x



Scala Check 1.10.x



SBT 1.12.x*



Systems



Two akka Services*



Graph Service



Sports Connect



Fanbase Connect*



One Spray Server



Hypermedia API



One Play Application*



HTML Front End



Admin Front End*



JSON Generation



Problem : How do you
isolate environment



Lambda: The Ultimate
Dependency Injection
Framework



Did my Head in



So what if …



Environment Readers
type
Settings
=
Set[EnvironmentSetting[_]]

!

type
EnvironmentReader[+A]
=
scalaz.Reader[Settings,
A]

object
EnvironmentReader
{

def
apply[A](f:
Settings
=>
A):EnvironmentReader[A]

=
scalaz.Reader(f)
}



Writing an Identity

protected
def
writeResourcePointer(identity:
ResourceIdentity)

:
EnvironmentReader[JObject]
=
for
{
!

href
<-‐
urlForIdentity(identity)

contentType
=
identity.documentType

representations
<-‐
writeRepresentations(identity)

views
<-‐
viewsForIdentity(identity)
!

}
yield
(":href"
-‐>
href)
~

(":type"
-‐>
contentType)
~

(":uri"
-‐>
identity.uri)
~

representations
~

(":views"
-‐>
views)



urlForIdentity
protected
def
urlForIdentity(identity:ResourceIdentity)

:
EnvironmentReader[JValue]
=
for
{
!

r
<-‐
URLResolverSetting.resolve(identity)

href
<-‐
mapOrJNothing(r)
{

ref
:
String
=>
pure(JString(ref))

}
!

}
yield
href



URLResolverSetting
case
class
URLResolverSetting(value:
Resolver[String])

extends
EnvironmentSetting[Resolver[String]]
{

val
key
=
"url-‐resolver"

}

!

object
URLResolverSetting
extends
ResolverSetting
({

case
setting:
URLResolverSetting
=>
setting.value

})



ResolverSetting
class
ResolverSetting[T](

finder
:
PartialFunction[

EnvironmentSetting[_],

Resolver[T]])
{
!

def
resolve(identity
:
ResourceIdentity)
:

EnvironmentReader[Option[T]]
=
EnvironmentReader

{
settings
=>

for
{

resolver
<-‐
settings.collectFirst(finder)

result
<-‐
resolver.resolve(identity)

}
yield
result

}

}



Wooh …
That's far more
complex than my …
blah blah in blah


Yup



Nope



Remember I
mentioned Runars talk
did my head in.*


It's actually really
easy to work with



Each components is
responsible for one
thing



Combining
components is trivial



You spend most of
your time doing
simple, safe,
composition


Spray



Elephants all the way
down



The Language
val
route
=
{

language
{
lang
=>

implicit
val
env
=
RequestEnvironmentProvider(

EnvironmentConfig(language
=
Some(lang))

)
…
!

}



The Directive*
def
language:
Directive1[Lang]
=
{

headerValue
{
!

case
HttpHeader("accept-‐language",
langString)
=>

langString.split(',').headOption.flatMap(firstLanguage
=>

Lang.get(firstLanguage))
!

case
_
=>
None
!

}
|
provide(Lang.defaultLang)

}



Creating a Response
def
getStream(implicit
env:
RequestEnvironmentProvider):

Route
=
{

path("stream"
/
StreamPath)
{
case
(stream,
afterStory)
=>

read(FetchStream(stream,
afterStory))
{
stream
=>

val
streamEnv
=
environmentForStreamWithTopics(

env,
stream.identity.references

)

respondWithHeaders(

`Cache-‐Control`(`public`,
`max-‐age`(60)))
{

complete
{

implicit
val
criteriaMarshaller:
Marshaller[StoryStream]

=
resourceMarshaller(streamEnv)

stream

}

}

}

}

}



A Little akka Helper



Reading the Graph
def
read[T](message
:
GraphRequest[T])(f
:
T
=>
Route):Route
=
{
ctx
=>

val
response:
Future[T]
=

Graph.reader.ask(message).asInstanceOf[Future[T]]

response.map
{

case
Error(messages)
=>
{

ctx.complete(

InternalServerError,
messages.mkString("n")

)

}

case
result
=>
f(result)(ctx)

}

}



akka



Topic Management



Topic Actor
private
class
TopicActor
(system
:
GraphSystem,

topic
:
StreamComponentIdentity)

extends
GraphWorker(system)
with
Stash
{

!

override
def
preStart()
{

available(topic)
onAvailable
{

become(active)

}
onMissing
{

topicManager
!
UpdateTopic(topic)

become(pending)

}

}

!

…
}


Receive

def
receive
=
PartialFunction.empty



Pending
def
pending
:
Actor.Receive
=
{

case
message
@
LinkStory(story,
identity)

if
identity
==
topic
=>
{

stash()

}

!

case
identity
if
identity
==
topic
=>
{

unstashAll()

become(active)

}

}



Active

def
active
:
Actor.Receive
=
{

case
LinkStory(story,
identity)
if
identity
==
topic
=>
{

gs.write(associateWithSubject(story,
topic))

}

}



Baby Steps



Challenges



As the Carpenters
said …



We've Only Just Begun



Data Load



Sparsity



Propagation



Friends



500 Topics



10,000,000 Nodes



1,000,000 Topics



10,000,000,000 Nodes



Time Series Data



Distribution



Need to change the
way you think about
services



Actors are not Queues



Monitoring is Hard



"Real Time"



Balancing
Load vs Caching



Two Hard Problems



Naming



Cache Invalidation



Off by One Errors



Caching Stuff is Easy



Invalidation is Hard



Limited Use



Just be Fast



Payments



À la carte



Cross Platform



Come Join Us



Disrupting



Looking for the 'X'
factor



Hiring Selectively



Talk to Me or Andrea



You miss 100 percent
of the the shots you
don't take
– Wayne Gretzky


Questions?
Kevin O’Neill
CTO - PlayUp
@kevinoneill
kevin@playup.com



A Slice of Scala

Recommended

Recommended

More Related Content

What's hot

What's hot (6)

Viewers also liked

Viewers also liked (6)

Recently uploaded

Recently uploaded (20)

A Slice of Scala