An overview of the Fly/Flight space based systems, and an introduction into integrating with the Play 2 Framework in Scala

1. Take Flight : Fly Object Space
Nigel.Warren@underscoreconsulting.com

2. Themes
• Spaces
• Fly Object Space
• Flight
• Fly and Play Demo
• Roundup
https://github.com/fly-object-space google : Fly Object Space

3. Spaces
Tuple Spaces – Linda - a co-ordination language
Presents a boundary for signals between …
Threads - Processes – Machines - Systems
Minimal Interface
3 Essential Operations
Timely – Lease based
Immutable Only

4. Fly
Operations
op
op
op
write(entry, lease ): lease
read(template, lease): Option[entry]
take(template, lease): Option[entry]
Query By Template
Template
Entry
Car(Some("Red"), None) == Car(Some("Red"), Some(5))
Car(None, Some(7))
!= Car(Some("Red"), Some(5))
Car(None, None)
== Car(Some("Red"), Some(2))

5. Write Op
def write(entry: AnyRef, lease: Long) : Long

6. Read Op
def read[T<: AnyRef](template: T, lease: Long): Option[T]

7. Take Op
def take[T<: AnyRef](template: T, lease: Long): Option[T]

8. Time Passes
Leases Expire

9. Design Diamond
Minimise Interface
Complexity
Minimise Uses

12. Flight
Idea : Advances in core Scala libraries can be applied to Fly
Futures
Ops are time constrained
Ops may succeed or fail
FiniteDurations
Express only finite leases in nanos (prev millis)
Make a JVM local ( inter-thread ) version that has the new
features as a test bed.

13. Flight
Experimental
Image : Experimental Aircarft - Woolf Sverak – www.redbubble.com

14. Flight
write[T <: AnyRef](entry: T, lease: FiniteDuration) :
Future[FiniteDuration] = …
read[T <: AnyRef](template: T, lease: FiniteDuration) :
Future[T] = …
take[T <: AnyRef](template: T, lease: FiniteDuration) :
Future[T] = …

15. Flight – Example
import
import
import
import
scala.concurrent.ExecutionContext.Implicits.global
scala.concurrent._
scala.concurrent.duration._
scala.language.postfixOps
import com.zink.fly.{ Flight => flt }
import com.zink.fly.examples.Price
//case class Price(symbol: Option[String], value: Option[Int])

16. Flight – Example
val offer = Price(Some("ARM"),Some(123))
flt.write(offer, 100 seconds) onSuccess {
case lease => println(s"Offer written for $lease")
}
"Offer written for 100 seconds"

17. Flight – Example
val tmpl = Price(Some("ARM"), None)
flt.read(tmpl, 10 seconds) onComplete {
case Success(entry) => println(entry)
case Failure(t) => println("Sorry: " + t.getMessage)
}
flt.take(tmpl, 10 seconds) onSuccess {
case ent => println(s"Take matched $ent")
}

18. Flight – Prop
val tmplArm = Price(Some("ARM"),Some(123))
val tmplWlf = Price(Some("WLF"),Some(321))
val lse = 10 seconds
val futArm = flt.read(tmplArm, lse)
val futWlf = flt.read(tmplWlf, lse)
(futArm zip futWlf) onComplete {
case Success(e) => println("Deal")
case Failure(t) => println("No Deal")
}

19. Fly – Demo

20. Fly – Summary
• Timely Distributed Computing
• Core Scala Library only
• Work with everything else
• Flight (Experimental) Interface
• FiniteDurations
• Futures
https://github.com/fly-object-space google : Fly Object Space

21. >
<------------>
<--->
Thanks
www.flyobjectspace.com
www.twitter.com/flyobjectspace

22. Playing with Fly
Asher Glynn
asher.glynn@burrowingdeep.com

23. Overview



Used Play 2.0 framework last year for startup
Simple as possible – database and app servers
Adapted to leverage Fly

24. Starting out with play

My journey





Copy an example
Write logic in Action handlers
Write DAO layer
Discover you need an API
Rewrite with quite a lot of swearing

25. Abstracting the logic
def Story[Request, Response](request: Request)
(logic: Connection => Response)
(implicit
validators: Seq[Request => Connection =>
Option[StoryError]],
reactors: Request => Response =>
Seq[Notification]) : Either[StoryError, Response]
= {
… // Handle transaction
}

26. Notes on Story abstraction





Use request/response to (potentially) allow for Akka
distribution
Handles overall transaction
Validators execute sequentially prior to logic
Reactor executes afterwards and asynchronously writes
notifications (if any) outside transaction context
Might have gone too far with implicits

27. Notifications – with DB





Put an adapter in but never did anything with them
Inserting into DB easy
Querying reasonably painful
NoSQL + Message Bus non trivial + complex in fast
moving startup
Eventually disabled clients

28. Notifications - With Fly




Easy filters
Get notifications without polling with notifyWrite
Can listen for interesting notifications out
Trivial to implement via writeMany

29. Closing auctions




Auction system needs once only transaction
Late auction is a bad one (unlikely to replay in case of
long outage)
Only winning bids translate to transactions
All nodes capable of closing transaction (for redundancy)

30. Closing auction - database

Various options – easiest holding write lock through a
select for update
def pin(auctionId: UUID) = SQL ("Select * from
TA_LOCKS where LOCKABLE = 'CLOSER' FOR
UPDATE").execute




Locks out other nodes
Could be finer grained
Needs to have extra timeout code setup to execute
predictably
Only once by virtue of changed state in DB (need to
check)

31. Closing auction - Fly

Contrast in Fly
fly.take(new models.auction.Auction(id = auctionId),
1000L)


Timeouts “for free”
Only once by virtue of getting the take

32. Integrating Fly



Upgrade to current version of Play
Add fly-java lib to /lib folder
Add to dependencies in Build.sbt
"com.flyobjectspace" %% "flyscala" % "2.1.0SNAPSHOT”

And you are away

33. Modifications for Fly

Modify constructor of class for null fields
case class Auction (
id
: UUID = null,
accountId
: UUID = null,

Add trait to indicate well formedness
trait WellFormed {
def isWellFormed : Boolean
}

34. Modifications for Fly

And add check for WellFormedness to each story


Add to validation chain
Add to main part of story

Use notifications to invalidate cache

Write auctions to Fly, use take for once only

35. Mistakes integrating with Fly




Going nuts with Akka
Messing up lease times and objects disappearing
Lease time too short
Adding 2PC complexity to thinking rather than working
within the Fly idiom

36. Running out of talent…



Use either presence of object in Space to indicate lock –
or absence!
For Auction – absence of auction object
For Account modifications – presence of object



What if there are two
What if the owner disappears
What if the ownership changes on an edge (split brain)

37. Longer term modifications





Modify Story abstraction to work with Futures
Modify to work cleanly with account balances
Keep database but redesign to work nicely with Fly
Keep notifications out of database
Flatten notification structure

38. Final note on notifications

Currently using a deep class
case class Notice( id: UUID,
originatorId: UUID,
timestamp: Long,
attributes: Map[String, String],
to: Seq[UUID],
system: Boolean)

Will flatten to help leverage Fly filters
case class Notice (subject: String = null,
_predicate: String = null,
_object = String)
And correlate on receivers

39. Thanks