Presented at JRubyConf EU August 2012

1. CONCURRENCY
AND DISTRIBUTED
SYSTEMS IN JRUBY
@iconara

2. speakerdeck.com/u/iconara
(real time!)

3. Theo / @iconara

4. Chief Architect at

5. big data with Ruby since 2009

6. MAKING THIS

7. INTO THIS

8. big data with Ruby since 2009

9. APRIL 2011
stuck

10. MAY 2011
let’s try JRuby

11. JUNE 2011
wrapping Java libraries for fun and for profit

12. HOTBUNNIES
natural Ruby API for RabbitMQ

13. JULY 2011
threads are better than processes

14. JRUBY

15. JRUBY

16. J.U.C
(java.util.concurrent)

17. SHORTCUT
Java Concurrency in Practice

18. EXECUTORS
configurable thread pools, a better
abstraction than using Thread directly

19. thread_pool = Executors.new_fixed_thread_pool(16)
all_the_things.each do |thing|
thread_pool.submit do
crunch_crunch_crunch(thing)
end
end

20. BLOCKING QUEUES
easy producer/consumer patterns

21. J.U.C.ATOMIC
optimistic locking primitives,
compare-and-swap,
thread safe non-blocking counters

22. MORE AWESOMES
ConcurrentHashMap, CountDownLatch,
ForkJoinPool, Google Guava, LMAX distruptor

23. TL; DR
use j.u.c and avoid mutable state
and you’re safe, mostly

24. THREADS
enough rope

25. AUGUST 2011
threads are a mess, let’s try actors

26. ACTORS
a higher level abstraction of concurrency

27. AKKA
concurrency library for Scala,
famous actor implementation

28. MIKKA
Akka wrapper for JRuby

29. DO ALL THE THINGS
Programming Concurrency on the JVM

30. AUGUST 2011
async systems with no back pressure
mechanism will die of memory starvation

31. INPUT
AGGREGATION TIMER
STATE PERSISTENCE
OUTPUT

32. INPUT
AGGREGATION TIMER
! STATE PERSISTENCE
OUTPUT

33. AUGUST 2011
actors are cool, but not really
worth it when all the arrows in your
flow chart point in the same direction

34. DIVIDE DIVIDE DIVIDE
run multiple independent, synchronous workers,
tune to the number of CPUs & IO wait

35. SEPTEMBER 2011
blocking all the way down,
back pressure all the way up

36. AUGUST 2011
(short rant about MongoDB’s global write lock)

37. NEW SHINY TOYS
for building new shiny products

38. CASSANDRA
distributed database, mostly magic

39. ColumnFamilyUpdater<String, String> updater = template.createUpdater("a key");
updater.setString("domain", "www.datastax.com");
updater.setLong("time", System.currentTimeMillis());
try {
template.update(updater);
} catch (HectorException e) {
// hurgh
}
CASSANDRA
very verbose drivers

40. new_data = {:domain => 'www.datastax.com', :time => Time.now.to_i}
column_family.update('a key', new_data)
EURYDICE
JRuby nails it

41. STORM
stream processing framework written in Clojure

42. STORM
(it does the arrows, you do the blobs)

43. REDSTORM
Storm for JRuby

44. YOUR CODE
DON’T DO
THIS AT HOME KIDS!
the key to building concurrent and distributed
systems is to separate what you want to do from the
concurrent- and distributed-ness, and let someone
else take care of that

45. JRUBY IS A SLIPPERY
SLOPE TO JBOSS
let there be no doubt about it

46. SCALING IS HARD
go to Srdan’s talk tomorrow

47. KTHXBAI
@iconara
github.com/iconara
architecturalatrocities.com
burtcorp.com