This presentation from the November 17, 2011 Boundary Meetup takes us through the architecture of Boundary's stream processing infrastructure and how the architecture is pushing the bounds of JVM throughput.

1. Garbage, Garbage Everywhere
GC Strategies for Event Processing Systems on the JVM
C. Scott Andreas
Pizza, Beer, and Tech Talks
November 17, 2011

2. What’s ESP / CEP?
• Event Stream Processing
Selecting events on dimensions among a stream of moving
data, maintaining them for a brief period, emitting aggregations.
• Complex Event Processing
Identifying correlations between events, predicting trends,
and programmatically reacting to emergent trends.

3. ESP and Network Analytics
• Packet flows are event streams with many dimensions.
• Blast them into the engine, select over the stream,
emit aggregations based on queries.
• Ipfix data flows in, JSON comes out.

4. Back of the Envelope
• 500 Mbps / sec data comes into the JVM
xxx Mbps / sec data goes out of the JVM
• This memory must be allocated, retained
for processing, freed, and collected.
• Actual allocation rates far higher than data in / out
(Memory also used for deserializing, aggregations, etc).

7. Opening the Grimoire
• GC Tuning to the rescue
• Oracle guide with 84 -XX: options
• Stas’s guide: 830 options (OpenJDK debug builds)
http://www.oracle.com/technetwork/java/javase/tech/vmoptions-jsp-140102.html
http://stas-blogspot.blogspot.com/2011/07/most-complete-list-of-xx-options-for.html

8. Moment of Pause
• Don’t touch the knobs unless you need to
• Server defaults are a decent place to start for local development
• Defaults shipped with Cassandra decent for bimodal GC profiles
• Basic rule of thumb: if you’re aggressively tuning garbage
collection, you can trade hours of frustration for ~10% gain

9. Generational Garbage Collection
• Modern JVMs divide heap space up into multiple “generations.”
• Most applications have a lot of objects which live for a very short
time, and a lot which live (nearly) forever.
• Generational collection enables the JVM to collect unused
memory more efficiently by avoiding unnecessarily scanning
heap / object graphs for references or free regions.

10. first attempt: “deploy the g1”

11. Low-Pause Collector: The G1
v% => threshold violations. lower is better.

13. G1 Collector
• Hundreds of tiny 1ms collections / second rather than
ParNew’s ~100 - 200ms larger collections.
• Capable of meeting ambitious pause targets.
• Powered by a gang of threads working in parallel
• ...cooperating to chew through CPU like it’s free.

14. second attempt:
you’re gonna laugh

15. the unsafe

16. Unsafe
• A OpenJDK/HotSpot class exposing direct access to the
underlying VM, OS, and memory.
• This includes the ability to allocate, manage, and free memory.
• Perhaps we can outsmart the JVM and do a better job than it!

20. Learned While Astray
• Finalization occurs in a single thread.
• Jumping from native finalization back into Java is expensive.
• Attempting to outsmart the garbage collector by creating
hundreds of thousands of tiny ByteBuffers is...a thing.
• Java’s collectors are very good at collecting garbage.
Your home-grown in-app GC go-kart is probably not.

21. returning to earth ::
attempt 3[a]

22. Lessons from Science
• Your rate of “freeing” must be equal to or exceed your rate of
object allocation on the heap.
• High rates of allocation speed up heap fragmentation,
which compounds the problem.
• Creating less garbage reduces your rate of allocation
(and freeing).
• This means less work for the garbage collector.

23. best way to help out the gc ::

24. PRODUCE LESS
GARBAGE

25. Breaking out YourKit

26. attempt 3[b]:
responsible tuning of the old hat

27. Optimizing for Infant Mortality
default newgen ratios in java 6
• Java 6 AMD64 (server) defaults to allocating
1/3 of heap to the new gen, 2/3 to the old gen.
• ESP/CEP workloads place tremendous pressure on the newgen.
The vast majority of objects survive less than five seconds.
• Experiment: Allocate 80% of heap to the new gen, set a higher
tenuring threshold, and lean hard on the ParNew collector.

28. CMS Collector
• Guardian of the tenured generation, favorite workhorse for years.
• Primarily parallel, easier on the CPU than the G1.
• ...But contains a significant pause phase, is less suited to
meeting low pause targets.

29. ParNew Collector
• Designed for the small, but works great in the large.
Excellent throughput, parallel collection.
• Can collect ~5GB in ~200ms on a quad-core Xeon w/HT.
• 200ms pause every several seconds favorable compared to
less frequent multi-second pauses and promotion failures.

30. Explosions in the Barrel

33. “real-time” and the jvm

34. Real Time and the JVM
• Real Time
Ability to meet specific targets with low variance is critical to the
bare minimum functionality of the product (e.g., air bags).
• “Soft” Real Time
Ability to meet targets important but not critical. Value of
system’s functionality is diminished but not eliminated by delay.

35. Real Time and “The Pause”
• To what extent can a system which can endure pauses of
unpredictable duration be considered “real-time”?
• Is it sufficient to mitigate the frequency and duration of pauses
for a system to still deliver value as “soft real-time”?
• Is the alternative worth the cost?

36. what does your app sound like?

37. Garbage, Garbage Everywhere
GC Strategies for Event Processing Systems on the JVM
C. Scott Andreas
Pizza, Beer, and Tech Talks
November 17, 2011