Transparent failover as a technique to improve reliability of mission critical systems is not usable with web applications that rely on additional threads in their HTTP sessions. RAP applications fall into this category of web applications. Sovereign is a research project by EclipseSource in conjunction with the Technical University Munich to develop a solution for the above problem. In this session we discuss two solutions under development in the Sovereign project. The first solution utilizes byte code weaving with AspectJ in order to connect the RAP application with Terracotta, a distributed heap for Java objects. The second solution being explored is to extend the Maxine VM by a feature to replicate/migrate objects and threads between VM instances. We describe their advantages and disadvantages for overcoming the problem of immobile Java threads that prevent transparent failover for RAP applications. We demonstrate how to transfer the state of a running RAP application from one server to another using both methods.

1. Strategies Recording Modified VM Outlook
Sovereign:
Migrating Java Threads
to Improve Availability of Web Applications
Johannes Eickhold, Technical University Munich
Markus Knauer, EclipseSource
Dr. Thomas Fuhrmann, Technical University Munich
March 22, 2011
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 1 / 34

2. Strategies Recording Modified VM Outlook
What is high availability for web applications?
Load Balancer
Cluster
Servlet Container
Every node can take over
Only works if app is stateless
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 2 / 34

3. Strategies Recording Modified VM Outlook
What is RAP?
RAP supports
Reusing RCP’s UI thread → single sourcing
Job framework eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 3 / 34

4. Strategies Recording Modified VM Outlook
Challenge: Transfer Server Side State
Load Balancer
sticky sessions!
no-fail-over = true
Cluster
S S S
Servlet Container
UIThread
UIThread
UIThread
Goal
Enable failsave operation of all RAP features
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 4 / 34

5. Strategies Recording Modified VM Outlook
Overview
1 Strategies
2 Recording
Strategy Explained
Example
Demo
3 Modified VM
Strategy Explained
Demo
4 Outlook
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 5 / 34

6. Strategies Recording Modified VM Outlook
Overview
1 Strategies
2 Recording
Strategy Explained
Example
Demo
3 Modified VM
Strategy Explained
Demo
4 Outlook
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 6 / 34

7. Strategies Recording Modified VM Outlook
Two Strategies for Enabling Session Failover
1 Recording Strategy
Records the widget state to shared heap
Operates on application level
2 Modified VM Strategy
Enables thread migration on the JVM level
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 7 / 34

8. Strategies Recording Modified VM Outlook
Recording Strategy
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 8 / 34

9. Strategies Recording Modified VM Outlook
Modified VM Strategy
licate threads
rep
primary VM secondary VM
freeze thread + replicate state and data + continue thread =
thread migration eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 9 / 34

10. Strategies Recording Modified VM Outlook
Recording Strategy vs. Modified VM Strategy
UIThread UIThread
record replay
byte code weaving unmodified application
unmodified Hot Spot VM enhanced Maxine VM
Terracotta shared heap object and thread replication
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 10 / 34

11. Strategies Recording Modified VM Outlook Strategy Explained Example Demo
Overview
1 Strategies
2 Recording
Strategy Explained
Example
Demo
3 Modified VM
Strategy Explained
Demo
4 Outlook
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 11 / 34

12. Strategies Recording Modified VM Outlook Strategy Explained Example Demo
Recording Strategy
Modifications to existing RAP framework
1 Record widget creation order on primary
2 Replay recorded widgets during creation phase on
secondary
3 Cluster the session including widget tree
4 Remove non-serializable UI thread from session
Incrementally add failover support for additional RAP widgets
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 12 / 34

13. Strategies Recording Modified VM Outlook Strategy Explained Example Demo
Lifecycle of a RAP application
UI Thread Shared Heap
request
Client
Initial request creates the UI thread
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 13 / 34

14. Strategies Recording Modified VM Outlook Strategy Explained Example Demo
Lifecycle of a RAP application
UI Thread Shared Heap
request
Client
UI thread executes createUI() and creates widget graph
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 13 / 34

15. Strategies Recording Modified VM Outlook Strategy Explained Example Demo
Lifecycle of a RAP application
UI Thread Shared Heap
record
request
Client
The widget graph is stored on the shared heap
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 13 / 34

16. Strategies Recording Modified VM Outlook Strategy Explained Example Demo
Lifecycle of a RAP application
UI Thread Shared Heap
response
Client
Servlet responds with rendered UI to client.
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 13 / 34

17. Strategies Recording Modified VM Outlook Strategy Explained Example Demo
Hot Standby Takes Over
UI Thread Shared Heap
FAILURE
request
Client
Primary servlet container fails and secondary jumps in
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 14 / 34

18. Strategies Recording Modified VM Outlook Strategy Explained Example Demo
Hot Standby Takes Over
UI Thread UI Thread Shared Heap
FAILURE
request
Client
Primary servlet container fails and secondary jumps in
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 14 / 34

19. Strategies Recording Modified VM Outlook Strategy Explained Example Demo
Hot Standby Takes Over
UI Thread UI Thread Shared Heap
FAILURE
request
Client
Skip widget creation in new UI thread!
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 14 / 34

20. Strategies Recording Modified VM Outlook Strategy Explained Example Demo
Failover Situation
UI Thread UI Thread Shared Heap
FAILURE replay
request
Client
Widgets are restored from the shared heap
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 15 / 34

21. Strategies Recording Modified VM Outlook Strategy Explained Example Demo
Failover Situation
UI Thread UI Thread Shared Heap
FAILURE
response
Client
We achieved transparent failover!
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 15 / 34

22. Strategies Recording Modified VM Outlook Strategy Explained Example Demo
Implementation
You could implement this yourself
or
reuse existing technologies.
AspectJ is used to:
Record order in which widgets are created
Track method calls to enable method skipping
Terracotta provides a shared heap
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 16 / 34

23. Strategies Recording Modified VM Outlook Strategy Explained Example Demo
Recap of Recording Strategy
Share Record
TIMs: Equinox, Tracing widget
Jetty creation with
Configure RAP AspectJ
(tc-config.xml) Skip methods
on secondary
No silver bullet!
RAP application has to be designed for failover
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 17 / 34

24. Strategies Recording Modified VM Outlook Strategy Explained Example Demo
Jump to the example
Example RAP application: "swt layout"
Aspects implementing the recording/replay
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 18 / 34

25. Strategies Recording Modified VM Outlook Strategy Explained Example Demo
SWT Layout Example
1 Label l = new Label (shell , SWT.NONE );
2 l. setText (" relocate me!");
3 ...
4 Button b = new Button (shell , SWT.PUSH );
5 b. setText ("left");
6 b. addSelectionListener (new SelectionAdapter () {
7 public void widgetSelected ( SelectionEvent e) {
8 localCounter ++;
9 globalCounter ++;
10 l. setText (" relocated "
11 + localCounter + "/" + globalCounter
12 + " times ");
13 }
14 });
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 19 / 34

26. Strategies Recording Modified VM Outlook Strategy Explained Example Demo
Trace RAP Widget Creation and Method Calls
UIThread Button shared heap
UIThread Button shared heap
new()
new()
store
restore
setText()
setText()
update
On secondary
On primary
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 20 / 34

27. Strategies Recording Modified VM Outlook Strategy Explained Example Demo
Recording Demo
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 21 / 34

28. Strategies Recording Modified VM Outlook Strategy Explained Demo
Overview
1 Strategies
2 Recording
Strategy Explained
Example
Demo
3 Modified VM
Strategy Explained
Demo
4 Outlook
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 22 / 34

29. Strategies Recording Modified VM Outlook Strategy Explained Demo
Modified VM Strategy
Extensions to the Maxine research VM
Object replication
Thread replication
All VMs in web cluster execute:
MigrationAgent + target application
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 23 / 34

30. Strategies Recording Modified VM Outlook Strategy Explained Demo
Maxine VM Internals
Maxine VM is written in Java (metacircular)
Threads
Uses pthreads
One-on-one mapping: Java ↔ pthread
Safepoint operations inserted at backward branches
Freezing of threads only via VmOperation
Compilers
Baseline: template JIT
Optimizing: port of C1X from HotSpot or CPS
Details in wiki: http://wikis.sun.com/display/MaxineVM
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 24 / 34

31. Strategies Recording Modified VM Outlook Strategy Explained Demo
Migration via Agents
RAP clustering
Agents run next to Equinox in same VM
Replication only used for UI threads of RAP sessions
migrationAgent.migrate(uiThread); eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 25 / 34

32. Strategies Recording Modified VM Outlook Strategy Explained Demo
Continuous Thread Replication
On primary:
Client RequestThread Replicator UIThread Secondary
HTTP request
handle(request)
notify(request)
handle(request)
finished
replicate(UIThread)
frames & objects
HTTP response
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 26 / 34

33. Strategies Recording Modified VM Outlook Strategy Explained Demo
Replication of Stack Frames
On secondary:
stack of ThreadStub
nativeThreadCreate()
VmThread.run()
ThreadStub.run()
receiving prepare
migration wait
Thread agent
receives
replicated thread
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 27 / 34

34. Strategies Recording Modified VM Outlook Strategy Explained Demo
Replication of Stack Frames
On secondary:
stack buffer
receiving MigratedThread.run()
migration create
frame n ... frame 2 frame 1 agent & rewire method1()
method2()
receives
debug information
...
methodN()
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 27 / 34

35. Strategies Recording Modified VM Outlook Strategy Explained Demo
Replication of Stack Frames
On secondary:
stack of ThreadStub
nativeThreadCreate()
VmThread.run()
ThreadStub.run()
receiving
migration resize
frame n ... frame 2 frame 1 agent
receives
debug information
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 27 / 34

36. Strategies Recording Modified VM Outlook Strategy Explained Demo
Replication of Stack Frames
On secondary:
stack of ThreadStub
nativeThreadCreate()
VmThread.run()
ThreadStub.run()
receiving
migration
frame n ... frame 2 frame 1 agent resize
receives
debug information
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 27 / 34

37. Strategies Recording Modified VM Outlook Strategy Explained Demo
Replication of Stack Frames
On secondary:
stack of ThreadStub
nativeThreadCreate()
VmThread.run()
ThreadStub.run()
receiving MigratedThread.run()
migration insert
frame n ... frame 2 frame 1 agent buffer method1()
method2()
receives
debug information
...
methodN()
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 27 / 34

38. Strategies Recording Modified VM Outlook Strategy Explained Demo
Replication of Stack Frames
On secondary:
stack of ThreadStub
nativeThreadCreate()
VmThread.run()
ThreadStub.run()
receiving MigratedThread.run()
migration
frame n ... frame 2 frame 1 agent rewire method1()
method2()
receives
debug information
...
methodN()
ThreadStub.patch()
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 27 / 34

39. Strategies Recording Modified VM Outlook Strategy Explained Demo
Inspecting the Migration Algorithm
RCP based tooling:
Inspect migrated entities at VM level
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 28 / 34

40. Strategies Recording Modified VM Outlook Strategy Explained Demo
Migration Demo
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 29 / 34

41. Strategies Recording Modified VM Outlook
Overview
1 Strategies
2 Recording
Strategy Explained
Example
Demo
3 Modified VM
Strategy Explained
Demo
4 Outlook
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 30 / 34

42. Strategies Recording Modified VM Outlook
Conclusion
Clustering RAP’s UIThreads is possible
Achieves transparent failover
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 31 / 34

43. Strategies Recording Modified VM Outlook
Future Work
1 Future directions for clustered RAP:
Support more SWT widgets
JFace and Workbench
Support alternatives to Terracotta
2 Potential for Java VMs with thread migration:
Distributed Equinox
Dynamic scaling for the cloud
Vertical scaling of running applications
3 Improvements for migration agents:
Migrate locked objects
Migrate interlocked threads
Migratable Resources: Sockets, Files
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 32 / 34

44. Strategies Recording Modified VM Outlook
Thank you for listening!
Questions?
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 33 / 34

45. Strategies Recording Modified VM Outlook
References
http://www.so.in.tum.de/sovereign
http://www.terracotta.org
http://www.eclipse.org/aspectj
http://wikis.sun.com/display/MaxineVM
Quitadamo, Mobile JikesRVM: A framework to support
transparent Java thread migration, 2008
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 34 / 34

46. TraceRAPWidgetCreation.aspectj
1 Object around () : traceRAPWidgetCreation () {
2 if ( primary ) {
3 widget = proceed ();
4 // record object
5 recordObject ( widget );
6 return widget ;
7 } else {
8 // replay object
9 widget = replayObject ();
10 return widget ;
11 }
12 }
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 1/2

47. TraceRAPMethodCalls.aspectj
1 Object around () : traceRAPMethodCalls () {
2 if ( primary ) {
3 // normal operation
4 return proceed ();
5 } else {
6 // do nothing
7 return null;
8 }
9 }
eclipsesource-lo
jeick@so.in.tum.de Sovereign March 22, 2011 2/2