Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
©2013 Azul Systems, Inc.	
 	
 	
 	
 	
 	
Priming Java

for Speed at Market Open

!
Getting Fast & Staying Fast

Gil Tene, ...
InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese an...
Presented at QCon New York
www.qconnewyork.com
Purpose of QCon
- to empower software development by facilitating the sprea...
©2012 Azul Systems, Inc.	
 	
 	
 	
 	
 	
High level agenda
Intro

Java realities at Market Open

A whole bunch of compiler...
©2013 Azul Systems, Inc.	
 	
 	
 	
 	
 	
©2013 Azul Systems, Inc.	
 	
 	
 	
 	
 	
About me: Gil Tene
co-founder, CTO @Azul...
©2013 Azul Systems, Inc.	
 	
 	
 	
 	
 	
Are you fast at Market Open?
©2013 Azul Systems, Inc.	
 	
 	
 	
 	
 	
Java at Market Open
. . .
Market Open
©2013 Azul Systems, Inc.	
 	
 	
 	
 	
 	
Java’s “Just In Time” Reality
Starts slow, learns fast

Lazy loading & initializa...
Compiler Stuff
Some simple compiler tricks
©2012 Azul Systems, Inc.	
 	
 	
 	
 	
 	
Code can be reordered...
int doMath(int x, int y, int z) {

int a = x + y;

int b...
©2012 Azul Systems, Inc.	
 	
 	
 	
 	
 	
Dead code can be removed
int doMath(int x, int y, int z) {

int a = x + y;

int b...
©2012 Azul Systems, Inc.	
 	
 	
 	
 	
 	
Values can be propagated
int doMath(int x, int y, int z) {

int a = x + y;

int b...
©2012 Azul Systems, Inc.	
 	
 	
 	
 	
 	
Math can be simplified
int doMath(int x, int y, int z) {

int a = x + y;

int b = ...
Some more compiler tricks
©2012 Azul Systems, Inc.	
 	
 	
 	
 	
 	
Reads can be cached
int distanceRatio(Object a) {

int distanceTo = a.getX() - st...
©2012 Azul Systems, Inc.	
 	
 	
 	
 	
 	
Reads can be cached
void loopUntilFlagSet(Object a) {

while (!a.flagIsSet()) {

l...
©2012 Azul Systems, Inc.	
 	
 	
 	
 	
 	
Writes can be eliminated
Intermediate values might never be visible

void updateD...
©2012 Azul Systems, Inc.	
 	
 	
 	
 	
 	
Writes can be eliminated
Intermediate values might never be visible

void updateD...
©2012 Azul Systems, Inc.	
 	
 	
 	
 	
 	
Inlining...
public class Thing {

private int x;

public final int getX() { return...
Speculative compiler tricks
JIT compilers can do things that
static compilers can have
a hard time with…
©2012 Azul Systems, Inc.	
 	
 	
 	
 	
 	
Class Hierarchy Analysis (CHA)
Can perform global analysis on currently loaded co...
©2012 Azul Systems, Inc.	
 	
 	
 	
 	
 	
Inlining works without “final”
public class Thing {

private int x;

public int ge...
©2012 Azul Systems, Inc.	
 	
 	
 	
 	
 	
The power of the “uncommon trap”

Being able throw away wrong code is very useful...
©2012 Azul Systems, Inc.	
 	
 	
 	
 	
 	
Untaken path example
“Never taken” paths can be optimized away with benefits:

voi...
©2012 Azul Systems, Inc.	
 	
 	
 	
 	
 	
Implicit Null Check example
All field and array access in Java is null checked

x ...
Deoptimization
©2012 Azul Systems, Inc.	
 	
 	
 	
 	
 	
Deoptimization:

Adaptive compilation is… adaptive
Micro-benchmarking is a black ...
©2012 Azul Systems, Inc.	
 	
 	
 	
 	
 	
Warmup often doesn’t cut it…
Common Example:

Trading system wants to have the fir...
©2012 Azul Systems, Inc.	
 	
 	
 	
 	
 	
Fun In a Box:

A simple Deoptimization example
Deoptimization due to lazy loading...
public	
  class	
  FunInABox	
  {	
  
	
  	
  	
  	
  static	
  final	
  int	
  THING_ONE_THREASHOLD	
  =	
  20000000	
  
...
Lumpy.local-40% !
Lumpy.local-40% java -XX:+PrintCompilation FunInABox
77 1 java.lang.String::hashCode (64 bytes)!
109 2 s...
 	
  	
  	
  .	
  
	
  	
  	
  	
  .	
  
	
  	
  	
  	
  .	
  
	
  	
  	
  	
  public	
  static	
  <T>	
  Class<T>	
  forc...
Lumpy.local-41% !
Lumpy.local-41% java -XX:+PrintCompilation FunInABox KeepThingsTame!
75 1 java.lang.String::hashCode (64...
©2012 Azul Systems, Inc.	
 	
 	
 	
 	
 	
Example causes for depotimization

at Market Open
First calls to method in an uni...
©2013 Azul Systems, Inc.	
 	
 	
 	
 	
 	
Java’s “Just In Time” Reality
Starts slow, learns fast

Lazy loading & initializa...
©2013 Azul Systems, Inc.	
 	
 	
 	
 	
 	
Starts slow, learns fast

Lazy loading & initialization

Aggressively optimized f...
©2012 Azul Systems, Inc.	
 	
 	
 	
 	
 	
What can we do about it?
Zing has a new feature set called “ReadyNow!”

Focused o...
©2013 Azul Systems, Inc.	
 	
 	
 	
 	
 	
Java at Market Open
. . .
Market Open
Deoptimization
ReadyNow!
avoids
deoptimizat...
©2013 Azul Systems, Inc.	
 	
 	
 	
 	
 	
Java at Market Open
. . .
Market Open
With Zing & ReadyNow!
©2013 Azul Systems, Inc.	
 	
 	
 	
 	
 	
Java at Market Open
. . .
Market Open
With ReadyNow!
Warmup?
Avoids
Restarts
©2013 Azul Systems, Inc.	
 	
 	
 	
 	
 	
. . .
Market Open
With ReadyNow! and
No Overnight Restarts
!
Start Fast & Stay Fa...
©2013 Azul Systems, Inc.	
 	
 	
 	
 	
 	
One liner takeaway
. . .
Zing: A cure for the Java hiccups
Market Open
Watch the video with slide synchronization on
InfoQ.com!
http://www.infoq.com/presentations/jvm-
dynamic-optimizations
Prochain SlideShare
Chargement dans…5
×

Priming Java for Speed at Market Open

593 vues

Publié le

Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/1olRBPG.

Sponsored by Azul. Gil Tene discusses issues with dynamically optimized environments used for trading systems along with techniques for dealing with them, including JVM performance tune up tricks. Filmed at qconnewyork.com.

Gil Tene is CTO and co-founder of Azul Systems. Gil pioneered Azul's Continuously Concurrent Compacting Collector (C4), Java Virtualization, Elastic Memory, and various managed runtime and systems stack technologies that combine to deliver the industry's most scalable and robust Java platforms.

Publié dans : Technologie
  • Soyez le premier à commenter

Priming Java for Speed at Market Open

  1. 1. ©2013 Azul Systems, Inc. Priming Java for Speed at Market Open ! Getting Fast & Staying Fast Gil Tene, CTO & co-Founder, Azul Systems
  2. 2. InfoQ.com: News & Community Site • 750,000 unique visitors/month • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • News 15-20 / week • Articles 3-4 / week • Presentations (videos) 12-15 / week • Interviews 2-3 / week • Books 1 / month Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations /jvm-dynamic-optimizations
  3. 3. Presented at QCon New York www.qconnewyork.com Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide
  4. 4. ©2012 Azul Systems, Inc. High level agenda Intro Java realities at Market Open A whole bunch of compiler optimization stuff Deoptimization… What we can do about it
  5. 5. ©2013 Azul Systems, Inc. ©2013 Azul Systems, Inc. About me: Gil Tene co-founder, CTO @Azul Systems Have been working on “think different” GC approaches since 2002 At Azul we make JVMs that dramatically improve latency behavior As a result, we end up working a lot with low latency trading systems And (after GC is solved) the Market Open problem seems to dominate concerns * working on real-world trash compaction issues, circa 2004
  6. 6. ©2013 Azul Systems, Inc. Are you fast at Market Open?
  7. 7. ©2013 Azul Systems, Inc. Java at Market Open . . . Market Open
  8. 8. ©2013 Azul Systems, Inc. Java’s “Just In Time” Reality Starts slow, learns fast Lazy loading & initialization Aggressively optimized for the common case (temporarily) Reverts to slower execution to adapt Warmup Deoptimization . . .
  9. 9. Compiler Stuff
  10. 10. Some simple compiler tricks
  11. 11. ©2012 Azul Systems, Inc. Code can be reordered... int doMath(int x, int y, int z) { int a = x + y; int b = x - y; int c = z + x; return a + b; } Can be reordered to: int doMath(int x, int y, int z) { int c = z + x; int b = x - y; int a = x + y; return a + b; }
  12. 12. ©2012 Azul Systems, Inc. Dead code can be removed int doMath(int x, int y, int z) { int a = x + y; int b = x - y; int c = z + x; return a + b; } ! Can be reduced to: ! int doMath(int x, int y, int z) { int a = x + y; int b = x - y; return a + b; }
  13. 13. ©2012 Azul Systems, Inc. Values can be propagated int doMath(int x, int y, int z) { int a = x + y; int b = x - y; int c = z + x; return a + b; } ! Can be reduced to: ! int doMath(int x, int y, int z) { return x + y + x - y; } !
  14. 14. ©2012 Azul Systems, Inc. Math can be simplified int doMath(int x, int y, int z) { int a = x + y; int b = x - y; int c = z + x; return a + b; } ! Can be reduced to: ! int doMath(int x, int y, int z) { return x + x; } !
  15. 15. Some more compiler tricks
  16. 16. ©2012 Azul Systems, Inc. Reads can be cached int distanceRatio(Object a) { int distanceTo = a.getX() - start; int distanceAfter = end - a.getX(); return distanceTo/distanceAfter; } Is the same as int distanceRatio(Object a) { int x = a.getX(); int distanceTo = x - start; int distanceAfter = end - x; return distanceTo/distanceAfter; }
  17. 17. ©2012 Azul Systems, Inc. Reads can be cached void loopUntilFlagSet(Object a) { while (!a.flagIsSet()) { loopcount++; } } Is the same as: void loopUntilFlagSet(Object a) { boolean flagIsSet = a.flagIsSet(); while (!flagIsSet) { loopcount++; } } ! That’s what volatile is for...
  18. 18. ©2012 Azul Systems, Inc. Writes can be eliminated Intermediate values might never be visible void updateDistance(Object a) { int distance = 100; a.setX(distance); a.setX(distance * 2); a.setX(distance * 3); } Is the same as void updateDistance(Object a) { a.setX(300); }
  19. 19. ©2012 Azul Systems, Inc. Writes can be eliminated Intermediate values might never be visible void updateDistance(Object a) { a. setVisibleValue(0); for (int i = 0; i < 1000000; i++) { a.setInternalValue(i); } a.setVisibleValue(a.getInternalValue()); } Is the same as void updateDistance(Object a) { a.setInternalValue(1000000); a.setVisibleValue(1000000); }
  20. 20. ©2012 Azul Systems, Inc. Inlining... public class Thing { private int x; public final int getX() { return x }; } ... myX = thing.getX(); Is the same as Class Thing { int x; } ... myX = thing.x;
  21. 21. Speculative compiler tricks JIT compilers can do things that static compilers can have a hard time with…
  22. 22. ©2012 Azul Systems, Inc. Class Hierarchy Analysis (CHA) Can perform global analysis on currently loaded code Deduce stuff about inheritance, method overrides, etc. Can make optimization decisions based on assumptions Re-evaluate assumptions when loading new classes Throw away code that conflicts with assumptions before class loading makes them invalid
  23. 23. ©2012 Azul Systems, Inc. Inlining works without “final” public class Thing { private int x; public int getX() { return x }; } ... myX = thing.getX(); Is the same as Class Thing { int x; } ... myX = thing.x; As long as there is only one implementer of getX()
  24. 24. ©2012 Azul Systems, Inc. The power of the “uncommon trap” Being able throw away wrong code is very useful E.g. Speculatively assuming callee type polymorphic can be “monomorphic” or “megamorphic” E.g. Can make virtual calls direct even without CHA E.g. Can speculatively inline things E.g. Speculatively assuming branch behavior We’ve only ever seen this thing go one way, so.... More Speculative stuff
  25. 25. ©2012 Azul Systems, Inc. Untaken path example “Never taken” paths can be optimized away with benefits: void computeMagnitude(int val) { if (val > 10) { scale = computeScale(val); else { scale = 1; } return scale + 9; } When all values so far were <= 10 can be compiled to: void computeMagnitude(int val) { if (val > 10) uncommonTrap(); return 10; }
  26. 26. ©2012 Azul Systems, Inc. Implicit Null Check example All field and array access in Java is null checked x = foo.x; is (in equivalent required machine code): if (foo == null) throw new NullPointerException(); x = foo.x; ! But compiler can “hope” for non-nulls, and handle SEGV <Point where later SEGV will appear to throw> x = foo.x; This is faster *IF* no nulls are encountered…
  27. 27. Deoptimization
  28. 28. ©2012 Azul Systems, Inc. Deoptimization: Adaptive compilation is… adaptive Micro-benchmarking is a black art So is the art of the Warmup Running code long enough to compile is just the start… Deoptimization can occur at any time often occur after you *think* the code is warmed up. Many potential causes
  29. 29. ©2012 Azul Systems, Inc. Warmup often doesn’t cut it… Common Example: Trading system wants to have the first trade be fast So run 20,000 “fake” messages through the system to warm up let JIT compilers optimize, learn, and deopt before actual trades What really happens Code is written to do different things “if this is a fake message” e.g. “Don’t send to the exchange if this is a fake message” JITs optimize for fake path, including speculatively assuming “fake” First real message through deopts...
  30. 30. ©2012 Azul Systems, Inc. Fun In a Box: A simple Deoptimization example Deoptimization due to lazy loading/initialization Two classes: ThingOne and ThingTwo Both are actively called in the same method But compiler kicks in before one is ever called JIT cannot generate call for uninitialized class So it leaves an uncommon trap on that path… And deopts later if it ever gets there. https://github.com/giltene/GilExamples/blob/master/src/main/java/FunInABox.java
  31. 31. public  class  FunInABox  {          static  final  int  THING_ONE_THREASHOLD  =  20000000          .          .          .                  static  public  class  ThingOne  {                  static  long  valueOne  =  0;   !                static  long  getValue()  {  return  valueOne++;  }          }   !        static  public  class  ThingTwo  {                  static  long  valueTwo  =  3;   !                static  long  getValue()  {  return  valueTwo++;  }          }   !        public  static  long  testRun(int  iterations)  {                  long  sum  =  0;   !                for(int  iter  =  0;  iter  <  iterations;  iter++)  {                          if  (iter  >  THING_ONE_THREASHOLD)                                  sum  +=  ThingOne.getValue();                          else                                  sum  +=  ThingTwo.getValue();                  }                  return  sum;          } Deopt example: Fun In a Box
  32. 32. Lumpy.local-40% ! Lumpy.local-40% java -XX:+PrintCompilation FunInABox 77 1 java.lang.String::hashCode (64 bytes)! 109 2 sun.nio.cs.UTF_8$Decoder::decodeArrayLoop (553 bytes)! 115 3 java.math.BigInteger::mulAdd (81 bytes)! 118 4 java.math.BigInteger::multiplyToLen (219 bytes)! 121 5 java.math.BigInteger::addOne (77 bytes)! 123 6 java.math.BigInteger::squareToLen (172 bytes)! 127 7 java.math.BigInteger::primitiveLeftShift (79 bytes)! 130 8 java.math.BigInteger::montReduce (99 bytes)! 140 1% java.math.BigInteger::multiplyToLen @ 138 (219 bytes)! Starting warmup run (will only use ThingTwo):! 147 9 sun.security.provider.SHA::implCompress (491 bytes)! 153 10 java.lang.String::charAt (33 bytes)! 154 11 FunInABox$ThingTwo::getValue (10 bytes)! 154 2% FunInABox::testRun @ 4 (38 bytes)! 161 12 FunInABox::testRun (38 bytes)! Warmup run [1000000 iterations] took 27 msec..! ! ...Then, out of the box! Came Thing Two and Thing One!! And they ran to us fast! They said, "How do you do?"...! ! Starting actual run (will start using ThingOne a bit after using ThingTwo):! 5183 12 made not entrant FunInABox::testRun (38 bytes)! 5184 2% made not entrant FunInABox::testRun @ -2 (38 bytes)! 5184 3% FunInABox::testRun @ 4 (38 bytes)! 5184 13 FunInABox$ThingOne::getValue (10 bytes)! Test run [200000000 iterations] took 1299 msec... Deopt example: Fun In a Box
  33. 33.        .          .          .          public  static  <T>  Class<T>  forceInit(Class<T>  klass)  {                  //  Forces  actual  initialization  (not  just  loading)  of  the  class  klass:                  try  {                          Class.forName(klass.getName(),  true,  klass.getClassLoader());                  }  catch  (ClassNotFoundException  e)  {                          throw  new  AssertionError(e);    //  Can't  happen                  }                  return  klass;          }   !        public  static  void  tameTheThings()  {                  forceInit(ThingOne.class);                  forceInit(ThingTwo.class);          }          .          .          . Deopt example: Fun In a Box https://github.com/giltene/GilExamples/blob/master/src/main/java/FunInABox.java
  34. 34. Lumpy.local-41% ! Lumpy.local-41% java -XX:+PrintCompilation FunInABox KeepThingsTame! 75 1 java.lang.String::hashCode (64 bytes)! 107 2 sun.nio.cs.UTF_8$Decoder::decodeArrayLoop (553 bytes)! 113 3 java.math.BigInteger::mulAdd (81 bytes)! 115 4 java.math.BigInteger::multiplyToLen (219 bytes)! 119 5 java.math.BigInteger::addOne (77 bytes)! 121 6 java.math.BigInteger::squareToLen (172 bytes)! 125 7 java.math.BigInteger::primitiveLeftShift (79 bytes)! 127 8 java.math.BigInteger::montReduce (99 bytes)! 133 1% java.math.BigInteger::multiplyToLen @ 138 (219 bytes)! Keeping ThingOne and ThingTwo tame (by initializing them ahead of time):! Starting warmup run (will only use ThingTwo):! 140 9 sun.security.provider.SHA::implCompress (491 bytes)! 147 10 java.lang.String::charAt (33 bytes)! 147 11 FunInABox$ThingTwo::getValue (10 bytes)! 147 2% FunInABox::testRun @ 4 (38 bytes)! 154 12 FunInABox::testRun (38 bytes)! Warmup run [1000000 iterations] took 24 msec..! ! ...Then, out of the box! Came Thing Two and Thing One!! And they ran to us fast! They said, "How do you do?"...! ! Starting actual run (will start using ThingOne a bit after using ThingTwo):! 5178 13 FunInABox$ThingOne::getValue (10 bytes)! Test run [200000000 iterations] took 2164 msec... Deopt example: Fun In a Box
  35. 35. ©2012 Azul Systems, Inc. Example causes for depotimization at Market Open First calls to method in an uninitialized class First call to a method in a not-yet-loaded class Dynamic branch elimination hitting an unexpected path Implicit Null Check hitting a null
  36. 36. ©2013 Azul Systems, Inc. Java’s “Just In Time” Reality Starts slow, learns fast Lazy loading & initialization Aggressively optimized for the common case (temporarily) Reverts to slower execution to adapt Warmup Deoptimization . . .
  37. 37. ©2013 Azul Systems, Inc. Starts slow, learns fast Lazy loading & initialization Aggressively optimized for the common case (temporarily) Reverts to slower execution to adapt What we have What we want No Slow Trades ReadyNow! to the rescue Java’s “Just In Time” Reality
  38. 38. ©2012 Azul Systems, Inc. What can we do about it? Zing has a new feature set called “ReadyNow!” Focused on avoiding deoptimization while keeping code fast First of all, paying attention matters E.g. Some of those dynamic branch eliminations have no benefit… Adds optional controls for helpful things like: Aggressive class loading (load when they come into scope) Safe pre-initialization of classes with empty static initializers Per method compiler directives (e.g. disable ImplicitNullChecks) … The feature set keep growing…
  39. 39. ©2013 Azul Systems, Inc. Java at Market Open . . . Market Open Deoptimization ReadyNow! avoids deoptimization
  40. 40. ©2013 Azul Systems, Inc. Java at Market Open . . . Market Open With Zing & ReadyNow!
  41. 41. ©2013 Azul Systems, Inc. Java at Market Open . . . Market Open With ReadyNow! Warmup? Avoids Restarts
  42. 42. ©2013 Azul Systems, Inc. . . . Market Open With ReadyNow! and No Overnight Restarts ! Start Fast & Stay Fast Java at Market Open
  43. 43. ©2013 Azul Systems, Inc. One liner takeaway . . . Zing: A cure for the Java hiccups Market Open
  44. 44. Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations/jvm- dynamic-optimizations

×