Un monde où 1 ms vaut 100 M€ - Devoxx France 2015

1. @Alex_Victoor @ThierryAbalea#sginsideit
Un monde où 1ms vaut 100M d’euros

2. @YourTwitterHandle#DVXFR14{session hashtag} @Alex_Victoor @ThierryAbalea#sginsideit
Speakers
Alexandre Victoor
@Alex_Victoor
Thierry Abaléa
@ThierryAbalea

3. 5Kstatus updates / Sec
6K
Tweets/ Sec
1,6M
Mails / SEC
40K
Searches / SEC
740K
Messages / SEC
Big Data != Web

4. 5Kstatus updates / Sec
6K
Tweets/ Sec
1,6M
Mails / SEC
40K
Searches / SEC
740K
Messages / SEC
1.1M
US OPTIONS
Trades & quotes / SEC
Big Data != Web

5. Plus vite que la lumière

8. @YourTwitterHandle@YourTwitterHandle@Alex_Victoor @ThierryAbalea#sginsideit
Latence à tous les
niveaux
APPLICATIF
JVM
OS
RESEAU
DISQUE
CPU
MEMOIRE

9. @YourTwitterHandle@YourTwitterHandle@Alex_Victoor @ThierryAbalea#sginsideit
Un Quiz pour s’échauffer

10. int SIZE = 1000000;
int NB_ARRAY = 50;
long[][] longs
= new long[NB_ARRAY][SIZE];
long result = 0;
for (int j=0; j<SIZE; j++)
for (int i=0; i<NB_ARRAY; i++)
result += longs[i][j];
int SIZE = 1000000;
int NB_ARRAY = 50;
long[][] longs
= new long[NB_ARRAY][SIZE];
long result = 0;
for (int i=0; i<NB_ARRAY; i++)
for (int j=0; j<SIZE; j++)
result += longs[i][j];
Qui est le plus rapide ?
1 2

11. @YourTwitterHandle@YourTwitterHandle@Alex_Victoor @ThierryAbalea#sginsideit
Memory Layout
Le premier programme est le plus rapide !
47ms
796ms
m
esurer

12. Cache L3
Unités d’exécution
Cache L1
Cache L2
Coeur 1
Registres
Cache L1
Cache L2
Coeur 2
Processeur
< 1 ns
~ 1 ns
~ 3 ns
~ 12 ns
Registres
Unités d’exécution

13. int SIZE = 1000000;
int NB_ARRAY = 50;
long[][] longs
= new long[NB_ARRAY][SIZE];
long result = 0;
for (int i=0; i<NB_ARRAY; i++)
for (int j=0; j<SIZE; j++)
result += longs[i][j];
Friendly

14. int SIZE = 1000000;
int NB_ARRAY = 50;
long[][] longs
= new long[NB_ARRAY][SIZE];
long result = 0;
for (int j = 0; j<SIZE;j++)
for (int i=0; i<NB_ARRAY; i++)
result += longs[i][j];
Pas friendly du tout

15. @YourTwitterHandle@YourTwitterHandle@Alex_Victoor @ThierryAbalea#sginsideit
Mesurer
micro benchmarks

16. @YourTwitterHandle@YourTwitterHandle@Alex_Victoor @ThierryAbalea#sginsideit
OpenJDK JMH
$ mvn archetype:generate
-­DinteractiveMode=false
-­DarchetypeGroupId=org.openjdk.jmh
-­DarchetypeArtifactId=jmh-­java-­benchmark-­archetype
-­DgroupId=org.sample
-­DartifactId=devoxx-­bench
-­Dversion=1.0

17. @YourTwitterHandle@YourTwitterHandle@Alex_Victoor @ThierryAbalea#sginsideit
Mise en situation

18. La vente de produit financiers (avant)

19. RFQ: Request For Quote

20. API - en direct
CLIENTBANQUE
WEB Marchés
SALES +
Trading
Risk (contrôles)Booking

21. @YourTwitterHandle@YourTwitterHandle@Alex_Victoor @ThierryAbalea#sginsideit
Java ?

22. @YourTwitterHandle@YourTwitterHandle@Alex_Victoor @ThierryAbalea#sginsideit
Simple pour démarrer ?

25. Random rand = new Random();
IntStream stream = rand.ints(10 * 1024 * 1024, 0, 2);

26. Random rand = new Random();
IntStream stream = rand.ints(10 * 1024 * 1024, 0, 2);
int sum = stream.sum();

27. Random rand = new Random();
IntStream stream = rand.ints(10 * 1024 * 1024, 0, 2);
int sum = stream.sum();
int sum = stream.parallel().sum();

28. Random rand = new Random();
IntStream stream = rand.ints(10 * 1024 * 1024, 0, 2);
int sum = stream.sum();
int sum = stream.parallel().sum();
113 ms

29. Random rand = new Random();
IntStream stream = rand.ints(10 * 1024 * 1024, 0, 2);
int sum = stream.sum();
int sum = stream.parallel().sum(); 1167 ms
157 ms

31. Première archi
Event loop

32. Première archi
Event loopI/O

33. Première archi
Event loop I/OI/O

35. @YourTwitterHandle@YourTwitterHandle@Alex_Victoor @ThierryAbalea#sginsideit
Queue

36. Lock

39. Nos heures de pointe
Open & Close

40. Coût du lock
Appels système

41. Coût du lock
Changements de contexte

42. Cache L3
Unités d’exécution
Cache L1
Cache L2
Coeur 1
Registres
Cache L1
Cache L2
Coeur 2
Processeur
< 1 ns
~ 1 ns
~ 3 ns
~ 12 ns
Registres
Unités d’exécution

43. Algorithmes non bloquants
Au moins un thread progresse

44. Algorithmes non bloquants
Pas de ‘section critique’, locks, mutexes, spin-­locks, …

45. j.u.c.ConcurrentLinkedQueue

46. P1
P2
P3
C
Queue MPSC

47. P1
P2
P3
C
Queues SPSC

48. P1
P2
P3
C
Queues SPSC
Single Writer Principle

49. Deuxième archi
I/OI/O

50. producerIndex = 42
Concurrent Reading and Writing, Leslie Lamport, 1977
E E null null null null null null null E
offset = 2
consumerIndex = 39 offset = 9
Lamport Queue

51. producerIndex = 42
offset = 2
consumerIndex = 39
offset = 9
Lamport Queue
nullnull
E
E
E
nullnull
null
nullnull
Concurrent Reading and Writing, Leslie Lamport, 1977

52. import java.util.AbstractQueue;
public final class LamportQueue1<E> extends AbstractQueue<E> {
private final E[] buffer;
private volatile long producerIndex = 0;
private volatile long consumerIndex = 0;
public LamportQueue1(int capacity) {
buffer = (E[]) new Object[capacity];
}
@Override
public int size() {
return (int) (producerIndex -­‐ consumerIndex);
}

53. @Override
public boolean offer(final E e) {
if (size() == buffer.length) {
return false;
}
final int offset = (int)(producerIndex % buffer.length);
buffer[offset] = e;
producerIndex++;
return true;
}

54. @Override
public boolean offer(final E e) {
if (size() == buffer.length) {
return false;
}
final int offset = (int)(producerIndex % buffer.length);
buffer[offset] = e;
producerIndex++;
return true;
}

55. @Override
public boolean offer(final E e) {
if (size() == buffer.length) {
return false;
}
final int offset = (int)(producerIndex % buffer.length);
buffer[offset] = e;
producerIndex++;
return true;
}

56. @Override
public boolean offer(final E e) {
if (size() == buffer.length) {
return false;
}
final int offset = (int)(producerIndex % buffer.length);
buffer[offset] = e;
producerIndex++;
return true;
}

57. @Override
public E poll() {
if (consumerIndex == producerIndex) {
return null;
}
final int offset = (int)(consumerIndex % buffer.length);
final E e = buffer[offset];
buffer[offset] = null;
consumerIndex++;
return e;
}

58. @Override
public E poll() {
if (consumerIndex == producerIndex) {
return null;
}
final int offset = (int)(consumerIndex % buffer.length);
final E e = buffer[offset];
buffer[offset] = null;
consumerIndex++;
return e;
}

59. @Override
public E poll() {
if (consumerIndex == producerIndex) {
return null;
}
final int offset = (int)(consumerIndex % buffer.length);
final E e = buffer[offset];
buffer[offset] = null;
consumerIndex++;
return e;
}

60. @Override
public E poll() {
if (consumerIndex == producerIndex) {
return null;
}
final int offset = (int)(consumerIndex % buffer.length);
final E e = buffer[offset];
buffer[offset] = null;
consumerIndex++;
return e;
}

61. @Override
public E poll() {
if (consumerIndex == producerIndex) {
return null;
}
final int offset = (int)(consumerIndex % buffer.length);
final E e = buffer[offset];
buffer[offset] = null;
consumerIndex++;
return e;
}

62. @Override
public E poll() {
if (consumerIndex == producerIndex) {
return null;
}
final int offset = (int)(consumerIndex % buffer.length);
final E e = buffer[offset];
buffer[offset] = null;
consumerIndex++;
return e;
}

63. ,4
,16
,0
,5
,10
,15
,20
BQ LQ1
16
Performance (MOps/s) x 4
BlockinqQueue Lamport v1
20
15
10
5
0
4

64. Correct ?

65. private final E[] buffer;
private volatile long producerIndex = 0;
private volatile long consumerIndex = 0;

66. @Override
public boolean offer(final E e) {
if (size() == buffer.length) {
// queue is full
return false;
}
final int offset =
(int)(producerIndex
% buffer.length);
buffer[offset] = e;
producerIndex++;
return true;
}
@Override
public E poll() {
if (consumerIndex == producerIndex) {
// queue is empty
return null;
}
final int offset = (int)(consumerIndex
% buffer.length);
final E e = buffer[offset];
buffer[offset] = null;
consumerIndex++;
return e;
}
Producer Thread Consumer Thread

67. // consumerIndex = 2
// producerIndex = 2
// e = 777
Producer Thread Consumer Thread

68. // consumerIndex = 2
// producerIndex = 2
// e = 777
final int offset =
(int)(producerIndex
% buffer.length); // 2
Producer Thread Consumer Thread

69. // consumerIndex = 2
// producerIndex = 2
// e = 777
final int offset =
(int)(producerIndex
% buffer.length); // 2
buffer[offset] = e; // buff[2] = 777
Producer Thread Consumer Thread

70. // consumerIndex = 2
// producerIndex = 2
// e = 777
final int offset =
(int)(producerIndex
% buffer.length); // 2
buffer[offset] = e; // buff[2] = 777
producerIndex++; // 3
Producer Thread Consumer Thread

71. if (consumerIndex == producerIndex) { // 2 != 3
// queue is empty
return null;
}
Producer Thread Consumer Thread
// consumerIndex = 2
// producerIndex = 2
// e = 777
final int offset =
(int)(producerIndex
% buffer.length); // 2
buffer[offset] = e; // buff[2] = 777
producerIndex++; // 3

72. if (consumerIndex == producerIndex) { // 2 != 3
// queue is empty
return null;
}
final int offset = (int)(consumerIndex
% buffer.length); // 2
Producer Thread Consumer Thread
// consumerIndex = 2
// producerIndex = 2
// e = 777
final int offset =
(int)(producerIndex
% buffer.length); // 2
buffer[offset] = e; // buff[2] = 777
producerIndex++; // 3

73. if (consumerIndex == producerIndex) { // 2 != 3
// queue is empty
return null;
}
final int offset = (int)(consumerIndex
% buffer.length); // 2
final E e = buffer[offset]; // null (buffer[2])
…
return e; // return null
Producer Thread Consumer Thread
// consumerIndex = 2
// producerIndex = 2
// e = 777
final int offset =
(int)(producerIndex
% buffer.length); // 2
buffer[offset] = e; // buff[2] = 777
producerIndex++; // 3

74. if (consumerIndex == producerIndex) { // 2 != 3
// queue is empty
return null;
}
final int offset = (int)(consumerIndex
% buffer.length); // 2
final E e = buffer[offset]; // null (buffer[2])
…
return e; // return null
Producer Thread Consumer Thread
Java 1.4
// consumerIndex = 2
// producerIndex = 2
// e = 777
final int offset =
(int)(producerIndex
% buffer.length); // 2
buffer[offset] = e; // buff[2] = 777
producerIndex++; // 3

75. if (consumerIndex == producerIndex) { // 2 != 3
// queue is empty
return null;
}
final int offset = (int)(consumerIndex
% buffer.length); // 2
final E e = buffer[offset]; // 777 (buffer[2])
…
return e; // return 777
Producer Thread Consumer Thread
Java 1.5 & +
// consumerIndex = 2
// producerIndex = 2
// e = 777
final int offset =
(int)(producerIndex
% buffer.length); // 2
buffer[offset] = e; // buff[2] = 777
producerIndex++; // 3
Happens-­Before

76. @YourTwitterHandle@YourTwitterHandle@Alex_Victoor @ThierryAbalea#sginsideit
Encore un Quiz !

77. Lequel est le plus rapide ?
public static final int SIZE = 256 * 1024;
private int[] data = new int[SIZE];
@Setup
public void init() {
Random rand = new Random();
for (int i = 0; i < SIZE; i++) {
data[i] = rand.nextInt(100) -­‐ 50;
}
}

78. @Benchmark
public int mathAbs() {
int sum = 0;
for (int x : data) {
sum += Math.abs(x);
}
return sum;
}
@Benchmark
public int customAbs() {
int sum = 0;
for (int x : data) {
if (x < 0) {
sum -­‐= x;
} else {
sum += x;
}
}
return sum;
}
1 2
Lequel est le plus rapide ?

79. @YourTwitterHandle@YourTwitterHandle@Alex_Victoor @ThierryAbalea#sginsideit
Instruction Pipeline
Le premier programme est le plus rapide !
273us
1180us
m
esurer

80. Fetch Decode Write-backExecute
Instruction
Instruction
Instruction
Instruction
Waiting
Program Order Cycle d’horloge 1

81. Fetch Decode Write-backExecute
Instruction
Instruction
Instruction
Instruction
Waiting
Program Order Cycle d’horloge 2

82. Fetch Decode Write-backExecute
Instruction
Instruction
Instruction
Instruction
Waiting
Program Order Cycle d’horloge 3

83. Fetch Decode Write-backExecute
Instruction
Instruction
Instruction
Instruction
Waiting
Program Order Cycle d’horloge 4

84. Fetch Decode Write-backExecute
Instruction
Instruction
Instruction
Instruction
Waiting
Cycle d’horloge 5Program Order

85. Fetch Decode Write-backExecute
Instruction
Instruction
Instruction
Waiting
Cycle d’horloge 6Program Order

86. Fetch Decode Write-backExecute
jump if sign
Waiting
Program Order Cycle d’horloge 1

87. Fetch Decode Write-backExecute
jump if sign
Waiting
Program Order Cycle d’horloge 2

88. Fetch Decode Write-backExecute
jump if sign
Waiting
Program Order Cycle d’horloge 3

89. Fetch Decode Write-backExecute
jump if sign
Waiting
Program Order Cycle d’horloge 4

90. Fetch Decode Write-backExecute
jump if sign
Waiting
Program Order Cycle d’horloge 4
Poubelle
Bad Prediction

91. @YourTwitterHandle#DVXFR14{session hashtag} @Alex_Victoor @ThierryAbalea#sginsideit
Mesurer la latence

92. moyenne
4
Latence
Temps
Centile (percentile)

93. Latence
Temps
Centile (percentile)

94. Latence
Temps
Centile (percentile)

95. 3.5
50%
Latence
Temps
Centile (percentile)

96. 3.5
6
50%
90%
Latence
Temps
Centile (percentile)

97. HdrHistogram
Histogram histo = new Histogram(5);;
histo.recordValue(end-­start);;
histo.getValueAtPercentile(0.99);;
histo.outputPercentileDistribution(os, 1D);;

99. @YourTwitterHandle@YourTwitterHandle@Alex_Victoor @ThierryAbalea#sginsideit
Queue v2

100. Coût du volatile

101. Unités d’exécution
Cache L1
Cache L2
Coeur
Registres
Store Buffer

102. Unités d’exécution
Cache L1
Cache L2
Coeur
Registres
Store Buffer
Volatile store

103. Unités d’exécution
Cache L1
Cache L2
Coeur
Registres
S5
S4
S3
S2
S1
Store Buffer
Volatile store

104. import java.util.AbstractQueue;
public final class LamportQueue2<E> extends AbstractQueue<E> {
private final E[] buffer;
private final AtomicLong producerIndex = new AtomicLong();
private final AtomicLong consumerIndex = new AtomicLong();
public LamportQueue2(int capacity) {
buffer = (E[]) new Object[capacity];
}
@Override
public int size() {
return (int) (producerIndex.get() -­‐ consumerIndex.get());
}

105. @Override
public boolean offer(final E e) {
if (size() == buffer.length) {
return false;
}
final int offset = (int)(producerIndex % buffer.length);
buffer[offset] = e;
producerIndex.lazySet(producerIndex.get() + 1);
return true;
}

106. @Override
public E poll() {
if (consumerIndex == producerIndex) {
return null;
}
final int offset = (int)(consumerIndex.get() % buffer.length);
final E e = buffer[offset];
buffer[offset] = null;
consumerIndex.lazySet(consumerIndex.get() + 1);
return e;
}

107. Performance (MOps/s)
,4
,16
,43
,0
,10
,20
,30
,40
,50
BQ LQ1 LQ2
X 10
BlockinqQueue Lamport v1 Lamport v2
50
40
30
20
10
0
4
16
43

109. @YourTwitterHandle@YourTwitterHandle@Alex_Victoor @ThierryAbalea#sginsideit
LES LOGS

110. Les logs, plusieurs approches
• Appender fichier standard, écritures bloquantes

111. Les logs, plusieurs approches
• Appender fichier standard, écritures bloquantes
• Appender bufferisé

112. Les logs, plusieurs approches
• Appender fichier standard, écritures bloquantes
• Appender bufferisé
• Appender asynchrone

113. Les logs, plusieurs approches
• Appender fichier standard, écritures bloquantes
• Appender bufferisé
• Appender asynchrone
• Appender memory map file

114. Memory map file
Mem virtuelle

115. Memory map file
Mem virtuelle Mem physique

116. Memory map file
Mem virtuelle DisqueMem physique

117. Logs & mmap
MEMOIRE (OFF HEAP)
BUFFER
FICHIER
Données vide

118. Logs & mmap
MEMOIRE (OFF HEAP)
BUFFER
FICHIER
Données vide
Appel systèm
e

119. <configuration>
<appender name = "TXT-­‐CHRONICLE"
class = "net.openhft.chronicle.logger.logback.TextIndexedChronicleAppender">
<path>perf-­‐test-­‐txt-­‐chronicle</path>
</appender>
</configuration>
Chronicle Logback Appender

120. Texte vs Binaire – données de marché
• Code ISIN sur 12 caractères
• Un prix d’achat (bid)
• Un prix de vente (ask)
Isin=FR0000120271 Bid=46.575 Ask=46.590

121. Texte vs Binaire
Isin=FR0000120271 Bid=46.575 Ask=46.590
Texte (UTF-­8)
49 73 69 6E 3D 46 52 30
30 30 30 31 32 30 32 37
31 20 42 69 64 3D 34 36
2E 35 37 35 20 41 73 6B
3D 34 36 2E 35 39 30
39 octets
Binaire
46 52 30 30 30 30 31 32
30 32 37 31 40 47 49 99
99 99 99 9A 40 47 4B 85
1E B8 51 EC
28 octets (20 avec des int)

122. Serialization vs toString()
public class Quote {
String code;
double bid;
double ask;
}
return "Quote {" +
"code='" + code + ''' +
", bid=" + bid +
", ask=" + ask +
'}';

123. Serialization vs toString()
public class Quote {
String code;
double bid;
double ask;
}
ByteBuffer buffer =
ByteBuffer.allocate(28);
buffer.put(code.getBytes());
buffer.putDouble(bid);
buffer.putDouble(ask);
buffer.flip();
out.write(buffer.array());

124. <appender name = "TXT-­‐CHRONICLE"
class = "net.openhft.chronicle.logger.logback.TextIndexedChronicleAppender">
<path>perf-­‐test-­‐txt-­‐chronicle</path>
</appender>
<appender name = "BIN-­‐CHRONICLE"
class = "net.openhft.chronicle.logger.logback.BinaryIndexedChronicleAppender">
<path>perf-­‐test-­‐bin-­‐chronicle</path>
</appender>
Logger en binaire avec Chronicle

125. logger.info("New quote received -­‐ {}", quote);

126. Bench logger
• Log de 100 000 messages de cotation
• Mesure de la latence induite par l’appel au logger
• System Under Test
CPU : 2x (Xeon E5-2630, 2.3GHz – 2.8 GHz, 6 cores)
RAM : 64 GB DDR3 PC10600
HDD : 2x 300GB en RAID1, 10K tr/min, SAS 6 Gb/s, 1 GB cache
Network 1GbE
OS: RHEL 6.4 avec un kernel 2.6.32

127. FileAppender vs BinaryIndexedChronicleAppender
1us
5us
2us
10us

128. @YourTwitterHandle@YourTwitterHandle@Alex_Victoor @ThierryAbalea#sginsideit
Conclusion

129. Dernier Quiz : ce qu’il faut retenir…
C’est toujours le premier programme le plus rapide
Il faut se méfier de son intuition, en matière de perf, il faut mesurer !!!
Soyez curieux, le hardware ce n’est pas sale

130. DÉCOUVREZ TOUTES NOS OFFRES SUR
C A R E E R S . S O C I E T E G E N E R A L E . C O M

131. @YourTwitterHandle@YourTwitterHandle@Alex_Victoor @ThierryAbalea#sginsideit
Questions ?

132. @Alex_Victoor @ThierryAbalea#sginsideit
Références
Talk ”Lock-­free Algorithms for Ultimate Performance”, Martin Thompson
https://yow.eventer.com/yow-­2012-­1012/lock-­free-­algorithms-­for-­ultimate-­performance-­by-­martin-­thompson-­1250
Talk “Queue evolution: from 10M to 470M ops/sec“, Nitsan Wakart
https://vimeo.com/100197431 / https://github.com/nitsanw/QueueEvolution
Talk “How NOT to Measure Latency”, Gil Tene
http://www.infoq.com/presentations/latency-­pitfalls
JMH, THE Micro Benchmark Tool for Java http://openjdk.java.net/projects/code-­tools/jmh/
HdrHistogram, THE Latency Measurement & Plotting Tool
https://github.com/HdrHistogram/HdrHistogram
Blog posts related to Mechanical Sympathy, Martin Thompson
http://mechanical-­sympathy.blogspot.com/
Blog posts related to Java Performance, Nitsan Wakart http://psy-­lob-­saw.blogspot.com/
Blog posts related to Java Performance, Peter Lawrey http://vanillajava.blogspot.com/
Chronicle, OpenHFT’s Tools http://openhft.net/
The “Mechanical Sympathy” Forum https://groups.google.com/forum/#!forum/mechanical-­sympathy
Code de cette présentation https://github.com/ThierryAbalea/high-­performance-­2015-­talk
Slides de cette présentation http://www.slideshare.net/ThierryAbalea/un-­monde-­o-­1-­ms-­vaut-­100-­m-­devoxx-­france-­2015
slideshare