"Quantum" Performance Effects

“Quantum” Performance Effects
v 3.0; February 2015
Sergey Kuksenko
sergey.kuksenko@oracle.com, @kuksenk0

The following is intended to outline our general product direction. It
is intended for information purposes only, and may not be
incorporated into any contract. It is not a commitment to deliver any
material, code, or functionality, and should not be relied upon in
making purchasing decisions. The development, release, and timing
of any features or functionality described for Oracle’s products
remains at the sole discretion of Oracle.
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 2/52

Intro

Intro: performance engineering
1. Computer Science → Software Engineering
– Build software to meet functional requirements
– Mostly don’t care about HW and data specifics
– Abstract and composable, “formal science”
2. Performance Engineering
– “Real world strikes back!”
– Exploring complex interactions between hardware, software, and data
– Based on empirical evidence, i.e. “natural science”

Intro: what’s the difference?
architecture vs microarchitecture

Intro: what’s the difference?
architecture vs microarchitecture
x86
AMD64(x86-64/Intel64)
ARMv7
....
Nehalem
Sandy Bridge
Bulldozer
Bobcat
Cortex-A9

Intro: SUTs1
∙ Intel® Core— i5-4300M [2.6 GHz] 1x2x2
– 𝜇arch: Haswell
– launched: Q4’2013
– OS: Xubuntu 14.04 (64-bits)
1System Under Test

Intro: SUTs1
∙ Intel® Core— i5-4300M [2.6 GHz] 1x2x2
– 𝜇arch: Haswell
– OS: Xubuntu 14.04 (64-bits)
∙ Samsung Exynos 4412, ARMv7 [1.6 GHz] 1x4x1
– 𝜇arch: Cortex-A9
– launched: 2011
– OS: Linaro 12.11
1System Under Test

Intro: SUTs (cont.)
∙ AMD Opteron— 4274HE [2.5 GHz] 2x8x1
– 𝜇arch: Bulldozer/Valencia
– OS: Oracle Linux Server release 6.0 (64-bits)
∙ Intel® Xeon® CPU E5-2680 [2.70 GHz] 2x8x2
– 𝜇arch: Sandy Bridge
– OS: Oracle Linux Server release 6.3 (64-bits)

Intro: JVM
∙ Java HotSpot— “1.8.0_25” 32-bits
∙ Java HotSpot— “1.8.0_25” 64-bits
∙ Java HotSpot— Embedded “1.8.0-ea-b79”

Intro: Demo code
https://github.com/kuksenko/quantum

Intro: Demo code
https://github.com/kuksenko/quantum
∙ Required: JMH (Java Microbenchmark Harness)
– http://openjdk.java.net/projects/code-tools/jmh/

Core

Demo1: double sum
private double [] A = new double [2048];
@Benchmark
public double test1 () {
double sum = 0.0;
for (int i = 0; i < A.length; i++) {
sum += A[i];
}
return sum;
}
@Benchmark
public double manualUnroll () {
double sum = 0.0;
for (int i = 0; i < A.length; i += 4) {
sum += A[i] + A[i + 1] + A[i + 2] + A[i + 3];
}
return sum;
}

Demo1: double sum
private double [] A = new double [2048];
@Benchmark
double sum = 0.0;
sum += A[i];
}
return sum;
}
@Benchmark
double sum = 0.0;
sum += A[i] + A[i + 1] + A[i + 2] + A[i + 3];
}
return sum;
}
426 𝑜𝑝𝑠
𝑚𝑠
1120 𝑜𝑝𝑠
𝑚𝑠

Demo1: looking into asm, test1
loop: vaddsd 0x10(%edi ,%eax ,8),%xmm0 ,%xmm0
vaddsd 0x18(%edi ,%eax ,8),%xmm0 ,%xmm0
add $0x10 ,%eax
cmp %ebx ,%eax
jl loop:

Demo1: looking into asm, manualUnroll
loop: vmovsd 0x48(%eax ,%edx ,8),% xmm0
vmovsd %xmm0 ,(% esp)
vmovsd 0x40(%eax ,%edx ,8),% xmm0
vmovsd %xmm0 ,0x8(%esp)
vaddsd 0x70(%eax ,%edx ,8),%xmm0 ,%xmm1
vaddsd %xmm2 ,%xmm1 ,%xmm1
vaddsd 0x10(%esp),%xmm1 ,%xmm1
vmovsd 0x18(%esp),%xmm0
vaddsd (%esp),%xmm0 ,%xmm0
add $0x10 ,%edx
cmp %ebx ,%edx
jl loop:

Demo1: measure time
@Benchmark
@BenchmarkMode(Mode.AverageTime)
@OperationsPerInvocation (2048)
double sum = 0.0;
sum += A[i];
}
return sum;
}
@Benchmark
double sum = 0.0;
sum += A[i] + A[i + 1] + A[i + 2] + A[i + 3];
}
return sum;
}

Demo1: measure time
@Benchmark
double sum = 0.0;
sum += A[i];
}
return sum;
}
@Benchmark
double sum = 0.0;
sum += A[i] + A[i + 1] + A[i + 2] + A[i + 3];
}
return sum;
}
𝑡𝑖𝑚𝑒 = 1.15 𝑛𝑠
𝑜𝑝
𝑜𝑝

Demo1: measure time
@Benchmark
double sum = 0.0;
sum += A[i];
}
return sum;
}
@Benchmark
double sum = 0.0;
sum += A[i] + A[i + 1] + A[i + 2] + A[i + 3];
}
return sum;
}
𝑜𝑝
𝐶𝑃 𝐼 =∼ 2.5
𝑜𝑝
𝐶𝑃 𝐼 =∼ 0.5
Cycles Per Instruction

𝜇arch: x86
CISC vs RISC

𝜇arch: x86
CISC and RISC
modern x86 CPU is not what it seems
All instructions (CISC) are dynamically translated into RISC-like
microoperations ( 𝜇ops).

𝜇arch: Intel’s internals
http://commons.wikimedia.org/wiki/
File:Intel_Nehalem_arch.svg
(c) Appaloosa, CC BY-SA 3.0

𝜇arch: simplified scheme

𝜇arch: looking into instruction tables2
Operation Latency 1
𝑇ℎ𝑟𝑜𝑢𝑔ℎ𝑝𝑢𝑡
addition (floating-point) 3 1
multiplication (floating-point) 5 0.5
addition (integer) 1 0.25
multiplication (integer) 3 1
2Haswell 𝜇arch

Demo1: test1, looking into asm again
loop: vaddsd 0x10(%edi ,%eax ,8),%xmm0 ,%xmm0
add $0x10 ,%eax
cmp %ebx ,%eax
jl loop:
𝑜𝑝
𝐶𝑃 𝐼 =∼ 2.5
∼ 3 𝑐𝑦𝑐𝑙𝑒𝑠
𝑜𝑝
𝑢𝑛𝑟𝑜𝑙𝑙𝑒𝑑 𝑏𝑦 16
19 𝑖𝑛𝑠𝑡𝑟𝑢𝑠𝑡𝑖𝑜𝑛𝑠

Demo1: test1, structural view
𝑜𝑝
𝐶𝑃 𝐼 =∼ 2.5
∼ 3 𝑐𝑦𝑐𝑙𝑒𝑠
𝑜𝑝
𝑢𝑛𝑟𝑜𝑙𝑙𝑒𝑑 𝑏𝑦 16

Demo1: manualUnroll

Demo1: manualUnroll, structural view
𝑜𝑝
𝐶𝑃 𝐼 =∼ 0.5
∼ 1.14 𝑐𝑦𝑐𝑙𝑒𝑠
𝑜𝑝
𝑢𝑛𝑟𝑜𝑙𝑙𝑒𝑑 𝑏𝑦 4 * 4

𝜇arch: Dependences
Performance ILP
3
of many programs is limited by natural data
dependencies.
3Instruction Level Parallelism

𝜇arch: Dependences
Performance ILP
3
of many programs is limited by natural data
dependencies.
What to do?
Break Dependency Chains!
3Instruction Level Parallelism

Demo1(cont.): breaking chains in a “right” way

...
sum += A[i];
}
return sum;

...
sum += A[i];
}
return sum;
...
sum0 += A[i];
sum1 += A[i + 1];
}
return sum0 + sum1;

...
sum += A[i];
}
return sum;
...
sum0 += A[i];
sum1 += A[i + 1];
}
return sum0 + sum1;
...
for (int i = 0; i < array.length; i += 4) {
sum0 += A[i];
sum1 += A[i + 1];
sum2 += A[i + 2];
sum3 += A[i + 3];
}
return (sum0 + sum1) + (sum2 + sum3);

Demo1(cont.): double sum final results
Haswell AMD ARM
manualUnroll 0.44 0.45 3.30
test1 1.15 1.50 6.60
test2 0.58 0.80 4.25
test4 0.39 0.43 4.25
test8 0.39 0.25 2.55
time, ns/op

Demo2: results
Haswell AMD ARM
DoubleMul.test1 2.84 2.52 8.17
IntMul.test1 1.14 1.16 10.04
IntMul.test2 0.58 0.75 7.38
IntMul.test4 0.38 0.67 4.64
IntSum.test1 0.39 0.32 8.92
IntSum.test2 0.24 0.48 6.12
time, ns/op

Branches: to jump or not to jump
public int absSumBranch(int a[]) {
int sum = 0;
for (int x : a) {
if (x < 0) {
sum -= x;
} else {
sum += x;
}
}
return sum;
}
loop: mov 0xc(%ecx ,%ebp ,4),%ebx
test %ebx ,%ebx
jl L1
add %ebx ,%eax
jmp L2
L1: sub %ebx ,%eax
L2: inc %ebp
cmp %edx ,%ebp
jl loop

Branches: to jump or not to jump
public int absSumPredicated(int a[]) {
int sum = 0;
for (int x : a) {
sum += Math.abs(x);
}
return sum;
}
loop: mov 0xc(%ecx ,%ebp ,4),%ebx
mov %ebx ,%esi
neg %esi
test %ebx ,%ebx
cmovl %esi ,%ebx
add %ebx ,%eax
inc %ebp
cmp %edx ,%ebp
jl Loop

Demo3: results
Regular Pattern = (+, –)*
Nehalem Haswell AMD ARM
branch_sorted 0.9 0.5 1.0 5.0
branch_regular 0.9 0.5 0.8 5.0
branch_shuffled 6.4 1.0 2.8 9.4
predicated_sorted 1.3 0.8 0.9 5.6
predicated_regular 1.3 0.8 0.9 5.3
predicated_shuffled 1.3 0.8 0.9 9.3
time, ns/op

Demo3: results
Regular Pattern = (+, +, –, +, –, –, +, –, –, +)*
Nehalem Haswell AMD ARM
branch_sorted 0.9 0.5 1.0 5.0
branch_regular 1.6 0.9 1.0 5.0
branch_shuffled 6.4 1.0 2.3 9.5
predicated_sorted 1.3 0.8 0.9 5.6
predicated_regular 1.3 0.8 0.9 5.3
predicated_shuffled 1.3 0.8 0.9 9.3
time, ns/op

Demo4: && vs &
public int countConditional(boolean [] f0 , boolean [] f1) {
int cnt = 0;
for (int j = 0; j < SIZE; j++) {
for (int i = 0; i < SIZE; i++) {
if (f0[i] && f1[j]) {
cnt ++;
}
}
}
return cnt;
}
&&
shuffled 1.8 ns/op
sorted 0.6 ns/op

Demo4: && vs &
public int countLogical(boolean [] f0 , boolean [] f1) {
int cnt = 0;
for (int j = 0; j < SIZE; j++) {
for (int i = 0; i < SIZE; i++) {
if (f0[i] & f1[j]) {
cnt ++;
}
}
}
return cnt;
}
&&
shuffled 1.8 ns/op
sorted 0.6 ns/op
&
shuffled 1.2 ns/op
sorted 1.2 ns/op

Demo5: interface invocation cost
public interface I { public int amount (); }
...
public class C0 implements I { public int amount (){ return 0; } }
...
@Benchmark
@OperationsPerInvocation(SIZE)
public int sum(I[] a) {
int s = 0;
for (I i : a) {
s += i.amount ();
}
return s;
}

Demo5: results
1 target 2 targets 3 targets 4 targets
sorted 0.8 0.8 4.9 5.0
regular 0.8 4.9 5.0
shuffled 1.0 17.5 19.1
time, ns/op

Not a Real Core

Not a Real Core: HW Multithreading
∙ Simultaneous multithreading, SMT
e.g. Intel® Hyper-Threading Technology
∙ Fine-grained temporal multithreading
e.g. CMT, Sun/Oracle ULTRASparc T1, T2, T3, T4, T5 ...

Back to Demo1: Execution Units Saturation
1 thread 2 threads 2 threads 4 threads
-cpu 1,3 -cpu 2,3
DoubleSum.test1 426 850 840 1660
DoubleSum.test2 845 1690 1260 2500
DoubleSum.test4 1260 2513 1260 2520
DoubleSum.manualUnroll 1120 2240 1260 2504
overall throughput, ops/ms

-cpu 1,3 -cpu 2,3
DoubleSum.test2 845 1690 1260 2500
DoubleSum.test4 1260 2513 1260 2520
Max single core throughput

-cpu 1,3 -cpu 2,3
DoubleSum.test2 845 1690 1260 2500
DoubleSum.test4 1260 2513 1260 2520
Max system throughput

Demo6: Map.get()
private Map <Integer , Integer > jdk_map;
private int[] keys;
@Benchmark
public int testJdkPrimitive () {
int s = 0;
for (int key : keys) {
s += jdk_map.get(key);
}
return s;
}

Demo6: Map.get()
private Map <Integer , Integer > jdk_map;
private Integer [] boxedKeys;
@Benchmark
public int testJdkBoxed () {
int s = 0;
for (Integer key : boxedKeys) {
s += jdk_map.get(key);
}
return s;
}

Demo6: Map.get()
private TIntIntMap third_party_map;
private int[] keys;
@Benchmark
public int test3dParty () {
int s = 0;
for (int key : keys) {
s += third_party_map.get(key);
}
return s;
}

Demo6: Map.get() results
-cpu 1
JdkPrimitive
JdkBoxed
3dParty
throughput, ops/ms

-cpu 1
JdkPrimitive 47
JdkBoxed
3dParty
throughput, ops/ms

-cpu 1
JdkPrimitive 47
JdkBoxed 71
3dParty
throughput, ops/ms

-cpu 1
JdkPrimitive 47
JdkBoxed 71
3dParty 74
throughput, ops/ms

-cpu 1 -cpu 2,3
JdkPrimitive 47
JdkBoxed 71
3dParty 74
throughput, ops/ms

-cpu 1 -cpu 2,3
JdkPrimitive 47 (25, 25)
JdkBoxed 71
3dParty 74
throughput, ops/ms

-cpu 1 -cpu 2,3
JdkBoxed 71 (40, 40)
3dParty 74
throughput, ops/ms

-cpu 1 -cpu 2,3
3dParty 74 (43, 43)
throughput, ops/ms

-cpu 1 -cpu 2,3 -cpu 2
(?) on -cpu 3
3dParty 74 (43, 43)
throughput, ops/ms

(?) on -cpu 3
JdkPrimitive 47 (25, 25) (30, ?)
3dParty 74 (43, 43)
throughput, ops/ms

(?) on -cpu 3
JdkPrimitive 47 (25, 25) (30, ?)
JdkBoxed 71 (40, 40) (50, ?)
3dParty 74 (43, 43)
throughput, ops/ms

(?) on -cpu 3
JdkPrimitive 47 (25, 25) (30, ?)
JdkBoxed 71 (40, 40) (50, ?)
3dParty 74 (43, 43) (16, ?)
throughput, ops/ms

Demo6: Hyper.troll()
public static double d0;
@Benchmark
public double troll () {
return (d0 / d2) / ((d1 / d2) / (d0 / d1));
}

Demo6: division results on Haswell
1 thread
int 250
double 180
throughput, ops/ 𝜇s
-cpu 1,3 -cpu 2,3 -cpu 3
(int, int) (250, 250) (125, 125) (125, 125)
(double, double) (180, 180) (90, 90) (90, 90)
(double, int) (180, 250) (150, 57) (90, 125)

Demo6: division results on AMD
1 thread
int 128
double 300
-cpu 0,1 -cpu 0,2 -cpu 0,8 -cpu 0
(int, int) (92, 92) (128, 128) (128, 128) (64, 64)
(double, double) (150, 150) (300, 300) (300, 300) (150, 150)
(double, int) (280, 120) (290, 128) (300, 128) (120, 64)

Conclusion

Enlarge your knowledge with these simple
tricks!
Reading list:
∙ “Computer Architecture: A Quantitative Approach”
John L. Hennessy, David A. Patterson
∙ CPU vendors documentation
∙ http://www.agner.org/optimize/
∙ etc.

Thanks!

Q & A ?

Appendix

Appendix: Frequency Variance
∙ Dynamic CPU Frequency
– TurboBoost and similar
"The processor must be working in the power, temperature, and
specification limits of the thermal design power (TDP)."©Intel

Appendix: TurboBoost in action
max normal freq.
measured freq.

Appendix: Set Fixed Frequency!
e.g.
cpufreq-set -u 2600000 -g performance

"Quantum" Performance Effects

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (11)

Similaire à "Quantum" Performance Effects

Similaire à "Quantum" Performance Effects (20)

Dernier

Dernier (20)

"Quantum" Performance Effects