Why is it so important? Why many languages adopt such kind of model? What should we care about when doing multi-threading applications?

1. Philip Yankov
x8academy

2. l Who am I?
l Current Hardware + performance
l What is Java Memory Model
l Happens-before
l Memory barriers
Plan

3. Philip Yankov
• Sofia University – CS and AI
• Previous experience:
– SAP Labs Bulgaria, MicrosoN, VMware
– Chobolabs and other startups
– Toptal
• Global winner of NASA Space Apps Challenge
with a prototype for Smart Glove
• x8academy – an Academy for BETTER soNware
engineers
– soon AI and MulU-threading courses

4. x = y = 0
x = 1
j = y
y = 1
i = x
Thread 1 Thread 2
What could be the result?
l Compiler can reorder
instrucUons.
l Compiler can keep
values in registers.
l Processor can reorder
instrucUons.
l Values may not be
synchronized to main
memory.

5. l i = 1; j = 1
l i = 0; j = 1
l i = 1; j = 0
l i = 0; j = 0
So in order to develop a mulU-threaded applicaUon we
need to understand:
• In what order the acUons are executed in the
applicaUon;
• How does the data sharing between threads?
Answer(s)

7. Intel processor

8. Processor

9. ac#on approximate #me (ns)
typical processor instrucUon 1
fetch from L1 cache 0.5
branch mispredicUon 5
fetch from L2 cache 7
mutex lock/unlock 25
fetch from main memory 100
2 kB via 1 GB/s 20.000
seek for new disk locaUon 8.000.000
read 1 MB sequenUally from disk 20.000.000
Source: h*ps://gist.github.com/jboner/2841832
ac#on approximate #me (ns)
typical processor instrucUon 1
fetch from L1 cache 0.5
branch mispredicUon 5
fetch from L2 cache 7
mutex lock/unlock 25
fetch from main memory 100
2 kB via 1 GB/s 20.000
seek for new disk locaUon 8.000.000
read 1 MB sequenUally from disk 20.000.000
Memory access Ume

10. JVM

11. Language specificaUons
• Before – the languages specificaUons do not make
reference to any parUcular compiler, opera#ng
system, or CPU. They make reference to an abstract
machine that is a generalizaUon of actual systems.
– the job of the programmer is to write code for the abstract
machine
– the job of the compiler is to actualize that code on a
concrete machine.
• By coding rigidly to the spec, you can be certain that
your code will compile and run without modificaUon
on any system, whether today or 50 years from now.

12. Language specificaUons
• The abstract machine in previous specificaUons is
fundamentally single-threaded. So it is not possible to
write mulU-threaded code that is "fully portable" with
respect to the spec. The spec does not even say
anything about the atomicity of memory loads and
stores or the order in which loads and stores might
happen.
• Nowadays, the abstract machine is mul#-threaded by
design. It also has a well-defined memory model; that
is, it says what the compiler may and may not do when
it comes to accessing memory.

13. • The Java memory model (JMM) is the model that describes the
behavior of memory in Java program (how threads interact)
• JMM is a contract between hardware, compiler and
programmers.
– A promise for programmers to enable implementaUon-independent
reasoning about programs
– A promise for security to prevent unintended informaUon leakage
– A promise for compilers: common hardware and soNware
opUmizaUons should be allowed as far as possible without violaUng
the first two requirements.
Java Memory Model

14. Why do we need a it?
• Different (hardware) plagorm memory models (none of
them match the JMM!!!)
• Many JVM implementaUons,
• It is not easy to program concurrently,
• Programmers (have to): write reliable and mulUthreaded
code,
• Compiler writers: implement opUmizaUon which will be a
legal, opUmizaUon according to the JLS
• Compiler (have to): produce fast and opUmal naUve code,
• Programmability, portability, performance

15. History of JMM
• The Java Memory Model (Manson, Pugh and
Adve, POPL 2005) was introduced aNer the
original memory model was found to be “fatally
flawed” (Pugh, 2000). The main flaws were:
– Many opUmisaUons were illegal (including CSE),
– Final fields could be observed to change,
– Unclear semanUcs of finalizaUon.
• The JMM aims to fix these problems with 3
different fixes.
• The core of the JMM only deals with the first
problem.

16. OpUmizaUons are applied almost
exclusively aNer handing responsibility
to the
JVM’s runUme where they are difficult
to comprehend.
A JVM is allowed to alter the executed
program as long as it remains correct.
The Java memory model describes a
contract for what a correct program is
(in the context of mulU-threaded
applicaUons).

17. Program order:
int a = 1;
int b = 2;
int c = 3;
int d = 4;
int e = a + b;
int f = c – d;
ExecuUon order:
int d = 4;
int c = 3;
int f = c – d;
int b = 2;
int a = 1;
int e = a + b;
• We would like all acUons in the program to be executed in the
same order as the one wripen by a programmer in the iniUal code
— Program Order. But some types of opUmizaUon require certain
changes, leading to ExecuUon Order
InstrucUons order

18. Two actions can be ordered by a happens-
before relationship. If one action happens-
before another, then the first is visible to and
ordered before the second.
Java Language Specification, Java SE 7 Edition
Happens-before order

19. “Happens-before” relaUon rules
• Program order rule: each acUon in a thread happens-before every
acUon in that thread that comes later in the program order
• Lock rule: an unlock of a lock happens-before every subsequent
lock on that same lock (this applies on library locks as well as on
intrinsic locks)
• Vola#le variable rule: a write to a volaUle field happens-before
every subsequent read of that same field (this applies on atomic
variables too)
• Thread start rule: a request to start a thread happens before every
acUon in the started thread
• Thread termina#on rule: any acUon in a thread happens-before
any other thread detects that thread has terminated (== any other
thread successfully returns from join() on that thread)

20. “Happens-before” relaUon rules
• Interrup#on rule: a thread requesUng interrupUon on
another thread happens-before the interrupted thread
detects the interrupt
• Finalizer rule: the end of a constructor of an object happens-
before the start of the finalizer for that object
• Constructor rule: serng default values for variables, serng
value to a final field in the constructor happens-before the
constructor ends
• Atomicity rule: write to an Atomic variable happens-before
read from that variable
• Happens before is a transi#ve rela#on.

21. field-scoped method-scoped
final
volatile
synchronized (method/block)
java.util.concurrent.locks.Lock
Using the above keywords, a programmer can indicate that a JVM should
refrain from op#miza#ons that could otherwise cause concurrency issues.
In terms of the Java memory model, the above concepts introduce addiUonal
synchroniza#on ac#ons which introduce addiUonal (parUal) orders. Without
such modifiers, reads and writes might not be ordered (weak memory
model) what results in a data race.
A memory model is a trade-off between a language’s simplicity
(consistency/atomicity) and its performance.
JMM building blocks

22. • VolaUle reads/writes can not be reordered - Any write to a
volaUle/synchronizaUon variable establishes a happens-
before relaUonship with subsequent reads of that same
variable.
SynchronizaUon AcUons

23. What does volaUle do?
• Compilers and runUme are not allowed to allocate volaUle
variables in registers
• VolaUle longs and doubles are atomic
• VolaUle reads are very cheep (no locks compared to
synchronized)
• VolaUle increment is not atomic (!!!)
• Elements in volaUle collecUon are not volaUle (for example
volaUle int[])
• Consider using java.u6l.concurrent

24. • Also SA elements help to connect the
sequences of acUons between different
threads, in such way forming SynchronizaUon
Order:
SynchronizaUon AcUons

25. • But let’s examine the case, when we have
acUons not only at SA elements, what will
happen with x?
SynchronizaUon AcUons

26. • What if reading of v1 have idenUfied 5 in y?
SynchronizaUon AcUons

27. • What if reading of v1 have idenUfied 0 in y?
SynchronizaUon AcUons

28. • Now focus on another case:
SynchronizaUon AcUons

29. • And one more case:
SynchronizaUon AcUons

33. class ArrayTest {
volatile boolean[] ready = new boolean[] { false };
int answer = 0;
void thread1() {
while (!ready[0]);
assert answer == 42;
}
void thread2() {
answer = 42;
ready[0] = true;
}
}
Declaring an array to be volaUle does not make its elements vola#le! In the above
example, there is no write-read edge because the array is only read by any thread.
For such volaUle element access, use java.u#l.concurrent.atomic.AtomicIntegerArray.
Array Elements

34. • High level
– java.uUl.concurrent
• Low level
– synchronized() blocks and methods,
– java.uUl.concurrent.locks
• Low level primi#ves
– volaUle variables
– java.uUl.concurrent.atomic
SynchronizaUon

38. class ThreadLifeCycle {
int foo = 0;
void method() {
foo = 42;
new Thread() {
@Override
public void run() {
assert foo == 42;
}
}.start();
}
}
Thread start

43. instance = <allocate>;
instance.foo = 42;
<freeze instance.foo>
if (instance != null) {
assert instance.foo == 42;
}
Ame
. . .
. . .
happens-before order
dereference order
When a thread creates an instance, the instance’s final fields are frozen. The Java
memory model requires a field’s ini#al value to be visible in the iniUalized form to
other threads.
This requirement also holds for properUes that are dereferenced via a final field,
even if the field value’s properUes are not final themselves (memory-chain order).
constructor
Does not apply for (reflec#ve) changes outside of a constructor / class iniUalizer.
Final fields operaUons order

44. class FinalFieldExample {
final int x;
int y;
static FinalFieldExample f;
public FinalFieldExample() {
x = 3;
y = 4;
}
static void writer() {
f = new FinalFieldExample();
}
static void reader() {
if (f != null) {
int i = f.x;
int j = f.y;
}
}
}
Guaranteed value 3
4 or 0 !!
Final fields

45. What operaUons in Java are atomic?
• Read/write on variables of primiUve types
(except of long and double – Word Tearing
problem),
• Read/write on volaUle variables of primiUve
type (including long and double),
• All read/writes to references are always
atomic (hpp://bit.ly/2c8kn8i),
• All operaUons on java.uUl.concurrent.atomic
types,

46. Java Memory Model: Access atomicity
• In order to provide atomicity, we need to have
this support from hardware.
• Possible problems:
– The absence of hardware operaUons for read/
record of big values;
– The request to memory’s sub-system, for
example, at х86 you can’t place the data point at
crossing of cashlines.

47. foo/1 = 0x0000
foo/2 = 0xFFFF foo/2 = 0x0000
class WordTearing {
long foo = 0L;
void thread1() {
foo = 0x0000FFFF;
// = 2147483647
}
void thread2() {
foo = 0xFFFF0000;
// = -2147483648
}
}
main memory (32 bit)
processor cache (32 bit)
1
2
foo/1 = 0xFFFF
foo/2 = 0x0000
foo/2 = 0xFFFF
foo/1 = 0x0000
processor cache (32 bit)
foo/1 = 0xFFFF
Atomicity

50. Source: h*p://shipilev.net/blog/2014/safe-public-construcAon/
x86 ARM
1 thread 8 threads 1 thread 4 threads
final wrapper 2.256 2.485 28.228 28.237
enum holder 2.257 2.415 13.523 13.530
double-checked 2.256 2.475 33.510 29.412
synchronized 18.860 302.346 77.560 1291.585
Problem: how to publish an instance of a class that does not define its fields to be final?
measured in ns/op; conAnuous instance requests
Besides plain synchroniza#on and the double-checked locking idiom, Java offers:
1. Final wrappers: Where double-checked locking requires volaUle field access, this
access can be avoided by wrapping the published instance in a class that stores
the singleton in a final field.
2. Enum holder: By storing a singleton as a field of an enumeraUon, it is guaranteed
to be iniUalized due to the fact that enumeraUons guarantee full iniUalizaUon.
Safe iniUalizaUon and publicaUon

51. What other acUons need specific
order?

52. class Externalization {
int foo = 0;
void method() {
foo = 42;
jni();
}
native void jni(); /* {
assert foo == 42;
} */
}
A JIT-compiler cannot determine the side-effects of a naUve operaUon. Therefore,
external ac#ons are guaranteed to not be reordered.
External acUons include JNI, socket communicaUon, file system operaUons or
interacUon with the console (non-exclusive list).
program
order
External acUon

53. l LoadLoad
l StoreStore
l LoadStore
l StoreLoad
Memory Barriers

54. Required
barriers
2nd operation
1st operation
Normal Load Normal Store Volatile Load
MonitorEnter
Volatile Store
MonitorExit
Normal Load LoadStore
Normal Store StoreStore
Volatile Load
MonitorEnter
LoadLoad LoadStore LoadLoad LoadStore
Volatile Store
MonitorExit
StoreLoad StoreStore
The JSR-133 Cookbook for Compiler Writers
Memory Barriers

55. Processor

56. ARM PowerPC SPARC TSO x86 AMD64
load-load yes yes no no no
load-store yes yes no no no
store-store yes yes no no no
store-load yes yes yes yes yes
ARM
x86
Source: Wikipedia
Processor opUmizaUons

57. 64%
78% 42%
Americans owning a parUcular device in 2014.
Source: Pew Research center
Devices distribuUon

58. To sum up...
• Concurrent programming isn’t easy,
• Design your code for concurrency (make it right
before you make it fast),
• Do not code against the implementaUon. Code
against the specificaUon,
• Use higher level synchronizaUon wherever
possible,
• Watch out for useless synchronizaUon,
• Use Thread Safe Immutable objects
• Keep in mind the Happens-before rules

60. Further reading
• Aleksey Shipilëv: One Stop Page (hpp://bit.ly/2cqBt4x,
hpps://shipilev.net/blog/2014/jmm-pragmaUcs/ )
• Brian Goetz: Java Concurrency in PracUce (
hpp://amzn.to/2cloe76)
• Pugh’s “Fixing the JAVA Memory Model”
• Adve’s “Shared Memory Consistency Models: A
Tutorial”
• Dubois’ “Memory Access Buffering in MulUprocessors”
• Boehm’s “Threads cannot be implemented as a library”

61. l Loads are not reordered with other loads.
l Stores are not reordered with other stores.
l Stores are not reordered with older loads.
l Loads may be reordered with older stores to different loca#ons but not with
older stores to the same loca#on.
l In a mulUprocessor system, memory ordering obeys causality (memory ordering
respects transiUve visibility).
l In a mulUprocessor system, stores to the same locaUon have a total order.
l In a mul#processor system, locked instruc#ons have a total order.
l Loads and stores are not reordered with locked instruc#ons.
Intel x86/64 Memory model details

62. Memory barrier - LoadLoad
• The sequence: Load1; LoadLoad; Load2
• Ensures that Load1's data are loaded before data
accessed by Load2 and all subsequent load
instrucUons are loaded. In general, explicit
LoadLoad barriers are needed on processors that
perform speculaUve loads and/or out-of-order
processing in which waiUng load instrucUons can
bypass waiUng stores. On processors that
guarantee to always preserve load ordering, the
barriers amount to no-ops.
The JSR-133 Cookbook for Compiler Writers

63. Memory barrier - StoreStore
• The sequence: Store1; StoreStore; Store2
• Ensures that Store1's data are visible to other
processors (i.e., flushed to memory) before the
data associated with Store2 and all subsequent
store instrucUons. In general, StoreStore barriers
are needed on processors that do not otherwise
guarantee strict ordering of flushes from write
buffers and/or caches to other processors or main
memory.
The JSR-133 Cookbook for Compiler Writers

64. Memory barrier - LoadStore
• The sequence: Load1; LoadStore; Store2
• Ensures that Load1's data are loaded before
all data associated with Store2 and subsequent
store instrucUons are flushed. LoadStore
barriers are needed only on those out-of-order
processors in which waiUng store instrucUons
can bypass loads.
The JSR-133 Cookbook for Compiler Writers

65. Memory barrier - StoreLoad
• The sequence: Store1; StoreLoad; Load2
• Ensures that Store1's data are made visible to other processors (i.e.,
flushed to main memory) before data accessed by Load2 and all subsequent
load instrucUons are loaded. StoreLoad barriers protect against a
subsequent load incorrectly using Store1's data value rather than that from
a more recent store to the same locaUon performed by a different
processor. Because of this, on the processors discussed below, a StoreLoad
is strictly necessary only for separaUng stores from subsequent loads of the
same locaUon(s) as were stored before the barrier. StoreLoad barriers are
needed on nearly all recent mulUprocessors, and are usually the most
expensive kind. Part of the reason they are expensive is that they must
disable mechanisms that ordinarily bypass cache to saUsfy loads from write-
buffers. This might be implemented by lerng the buffer fully flush, among
other possible stalls.
The JSR-133 Cookbook for Compiler Writers

66. OperaUons order

67. Processor LoadStore LoadLoad StoreStore StoreLoad Data
dependency
orders
loads?
Atomic
Conditional
Other
Atomics
Atomics
provide
barrier?
sparc-TSO no-op no-op no-op membar
(StoreLoad)
yes CAS:
casa
swap,
ldstub
full
x86 no-op no-op no-op mfence or
cpuid or
locked
insn
yes CAS:
cmpxchg
xchg,
locked
insn
full
ia64 combine
with
st.rel or
ld.acq
ld.acq st.rel mf yes CAS:
cmpxchg
xchg,
fetchadd
target +
acq/rel
arm dmb
(see below)
dmb
(see below)
dmb-st dmb indirection
only
LL/SC:
ldrex/strex
target
only
ppc lwsync
(see below)
lwsync
(see below)
lwsync hwsync indirection
only
LL/SC:
ldarx/stwcx
target
only
alpha mb mb wmb mb no LL/SC:
ldx_l/stx_c
target
only
pa-risc no-op no-op no-op no-op yes build
from
ldcw
ldcw (NA)
The JSR-133 Cookbook for Compiler Writers
* The x86 processors supporting "streaming SIMD" SSE2 extensions require LoadLoad "lfence" only only in connection with
these streaming instructions.
Memory barriers - architecture