Non-blocking synchronization — what is it and why we (don't?) need it @ Riga Dev Day 2016

1. Alexey Fyodorov
Odnoklassniki
@23derevo
Non-blocking synchronization
what is it and why we (don't?) need it

2. What is this talk about?

3. 3
Concurrency

4. Who is this talk for?

5. 5
For concurrency
beginners
Sorry
Please go to another room

6. 6
For concurrency
beginners
Sorry
Please go to another room
For non-blocking
programming beginners
A short introduction
For advanced concurrent
programmers
We will talk about CAS/atomics
implementation details!
For hipsters We will cover internet hype!
Immutable vs. Mutable

7. Intro. Locking

8. 8

9. 9
Main Models
Shared Memory Messaging
write + read send + onReceive
Similar to how we program it Similar to how a real hardware works
Distributed ProgrammingConcurrent Programming

10. 10
Advantages of Parallelism
Resource utilization
Async handling
Simplicity
Utilization of several cores/CPUs
aka PERFORMANCE
Complexity goes to magic frameworks
• ArrayBlockingQueue
• ConcurrentHashMap
• Akka
Responsible services, Responsible UI

11. 11
Lock lock = new ReentrantLock()
Lock lock = new ReentrantLock(true)

12. 12
Fairness
Lock lock = new ReentrantLock(true)

13. 13
Locking in Java
Old School
wait()
notify()
notifyAll()
synchronized {
doSomething();
}
public synchronized foo() {
doSomethingElse();
} Lock lock = new ReentrantLock();
try {
lock.lock();
doSomething();
} finally {
lock.unlock();
} Since Java 5
In the Language In the Standard Library (JDK)
java.util.concurrent.*
java.util.concurrent.atomic.*

14. 14
Counter
public interface Counter {
long get();
void increment();
}

15. 15
Simple Counter
public interface Counter {
long get();
void increment();
}
public class SimpleCounter implements Counter {
long value = 0;
public long get() {
return value;
}
public void increment() {
value++;
}
}

16. 16
Volatile Counter
public class VolatileCounter implements Counter {
volatile long value = 0;
public long get() {
return value;
}
public void increment() {
value++;
}
}
public interface Counter {
long get();
void increment();
}

17. 17
Synchronized Counter
public class SynchronizedCounter implements Counter {
long value = 0;
public synchronized long get() {
return value;
}
public synchronized void increment() {
value++;
}
}
public interface Counter {
long get();
void increment();
}

18. 18
Disadvantages of Locking
• Deadlocks
• Priority Inversion
• Reliability
- What will happen if lock owner die?
• Performance
- Scheduler can push lock owner out
- No parallelism inside a critical section!

19. 19
Amdahl’s Law
α non-parallelizable part of the computation
1-α parallelizable part of the computation
p number of threads
Sp=
"
α#
%&α
'

20. We need something else!

21. Welcome to Non-blocking Synchronization

22. 22
If-Modify-Write
volatile int value = 0;
if (value == 0) {
value = 42;
}
No Atomicity
in terms of JMM

23. 23
Compare and Swap
int value = 0;
synchronized (...) {
if (value == 0) {
value = 42;
}
}

24. 24
Compare and Swap
int value = 0;
i.compareAndSet(0, 42);

25. 25
CAS Semantics
public class PseudoCAS {
private long value;
public synchronized long get() {
return value;
}
public synchronized long compareAndSwap(long expectedValue, long newValue) {
long oldValue = value;
if (oldValue == expectedValue) {
value = newValue;
}
return oldValue;
}
public synchronized boolean compareAndSet(long expectedValue, long newValue) {
return expectedValue == compareAndSwap(expectedValue, newValue);
}
}

26. 26
Pseudo-CAS Counter
public class PseudoCasLoopCounter implements Counter {
private PseudoCAS value = new PseudoCAS();
public long get() {
return value.get();
}
public void increment() {
long v;
do {
v = value.get();
} while (value.compareAndSet(v, v + 1));
}
}
public interface Counter {
long get();
void increment();
}

27. 27
Pseudo-CAS Counter
public class PseudoCasLoopCounter implements Counter {
private PseudoCAS value = new PseudoCAS();
public long get() {
return value.get();
}
public void increment() {
long v;
do {
v = value.get();
} while (value.compareAndSet(v, v + 1));
}
}
public interface Counter {
long get();
void increment();
}

28. CAS in Java

29. 29
CAS in Java
Since Java 5, JSR166
java.util.concurrent.atomic
• Scalars
• Field updaters
• Arrays
• Compound variables
• Accumulators/Adders
since Java 8

30. 30
Scalars
• AtomicBoolean
• AtomicInteger
• AtomicLong
• AtomicReference

31. 31
Scalars
• AtomicBoolean
• AtomicInteger
• AtomicLong
• AtomicReference

32. 32
• boolean compareAndSet(long expect, long update)
• long addAndGet(long delta)
• long getAndAdd(long delta)
• long getAndDecrement()
• long getAndIncrement()
• long incrementAndGet()
• …
AtomicLong

33. 33
• boolean compareAndSet(long expect, long update)
• long addAndGet(long delta)
• long getAndAdd(long delta)
• long getAndDecrement()
• long getAndIncrement()
• long incrementAndGet()
• …
AtomicLong

34. 34
CAS Counter
public interface Counter {
long get();
void increment();
}
public class CasLoopCounter implements Counter {
private AtomicLong value = new AtomicLong();
public long get() {
return value.get();
}
public void increment() {
long v;
do {
v = value.get();
} while (value.compareAndSet(v, v + 1));
}
}

35. 35
Compare and Swap — Hardware Support
compare-and-swap
CAS
load-link / store-conditional
LL/SC
cmpxchg
ldrex/strex lwarx/stwcx

36. 36
CAS Disadvantages
Contended CAS —> tons of useless CPU cycles
do {
v = value.get();
} while (value.compareAndSet(v, v + 1));
Writing fast and correct CAS algorithms requires an expertise

37. 37
• boolean compareAndSet(long expect, long update)
• long addAndGet(long delta)
• long getAndAdd(long delta)
• long getAndDecrement()
• long getAndIncrement()
• long incrementAndGet()
• …
AtomicLong

38. 38
Get-and-Add Counter
public interface Counter {
long get();
void increment();
}
public class CasLoopCounter implements Counter {
private AtomicLong value = new AtomicLong();
public long get() {
return value.get();
}
public void increment() {
value.getAndAdd(1);
}
}

39. 39
39
atomicLong.getAndAdd(5)
JDK 7u95 JDK 8u74
83
46
15 11
132
105
45 43
1 2 3 4
ops / μs
threads

40. 40
40
atomicLong.getAndAdd(5)
JDK 7u95 JDK 8u74
83
46
15 11
132
105
45 43
1 2 3 4
ops / μs
threads

41. 41
41
atomicLong.getAndAdd(5)
lock addq $0x5,0x10(%rbp))loop:
mov 0x10(%rbx),%rax
mov %rax,%r11
add $0x5,%r11
lock cmpxchg %r11,0x10(%rbx)
sete %r11b
movzbl %r11b,%r11d
test %r10d,%r10d
je loop
JDK 7u95 -XX:+PrintAssembly JDK 8u74 -XX:+PrintAssembly
83
46
15 11
132
105
45 43
1 2 3 4
ops / μs
threads

42. 42
AtomicLong.getAndAdd() — JDK 7
cmpxchg

43. 43
AtomicLong.getAndAdd() — JDK 8
lock addq
JVM
Intrinsic

44. 44
Multivariable Invariant

45. 45
Multivariable Invariant

46. 46
Field Updaters
• AtomicIntegerFieldUpdater
- Reflection-based updater for volatile int
• AtomicLongFieldUpdater
- Reflection-based updater for volatile long
• AtomicReferenceFieldUpdater
- Reflection-based updater for volatile object

47. 47
AtomicLongFieldUpdater
long addAndGet(T obj, long delta)
boolean compareAndSet(T obj, long exp, long upd)
long getAndAdd(T obj, long delta)
long incrementAndGet(T obj)

48. 48
Volatile Counter
public class VolatileCounter implements Counter {
volatile long value = 0;
public long get() {
return value;
}
public void increment() {
value++;
}
}
public interface Counter {
long get();
void increment();
}

49. 49
AtomicLongFieldUpdater-based Counter
public class AFUCounter implements Counter {
private final VolatileCounter counter = new VolatileCounter();
AtomicLongFieldUpdater<VolatileCounter> updater
= AtomicLongFieldUpdater.newUpdater(VolatileCounter.class, "value");
public AFUCounter() throws NoSuchFieldException {
Field field = VolatileCounter.class.getDeclaredField("value");
field.setAccessible(true);
}
public long get() {
return updater.get(counter);
}
public void increment() {
updater.addAndGet(counter, 1);
}
}

50. 50
AtomicLongFieldUpdater

51. 51
AtomicLongFieldUpdater

52. 52
AtomicArrays
AtomicIntegerArray
AtomicLongArray
AtomicReferenceArray
• long addAndGet(int i, long delta)
• long getAndAdd(int i, long delta)
• boolean compareAndSet(
int i, long exp, long upd)
• long incrementAndGet(int i)
• …

53. 53
Compound Variables
AtomicMarkableReference
V сompareAndSet( V expectedRef, V newRef, boolean expectedMark, boolean newMark)
AtomicStampedReference
boolean compareAndSet( V expectedRef, V newRef, int expectedStamp, int newStamp)

54. 54
Accumulators
• DoubleAccumulator
• DoubleAdder
• LongAccumulator
• LongAdder
• (Striped64)
• void accumulate(long x)
• long get()
• long getThenReset()
• void reset()

55. 55
Non-blocking Guarantees
Wait-Free Per-thread progress is guaranteed
Lock-Free Overall progress is guaranteed
Obstruction-Free Overall progress is guaranteed if threads
don’t interfere with each other

56. 56
AtomicLong-based Counter
public interface Counter {
long get();
void increment();
}
public class CasCounter implements Counter {
private AtomicLong value = new AtomicLong();
public long get() {
return value.get();
}
public void increment() {
long v;
do {
v = value.get();
} while (value.compareAndSet(v, v + 1));
}
}
Any guarantees?
A. Wait-Free
B. Lock-Free
C. Obstruction-Free
D. No guarantees

57. 57
AtomicLong-based Counter
public interface Counter {
long get();
void increment();
}
public class CasCounter implements Counter {
private AtomicLong value = new AtomicLong();
public long get() {
return value.get();
}
public void increment() {
long v;
do {
v = value.get();
} while (value.compareAndSet(v, v + 1));
}
}
Any guarantees?
A. Wait-Free
B. Lock-Free
C. Obstruction-Free
D. No guarantees

58. 58
AtomicLong-based Counter
public interface Counter {
long get();
void increment();
}
public class CasCounter implements Counter {
private AtomicLong value = new AtomicLong();
public long get() {
return value.get();
}
public void increment() {
long v;
do {
v = value.get();
} while (value.compareAndSet(v, v + 1));
}
}
Any guarantees on x64?
A. Wait-Free
B. Lock-Free
C. Obstruction-Free
D. No guarantees

59. 59
AtomicLong-based Counter
public interface Counter {
long get();
void increment();
}
public class CasCounter implements Counter {
private AtomicLong value = new AtomicLong();
public long get() {
return value.get();
}
public void increment() {
long v;
do {
v = value.get();
} while (value.compareAndSet(v, v + 1));
}
}
Any guarantees on x64?
A. Wait-Free
B. Lock-Free
C. Obstruction-Free
D. No guarantees

60. Non-blocking Data Structures

61. 61
Non-blocking Stack

62. 62
Non-blocking queue
Michael and Scott, 1996
https://www.research.ibm.com/people/m/michael/podc-1996.pdf
Threads help each other

63. References

64. 64
Books

65. 65
Concurrency-interest
http://altair.cs.oswego.edu/mailman/listinfo/concurrency-interest
Doug Lee and Co

66. 66
Q & A

67. 67
Thank you!