Life as a software engineer is so exciting! Computing power continue to rise exponentially, software demands continue to rise exponentially as well, so far so good. The bad news are that in the last decade the computing power of single threaded application remains almost flat.
If you decide to continue ignoring concurrency and multi-threading the gap between the problems you are able to solve and your hardware capabilities will continue to rise. In this session we will discuss different approaches for taming the concurrency beast, such as shared mutability,shared immutability and isolated mutability actors, STM, etc we will discuss the shortcomings and the dangers of each approach and we will compare different programming languages and how they choose to tackle/ignore concurrency.
2. About
Me:
Haim
Yadid
•21 Years of SW development experience
•Performance Expert
•Consulting R&D Groups
•Training: Java Performance Optimization
3.
4. Moore’s
Law
•Number of transistors doubles constantly
•CPU frequency à
stalled.
•Performance boost through parallelism
5. Yes….
But
•Performance is not a problem anymore
•We prefer commodity hardware
•We have Hadoop and Big Data!!!
•Hardware is cheap.
•Several processes will do
•My programming language
will protect me
7. Concurrency
•Decomposition of your program
into independently executing processes
•About structure
•About design
•It is not something you code it is
something you architect
14. Liveliness
Problems
•Deadlock
•Starvation
•Waiting for a train that will never come
func count() {
for i := 0; i < 1000; i++ {
fmt.Println(i)
time.Sleep(10 * time.Millisecond)
}
}
func main() {
go count()
time.Sleep(3000 * time.Millisecond)
for {}
GOMAXPROCS=2
}
Go
15. Data
Races
•Inopportune interleaving
•Stale values
•Loosing updates
•Infinite loops
class Foo {
private HashSet h = new HashSet();
!
}
!
boolean introduceNewVal(Object v) {
if (!h.contains(v)) {h.add(v); return true; }
return false;
}
Java
17. Whats
wrong
here?
•IncrementX accessed from ThreadX
•IncrementY accessed from Thread Y
•An aggregator thread will read both
Class
T
{
volatile
int
x
=
0;
volatile
int
y=0;
!
long incrementX() { x++; }
long incrementY() { y++; }
}
False
sharing:
cache
coherency
-‐>
hitting
same
cache
line
Java
23. Serial
Execution
•Three queries to DB executed
•In Serial
•Long response time
doGet(req,resp) {
rs1 = runQuery1()
rs2 = runQuery2()
rs3 = runQuery3()
resp.write(mergeResults(rs1,rs2,rs3))
}
Can
be
parallelized
Pseudo(Java)
24. Create
Threads
•Run three queries in parallel…..
•but:
doGet(req,resp) {
q1 = new Query();
new Thread(q1).start();
q2 = new Query();
new Thread(q2).start();
q3 = new Query();
new Thread(q3).start();
!
}
rs1 = q1.getRs();
rs2 = q2.getRs();
rs3 = q3.getRs();
resp.write(mergeResults(rs1,rs2,rs3));
Thread
leak
+
Thread
creation
overhead
+
data
races
Pseudo(Java)
25. Thread
Pool
doGet(req,resp) {
ExecutorService e = Executors.newFixedThreadPool(3);
ArrayList tasks = …;
tasks.add(new QueryTask(Query1)) …. 3
List<Future<Integer>> fs;
fs = e.invokeAll(tasks); // invoke all in parallel
ArrayList results = …
for (Future<Integer> f : l) { // collect results
if (f.isDone()) {
results.add(f.get());
}
}
resp.write(mergeResults(results));
}
Thread
pool
leak
+
Thread
pool
creation
overhead
Pseudo(Java)
26. Thread
Pool
2
static ExecutorService e = Executors.newFixedThreadPool(3);
!
doGet(req,resp) {
ArrayList tasks = …;
tasks.add(new QueryTask(Query1)) …. 3
List<Future<Integer>> fs;
fs = e.invokeAll(tasks); // invoke all in parallel
!
}
ArrayList results = …
for (Future<Integer> f : l) { // collect results
if (f.isDone()) {
res.add(f.get());
}
}
resp.write(mergeResults(results));
Size
?/
share
thread
pool?
/
name
thread
pool
threads
Pseudo(Java)
27. Same
Example
With
Go
func
execQuery(query
string,
c
chan
*Row)
{
c
<-‐
db.Query(query)
}
!
func
doGet(req,resp) {
c := make(chan *Row)
go execQuery(query1,c)
go execQuery(query2,c)
go execQuery(query3,c)
for i := 0; i<3 ; i++
combineRs(rs <-c)
}
Pseudo(Go)
28.
29. State
Management
•Eventually we need to have state
•It is easy to deal with state when we have
one Sid
•But what happens when there are several
•We have three approaches
•Most are familiar with only one
33. Visibility
•Change made by sid1 is visible to sid2?
•Solutions
•volatile keyword
•Memory Model(Happens before)
Memory
L3
cache
L2
cache
L1
cache
•compiler reordering
Registers
•Caches
CPU
•Not so simple
34. Atomicity
•What can be done in a single step ?
•CAS constructs (Compare and swap)
•AtomicInteger
•AtomicLong
•ConcurrenctHashMap putIfAbsent
35. Atomicity
is
not
Viral
•An (almost) real example
•A non transactional database
•Balance per user
•Use atomicity to solve the problem
class
User
{
private
AtomicLong
balance
=
…..
!
int updateBalance(int diff) {
long temp = balance.addAndGet(diff);
setToDB(temp);
}
}
Java
39. STM
(Optimistic)
•Software Transactional Memory
•Transactional semantics ACI: (not Durable)
•Atomic,
•Consistent and
•Isolated
•No deadlocks - when collision retry!
•Clojure refs , Akka refs
STM
performance
problem
when
there
are
too
many
mutations.
41. Mutliverse
STM
Example
import org.multiverse.api.references.*;
import static org.multiverse.api.StmUtils.*;
!
!
!
public class Account{
private final TxnRef<Date> lastUpdate;
private final TxnInteger balance;
public Account(int balance){
this.lastUpdate = newTxnRef<Date>(new Date());
this.balance = newTxnInteger(balance);
}
Java
42. Mutliverse
STM
Example
public void incBalance(final int amount, final Date date){
atomic(() ->{
balance.inc(amount);
lastUpdate.set(date);
!
if(balance.get()<0){
throw new IllegalStateException("Not enough money");
}
});
}
}
Java8
43. Mutliverse
STM
Example
!
public static void transfer(final Account from, final Account to, final int amount){ Java8
atomic(()->{
Date date = new Date();
from.incBalance(-amount, date);
to.incBalance(amount, date);
});
}
Retry
ahead
beware
of
side
effects
44.
45. Pure
Immutability
•We have shared state
•But Shared state is read only (after
construction)
•No concurrency issues
•No deadlocks
•No race conditions
•No stale values
•Optimal for cache
47. Immutable
Object
Example
Object cannot be changed after
construction
all fields are final
public final Class MySet {
private final Set<String> vals = new HashSet<String>();
public MySet(String names[]) {
for(name:names) vals.add(name);
}
public boolean containsVal(String name);…..
}
Java
48. CopyOnWrite
Collections
•Any changes to it will create a new copy
•Safe
•Fast read, read without synchronisation
•Iteration is fast
•do not support remove() set() add()
Bad
performance
when
mutation
rate
is
high
52. Example
Customization
Cache
•A Web application server
•Serving Complicated and customisable UI
•Each user has it’s own customization
(potentially)
•Classic for immutable collection
•Low mutation rate
•High read rate
53. Example
Customization
Cache
•Customization Data is immutable
•Customization Data HashMap is a
Persistent Map
•Cache is represented by a single STM
reference
•Update will fail if two are performing it
at once
54. Immutability
and
GC
•Immutability is great
•But: Generates of a lot of objects
•When done for short lived objects GC can
cope with it
•Long lived immutable objects/collections
which change frequently may cause GC to
have low throughput and high pause times
55.
56. Isolated
Mutability
•No shared state
•Each Sid has its own pile of sand
•Message passing between Sids
•Prefer passing immutable objects
60. Building
a
Monitoring
System
•Monitoring System (e.g. Nagios)
•~100k of monitors
•running periodically
•Each one has a state.
•Consumers are able to query state.
•Some monitors may affect other monitor
state
62. Monitor
Actors
•MonitorStateActor(MSA)
•Alway readys to be queried
•State updated by message from MRA
•Stateless
•MonitorRecalculateActor(MRA)
•Maybe recalculating and not responsive
•Stateful
•Supervises MSA
63. MonitorsCache
•An immutable cache - holds all actor refs
•Single view of the world.
•Used by SchedulerActor and Query
Actor
•May have several objects managed By
STM
65. Further
Reading
•Java Concurrency In Practice /Brian Goetz
•Effective Akka / Jamie Allen
•Clojure High Performance Programming /
Shantanu Kumar
•Programming Concurrency on the JVM:
Mastering Synchronization, STM, and
Actors /Subramaniam, Venkat