Herb Sutter (GotW.ca) says that the concept of Concurrency is easier understood if split into three sub concepts; scalability, responsiveness and consistency. This presentation is the first of three covering these concepts, starting off with everyone’s favorite: Scalability – i.e. splitting a CPU-bound problem onto several cores in order to solve the problem faster. I will show what tools what .NET offer but also performance pitfalls that arise from an escalating problem that plagued computer architecture for the last 20 years.

1. Mårten Rånge
WCOM AB
@marten_range

2. Concurrency
Examples for .NET

4. Responsive

5. Performance
Scalable algorithms

6. Three pillars of Concurrency
 Scalability (CPU)
 Parallel.For
 Responsiveness
 Task/Future
 async/await
 Consistency




lock/synchronized
Interlocked.*
Mutex/Event/Semaphore
Monitor

7. Scalability

9. Which is fastest?
var ints = new int[InnerLoop];
var random = new Random();
for (var inner = 0; inner < InnerLoop; ++inner)
{
ints[inner] = random.Next();
}
// -----------------------------------------------var ints = new int[InnerLoop];
var random = new Random();
Parallel.For(
0,
InnerLoop,
i => ints[i] = random.Next()
);

10. SHARED STATE  Race condition
var ints = new int[InnerLoop];
var random = new Random();
for (var inner = 0; inner < InnerLoop; ++inner)
{
ints[inner] = random.Next();
}
// -----------------------------------------------var ints = new int[InnerLoop];
var random = new Random();
Parallel.For(
0,
InnerLoop,
i => ints[i] = random.Next()
);

11. SHARED STATE  Poor performance
var ints = new int[InnerLoop];
var random = new Random();
for (var inner = 0; inner < InnerLoop; ++inner)
{
ints[inner] = random.Next();
}
// -----------------------------------------------var ints = new int[InnerLoop];
var random = new Random();
Parallel.For(
0,
InnerLoop,
i => ints[i] = random.Next()
);

13. Then and now
Metric
VAX-11/750 (’80)
Today
Improvement
MHz
6
3300
550x
Memory MB
2
16384
8192x
Memory MB/s
13
R ~10000
W ~2500
770x
190x

14. Then and now
Metric
VAX-11/750 (’80)
Today
Improvement
MHz
6
3300
550x
Memory MB
2
16384
8192x
Memory MB/s
13
Memory nsec
225
R ~10000
W ~2500
70
770x
190x
3x

15. Then and now
Metric
VAX-11/750 (’80)
Today
Improvement
MHz
6
3300
550x
Memory MB
2
16384
8192x
Memory MB/s
13
Memory nsec
225
R ~10000
W ~2500
70
770x
190x
3x
Memory cycles
1.4
210
-150x

16. 299,792,458 m/s

18. Speed of light is too slow

19. 0.09 m/c

21. 99% - latency mitigation
1% - computation

22. 2 Core CPU
CPU1
CPU2
L1
L1
L2
L2
L3
RAM

23. 2 Core CPU – L1 Cache
CPU1
L1
CPU2
new Random ()
new int[InnerLoop]
L1

24. 2 Core CPU – L1 Cache
CPU1
CPU2
Random object
Random object
L1
L1

25. 2 Core CPU – L1 Cache
CPU1
CPU2
Random object
Random object
L1
L1

26. 2 Core CPU – L1 Cache
CPU1
CPU2
Random object
Random object
L1
L1

27. 2 Core CPU – L1 Cache
CPU1
CPU2
Random object
Random object
L1
L1

28. 2 Core CPU – L1 Cache
CPU1
CPU2
Random object
Random object
L1
L1

29. 2 Core CPU – L1 Cache
CPU1
CPU2
Random object
Random object
L1
L1

30. 4 Core CPU – L1 Cache
CPU1
L1
CPU2
L1
CPU3
new Random ()
new int[InnerLoop]
L1
CPU4
L1

31. 2x4 Core CPU
CPU1 CPU2 CPU3 CPU4
CPU5 CPU6 CPU7 CPU8
L1
L1
L1
L1
L1
L1
L1
L1
L2
L2
L2
L2
L2
L2
L2
L2
L3
L3
RAM

32. Solution 1 – Locks
var ints = new int[InnerLoop];
var random = new Random();
Parallel.For(
0,
InnerLoop,
i => {lock (ints) {ints[i] = random.Next();}}
);

33. Solution 2 – No sharing
var ints = new int[InnerLoop];
Parallel.For(
0,
InnerLoop,
() => new Random(),
(i, pls, random) =>
{ints[i] = random.Next(); return random;},
random => {}
);

34. Parallel.For adds overhead
Level2
Level1
Level2
Level0
Level2
Level1
Level2
ints[0]
ints[1]
ints[2]
ints[3]
ints[4]
ints[5]
ints[6]
ints[7]

35. Solution 3 – Less overhead
var ints = new int[InnerLoop];
Parallel.For(
0,
InnerLoop / Modulus,
() => new Random(),
(i, pls, random) =>
{
var begin
= i * Modulus
;
var end
= begin + Modulus
;
for (var iter = begin; iter < end; ++iter)
{
ints[iter] = random.Next();
}
return random;
},
random => {}
);

36. var ints = new int[InnerLoop];
var random = new Random();
for (var inner = 0; inner < InnerLoop; ++inner)
{
ints[inner] = random.Next();
}

37. Solution 4 – Independent runs
var tasks = Enumerable.Range (0, 8).Select (
i => Task.Factory.StartNew (
() =>
{
var ints = new int[InnerLoop];
var random = new Random ();
while (counter.CountDown ())
{
for (var inner = 0; inner < InnerLoop; ++inner)
{
ints[inner] = random.Next();
}
}
},
TaskCreationOptions.LongRunning))
.ToArray ();
Task.WaitAll (tasks);

38. Parallel.For
Only for CPU bound problems

39. Sharing is bad
Kills performance
Race conditions
Dead-locks

40. Cache locality
RAM is a misnomer
Class design
Avoid GC

41. Natural concurrency
Avoid Parallel.For

42. Act like an engineer
Measure before and after

43. One more thing…

44. http://tinyurl.com/wcom-cpuscalability

45. Mårten Rånge
WCOM AB
@marten_range