2. Concurrency
Examples for .NET

4. Responsive

5. Performance
Scalable algorithms

6. Three pillars of Concurrency
 Scalability (CPU)
 Parallel.For
 Responsiveness
 Task
 async/await
 Consistency
 lock
 Interlocked.*
 Mutex/Event/Semaphore
 Monitor

7. Scalability

9. Which is fastest?
var ints = new int[InnerLoop];
var random = new Random();
for (var inner = 0; inner < InnerLoop; ++inner)
{
ints[inner] = random.Next();
}
// ------------------------------------------------
var ints = new int[InnerLoop];
var random = new Random();
Parallel.For(
0,
InnerLoop,
i => ints[i] = random.Next()
);

10. SHARED STATE  Race condition
var ints = new int[InnerLoop];
var random = new Random();
for (var inner = 0; inner < InnerLoop; ++inner)
{
ints[inner] = random.Next();
}
// ------------------------------------------------
var ints = new int[InnerLoop];
var random = new Random();
Parallel.For(
0,
InnerLoop,
i => ints[i] = random.Next()
);

11. SHARED STATE  Poor performance
var ints = new int[InnerLoop];
var random = new Random();
for (var inner = 0; inner < InnerLoop; ++inner)
{
ints[inner] = random.Next();
}
// ------------------------------------------------
var ints = new int[InnerLoop];
var random = new Random();
Parallel.For(
0,
InnerLoop,
i => ints[i] = random.Next()
);

13. Then and now
Metric VAX-11/750 (’80) Today Improvement
MHz 6 3300 550x
Memory MB 2 16384 8192x
Memory MB/s 13 R ~10000
W ~2500
770x
190x

14. Then and now
Metric VAX-11/750 (’80) Today Improvement
MHz 6 3300 550x
Memory MB 2 16384 8192x
Memory MB/s 13 R ~10000
W ~2500
770x
190x
Memory nsec 225 70 3x

15. Then and now
Metric VAX-11/750 (’80) Today Improvement
MHz 6 3300 550x
Memory MB 2 16384 8192x
Memory MB/s 13 R ~10000
W ~2500
770x
190x
Memory nsec 225 70 3x
Memory cycles 1.4 210 -150x

16. 299,792,458 m/s

18. Speed of light is too slow

19. 0.09 m/c

21. 99% - latency mitigation
1% - computation

22. 2 Core CPU
RAM
L3
L2
L1
CPU
L2
L1
CPU

23. 2 Core CPU – L1 Cache
L1
CPU
L1
CPU
new Random ()
new int[InnerLoop]

24. 4 Core CPU – L1 Cache
L1
CPU
L1
CPU
L1
CPU
L1
CPU
new Random ()
new int[InnerLoop]

25. 2x4 Core CPU
RAM
L3
L2
L1
CPU
L2
L1
CPU
L2
L1
CPU
L2
L1
CPU
L3
L2
L1
CPU
L2
L1
CPU
L2
L1
CPU
L2
L1
CPU

26. Solution 1 – Locks
var ints = new int[InnerLoop];
var random = new Random();
Parallel.For(
0,
InnerLoop,
i => {lock (ints) {ints[i] = random.Next();}}
);

27. Solution 2 – No sharing
var ints = new int[InnerLoop];
Parallel.For(
0,
InnerLoop,
() => new Random(),
(i, pls, random) =>
{ints[i] = random.Next(); return random;},
random => {}
);

28. Parallel.For adds overhead
Level0
Level1
Level2
ints[0]
ints[1]
Level2
ints[2]
ints[3]
Level1
Level2
ints[4]
ints[5]
Level2
ints[6]
ints[7]

29. Solution 3 – Less overhead
var ints = new int[InnerLoop];
Parallel.For(
0,
InnerLoop / Modulus,
() => new Random(),
(i, pls, random) =>
{
var begin = i * Modulus ;
var end = begin + Modulus ;
for (var iter = begin; iter < end; ++iter)
{
ints[iter] = random.Next();
}
return random;
},
random => {}
);

30. var ints = new int[InnerLoop];
var random = new Random();
for (var inner = 0; inner < InnerLoop; ++inner)
{
ints[inner] = random.Next();
}

31. Solution 4 – Independent runs
var tasks = Enumerable.Range (0, 8).Select (
i => Task.Factory.StartNew (
() =>
{
var ints = new int[InnerLoop];
var random = new Random ();
while (counter.CountDown ())
{
for (var inner = 0; inner < InnerLoop; ++inner)
{
ints[inner] = random.Next();
}
}
},
TaskCreationOptions.LongRunning))
.ToArray ();
Task.WaitAll (tasks);

32. Parallel.For
Only for CPU bound problems

33. Sharing is bad
Kills performance
Race conditions
Dead-locks

34. Servers have natural concurrency
Avoid Parallel.For

35. Act like an engineer
Measure before and after

36. One more thing…

37. Mårten Rånge
marran@wcom.se