1. PARALLEL & ASYNC
PROCESSING USING TPL
DATAFLOW
Petru Rebeja

3. AGENDA
• What is Dataflow?
• When to use it?
• How to use it?
• Q&A

4. THE BIG PICTURE
CLR Thread Pool
Tasks
PLINQ Parallel Loops
Concurrent Collections
Dataflow

5. DATAFLOW BENEFITS
• Effortless use of multi-threading
• Performance boost via painless optimization
• Development focus is on the ‘what’ rather than ‘how’

6. DATAFLOW USAGES
High throughput, low-latency scenarios
Robotics
Manufacturing
Imaging Biology
Oil & Gas
Finance

7. PROGRAMMING MODEL
• Actor-based programming
• In-process message passing
• Components (blocks) for creating data processing pipelines

8. ARCHITECTURE
IDataflowBlock
ISourceBlock<TOutput> ITargetBlock<TInput>
IPropagatorBlock<Tinput,Toutput>

9. COMPOSITION
Source
Target
Propagator
Optional
Transform

10. BUFFERING BLOCKS
BufferBlock<T>
BroadcastBlock<T>
WriteOnceBlock<T>

11. EXECUTION BLOCKS
ActionBlock<T>
TransformBlock<T,V>
TransformManyBlock<T,V>

12. GROUPING BLOCKS
BatchBlock<T>
JoinBlock<T1,T2,…>
BatchedJoinBlock<T1,T2>

13. BEHAVIOR CONFIGURATION OPTIONS
• BufferBlock<T>
• BroadcastBlock<T>
• WriteOnceBlock<T>
DataflowBlockOptions
• ActionBlock<T>
• TransformBlock<TIn, TOut>
• TransformManyBlock<TIn, TOut>
ExecutionDataflowBlockOptions
• BatchBlock<T>
• JoinBlock<T1, T2[, T3]>
• BatchedJoinBlock<T1, T2>
GroupingDataflowBlockOptions

14. COMPLETION & CANCELLATION
• To know when a block completes await block.Completion
or add a continuation task to it
• To propagate completion from source to target, set
DataflowLinkOptions.PropagateCompletion when
linking
• Set DataflowBlockOptions.CancellationToken to
enable cancellation

15. ERROR HANDLING
• If the exception does not affect the integrity of the
pipeline – use a try/catch inside the block
• Otherwise, handle errors outside of the pipeline by
• Adding a continuation to block.Completion
• Propagating errors through the pipeline

16. DEALING WITH CONCURRENCY
• Rule of thumb: avoid shared state whenever possible.
• Use ConcurrentExclusiveSchedulerPair to perform
updates on shared state
• Be aware of the caveats with
ConcurrentExclusiveSchedulerPair

17. CREATING CUSTOM BLOCKS
The easy way:
DataflowBlock.Encapsulate<TInput, TOutput>(
target, source)

18. CREATING CUSTOM BLOCKS
The hard(core) way:
class CustomBlock:
IPropagatorBlock<TInput, TOutput>
{
}

19. CREATING CUSTOM BLOCKS
Either way you choose, don’t forget to:
• Propagate completion
• Pool for cancellation

20. REFERENCES & FURTHER READING
Dataflow (Task Parallel Library) http://msdn.microsoft.com/en-us/library/hh228603(v=vs.110).aspx
Stephen Toub
TPL Dataflow Tour
http://channel9.msdn.com/posts/TPL-Dataflow-Tour
Joseph Albahari
The Future of .NET Parallel
Programming
http://channel9.msdn.com/events/TechEd/Australia/Tech-Ed-Australia-
2011/DEV308
Stephen Toub
Inside TPL Dataflow
http://channel9.msdn.com/Shows/Going+Deep/Stephen-Toub-Inside-TPL-
Dataflow
Alexey Kursov
Pipeline TPL Dataflow Usage examples
https://www.youtube.com/watch?v=AI9KxgDF43k
https://www.youtube.com/watch?v=AI9KxgDF43k
Richard Blewett, Andrew Clymer
Pro Asynchronous Programming with
.NET
APRESS 2013
ISBN: 978-1430259206
AKKA.NET http://getakka.net/

21. QUESTIONS?

22. THANK YOU!
Petru.Rebeja@gmail.com
Parallel & Async Processing using TPL
Dataflow