More Related Content Similar to Asynchronous Event Streams – when java.util.stream met org.osgi.util.promise! - Tim Ward (20) Asynchronous Event Streams – when java.util.stream met org.osgi.util.promise! - Tim Ward1. Copyright © 2005 - 2015 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event - Asynchronous Event Streams Nov 2015
Asynchronous Event Streams – when
java.util.stream met org.osgi.util.promise!
Tim Ward
http://www.paremus.com
info@paremus.com
2. Copyright © 2005 - 2015 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event - Asynchronous Event Streams Nov 2015
•Chief Technology Officer at Paremus
•7 years developing OSGi specifications
•Chair of the OSGi IoT Expert Group
•Interested in Asynchronous Distributed Systems
•Author of Manning’s Enterprise OSGi in Action
•http://www.manning.com/cummins
Who is Tim Ward?
@TimothyWard
3. Copyright © 2005 - 2015 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event - Asynchronous Event Streams Nov 2015
Data Structure and Algorithms
4. Copyright © 2005 - 2015 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event - Asynchronous Event Streams Nov 2015
You all remember Computer Science lectures, right?
Whether or not we think about them much, data structures are really important
Knowing when you want a List, Set or Map is a good start
Knowing the drawbacks of various implementation types is better!
Thankfully, most of the time it doesn’t matter which implementation you use
N is typically small enough for it not to matter
Modern computers are so fast that even a bad algorithm is “instant”
Most code paths aren’t super hot
The JIT Compiler is often indistinguishable from magic
5. Copyright © 2005 - 2015 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event - Asynchronous Event Streams Nov 2015
Working with simple data structures
At their most basic level data structures are “holders” for data
All you can really do is add/remove a value, or look at the existing value(s)
17 23 11 1931
The order in which the values appear is typically determined by the structure
Some data structures restrict visibility to a first/last value
We typically refer to processing the values in a data structure as “iteration”
6. Copyright © 2005 - 2015 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event - Asynchronous Event Streams Nov 2015
Iteration in Java
We’ve probably all written this code:
for(int i =0; i < list.size(); i ++) {
MyData data = list.get(i);
…
} and this code:
Iterator<MyData> it = list.iterator();
while(it.hasNext()) {
MyData data = it.next();
…
}and this code:
for(MyData data : list) {
…
}
Java’s collections make iteration easy
7. Copyright © 2005 - 2015 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event - Asynchronous Event Streams Nov 2015
Problems with iteration
Whilst Java iteration is easy, it’s still easy to make mistakes:
This is bad for Linked Lists!
for(int i =0; i < list.size(); i ++) {
MyData data = list.get(i);
…
}
“External” iteration also pushes control logic into your code!
Java 8 updated the Collections API to support “internal” iteration
Powerful functional concepts were added via the Stream API
8. Copyright © 2005 - 2015 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event - Asynchronous Event Streams Nov 2015
Iteration in Java 8
Streaming collections is very simple:
list.stream().forEach(data -> …);
Internal iteration separates the “what” from the “how” in your code
Parallel processing can occur implicitly if the list supports it
Functional pipelines allow for easy processing
list.stream()
.map(MyData::getAge)
.filter(i -> i < 15)
.count();
intermediate operations
terminal operation
9. Copyright © 2005 - 2015 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event - Asynchronous Event Streams Nov 2015
Properties of Java 8 Streams
Java 8 Streams have a number of useful properties:
They are lazy
Streams only process data on-demand
This is triggered by a “terminal operation”
They can “short circuit”
Some operations don’t need to see the whole stream
findFirst() and findAny() can return if an element is found
Importantly the Stream is “pulling” the data from the data structure
10. Copyright © 2005 - 2015 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event - Asynchronous Event Streams Nov 2015
Streams of data
11. Copyright © 2005 - 2015 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event - Asynchronous Event Streams Nov 2015
Streams - the overloaded concept
Before Java 8, a “stream” of data was a java.io.InputStream
Whilst you probably didn’t think of it that way, you still iterated
int read;
while((read = is.read()) != -1) {
byte data = (byte) read;
…
}
The big difference with an InputStream is that it may block
A thread may get “stuck” waiting for user input, or a slow network
Java NIO has non-blocking input, but is much harder to use
12. Copyright © 2005 - 2015 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event - Asynchronous Event Streams Nov 2015
Streams - the overloaded concept (2)
An InputStream behaves like an ordered Collection of bytes
Using a Java 8 Stream over these bytes could make sense
is.forEach(data -> …)
But the InputStream may block the thread indefinitely
If the input is asynchronous then resources are wasted by waiting
A slow function may not process data as fast as it arrives
Rapid bursts of data may overload the consumer
All of this is independent of the data, be it bytes or Objects
Java 8 Streams aren’t able to cope with asynchronous data
13. Copyright © 2005 - 2015 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event - Asynchronous Event Streams Nov 2015
Push-based asynchronous streams
14. •Asynchronous systems offer better performance
•This does assume that there isn’t a single bottleneck!
•Asynchronous Distributed systems typically scale better
•Less time spent waiting for high-latency calls
•More reasonable failure characteristics
•Parallelism is easier to exploit
•Network Infrastructure is intrinsically asynchronous
•Network Engineers have been working this way for decades
Copyright © 2005 - 2015 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event - Asynchronous Event Streams Nov 2015
Advantages of Asynchronous Programming
100% 100%
15. Copyright © 2005 - 2015 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event - Asynchronous Event Streams Nov 2015
Push-based Streams
Push-based-streams are fundamentally different from pull-based streams
The processing function is called when data arrives, not when the
previous entry has been processed
Terminal operations must be asynchronous and non-blocking
The Promise is the primitive of Asynchronous Programming
Rather than returning values a push-based stream returns a Promise
A Promise represents a delayed result that will be “resolved” later
OSGi Promises are a good option here
16. Copyright © 2005 - 2015 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event - Asynchronous Event Streams Nov 2015
Mapping Java 8 Streams to a push model
The Java 8 Stream has a rich and powerful API
Making it push-based is easier than you might think!
Change the return type of the terminal operations
Pull Push
long count() Promise<Long> count()
boolean anyMatch() Promise<Boolean> anyMatch()
boolean allMatch() Promise<Boolean> allMatch()
Optional<T> min() Promise<Optional<T>> min()
Optional<T> max() Promise<Optional<T>> max()
17. Copyright © 2005 - 2015 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event - Asynchronous Event Streams Nov 2015
Problems with this approach
This model actually works very well, but there are some problems
How do we know when an asynchronous stream has finished?
A pull-based model can simply indicate that there is no more data
Pull-based iteration offers a natural “brake” by processing elements in turn
Push-based systems can be overwhelmed by “Event Storms”
Even a single client thread can be problematic if it is too eager!
How do we cope with this?
18. Copyright © 2005 - 2015 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event - Asynchronous Event Streams Nov 2015
Using Events to communicate
Pushing the raw data into the consumer is simple, but insufficient
Consumers need a pushed event to indicate the end of a stream
Events should also be able to propagate failures
An Event is therefore a simple wrapper for data or metadata
public static enum EventType { DATA, ERROR, CLOSE };
public final class AsyncEvent<T> {
public EventType getType() { … }
public T getData() { … }
public Exception getFailure{ … }
}
19. Copyright © 2005 - 2015 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event - Asynchronous Event Streams Nov 2015
Learning lessons from Network Engineers
Push-based streams share a lot of concepts with computer networks
Asynchronous delivery
Producer and Consumer may run at different rates
TCP solves the “event storm” problem with back-pressure
The producer gets faster, but backs off if the consumer’s ACK rate drops
Our push-streams need to feed back to the data producer
The simplest way to do this is simply to say “don’t call me for a while”
20. Copyright © 2005 - 2015 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event - Asynchronous Event Streams Nov 2015
The AsyncConsumer
A simple functional interface for consuming data:
public interface AsyncConsumer {
long accept(AsyncEvent<T> event);
}
End of Stream events can be detected and back-pressure returned
If positive then the producer should wait at least that long
If zero then continue as soon as possible
If negative then close the stream
21. Copyright © 2005 - 2015 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event - Asynchronous Event Streams Nov 2015
Buffers, Windows and Circuit Breakers
22. An event consumer may receive events on many threads
The event consumer may also be slow to process data
Buffering allows a thread switch, freeing up the producer’s thread
It also allows the consumer to scale up or down the number of workers
Incoming data is queued until it can be processed
The buffer can return back-pressure based on how full it is
Buffering is a built-in feature of the push-based stream
Copyright © 2005 - 2015 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event - Asynchronous Event Streams Nov 2015
Buffering events
1723113119
23. Copyright © 2005 - 2015 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event - Asynchronous Event Streams Nov 2015
Buffering events (2)
What happens when the buffer gets full?
We could use an infinite buffer, but memory isn’t infinite…
Blocking is a possibility, but not very asynchronous!
A good option is simply to close the stream
This is called a “circuit breaker” - it trips if the consumer falls too far behind
This model protects against event storms
1723113119375?
24. Copyright © 2005 - 2015 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event - Asynchronous Event Streams Nov 2015
Windowing events
An event consumer may wish to receive batches of events to process
The consumer can then forward an aggregate event
Batches can be defined using an absolute number, or a time window
The underlying behaviour is similar to buffering
1723113119
17231131195 37Number
Time
25. Copyright © 2005 - 2015 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event - Asynchronous Event Streams Nov 2015
Producing data events
26. Copyright © 2005 - 2015 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event - Asynchronous Event Streams Nov 2015
Producing events
So far we’ve focussed on consuming and processing events
This is not very useful unless we can produce events!
Event producers are connected to consumers using the open() method
public interface AsyncEventSource<T> {
Closeable open(AsyncConsumer<? super T> event);
}
The returned Closeable can be used to “end” the stream
This is useful when the data stream is infinite!
27. Copyright © 2005 - 2015 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event - Asynchronous Event Streams Nov 2015
Helpful behaviours
Event Producers have to cope with multiple registrations
They also have to handle back-pressure from the consumer
Sometimes the producer has no choice about waiting!
Writing a producer should not be hard, so we provide help
Buffers/Circuit breakers mean that the producer can “ignore” back pressure
The RFC also describes a “helper” for managing multiple registrations
28. Copyright © 2005 - 2015 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event - Asynchronous Event Streams Nov 2015
Playing with streams
29. Copyright © 2005 - 2015 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event - Asynchronous Event Streams Nov 2015
Playing with push streams
Using push streams is a lot like creating Java 8 streams
Creating them is a little different!
A Java 8 stream is created directly from a collection (or Spliterator)
Currently Push Streams use a factory to turn a Source into a Stream
The stream does not open the source until a terminal operation is invoked
No events are pushed before your pipeline is ready!
30. Copyright © 2005 - 2015 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event - Asynchronous Event Streams Nov 2015
Examples
31. Copyright © 2005 - 2015 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event - Asynchronous Event Streams Nov 2015
Example streams
All examples use an Event Source which sends Integers from 0 to N-1
Find the biggest?
createStream(10)
.max(Integer::compare);
9 (wrapped in a Promise!)
Find the first odd number?
createStream(10)
.filter(x -> x % 2 == 1)
.findFirst();
1 (also in a Promise!)
Find the combined total?
createStream(10)
.reduce(0,(a,b)-> a + b);
45 (get the idea yet?)
Count the time windowed events?
createStream(100)
.window(200, Collection::size)
.collect(toList());
It depends!
32. Copyright © 2005 - 2015 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event - Asynchronous Event Streams Nov 2015
Demo (sort of!)
33. Copyright © 2005 - 2015 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event - Asynchronous Event Streams Nov 2015
•For more about OSGi...
• Specifications at http://www.osgi.org
• Enterprise OSGi in Action
• http://www.manning.com/cummins
•For more about the Push Streams
• http://github.com/osgi/design
•See the IoT Competition at 1745 today
Questions?
Thanks!
http://www.paremus.com
info@paremus.com
http://www.manning.com/cummins
34. Copyright © 2005 - 2015 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event - Asynchronous Event Streams Nov 2015