Scalable Event Processing – Pushing the limits with Push Streams! - Tim Ward

Copyright © 2005 - 2016 Paremus Ltd.
May not be reproduced by any means without express permission. All rights reserved.
OSGi Community Event Nov 2016
OSGi Community Event 2016
co-located at
EclipseCon Europe 2016
Scalable Event Processing
Pushing the limits with Push Streams!
Tim Ward
http://www.paremus.com
info@paremus.com

•Chief Technology Oﬃcer at Paremus

•8 years developing OSGi speciﬁcations

•Chair of the OSGi IoT Expert Group

•Interested in Asynchronous Distributed Systems

•Author of Manning’s Enterprise OSGi in Action

•http://www.manning.com/cummins
Who is Tim Ward?
@TimothyWard

Working with Data

Iteration in Java
We’ve probably all written this code:

for(int i =0; i < list.size(); i ++) {
MyData data = list.get(i);
…
} and this code:

Iterator<MyData> it = list.iterator();
while(it.hasNext()) {
MyData data = it.next();
…
}and this code:

for(MyData data : list) {
…
}
Data is everywhere, and Java’s collections make processing data easy

Problems with iteration
Whilst Java iteration is easy, it’s still easy to make mistakes:
This is bad for Linked Lists!

for(int i =0; i < list.size(); i ++) {
MyData data = list.get(i);
…
}
“External” iteration also pushes control logic into your code!

Parallelising the processing is a huge task

Java 8 updated the Collections API to support “internal” iteration

Powerful functional concepts were added via the Stream API

Iteration in Java 8
Streaming collections is very simple:

list.stream().forEach(data -> …);
Internal iteration separates the “what” from the “how” in your code

Parallel processing can occur implicitly if the list supports it

Functional pipelines allow for easy processing

list.stream()
.map(MyData::getAge)
.filter(i -> i < 15)
.count();
intermediate operations

terminal operation

Properties of Java 8 Streams
Java 8 Streams have a number of useful properties:
They are lazy

Streams only process data on-demand

This is triggered by a “terminal operation”
They can “short circuit”

Some operations don’t need to see the whole stream

findFirst() and findAny() can return if an element is found
Importantly the Stream is “pulling” the data from the data structure

Streams of data

Streams - the overloaded concept
Before Java 8, a “stream” of data was a java.io.InputStream

Whilst you probably didn’t think of it that way, you still iterated

int read;
while((read = is.read()) != -1) {
byte data = (byte) read;
…
}
The big diﬀerence with an InputStream is that it may block

A thread may get “stuck” waiting for user input, or a slow network

Java NIO has non-blocking input, but is much harder to use

Streams - the overloaded concept (2)
An InputStream behaves like an ordered Collection of bytes

Using a Java 8 Stream over these bytes could make sense

is.forEach(data -> …)
But the InputStream may block the thread indeﬁnitely

If the input is asynchronous then resources are wasted by waiting

A slow function may not process data as fast as it arrives

Rapid bursts of data may overload the consumer

All of this is independent of the data, be it bytes or Objects

Java 8 Streams aren’t able to cope with asynchronous data

Push-based asynchronous streams

Push-based Streams
Push-based-streams are fundamentally diﬀerent from pull-based streams

The processing function is called when data arrives, not when the
previous entry has been processed

Asynchronous operation allows for high throughput and parallelisation

Terminal operations must be asynchronous and non-blocking

The Promise is the primitive of Asynchronous Programming

Rather than returning values a push-based stream returns a Promise

A Promise represents a delayed result that will be “resolved” later

OSGi Promises are a good option here

Mapping Java 8 Streams to a push model
The Java 8 Stream has a rich and powerful API

Making it push-based is easier than you might think!

Change the return type of the terminal operations
Pull Push
long count() Promise<Long> count()
boolean anyMatch() Promise<Boolean> anyMatch()
boolean allMatch() Promise<Boolean> allMatch()
Optional<T> min() Promise<Optional<T>> min()
Optional<T> max() Promise<Optional<T>> max()

Problems with this approach
This model actually works very well, but there are some problems

How do we know when an asynchronous stream has ﬁnished?

A pull-based model can simply indicate that there is no more data

Pull-based iteration oﬀers a natural “brake” by processing elements in turn

Push-based systems can be overwhelmed by “Event Storms”

Even a single client thread can be problematic if it is too eager!

How do we cope with this?

Using Events to communicate
Pushing the raw data into the consumer is simple, but insuﬃcient

Consumers need a pushed event to indicate the end of a stream

Events should also be able to propagate failures

An Event is therefore a simple wrapper for data or metadata

public static enum EventType { DATA, ERROR, CLOSE };
public final class PushEvent<T> {
public EventType getType() { … }
public T getData() { … }
public Exception getFailure{ … }
}

Learning lessons from Network Engineers
Push-based streams share a lot of concepts with computer networks

Asynchronous delivery

Producer and Consumer may run at diﬀerent rates

TCP solves the “event storm” problem with back-pressure

The producer gets faster, but backs oﬀ if the consumer’s ACK rate drops

Our push-streams need to feed back to the data producer

The simplest way to do this is simply to say “don’t call me for a while”

The AsyncConsumer
A simple functional interface for consuming data:

public interface PushEventConsumer {
long accept(PushEvent<T> event);
}
End of Stream events can be detected and back-pressure returned

If positive then the producer should wait at least that long

If zero then continue as soon as possible

If negative then close the stream

Buffers, Windows and Circuit Breakers

An event consumer may receive events on many threads

The event consumer may also be slow to process data

Buffering allows a thread switch, freeing up the producer’s thread

It also allows the consumer to scale up or down the number of workers

Incoming data is queued until it can be processed

The buffer can return back-pressure based on how full it is

Buffering is a built-in feature of the push-based stream

Buffering events
1723113119

Buffering events (2)
What happens when the buffer gets full?

We could use an infinite buffer, but memory isn’t infinite…

Blocking is a possibility, but not very asynchronous!

A good option is simply to close the stream

This is called a “circuit breaker” - it trips if the consumer falls too far behind

This model protects against event storms
1723113119375?

Windowing events
An event consumer may wish to receive batches of events to process

The consumer can then forward an aggregate event

Batches can be deﬁned using an absolute number, or a time window

The underlying behaviour is similar to buﬀering
1723113119
17231131195 37Number
Time

Producing data events

Producing events
So far we’ve focussed on consuming and processing events

This is not very useful unless we can produce events!

Event producers are connected to consumers using the open() method

public interface PushEventSource<T> {
Closeable open(PushEventConsumer<? super T> event);
}
The returned Closeable can be used to “end” the stream

This is useful when the data stream is inﬁnite!

Helpful behaviours
Event Producers have to cope with multiple registrations

They also have to handle back-pressure from the consumer

Sometimes the producer has no choice about waiting!

Writing a producer should not be hard, so we provide help

The SimplePushEventSource provides buﬀered publishing

The Buﬀer/Circuit breaker allows the producer to “ignore” back pressure

A connect Promise allows event delivery to be lazily started

Playing with streams

Playing with push streams
We need an asynchronous source of events

The UK rail network has an open data API!

Events are batched (for performance) and delivered using STOMP

Events are JSON objects and so can easily be processed!

Let’s have a play!

•For more about OSGi...

• Speciﬁcations at http://www.osgi.org

• Enterprise OSGi in Action

• http://www.manning.com/cummins

•For more about the Push Streams

• http://github.com/osgi/design

•See the IoT Competition at 1745 today

Questions?
Thanks!
http://www.paremus.com
info@paremus.com
http://www.manning.com/cummins

Addendum

Demo Gremlins
This morning I was testing my demo

I couldn’t connect to the Network Rail Feed

They had Deactivated my account!

No demo = Unhappy Audience :(

But OSGi is amazing technology from the future!

I built a new module using MQTT to consume events from the OSGi Train

1 hour later - Working Demo. No other code had to change!

Network Rail reactivated my account 10 minutes later…

www.paremus.com @Paremus info@paremus.com

Scalable Event Processing – Pushing the limits with Push Streams! - Tim Ward

Recommandé

Recommandé

Contenu connexe

Plus de mfrancis

Plus de mfrancis (20)

Dernier

Dernier (20)

Scalable Event Processing – Pushing the limits with Push Streams! - Tim Ward