Micro-services introduce best practices to distributed systems, but how can we support micro-services with low consistent latencies.

1. Peter Lawrey
CEO of Higher Frequency Trading
New York 2015
Micro-services or monolith in a
reactive low latency Java applications

2. Peter Lawrey
Java Developer/Consultant for investment banks, hedge fund and
trading firms for 7 years, 16 years in Java, 23 years in IT.
Most answers for Java and JVM on stackoverflow.com
Founder of the Performance Java User’s Group.
Architect of Chronicle Software
Java Champion

3. Chronicle Software
Help companies migrate to high performance Java code.
Sponsor open source projects https://github.com/OpenHFT
Licensed solutions Chronicle-Enterprise and Chronicle-Fix
Offer one week proof of concept workshops, advanced Java
training, consulting and bespoke development.

5. Enterprise support for custom DSLs
We have a micro-services container which has one thread.
A service or strategy can;
• be written in a language which can be turned into Java and
compiled in memory.
• be deployed to an individual container/CPU pinned thread
• upgraded while running.
• run a strategy in test mode before switching control.
• be stopped.
• have it’s state examined or altered remotely.
• Micro-second latencies persisting every message.

6. Agenda
• Micro-Services and Reactive, what’s new?
• When does it make more sense to have micro-services or a
monolith?
• How can we get micro-second response times in a micro-
services architecture?
• How can micro-services help trading systems and real time
applications to be easier to deploy and manage?
• How can reactive streams help and what are the alternatives?
• How can we orchestrate and monitor these services?

7. Micro-Services and Reactive, what’s new?
These are hot topics which attempt to bring together a collection
of best practices in different domains.
These are not all or nothing.
You can implement a sub-set of these best practices, and most
likely you are already doing this to some degree.

8. UNIX Philosophy
• Small is beautiful.
• Make each program do one thing well.
• Build a prototype as soon as possible.
• Choose portability over efficiency.
• Store data in flat text files.
• Use software leverage to your advantage.
• Use shell scripts to increase leverage and portability.
• Avoid captive user interfaces.
• Make every program a filter.
https://en.wikipedia.org/wiki/Unix_philosophy#Mike_Gancarz:_The_UNIX_Philosophy

9. Old Micro-Services
Before there was HTTP, there was
• Daytime, uptime.
• DNS
• NTP
• FTP
• Syslog
• NFS
• SMTP

10. Monolith vs Micro-Services
The main difference is in deployment. Monolith means deploying
everything at once. Micro-Services supports (re)deploying
portions of a running system.
Monoliths are simpler up to a point. If your solution is simple
enough, you don’t need more than one service.
Micro-Services can scale better, esp across multiple machines.
You are more likely to be forced to follow best practices.

11. Micro-Services Best practices
• Modelled around the business domain.
• Culture of automation.
• Hide implementation details.
• Decentralize all things.
• Deployed independently, and monitored independently.
• Isolated failures.
Micro-Services forces you to think about these issues, however in
a monolith, this may require more discipline.

12. Micro-Services are a …
• Way to introduce best practices for distributed systems
• replacement monoliths which should have been a distributed
system.
• if it’s not distributed already, chances are it should continue to
be a monolith.

13. Monoliths are a …
• Easier to unit and functional test.
• Easier to debug.
• Easier to run on one machine.
To get the best of both worlds, make sure you micro-services can
be run as a monolith. This allows you to configure your service
components to run in a single thread, multiple thread, multiple
processes or multiple machines which suits your changing
requirements.

14. Unit Testing High Availability
• Create systems which communicate via asynchronous calls.
• Allow them to be configured as distributed components or in a
single thread.
• Create a unit tests where the listener of your primary service is
your secondary for the same service.
This allows you to unit test your redundant system behaves
correctly and is easy to debug when it doesn’t.

15. Reactive Streams
• an initiative to provide a standard for asynchronous stream
processing with non-blocking back pressure.
• follows the Reactive Manifesto in always being ready to handle
any requirements.
• assumes the producer exists to feed the consumers.
• producer is dependent on the slowest consumer.
• coupled testing between the producer and consumer.
Designed to handle high workloads which have the potential to
overwhelm your system.

16. Chronicle Queue
• A solution to provide a non blocking asynchronous stream
processing without back pressure via an unbounded queue.
• follows the Reactive Manifesto in always being ready to handle
any requirements.
• assumes the producer is consumer insensitive.
• producer and consumer are tested independently.
• assumes you benefit from a record of every message.
Designed to handle high workloads where a slow consumer
cannot or should not slow the producer. E.g. an exchange.

17. What is low latency?
The term “low latency” can applied to a wide range of situations.
A broad definition might be;
Low latency means you have a view on how much the response
time of a system costs your business.
In this talk I will assume;
Low latency means you care about latencies you can only
measure as even the worst latencies are too fast to see.

18. What is an ultra low GC?
In a generational collector you have multiple generations of
objects.
Small to medium sized objects are created in the Eden space.
When your Eden space fills up you trigger a Minor Collection.
So how big can your Eden space be and still have Compressed
OOPS?

19. Where does the Eden space fit in
Java 8 memory layout.

20. How can we get micro-second response times in a
micro-services architecture?
Using small web services can lead to high latencies. They use
HTTP/REST with JSON and the more hops between services, the
greater the overhead.
An alternative is to use low latency messaging. This can reduce
latency between threads and processes on the same machine to
as low at 1 micro-second, and between machines to single digit
micro-seconds.

21. Low latency messaging
• Tibco FTL – RDMA and shared memory.
• Aeron – UDP and shared memory.
• Chronicle Queue / Map – persisted shared memory and TCP.
Each of these solutions work in the single digit micro-seconds
between machines and sub micro second on the same machine.

22. Low latency micro-services
Passing a message from a producer to a worker and a worker to a
consumer, we get a typical latency of 4.5 micro-seconds on a
laptop.
We are looking to improve performance to be less than 1 micro-
second per hop for larger messages.

23. Chronicle Demo

24. Low latency serialization
The cost of serialization/deserialization can be greater than the
message itself.
• SBE – schema based binary protocol
• Chronicle Bytes – binary protocol.
• Chronicle Wire – meta data binary/text protocol. E.g. JSON.
• Protobuf, kryo, Cap’n proto – cross platform serialization.
A key advantage of SBE and Chronicle is re-use of objects in
deserialization to avoid creating garbage.

25. A low latency API which uses Lambdas
Timings are in micro-seconds with JMH.
* Data was read/written to native memory.
Wire Format Bytes 99.9 %tile 99.99 %tile 99.999 %tile worst
JSONWire 100* 3.11 5.56 10.6 36.9
Jackson 100 4.95 8.3 1,400 1,500
Jackson + Chronicle-Bytes 100* 2.87 10.1 1,300 1,400
BSON 96 19.8 1,430 1,400 1,600
BSON + Chronicle-Bytes 96* 7.47 15.1 1,400 11,600
BOON Json 100 20.7 32.5 11,000 69,000
"price":1234,"longInt":1234567890,"smallInt":123,"flag":true,"text":"Hello World!","side":"Sell"

26. A resizable buffer and a Wire format
// Bytes which wraps a ByteBuffer which is resized as needed.
Bytes<ByteBuffer> bytes = Bytes.elasticByteBuffer();
// YAML based wire format
Wire wire = new TextWire(bytes);
// or a binary YAML based wire format
Bytes<ByteBuffer> bytes2 = Bytes.elasticByteBuffer();
Wire wire2 = new BinaryWire(bytes2);
// or just data, no meta data.
Bytes<ByteBuffer> bytes3 = Bytes.elasticByteBuffer();
Wire wire3 = new RawWire(bytes3);

27. Low latency API using Lambdas (Wire)
message: Hello World number: 1234567890 code: SECONDS price: 10.5
wire.read(() -> "message").text(this, (o, s) -> o.message = s)
.read(() -> "number").int64(this, (o, i) -> o.number = i)
.read(() -> "timeUnit").asEnum(TimeUnit.class, this, (o, e) -> o.timeUnit = e)
.read(() -> "price").float64(this, (o, d) -> o.price = d);
wire.write(() -> "message").text(message)
.write(() -> "number").int64(number)
.write(() -> "timeUnit").asEnum(timeUnit)
.write(() -> "price").float64(price);
To write a message
To read a message

28. A resizable buffer and a Wire format
message: Hello World number: 1234567890 code: SECONDS price: 10.5
In the YAML based TextWire
Binary YAML Wire
message: Hello World
number: 1234567890
code: SECONDS
price: 10.5
00000000 C7 6D 65 73 73 61 67 65 EB 48 65 6C 6C 6F 20 57 ·message ·Hello W
00000010 6F 72 6C 64 C6 6E 75 6D 62 65 72 A3 D2 02 96 49 orld·num ber····I
00000020 C4 63 6F 64 65 E7 53 45 43 4F 4E 44 53 C5 70 72 ·code·SE CONDS·pr
00000030 69 63 65 90 00 00 28 41 ice···(A

29. Lambdas and Junit tests
message: Hello World number: 1234567890 code: SECONDS price: 10.5
To read the data
To check the data without a data structure
wire.read(() -> "message").text(this, (o, s) -> o.message = s)
.read(() -> "number").int64(this, (o, i) -> o.number = i)
.read(() -> "timeUnit").asEnum(TimeUnit.class, this, (o, e) -> o.timeUnit = e)
.read(() -> "price").float64(this, (o, d) -> o.price = d);
wire.read(() -> "message").text("Hello World", Assert::assertEquals)
.read(() -> "number").int64(1234567890L, Assert::assertEquals)
.read(() -> "timeUnit").asEnum(TimeUnit.class, TimeUnit.SECONDS,Assert::assertEquals)
.read(() -> "price").float64(10.5, (o, d) -> assertEquals(o, d, 0));

30. Interchanging Enums and Lambdas
message: Hello World number: 1234567890 code: SECONDS price: 10.5
Enums and lambdas can both implement an interface.
Wherever you have used a non capturing lambda you can also use
an enum.
enum Field implements WireKey {
message, number, timeUnit, price;
}
@Override
public void writeMarshallable(WireOut wire) {
wire.write(Field.message).text(message)
.write(Field.number).int64(number)
.write(Field.timeUnit).asEnum(timeUnit)
.write(Field.price).float64(price);
}

31. When to use Enums
message: Hello World number: 1234567890 code: SECONDS price: 10.5
Enums have a number of benefits.
• They are easier to debug.
• The serialize much more efficiently.
• Its easier to manage a class of pre-defined enums to implement
your code, than lambdas which could be any where
Under https://github.com/OpenHFT/Chronicle-Engine search for
MapFunction and MapUpdater

32. When to use Lambdas
message: Hello World number: 1234567890 code: SECONDS price: 10.5
Lambdas have a number of benefits.
• They are simpler to write
• They support generics better
• They can capture values.

33. Where can I try this out?
message: Hello World number: 1234567890 code: SECONDS price: 10.5
The source for these micro-benchmarks are test are available
https://github.com/OpenHFT/Chronicle-Queue
Chronicle Engine with live subscriptions
https://github.com/OpenHFT/Chronicle-Engine

34. Q & A
Peter Lawrey
@PeterLawrey
http://chronicle.software
http://vanillajava.blogspot.com