This will be a journey through the evolution of both the definition and implementations of Reactive Programming and how they have been converging in a quest to make building responsive applications a sane process.
We are still at the point where there is no complete consensus on what a reactive application is; I will explore how this consensus is evolving and what problems it brings.
I am going to talk about the past, present and possible futures of reactive programming and how you can survive it all.
This talk was made at http://2017.progscon.co.uk/
2. reactive
/rɪˈaktɪv/
adjective
1. showing a response to a stimulus.
"pupils are reactive to light"
2. acting in response to a situation rather than creating or
controlling it.
"a proactive rather than a reactive approach"
10. 1985: Reactive systems
On the development of reactive systems goo.gl/dbECw8
D. Harel and A. Pnueli from The Weizmann Institute
• Transformational vs reactive systems
• Describing reactive systems
• The development of a reactive system
13. 1991: Reactive Programming
Three paradigms:
1. Events update the state goo.gl/5tm66t
2. The state satisfies constraints goo.gl/vr2V3F
3. Processing event streams goo.gl/XcFgQL
14. 1. Events modify the state
• External events (activations)
• Reactions
Frédéric Boussinot
Reactive C: An Extension of C Reactive Systems
16. 2. The state should satisfy constraints
Bruce L. Horn
Siri : a constrained-object language
for reactive program implementation
• Visual metaphors
• Direct manipulation
• Reflection of changes in the data
26. 2012: Survey on RP
RP = abstraction over time management
The most comprehensive non-JS survey goo.gl/0Dhvnm
Compares:
• 2 categories: FRP siblings and RP cousins
• Evaluation model: push/pull
• Lifting
• Multidirectional
• Glitch avoidance
• Distributed model
I am Evgeny Poberezkin. I work as the Head of Development at MailOnline – one of the largest news websites better known as Daily Mail. I work with 25 excellent software developers and we do very cool stuff there.
Previously I’ve been extremely lucky to have the opportunity to build the second generation of MailOnline content management system. We’re now about to kick off another exciting JavaScript project and we’d be very happy to hire some experienced developers.
When David asked me to talk about Reactive Programming here, I think because I was one of the creators of reactive framework milo.js, I replied something like “the more I do it the less I understand what … on Earth … it is and maybe I can talk about this … lack of understanding. David said he liked the idea, so here I am.
Even though I still have more questions than answers, I’ve found some interesting stuff that I am going to share here.
What does the word “reactive” mean? …
The dictionary gives us two main options: showing the response and acting in response.
It is correct in relation to reactive programs, but it would also be correct for event-driven applications and we want to differentiate them from “reactive” applications somehow.
Google search reveals Reactive Manifesto. Unfortunately, it doesn’t make it any clearer what reactive application is. It states that it is indeed “event driven” and also has some other criteria. But any decent application should satisfy these criteria.
I asked a question – where did it come from. As they say, the key to the future lies in the past. So I’ve tried to dig deeper in the past to try to understand who created this thing we now call reactive programming and why.
It all seem to have started from this man, at least he seems to be the first one who used the term “reactive” in relation to computer system. Do you know who he is? …
In 1971 he said “The best way to predict the future is to invent it”.
In 1973 Xerox Alto computer was released based on graphical user interface with overlapping windows that this man architected. His name is Alan Kay and he is also one of the fathers of the idea of object-oriented programming.
The less known history is that in 1969, before he worked at Xerox, he wrote a PhD thesis called “The Reactive Engine” describing “the machines which can participate in an interactive dialogue”. He wrote about “the FLEX Machine, [where FLEX was the programming language he created]
a personal, reactive, minicomputer which communicates in text and pictures by means of keyboard, line-drawing CRT [that we call “a display” now] and tablet”.
To achieve it Alan Kay proposed the multi-process environment controlled by a single event queue managed by a scheduler. This picture from his thesis illustrates parallel processing with two sets of data that is merged.
While he didn’t describe reactive programming yet in any modern meaning of the term, he got quite close – this picture very much reminds me the visual presentation of two event streams that are merged.
Not only Alan Kay seemed to introduce the term “reactive” into computing. His work served as a blueprint for computing and programming industries for decades.
So what happens next?
In 1970s the concept of data-flow programming was actively developed. This approach suggests describing application as connected data graph consisting of several simple elements.
Paul Kosinski, an IBM fellow, has written several works on the subject. He was writing: “Operators in a data flow programming language functionally transform their inputs to their outputs without ever affecting the state of the rest of the program.” (http://csg.csail.mit.edu/pubs/memos/Memo-135/Memo-135new.pdf )
Another computer scientist, Jack Dennis, wrote: “Well behaved data flow program is always functional in the sense that a unique set of output values is determined by any set of input values”.
(http://publications.csail.mit.edu/lcs/pubs/pdf/MIT-LCS-TM-061.pdf)
While data flow programming at the time found its application mainly in signal processing, their works essentially described what we now call functional reactive programming – processing streams of events without side effects.
Fast forward to 1985. David Harel and Amir Pnueli from The Weizmann Institute in Israel wrote paper with the title. “On the development of reactive systems”.
They compared reactive systems with more traditional transformational systems and they discussed how reactive systems should be described and developed.
(http://www.wisdom.weizmann.ac.il/~harel/SCANNED.PAPERS/ReactiveSystems.pdf)
The behaviour of a reactive system can be described in terms of events and conditions that cause transitions between the states. Both states and transitions can be associated in with output events, called activities, which can be triggered either by executing a transition or by entering, exiting or simply being in a state.
So the concepts, or abstractions, in the reactive system according to this paper include: states, transitions, events, conditions (or constraints) and activities.
An interesting observation would be that to this day all reactive libraries and frameworks include only some of these concepts.
For development of reactive systems Harel and Pnueli suggested a simple idea: “The development of a reactive system is not a one-dimensional process where specification and design are two stages that follow each other, but a two-dimensional process that they call “a magic square”, where the specification and design are two dimensions themselves.
This work was fundamental but quite abstract – it didn’t prescribe any solution or development process, rather it described the criteria which the solution and development process should satisfy.
That may be the reason while most following paper seemed to have lost some part of these ideas.
There were three publications in 1991 that proposed three quite distinct approaches to how we do reactive programming:
the first based on events that update the state,
the second based on conditions that the state should satisfy,
the third based on processing streams of events
The first publication was “Reactive C: An Extension of C Reactive Systems” by Frédéric Boussinot
He wrote: Reactive systems are interactive programs that react continuously to sequences of activations coming from the external world. Reactive programming leads to a new programming style.
(http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.14.2178&rep=rep1&type=pdf)
Let’s see how we can construct a temperature converter that converts temperature form Celsius to Fahrenheit and back following this pattern when events update the state. We have a two values that emit events every time their state change. When this happens we can update the state in another value.
The second paper in the same year was written by Bruce Horn. He describes a constrained-object language for reactive program implementation. The language was called Siri.
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.829.2232&rep=rep1&type=pdf
About reactive programmes he writes: The interfaces for such programs are usually based on three concepts: accurate and appealing visual metaphors, interaction by direct manipulation, and immediate reflection of changes in the data. Programs based on these concepts are called reactive, after Alan Kay's Reactive Engine.
He also writes: A well-designed reactive program maintains an internal model that is kept consistent with an end user model.
The core of the language Siri is constraints pattern that allows to define conditions (or rules) that the language should maintain when the state changes in response to some events.
Defining temperature converter using this pattern is really easy. All you have to do is to explicitly write the constraints and let the language do the rest.
When C changes F updates. When F changes C updates too. The language Siri was able to solve systems of linear equations used in constraints, so doing this inverse transformation is an easy task for it.
The third paper was written by John Plaice: “RLucid, a general real-time dataflow language” (http://www.cse.unsw.edu.au/~plaice/archive/JAP/P-FTRTFTS92-rlucid.pdf).
He writes: “Stream processing seems natural to reactive programming: a program takes as input streams of events, and generates as output new streams of events. For this reason, many attempts have been made to use dataflow languages to program or specify reactive or real-time systems.”
It is interesting this quote has transformed to the statement that I heard quite often: “reactive programming IS stream processing”. We really are strongly opinionated people!
In this paradigm we can define our temperature converter as the transformations of reading temperature in one unit into the stream of readings in other units using, in this case, map function.
So, to recap what I was just talking about.
1991 seems to have been a pivotal year when three different paradigms for reactive programming have emerged:
Reactions to events update the state
The state satisfies constraints (or conditions)
Processing event streams
Each of these paradigms is a subset of a more holistic view proposed 6 years earlier.
What happened next?
In 1997 Conan Eliott from Microsoft Research Graphics Group and Paul Hudak from Yale University published a paper titled “Functional Reactive Animation” where they proposed a Haskell based system FRAN.
They modelled the system based on four concepts:
Temporal modelling of behaviours, hat vary over continuous time. Behaviours are first class values, and are built up compositionally.
Modelling of events that may refer to the outside world, for example button press, but also be based on some conditions applied to behaviours – they use collision detection in animations as an example.
Declarative reactivity: that is what makes it different from event driven patterns. Although behaviours are expressed in terms of reactions to events, the behaviours may explicitly depend on time and other behaviours rather than be imperative event-driven state changes.
Polymorphic media/views – while views should have their own unique methods they should have methods compatible with events and behaviours.
All the implementations of FRP use these or very similar concepts. Their implementation used lazy evaluation model, or “pull” when recomputation of values happens only when they are requested – it was natural as they’ve build their system using Haskell.
This picture illustrates two of four concepts I’ve just talked about – continuous behaviours, or signals, and discrete events.
The survey of FRP that was published in 2011 says:
The primary concepts of FRP are signals or behaviors (timevarying values) and events (collections of instantaneous values, or time-value pairs). Every instance of FRP conceptually includes both, though they may not be first-class values, and one may be defined in terms of the other. FRP achieves reactivity by providing constructs for specifying how signals or behaviors change in response to events. This may be the primary way of specifying and implementing behaviors (as in EFRP [21] and Frappe [5]) or behaviors may be semantic functions ´ of time which are replaced on event occurrences (this is the model in most other implementations).
So we can see that the concept of FRP that was formalised by FRAN in 1997 has remained essentially unchanged for 20 years.
Most of the research on reactive programming descends from Fran, a functional domain specific language developed in the late 1990s to ease the construction of graphics and interactive media applications.
Another paradigm of reactive programming that was based on constraints was also developed. We can also refer to constraints as rules, condition, explicit data bindings etc. – all these terms define a subset of a wider term “constraints”.
Two libraries based on this paradigm are worth mentioning: Trellis for Python and Cells for Common Lisp. I am not adding code samples here; you can easily find them.
By the way, what do you think is most widely used implementation of reactive programming?
It defines relations between elements using arbitrary expressions
It supports efficient dynamic re-computations.
It’s very visual.
In 2007 a software engineer Robert Wensman created a wikipedia page on reactive programming that focussed on the approach of creating excplicit contraints connecting “behaviours”
In 2012 a group of researchers conducted a survey on the existing implementations of reactive programming
ftp://mail.ssel.vub.ac.be/tech_report/2012/vub-soft-tr-12-13.pdf
An interesting conclusion they’ve formulated was that reactive programming languages abstract over time management, just as garbage collectors abstract over memory management. That is probably the most concise and yet very focussed definition of RP – an abstraction over time management.
They write: reactive programming tackles issues posed by event-driven applications by providing abstractions to express programs as reactions to external events and having the language automatically manage the flow of time (by conceptually supporting simultaneity), and data and computation dependencies.
They have analysed more than 15 implementations across 6 categories.
An observation here is that while some implementations are called “siblings” of functional reactive programming – that is their terminology, others are cousins of reactive programming, the implication here is that it is almost not reactive programming. That shows the divide of opinions even among researches who could have stayed more impartial.
When reactive programming is embedded in host languages (either as a library or as a language extension), existing language operators and user defined functions or methods must be converted to operate on behaviours and events. Such conversion of an ordinary operator to a variant that can operate on behaviours is called lifting.
The survey differentiates three ways lifting happens in different implementations:
- Implicit, when the same operators and functions are used for behaviours. It is possible in languages that support overloading. In is possible in JavaScript using pre-compilation – that’s the approach one framework uses – I will show it later
- Explicit – when an ordinary function is passed to a lifting function to create a lifted version of this function.
- Manual – when reactive programming implementation provides ways to access normal data encapsulated in behaviours and you just apply normal function to this data. Regreattably, this is the most common way in JavaScript. While it’s the simplest to implement it loses in expressiveness.
Glitch is the term used for update inconsistencies that may happen during data propagation.
When a computation is run before all its dependent expressions are evaluated, it may result in fresh values being combined with stale values, leading to a glitch. This can only happen in languages employing a push-based evaluation model.
Let’s look at the example …
Glitch avoidance is an important thing for implementation of RP.
JavaScript frameworks can also be categorized by the paradigm of reactive programming that is used.
The most common approach based on reactions to events is implemented in React framework and Flux pattern. It provides excellent abstractions for reactive apps of medium complexity.
The biggest issues that developers usually mention are:
State propagation can be slow
Diffing strategy should be efficient
You have to write a lot of boilerplate
From the wider point of view, the problem of this pattern is that it doesn’t provide abstractions for explicitly creating rules connecting models and models and views.
Processing event streams is implemented in ReactiveX and in BaconJS. Again, connecting behaviours with rules or constraints is rudimentary.
Another problem is that testing is more difficult here.
The third group of frameworks provides abstractions to connect models to views or even models to models, although the latter is less common. But then they may not very efficiently combine it with event management.