Chasing Bugs with the BeepBeep Event Stream Processor

Sylvain Hallé
Université du Québec à Chicoutimi
Canada
Chasing Bugs
with the BeepBeep
Event Stream Processor
TAROT 2016

System
Instrumentation
Trace
Events

System
Instrumentation
Trace
Events
Trace
validation

System
Runtime monitoring
Instrumentation

System
Runtime monitoring
Overhead
Instrumentation

Monitor
Property
Target system
Sensor
Internal
state
Event
Verdict
Feedback
(optional)

One trace
at a time
Don't care about
interacting with the SUT
?
Internal state can
be queried
Properties partially
specify behaviour

Possibility for the monitor to interact with the SUT
(enforcement monitors)
Opportunity to express
more complex properties
(ab)*
Overhead is a
concern

Part One
Use cases for monitoring

Sylvain Hallé
Cloud computingAjax web application

Sylvain Hallé
JavaScript

Sylvain Hallé
Bee G
Beatles
Camel
Caravan

Sylvain Hallé
Bee G
Beatles
Camel
Caravan
<a onclick=
>
"javascript:
findBand(’ ’)"Beatles

Sylvain Hallé
findBand(’Beatles’)

Sylvain Hallé
findBand(’Beatles’)
artist
beatles=

Sylvain Hallé
document.innerHTML = findBand(’Beatles’)
artist
beatles=

Sylvain Hallé
Does not need
to be a URL
Does not need
to be HTML
<SearchResults>
</SearchResults>
The Beatles
Rubber Soul
...
<Item>
</Item>
<Artist>
</Artist>
<Title>
</Title>
<Search>
</Search>
beatles
<Artist>
</Artist>

Sylvain Hallé
<Search>
</Search>
beatles
<Artist>
</Artist> XML
The eXtensible Markup
Language
?Nested collection of
elements
?Input/output data is
semi-structured
.
.
<SearchResults>
</SearchResults>
The Beatles
Rubber Soul
...
<Item>
</Item>
<Artist>
</Artist>
<Title>
</Title>

Sylvain Hallé
Cloud computingConceptually...

Sylvain Hallé
Cloud computingConceptually...
Web service
Web client

Sylvain Hallé
Main issue
Possible
between messages sent
and messages expected
mismatch
Not like traditional programming: all
input-output is exchanged unverified!

Sylvain Hallé
Defining message formats
?

Sylvain Hallé
1.
2.
...
?

Sylvain Hallé
1.
2.
...
<ItemSearch>
</ItemSearch>
beatles<Artist> </object>

Sylvain Hallé
<ItemSearch>
</ItemSearch>
<ItemSearchResponse>
<Items>
</Items>
</ItemSearchResponse>
Help!
The Beatles
<Item>
</Item>
...
<Title> </no>
<Artist> </Artist>

Sylvain Hallé
<ItemSearch>
</ItemSearch>
<Items>
</Items>
Help!
The Beatles
<Item>
</Item>
...
<Title> </no>
<Artist> </Artist>
XML request
XML response

Sylvain Hallé
<ItemSearch>
</ItemSearch>
ItemSearch[
[string]
]
Artist
<Items>
</Items>
Help!
The Beatles
<Item>
</Item>
...
<Title> </no>
<Artist> </Artist>
XML request
XML response

Sylvain Hallé
<ItemSearch>
</ItemSearch>
ItemSearch[
[string]
]
Artist
ItemSearchResponse[
[
Item[
Title[string],
Artist[string]
]{0,¥}
]
]
Items
<Items>
</Items>
Help!
The Beatles
<Item>
</Item>
...
<Title> </no>
<Artist> </Artist>
XML request
XML response

Sylvain Hallé
ItemSearch[
[string]
]
Artist
ItemSearchResponse[
[
Item[
Title[string],
Artist[string]
]{0,¥}
]
]
Items
?
!

Sylvain Hallé
WSDL: Web Service Description Language
ItemSearch[
[string]
]
Artist
CartCreate[
[int],
[int],
[
Item[
Title[string],
Artist[string]
]{0,¥}
]
]
Items
SessionKey
Items
?
?
ItemSearchResponse[
[
Item[
Title[string],
Artist[string],
]{0,¥}
]
]
Items
CartCreateResponse[
[int],
[int],
[
Item[
Title[string],
Artist[string]
]{0,¥}
]
]
SessionKey
CartId
Items
!
!
. . .

Sylvain Hallé
<ItemSearch>
</ItemSearch>
beatles
1234
<Artist> </Artist>
<Bizbiz> </Bizbiz>

Sylvain Hallé
<ItemSearch>
</ItemSearch>
beatles
1234
<Artist> </Artist>
<Bizbiz> </Bizbiz>
ItemSearch[
[string]
]
Artist vs.
?

Sylvain Hallé
<CartCreateResponse>
<SessionKey>
</SessionKey>
<CartId> </CartId>
<Items>
</Items>
1234
abc
...

Sylvain Hallé
<CartCreateResponse>
<SessionKey>
</SessionKey>
<CartId> </CartId>
<Items>
</Items>
1234
abc
...
CartCreateResponse[
[int],
[int],
[
Item[
Title[string],
Artist[string]
]{0,¥}
]
]
SessionKey
CartId
Items
vs.
!

Sylvain Hallé
?
What happened?

Sylvain Hallé
1.
2.
...
What happened?

Sylvain Hallé
2
What happened?

Sylvain Hallé
c
What happened?

Sylvain Hallé
2
c
What happened?

Sylvain Hallé
?
2
c
2
c
Interface contracts
All messages comply with the WSDL but...
1.
2.
...

Sylvain Hallé
?
2
c
2
c
Interface contracts
You cannot add the same item
twice to the shopping cart
1.
2.
...

Sylvain Hallé
?
2
c
2
c
Interface contracts
???
You cannot add the same item
twice to the shopping cart
1.
2.
...

Sylvain Hallé
Interface contracts
???

Sylvain Hallé
???
Interface contracts

Sylvain Hallé
The big question
Prevent
contract
violations

LOG
CODE
...
out.print("Lemming into Floater");
...

LOG
CODE
...
out.print("Lemming into Floater");
...
Game starts
Lemming into Blocker
...
Lemming into Floater
...

LOG
CODE
Game starts
Lemming into Blocker
...
Lemming into Floater
...
...
logger.log("Lemming into Floater",
Logging.LEVEL_DEBUG);
...

LOG
CODE
[10:24:31] INFO Game starts
[10:24:33] WARN Lemming into Blocker
...
[10:25:01] DEBG Lemming into Floater
...
...
logger.log("Lemming into Floater, id: "
+ lem._id, Logging.LEVEL_DEBUG);
...

LOG
CODE
...
[10:25:01] DEBG Lemming into Floater, id: 32
...
...
...

x?
y?
??
LOG
CODE
...
...
...
...

LOG
CODE
...
...
...
String msg = "Lemming into Floater ";
msg += "id: " + lem._id + ", ";
msg += "x: " + lem._x + ", ";
msg += "y: " + lem._y;
logger.log(msg, Logging.LEVEL_DEBUG);
...

LOG
CODE
...
msg += "id: " + lem._id + ", ";
msg += "x: " + lem._x + ", ";
msg += "y: " + lem._y;
...
...
[10:25:01] DEBG Lemming into Floater, id: 32, x: 320, y: 67
...

LOG
CODE
...
for (Lemming lm in lemmings) {
msg += "id: " + lm._id + ", ";
msg += "x: " + lm._x + ", ";
msg += "y: " + lm._y + "; ";
}
...
...
[10:25:01] DEBG Lemming into Floater, id: 32, x: 320, y: 67
...

LOG
CODE
...
for (Lemming lm in lemmings) {
msg += "id: " + lm._id + ", ";
msg += "x: " + lm._x + ", ";
msg += "y: " + lm._y + "; ";
}
...
...
[10:25:01] DEBG Lemming into Floater, id: 32, x: 320, y: 67 ;
id: 31, x: 450, y: 43 ; id: 23, x: 229, y: 40 ; ...
...

Game code
XML
template
T
Temporal
property
φ
{
}
Game
loop
1
3
2
Monitor
P
Named pipe,
TCP socket,
HTTP request
Verdict
XML event
Buﬀer

Part Two
Specifying properties with logic

Sylvain Hallé
Interface contracts
All possible sequences
of all possibles messages
with all possible values

Sylvain Hallé
Interface contracts
Constraints
on individual
messages

Sylvain Hallé
Interface contracts
Constraints
on sequencesConstraints
on individual
messages

Sylvain Hallé
Interface contracts
Constraints
on sequences
Data-aware
sequential constraints
Constraints
on individual
messages

Sylvain Hallé
Interface contracts
Interface contract =
valid (error-free) interactions
Constraints
on sequences
Data-aware
sequential constraints
Constraints
on individual
messages

Sylvain Hallé
Constraints on individual messages
Examples:
Three types of constraints (I)
<Message>
<Action>ItemSearch</Action>
<Results>5</Results>
<Keyword>beatles</Keyword>
<Page>1</Page>
</Message>

Sylvain Hallé
Examples:
1. The element must be an integer between 1 and 20.Page "/M
<Message>
<Page>1</Page>
</Message>

Sylvain Hallé
1. The element must be an integer between 1 and 20.
2. The element is mandatory only if is present,
otherwise it is forbidden.
Page
Page Results
"/M
<Message>
<Page>1</Page>
</Message>
Examples:

Sylvain Hallé
Constraints on message sequences
Examples:
2
<Message>
<Action>
Login
</Action>
...
</Message>
<Message>
<Action>
LoginResponse
</Action>
...
</Message>
<Message>
<Action>
CartCreate
</Action>
...
</Message>
Three types of constraints (II)

Sylvain Hallé
Examples:
2
3. The request cannot be resent if its response is
successful.
.
Login "/
"/
<Message>
<Action>
Login
</Action>
...
</Message>
<Message>
<Action>
LoginResponse
</Action>
...
</Message>
<Message>
<Action>
CartCreate
</Action>
...
</Message>
X

Sylvain Hallé
Examples:
2
3. The request cannot be resent if its response is
successful.
.
4. must follow a successful LoginResponse.
Login
CartCreate
"/
"/
"/
<Message>
<Action>
Login
</Action>
...
</Message>
<Message>
<Action>
LoginResponse
</Action>
...
</Message>
<Message>
<Action>
CartCreate
</Action>
...
</Message>
X

Sylvain Hallé
Data-aware sequential constraints
Examples:
2
5. There can be at most one active cart ID per session key."/
Three types of constraints (III)
<Message>
<SessionKey>123</SessionKey>
<CartId>789</CartId>
...
</Message>
<Message>
...
</Message>

Sylvain Hallé
Examples:
2
6. You cannot add the same item twice to the shopping cart."/
<Message>
<Action>CartAdd</Action>
<Items>
<Item>
<ItemId>567</ItemId>
...
<Message>
<Items>
<Item>
...
X

Mario cannot reach the end of a level without jumping
at least once

Mario can never jump higher than 20 pixels
20

X
If Mario crouches, he cannot jump right after

X
Mario cannot collide with an enemy while he is
holding a Koopa shell

% grep "Lemming into Floater" log

% grep -P "Lemming into Floater, .*? x: (d+?), y: 1" log

% grep -P "Lemming into Floater, .*? x: (d+?), y: 1" log
| sed 's/^.*id: ([0-9]+).*$/1/'

% grep -Pzo "(?s)Lemming into Basher, id: (d+).*Lemming into
Floater, id: 1 [^;]*?x: (d+?), y: 2" log
| sed -r 'N;s/^.*?id: ([0-9]+).x: ([0-9]+), y: ([0-9]+)$
/1, 2, 3/'

#!/usr/bin/python
import re
with open('log', 'r') as logfile:
log = logfile.read()
bm = re.findall(r'Basher, id: (d+)', log)
fm = re.findall(r'Floater, id: (d+), x: (d+), y: (d+)', log)
for lem in fm:
if lem[0] in bm:
print lem[0] + ', ' + lem[1] + ', ' + lem[2]

#!/usr/bin/python
import re, sys
bashers = {}
for line in sys.stdin:
res = re.match(r'^Lemming into (.*?), id: (d+), x: (d+),
y: (d+)', line)
if res:
if res.group(1) == 'Basher':
bashers[res.group(2)] = 1
else:
if res.group(1) == 'Floater' and res.group(2) in bashers:
print res.group(2) + ', ' + res.group(3) + ', '
+ res.group(4)

* A regexp (matches the unstructured event text, or the "msg" field
for CEE/Lumberjack structured events)
ceelog '/DHCP/'
* A field comparison (matches a CEE/Lumberjack field)
ceelog 'uid == "0"'
ceelog 'uid != "0"'
ceelog 'trusted!uid == "0"'
ceelog 'username ~ /^guest-/'
ceelog 'username !~ /^guest-/'
* A combination of the above
ceelog 'trusted!uid == "0" && username ~ /^guest-/'

* A regexp (matches the unstructured event text, or the "msg" field
for CEE/Lumberjack structured events)
ceelog '/DHCP/'
* A field comparison (matches a CEE/Lumberjack field)
ceelog 'uid == "0"'
ceelog 'uid != "0"'
ceelog 'trusted!uid == "0"'
ceelog 'username ~ /^guest-/'
ceelog 'username !~ /^guest-/'
* A combination of the above
ceelog 'trusted!uid == "0" && username ~ /^guest-/'
grep
{

Sylvain Hallé
Expressing data constraints
Simple XPath
Fetches portions of an XML document according to a
query path = sequence of tags
:set of messages
: set of XML query paths
: set of atomic values
: ´®2
Examples:
(‘‘/a/b/c’’, m) = {1,2,4}
(‘‘/a/b/d’’, m) = Æ
M
M
Q
Q
V
V
p
p
p m
{
<a>
<b>
<c>1</c>
<c>2</c>
</b>
<d>
<c>9</c>
</d>
<b>
<c>3</c>
</b>
</a>

Sylvain Hallé
XPath term
Expresses properties over values fetched by XPath expressions
For some message Î, path Î,
"x : j(x) Ûj(v) for every Î( , )
$x : j(x) Ûj(v) for some Î( , )
Examples:
"x : x < 5/a/b/c
$x :/a/b
$x : "y : y £x/a/b/c /a/b/c
m M
mq
mq
q Q
q
q
v
v
p
p
2
<a>
<b>
<c>1</c>
<c>2</c>
</b>
<d>
<c>9</c>
</d>
<b>
<c>3</c>
</b>
</a>
m
{

Sylvain Hallé
2
1. The element must be an integer between 1 and 20.
Page
Page Results
"/M
<Message>
<Page>1</Page>
</Message>

Sylvain Hallé
2
1. " x : x > 0 Ùx < 21/Message/Page
Page Results
<Message>
<Page>1</Page>
</Message>

Sylvain Hallé
2
1. " x : x > 0 Ùx < 21/Message/Page
2. $ x : Û$ y :/Message/Page /Message/Results
<Message>
<Page>1</Page>
</Message>

Sylvain Hallé
Linear Temporal Logic
Alphabet (A)
Set of possible messages
Trace (A*)
Sequence of messages

Sylvain Hallé
LTL formula = assertion on the of states in a tracesequence
a "always a"
a "a in the next"
a "eventually a"
a b "a until b"
G
X
F
W
G (a ®b)X (d cÚe) WØFALSE TRUE
. . .A A EC CDB B

Sylvain Hallé
Well-known results:
1. For every LTL formula j, there exists a Büchi automaton A
such that for every (infinite) trace s:
i.e. LTL describes languages
2. The alphabet symbols can be generalized to finite sets of
Boolean propositions
w-regular
ÞLet’s use XPath terms as our Boolean propositions
j
s|= jÛsÎL(A )j

Sylvain Hallé
Examples:
2
3. (" a : a = LoginResponse ®/Message/Action
( " a’ : a’ ¹Login))/Message/Action
.
G
X G
CartCreate "/
<Message>
<Action>
Login
</Action>
...
</Message>
<Message>
<Action>
LoginResponse
</Action>
...
</Message>
<Message>
<Action>
CartCreate
</Action>
...
</Message>
X

Sylvain Hallé
Examples:
2
.
G
X G
CartCreate "/
<Message>
<Action>
Login
</Action>
...
</Message>
<Message>
<Action>
LoginResponse
</Action>
...
</Message>
<Message>
<Action>
CartCreate
</Action>
...
</Message>
X
Xpath terms

Sylvain Hallé
Examples:
2
4. (" a : a ¹CartCreate)/Message/Action
(" a’ : a’ =LoginResponse)/Message/Action
G
X G
W
.
<Message>
<Action>
Login
</Action>
...
</Message>
<Message>
<Action>
LoginResponse
</Action>
...
</Message>
<Message>
<Action>
CartCreate
</Action>
...
</Message>
X
Xpath terms

Mario cannot reach the end of a level without jumping
at least once
F action = jump

Mario can never jump higher than 20 pixels
G (action = jump → height < 20)
20

X
If Mario crouches, he cannot jump right after
G (action = crouch → X action ≠ jump)

X
Mario cannot collide with an enemy while he is
holding a Koopa shell
G (action = haveShell → X action ≠ collision)
G (action = haveShell →
(action ≠ collision U action = dropShell))
or better

Sylvain Hallé
Runtime monitoring
Gerth, Peled, Vardi, Wolper (PSTV 1995): construction of a
Büchi automaton from a given LTL formula j
Benefit:
" ": automaton states are built as the
trace is read
on-the-fly

Sylvain Hallé
Runtime monitoring
j
Benefit:
trace is read
on-the-fly

Sylvain Hallé
Runtime monitoring
s=
j
Benefit:
trace is read
on-the-fly

Sylvain Hallé
Runtime monitoring
s=a
j
Benefit:
trace is read
on-the-fly

Sylvain Hallé
Runtime monitoring
s=a
j
a
Benefit:
trace is read
on-the-fly

Sylvain Hallé
Runtime monitoring
s=ab
j
a
Benefit:
trace is read
on-the-fly

Sylvain Hallé
Runtime monitoring
s=ab
j
a b
b
Benefit:
trace is read
on-the-fly

Sylvain Hallé
Runtime monitoring
s=aba
j
a b
b
Benefit:
trace is read
on-the-fly

Sylvain Hallé
Runtime monitoring
s=aba
j
a
a
a
b
b
Benefit:
trace is read
on-the-fly

Sylvain Hallé
Benefit:
trace is read
Dead end: formula is false
on-the-fly
Runtime monitoring
s=aba
j
a
a
a
b
b

Sylvain Hallé
Runtime monitoring
Algorithm overview:
1. An LTL formula is decomposed into nodes of the form
sub-formulas that
must be true now
sub-formulas that must
be true in the next state

Sylvain Hallé
Algorithm overview:
1. An LTL formula is decomposed into nodes of the form
Example:
sub-formulas that
must be true now
sub-formulas that must
be true in the next state
Runtime monitoring

Sylvain Hallé
2. Negations pushed inside (classical identities +
dual of U = V)
3. At the leaves, Gcontains atoms + negations of atoms:
we evaluate them
Verdict:
! All leaves contain : formula is false
! A leaf is : formula is true
! Otherwise:
4. Next event: Dcopied into Gand we continue
FALSE
empty
Runtime monitoring

Sylvain Hallé
Example: G (a ®b)X
Runtime monitoring

Sylvain Hallé
Example: G (a ®b)X
G (a ®b)X ?
Runtime monitoring

Sylvain Hallé
Example: G (a ®b)X
G (a ®b)X ?
a ®bX G (a ®b)X?
Runtime monitoring

Sylvain Hallé
Example: G (a ®b)X
G (a ®b)X ?
Øa G (a ®b)X?
a ®bX G (a ®b)X?
Runtime monitoring

Sylvain Hallé
Example: G (a ®b)X
G (a ®b)X ?
a, X b G (a ®b)X?Øa G (a ®b)X?
a ®bX G (a ®b)X?
Runtime monitoring

Sylvain Hallé
Example: G (a ®b)X
G (a ®b)X ?
a, X b G (a ®b)X?
a G (a ®b), bX?
Øa G (a ®b)X?
a ®bX G (a ®b)X?
Runtime monitoring

Sylvain Hallé
Example: G (a ®b)X
a G (a ®b), bX?
Øa G (a ®b)X?
Runtime monitoring

Sylvain Hallé
Example: G (a ®b)X
s= a
a G (a ®b), bX?
Øa G (a ®b)X?
Runtime monitoring

Sylvain Hallé
Example: G (a ®b)X
s= a
a G (a ®b), bX?
Runtime monitoring

Sylvain Hallé
Example: G (a ®b)X
s= a
G (a ®b), bX?
Runtime monitoring

Sylvain Hallé
Example: G (a ®b)X
s= a
?G (a ®b), bX
G (a ®b), bX?
Runtime monitoring

Sylvain Hallé
Example: G (a ®b)X
s= a
a, X b, b G (a ®b)X?
a, b G (a ®b), bX?
Øa, b G (a ®b)X?
a ®b, bX G (a ®b)X?
?G (a ®b), bX
Runtime monitoring

Sylvain Hallé
Example: G (a ®b)X
s= a
a, b G (a ®b), bX?
Øa, b G (a ®b)X?
Runtime monitoring

Sylvain Hallé
Example: G (a ®b)X
s= a
Øa, b G (a ®b)X?
Runtime monitoring

Sylvain Hallé
Example: G (a ®b)X
s= ac
Øa, b G (a ®b)X?
Runtime monitoring

Sylvain Hallé
Example: G (a ®b)X
s= ac
No way to extend the trace:
formula is false
Runtime monitoring

Sylvain Hallé
The verification can be separated in two steps
G
X
Ú
"$
$ G
F
®
1. Temporal step
Determine termporal
relationships to current
message
2. Data step
Evaluate relevant XPath
terms on message

Sylvain Hallé
Examples:
2
5. (" k : " c :/Message/SessionKey /Message/CartId
(" k’ : " c’ :/Message/SessionKey /Message/CartId
k = k’ ®c = c’))
G
G
<Message>
...
</Message>
<Message>
...
</Message>

Sylvain Hallé
2
k = k’ ®c = c’))
G
G

Sylvain Hallé
·XPath terms and temporal operators are
mixed
.
·Not just ‘‘LTL with syntactical sugar’’
.
·Not just a pathological case
k = k’ ®c = c’))
G
G
2
G
G
"
"
k

Sylvain Hallé
Examples:
2
6. (" a : a = CartAdd ®/Message/Action
" i : (" a’ :/Message/ItemId /Message/Action
a’ = CartAdd ®" i’ : i ¹i’ ))/Message/ItemId
G
X G
<Message>
<Items>
<Item>
...
<Message>
<Items>
<Item>
...
X

Sylvain Hallé
Quantification must be relative to the values in the current
message, and not the whole set V of possible values!
Example: ‘‘In every message, the a parameter must equal the b
parameter’’. Suppose V = {1,2}, and classical first-order
quantification.
Runtime monitoring
"x : "y : x = ya b
("y : 1 = y) Ù("y : 1 = y)b b
(1 = 1) Ù(1 = 2) Ù(1 = 1) Ù(1 = 2)
Contradiction
G
G G
G G G G

Sylvain Hallé
LTL-FO+
current
(Hallé & Villemaire, EDOC 2008)
Extension of LTL with (limited) first-order quantification on
message elements
·Boolean and LTL operators keep their original meaning
·An XPath term is always meant to refer to the
message in the trace
Runtime monitoring

Sylvain Hallé
Adaptation of the runtime monitoring algorithm to handle
LTL-FO+:
1. Atoms become equality tests
2. Decomposition rules for quantifiers
(and vice versa)
Runtime monitoring

Sylvain Hallé
Enforcing interface contracts at runtime
XMLHttpRequest
·JavaScript object
·Provided by the browser
·All communications to monitor
already centralized: ‘‘no’’
instrumentation

Sylvain Hallé
XMLHttpRequestBB

Sylvain Hallé
XMLHttpRequestBB
XMLHttpRequest
LTL-FO+
algorithm
·Wrapper around original
·Provides same methods
·Checks messages before
relaying them

Sylvain Hallé
Add BeepBeep to an application
myapplication.html
<html>
<head>
<title>
</title>
<script type=" "
href=" "/>
</head>
<body>
</body>
</html>
My Application
...
text/javascript
myapplication.js
?
Include BeepBeep
Copy BeepBeep in the application's directory
http://beepbeep.sourceforge.net

Sylvain Hallé
myapplication.html
<html>
<head>
<title>
</title>
<script type=" "
href=" "/>
</head>
<body>
</body>
</html>
My Application
...
text/javascript
myapplication.js
<script type="text/javascript"
href="beepbeep.js"/>
?
Include BeepBeep

Sylvain Hallé
myapplication.html myapplication.js
<html>
<head>
<title>
</title>
<script type=" "
href=" "/>
</head>
<body>
</body>
</html>
My Application
...
text/javascript
myapplication.js
// Initializations
= ();
()
{
( );
}
...
req XMLHttpRequest
...
abc
...
req. some_message
new
function
send
?
Include BeepBeep

Sylvain Hallé
beepstore.html beepstore.js
<html>
<head>
<title>
</title>
<script type=" "
href=" "/>
</head>
<body>
</body>
</html>
My Application
...
text/javascript
myapplication.js
// Initializations
= ();
()
{
( );
}
...
req
...
abc
...
req. some_message
new
function
send
XMLHttpRequestBB
Include BeepBeep
?


Sylvain Hallé
Create a with LTL-FO+ formulascontract file?
# -------------------------------------------------------
# BeepBeep contract file for the Beep Store
# -------------------------------------------------------
% The element Page must be an integer between 1 and 20.
% The element Page is mandatory only if Results is
present, otherwise it is forbidden.
% The Login request cannot be resent if its response
is successful.
; ( p /Message/Page (((p) > ({0})) ((p) < ({21}))))
; ( a /Message/Action (((a) = ({ItemSearch})) (
(( r /Message/Results ({TRUE}))
( p /Message/Page ({TRUE})))
(( p /Message/Page ({TRUE})) (
r /Message/Results ({TRUE}))))))
; ( a /Message/Action (((a) ({LoginResponse}))
( ( ( b /Message/Action ( ((b) ({Login}))))))))
G
G
G
X G
[ ]
[ ]
< >
< >
< >
< >
[ ]
[ ]
&
->
->
&
->
= ->
! =
Caption: used
when violations
are discovered
Plain-text
LTL-FO+
(automatically
parsed)
}

Sylvain Hallé
When loading the application, BeepBeep starts as a small
Java applet inside the page
The
Beep
Store
GO
Sign in or register
What is this?
Login
Ask for account
Contact us
Fault parameters
Search: Your Cart
Search results for ‘Beatles’
Rubber Soul
The Beatles
Yellow Submarine
The Beatles

?/?/?/?/?/?:0:0

Sylvain Hallé
BeepBeep’s visible interface
?/?/?/?/?/?:0:0
Current state of monitor
for each property
Number of
messages
processed
Cumulative
processing
time (in ms)
T: last message made it true
t: is true
F: last message made it false
f: is false
?: not yet true/false

Chocolate Doom
74 KLOC
Angry Bots
26 KLOC
Pacman Canvas
1 KLOC
Bos Wars
113 KLOC
Infinite Mario Bros.
6 KLOC
Pingus
40 KLOC

We still have to write the
properties, so what's the point?*
We're too busy to learn
another toolA*
Using this requires a higher level of
education. We'll have to increase wagesA*
*Really happened
We can already do all thisA*
(The Angry Practicioner)

OF COURSE.
This is a ﬁnitely computable problem.
It can be computed in ﬁnite time by a
Turing machine.
There exists a C/Python/Assembly/Haskell/JavaScript/Scala/Go/BrainFuck//Whatever
program that does that. We just need to
write it down.
So what's the big news???

Data
30,000
LINES OF
GLUE CODE
Untested
You wrote this
Hardly reusable
Result
HOW.

Data
Componentized
Reusable
Worth investing time testing
Result
HOW.
Engine
1,000
LINES OF
DSL

Alan Perlis
(1922-1990)
Beware of of the
Turing tar-pit in
which everything is
possible but nothing
of interest is easy. ,,
,,

G (∃ retAddrVal ∈ ./return-address : (
(./instruction = call) ∧ (¬ ((F( ((./instruction = mov)
∧ (./output/type = general-register)) →
(∃ regA ∈ ./output/name : (F (( ((./instruction = mov)
∧ (./output/type = general-register))
∧ (./input/type = litteral) ) →
(∃ regB ∈ ./output/name :
( ∃ constAddr ∈ ./input/value :
(F (((./instruction = cmp ) ∧
(./output/type = regA)) →
(∃ loc ∈ ./location : (F((
((./instruction = mov ) ∧
(./output/type = general-register)) ∧
(./output/name = regA ) ) ∧
((./input/name = regB ) ∧ (./input/type = ptr)))))) ))))) )))
U ((./instruction = return)
∧ (./fonction-returned = retAddrVal)) )) ))

EventsEvents
An event is an element e taken from some
set E, called the event type
Booleans
B
Numbers
R
2
3
4
π
Strings
S
abc
Functions
X Y→
Sets
X
2
Primitive
types
Composite
types

A sample logA sample log
A file (or stream) of events
[10:24:33] WARN Lemming into Blocker...[
[10:25:01] DEBG Lemming into Floater, id: 32,
x: 320, y: 67 ; id: 31, x: 450, y: 43 ;
id: 23, x: 229, y: 40 ; ... ...
Each event has one or more
data elements
Actual (physical) format not relevant
for us

Searching the logSearching the log
Select AVG(closingPrice)
From ClosingStockPrices
Where stockSymbol = `MSFT'
for (t = ST; t < ST+50, t+= 5) {
WindowIs(ClosingStockPrices, t - 4, t);
}

ProblemsProblems
Formal languages (e.g. logic, automata)
focus on event ordering; not so good at
performing computations over events
Complex Event Processing often reduces
to a thin layer over custom procedural
code
Goal: provide a formal and
non-procedural framework for
the processing of event streams

TracesTraces
An event trace (or event stream) is a potentially
infinite sequence of events of a given type:
2 0 6 3
4 9 . . .
Traces are symbolically denoted by:
e = e0 e1 e2 e3 ...
The set of all traces of type T is denoted as:
T*

ProcessorsProcessors
A processor is a function that takes 0 or more
event traces as input, and returns 0 or 1
event trace as output
1 : 1 processor
2 : 1 processor
. . . . . .

A high-level event trace can be produced by
composing ("piping") together one or more
processors from lower-level traces
CompositionComposition

Processor algebraProcessor algebra
Goal: come up with a "toolbox" of basic
processors sufficient to perform various
computations over traces
??

A few useful functionsA few useful functions
Identity function: returns an event if given one,
or t if passed the empty event ε
ιt(x) =
{t if x = ε
x otherwise
+(x) = {x}
Wrap function
-({x}) = x
Peel function
Path function: returns subtree at end
of path π
/π

SemanticsSemantics
Processors can be defined formally by
describing how their output trace is created
from their input trace(s)
e0, ..., en : φ(x0 , ..., xn)
Input trace(s)
Symbolic variables:
xi refers to the i-th trace
on the left

Constants as processorsConstants as processors
Any element t of type T can be lifted as a
0 : 1 processor producing the infinite trace
t t t t ...
t . . .t t
The constant
processor t e : t = t t t ...

Input/outputInput/output
0 : 1 processors can be used to produce an
event trace out of an external source (i.e.
standard input, a file, etc.)
Ditto for 1 : 0 processors
a . . .b
a . . .b

MutatorMutator
Returns t, but only as many times as the
number of events received so far
i.e. "mutates" input events into t
tte

Functions as processorsFunctions as processors
Any n-ary function f defined on individual
events can be lifted to an n:1 processor on
traces, by applying it successively to n-uples
2 0 6. . .
3 8 1
+ 7 8 5
. . .
. . .

Functions as processorsFunctions as processors
Any n-ary function f defined on individual
events can be lifted to an n:1 processor on
traces, by applying it successively to n-uples
e0, e1 : x0+x1
e00
+ e10
e01
+ e11
, e02
+ e12
, , . . .
=

FreezeFreeze
Returns the first event received, upon every
event received
abb. . . a a a . . .
e : x = e0 e0 e0 ...

DelayDelay
Returns every the input trace, starting from its
n-th event
abc. . . b . . .
e : x = en en+1 en+2 ...
2
n
e : xn=
c

DecimateDecimate
Returns every n-th event of the input trace
abc. . . a . . .
e : x = e0 en e2n ...
2
n
c
Ψ
Ψ
e : x=
ni
e : xn
Ψ
i

WindowWindow
Simulates the application of a "sliding
window" to a trace
Takes as arguments: another processor φ
and a window width n
Returns the result of φ after processing
events 0 to n-1...
Then the result of (a new instance of) φ
that processes events 1 to n...
...an so on
Υ φn

Example: execution of the processor
on the trace
WindowWindow
2 1 5 0
Υ ++2
Υ2
2 12 1 2 12 3
2 11 5 2 11 6
2 15 0 2 15 5
2 1 5 0 3 6 5

WindowWindow
The window processor can take any
processor as an argument...
...i.e. the sliding window can be applied to
anything.
Formally:
e : φ e : φ=
n-1iΥn
i

FilterFilter
Discards events from an input trace based
on a selection criterion
Takes as argument another processor φ
Evaluates φ on the trace that starts at event
0; returns that event if the first event
returned by φ is T
Same process on the trace that starts at
event 1...
...an so on
Φ φ

on the trace
FilterFilter
2 1 5 0
Φ∈2IN
Φ2 1 5 0 2 0
∈2IN
∈2IN2 1 5 0

FilterFilter
The filter can take any processor as an
argument...
...including a processor that requires multiple
input events before outputting something
Formally:
e : φ e : φ=Φ 1
ΦΦ(e, φ) ,
Φ(e, φ) =
{ e0 if
no event otherwise
e : φ = T
0

SpawnSpawn
Cumulative combination of a processor's
output for every suffix of a trace
Creates one new instance of processor
φ upon every new input event
Feeds each input event to all existing
instances of φ
Combines the value returned by each
instance using function f
...and outputs it
Σ φf

on the trace
SpaweSpawn
2 1 5 0
Σ+
Σ+2 1 5 0 2 3 8
x
x
x
8
2 1 5 0 2 1 5 0
x1 5 0 1 5 0
x5 0 5 0
++
+

SpawnSpawn
Formally:
e :
e :
=
1
Σ φf
e : φ
0
, f ( Σ φfe : φ
0
, e : φ
0
,e : φ
0
, )
Turns out to be a powerful device; depending
on φ and f, can provide many useful
processors...

SpawnSpawn
Count events Σ 1+
Cumulative sum Σ+
Set of all events Σ∪+
= #
= ++
=∪

These processors can be freely composed
Compute the statistical moment of order n
n
Σ+
Σ+
1
÷

Compute the statistical moment of order n
n
Σ+
Σ+
1
÷
= #

Return sum of two successive events,
only if it is greater than 5
++
Υ2
Φ
> 5

All together nowAll together now

Count pairs of successive events that are
more than one standard deviation from
the mean
E(X)
-

the mean
σ
E(X)
-
÷

the mean
σ
E(X)
-
÷ Φ
> 1

the mean
σ
E(X)
-
÷
X
Φ
> 1
Φ∧

the mean
#σ
E(X)
-
÷
X
Φ
> 1
Φ∧

AdvantagesAdvantages
No imperative constructs
No restrictions on what can be piped to
what (modulo type compatibility)
Streaming operation: outputs produced
as inputs are being consumed
Implicit handling of buffering, duplication,
etc.

A declarative event stream
query engine

abc. . . a . . .
n c
Ψ
abc. . . b . . .
n c
EVERY nTH OF (T)
TRIM n FROM (T)

WHERE2 1 5 0 2 0
C
C2 1 5 0
(T) WHERE condition

COMBINE (T) WITH f
Σ+2 1 5 0 2 3 8
f
f
8
2 1 5 0 2 1 5 0
f1 5 0 1 5 0
f5 0 5 0
++
+

a . . .b
a . . .b
FILE "filename"
SAVE (T) TO "filename"

WHEN @P IS A PROCESSOR:
THE SUM OF ( @P )
IS THE PROCESSOR
COMBINE (@P) WITH SUM.
Arbitrary
symbol
Grammar rule this symbol
must parse against
New grammar case
Grammar rule the case
is added to
Expression the new case
stands for

THE COUNT OF ( @P ) IS THE PROCESSOR
COMBINE (SELECT 1 FROM (@P)) WITH SUM.
THE SUM OF ( @P ) IS THE PROCESSOR
COMBINE (@P) WITH SUM.
THE AVERAGE OF ( @P ) IS THE PROCESSOR
SELECT (T.*) ÷ (U.*) FROM (
THE SUM OF (@P) AS T,
THE COUNT OF (@P) AS U).

On every fifth trading day starting today,
calculate the average closing price of MSFT
for the five most recent trading days, and keep
the query standing for fifty trading days.
timestamp stockSymbol closingPrice
0 APPL 1039.3
0 MSFT 950.0
0 GOGL 433.3
1 MSFT 951.2
1 APPL 1038.3
... ... ...

String line = br.readLine().trim();
if (!line.isEmpty()) {
String[] parts = line.split(",");
if (parts[0].compareTo("ABC") != 0) {
value_index++;
sum += Double.parseDouble(parts[1]);
if (value_index == 5) {
double average = sum / 5;
value_index = 0;
sum = 0;
return average;
}}}

SELECT afd FROM (
SELECT S1.timestamp AS ts,
AVG(S2.closingPrice) AS afd
FROM
(SELECT * FROM stocks
WHERE stockSymbol = "MSFT") AS S1,
WHERE stockSymbol = "MSFT") AS S2
WHERE (S2.timestamp - S1.timestamp) < 5
GROUP BY S1.timestamp) AS S3
WHERE MOD(ts, 5) = 0;

EVERY 5TH OF (
APPLY (THE AVERAGE OF (*)) TO (
SELECT closingPrice FROM stocks)
WHERE (stockSymbol) = ("MSFT"))))
ON A WINDOW OF 5).

Calculate how many times the closing price
of MSFT is greater than 20 and the next
day, its closing price is less than 10.

SELECT COUNT(*) FROM
WHERE stockSymbol = "MSFT") AS S1,
WHERE stockSymbol = "MSFT") AS S2
WHERE (S2.timestamp - S1.timestamp) = 1
AND S1.closingPrice > 20
AND S2.closingPrice < 10;

MY PATTERN IN ( @P ) IS THE PROCESSOR
(SELECT (closingPrice) LESS THAN (20) FROM (@P))
AND
(NEXT (SELECT (closingPrice) GREATER THAN (10) FROM (@P))).
THE COUNT OF ((
MY PATTERN IN (
(SELECT closingPrice FROM stocks)
WHERE (stockSymbol) = ("MSFT")))
WHERE (*) = (true)).
Linear
Temporal
Logic!

import ca.uqac.lif.cep.*;
public class MyProcessor extends SingleProcessor {
public Queue<Vector<Object>> compute(Vector<Object> inputs)
{
}
public void build(Stack<Object> s)
{
}
}
. . . Create output events from input . . .
. . . Instantiate processor from parse stack . . .
<processor> := . . .
<number> := . . .
<string> := . . .
Add new rules to any symbol from
the basic grammar

<processor> := <my_processor> ;
<my_processor> := INCREMENT ( <processor> ) BY <number> ;
Symbols already defined in basic grammar
Adds a new case to an existing rule

import ca.uqac.lif.cep.*;
public class MyProcessor extends SingleProcessor {
private int increment;
public Queue<Vector<Object>> compute(Vector<Object> inputs)
{
Queue<Vector<Object>> out = new Queue<Vector<Object>>();
Vector<Object> v = new Vector<Object>();
Integer i = (Integer) inputs.firstElement() + increment;
v.addElement(i);
out.put(v);
return out;
}
. . .

. . .
public void build(Stack<Object> s)
{
Number n = (Number) s.pop();
s.pop();
s.pop();
Processor p = (Processor) s.pop();
s.pop();
s.pop();
increment = n.intValue();
Connector.connect(p, this);
s.push(this);
}
}
Read contents of
parse stack
<number>
BY
(
<processor>
)
INCREMENT
Set processor's state
Pipe it to its input
Put on parse stack

Some pre-packaged grammar extensions:
Manipulation of name-value tuples
Set theory
Formatted input (CSV, XML, JSON)
Graphing (histograms, scatterplots, ...)
Basic signal processing (smoothing,
peak detection, ...)
Create your own!

It works
1500 Hz performance
Rapid prototyping -400 LOC3
What vs. how
https://liﬂab.github.io/
beepbeep-3

A few pointersvvv
Mv LeuckerE Cv Schallhartv y4KK9Hv A Brief Account of
Runtime Veriﬁcationv J. Log. and Alg. Prog. 78 y5Hv
Dv Luckhamv y4KK4Hv The Power of Events: An
Introduction to CEPv AddisonbWesleyv
Sv HalléE Rv Villemairev y4KG4Hv Runtime Enforcement
of Web Service Message Contracts with Datav IEEE
Trans. Services Computing 5y4Hv
Sv VarvaressosE Kv LavoieE Av Blondin MasséE Sv
GabouryE Sv Hallév y4KG:Hv Automated Bug Finding in
Video Games: A Case Study for Runtime Monitoringv
ICST 2014v

Chasing Bugs with the BeepBeep Event Stream Processor

Recommandé

Recommandé

Contenu connexe

En vedette

En vedette (14)

Similaire à Chasing Bugs with the BeepBeep Event Stream Processor

Similaire à Chasing Bugs with the BeepBeep Event Stream Processor (20)

Plus de Sylvain Hallé

Plus de Sylvain Hallé (20)

Dernier

Dernier (20)

Chasing Bugs with the BeepBeep Event Stream Processor