The term runtime logic verification defines a field that ranges from software verification for compliance with a set of specifications to assuring the adoption of good coding practices. Under this scope, we created lovpy, a novel metaprogramming library for python, that introduces to its ecosystem the capabilities of runtime logic verification. Definition of expected behavior is performed using the intuitive specifications language Gherkin, while using the library requires no code modifications. For its implementation we utilized a broad set of tools, ranging from the domains of graph theory, formal languages theory and temporal logic to deep learning, with specific focus on graph neural networks. We also, provided the mathematical foundation for a new type of graph, designed for representing temporal specifications. Based on it, we defined a set of mathematically proved logic algorithms. Then, we used these structures for implementing a novel theorem proving system, located at the heart of lovpy and ensuring the absolute validity of reported violations. We evaluated five different proving architectures, consisting from heuristics and simple neural models, to deep graph neural networks. For the training of neural systems, we implemented a mechanism for generating synthetic theorems, utilizing a series of mathematical properties. Finally, we used lovpy for detecting bugs in two popular open-source libraries, Django and Keras.

1. Python metaprogramming
in linear time language
for automated runtime verification
with graph neural networks
Diploma thesis by:
Dimitrios Karageorgiou (SRN: 8420)
soulrain@outlook.com
Supervisors:
• Andreas Symeonidis (Associate Professor)
• Emmanouil Krasanakis (PhD Candidate)
Faculty of Engineering
School of Electrical And Computers Engineering
Department of Electronics and Computer Engineering
Intelligent Systems and Software Engineering Labgroup
Friday, 12th November 2021

2. In other words…
Lovpy is a runtime logic verification library for Python.
2
Lovpy
Logic verification becomes mainstream!
Lovpy by Dimitrios
Karageorgiou

3. What is logic verification?
 Runtime verification approach at its heart.
► E.g. a deadlock can be detected only at runtime.
 Extends verification techniques outside of strict software verification domain.
 E.g. enforcing best practices to the users of a library is also a logic problem.
3
Instrumentation
Python
System
Monitor
Specifications
Verdicts
Events
Specification Violated
or
Specification Holds
e.g.
Lovpy by Dimitrios
Karageorgiou

4. Not another runtime verification library...
 Only complex and application specific runtime verification libraries existed for Python.
 Design goals:
 Minimal user effort to enable verification, without required code modifications.
 Specifications in an easy-to-learn and intuitive language.
 Never report a violation that does not exist (0% false-negatives).
 Report violations before they happen (prevent side-effects).
 Report the last provably correct line of code (all specifications hold).
4
Lovpy by Dimitrios
Karageorgiou

5. No code modifications required!
 Enable verification by just executing the library:
python –m lovpy <script.py>
5
Lovpy by Dimitrios
Karageorgiou
► Lovpy AOT Preprocessor handles the rest:
Python Code
Lovpy AOT
Preprocessor
Augmented Python
Code
Python Runtime
Python Objects Augmented Objects Lovpy Execution

6. Augmented Python Objects
 Each Python object is augmented to hold its execution state:
6
Current System State
Augmented Object 1
Object 1 State 1
Augmented Object 2
Object 2 State 2
Augmented Object N
Object N State N
 Objects can live anywhere
(multiple threads, processes etc.)
 Parallelization of original system is retained.
 Verification on per-object basis.
Lovpy by Dimitrios
Karageorgiou

7. Specifications in Gherkin 7
Gherkin is:
 Simple
 Easy-to-learn
 Intuitive
1. SCENARIO:
2. WHEN call acquire
3. THEN SHOULD NOT locked
4. AND locked
5.
6. SCENARIO:
7. GIVEN locked
8. WHEN call release
9. THEN NOT locked
Lovpy by Dimitrios
Karageorgiou

8. Everything is mathematically proved!
 Monitor utilizes an Automated Theorem Prover:
8
Monitor
Current System
State
Theorems
Automated Theorem
Prover
System State
Builder
Specifications
Parser
Properties to Prove
Execution Events
Specifications
Verdicts
Lovpy by Dimitrios
Karageorgiou

9. Automated Theorem Proving
9
Lovpy by Dimitrios
Karageorgiou

10. Everything is a Temporal Graph
 Execution States, Theorems and Properties are converted to Temporal Graphs.
 Temporal Graph:
o A kind of Abstract Syntax Graph.
o Nodes are either logical operators or predicates.
o Edges contain timestamps.
o Each timestamp is the most recent moment the
subgraph holds.
o Timestamps can be relative or absolute.
o Definition of mathematically proved logic algorithms (logic graph removal/addition, graph modus
ponens, etc.)
10
Lovpy by Dimitrios
Karageorgiou

11. Everything is a Temporal Graph
 Execution States, Theorems and Properties are converted to Temporal Graphs.
11
 …
 lock = threading.Lock()
 …
 lock.acquire()
 …
 lock.release()
 …
 lock.acquire()
 …
Lovpy by Dimitrios
Karageorgiou

12. Everything is a Temporal Graph
 Execution States, Theorems and Properties are converted to Temporal Graphs.
12
1. SCENARIO:
2. GIVEN locked
3. WHEN call release
4. THEN NOT locked
Lovpy by Dimitrios
Karageorgiou

13. Everything is a Temporal Graph
 Execution States, Theorems and Properties are converted to Timed Graphs.
13
1. SCENARIO:
2. WHEN call acquire
3. THEN SHOULD NOT locked
4. AND locked
 Part of conclusion that refers to the same time moment with assumption, is always proved and becomes a theorem.
Lovpy by Dimitrios
Karageorgiou

14. Proving process initialization 14
Lovpy by Dimitrios
Karageorgiou
Property to Prove
Execution State Graph

15. Theorem application #1 15
Lovpy by Dimitrios
Karageorgiou
Applied Theorem #1
Execution State Graph #1

16. Theorem application #2 16
Lovpy by Dimitrios
Karageorgiou
Applied Theorem #2
Execution State Graph #2

17. Property proved! 17
Lovpy by Dimitrios
Karageorgiou
Proved property
Final Execution State Graph

18. Violation detected! 18
Lovpy by Dimitrios
Karageorgiou
► Last correct line reported too!

19. Improve theorem proving capability
 Deterministic next theorem selection:
 Next theorem to apply is the one whose assumption uses the oldest predicates.
х Problem: Oldest theorem is not always the appropriate one to apply.
 Solution: Deep Learning and Graph Neural Networks
► Next theorem to apply is selected using a deep neural model.
 Use Graph Neural Networks to embed graph.
 Train model using synthetic theorems generated by:
 Lovpy Synthetic Theorems Generator
19
Lovpy by Dimitrios
Karageorgiou

20. Deep Neural Architecture Overview 20
Lovpy by Dimitrios
Karageorgiou
Concatenation
Current
State
Theorem
Instance
Goal
Property
Theorem N
Theorem 2
Theorem 1
Score N
Score 2
Score 1
Current
State Graph
Encoder
Theorem
Instance
Graph
Encoder
Goal
Property
Graph
Encoder
Current
State
Graph
Theorem
Graph
Goal
Graph

21. Neural Graph Encoder 21
Lovpy by Dimitrios
Karageorgiou

22. Evaluation 22
 Evaluated five different architectures on proving 2.5k synthetic theorems.
Correct Proofs
Heuristic 70.7%
MLP 59.25
MLP + Heuristic 70.7%
GNN 61.6%
GNN + Heuristic 73.8%
Lovpy by Dimitrios
Karageorgiou
► Detected common bugs of 5 different domains, in 20 erroneous python programs.
 Detected two bugs in popular open-source projects:
 Django Web Framework
 Keras

23. Future possibilities
► Public Lovpy’s Repository:
 Community written specifications for specific domains
(e.g. best practices for using Tensorflow)
 Community trained neural models for theorem selection.
 Specifications mining for eliminating the need for hand-written specifications.
 Natural language support in Gherkin rules.
23
Lovpy by Dimitrios
Karageorgiou

24. Questions???
24
Thanks for watching!
Lovpy by Dimitrios
Karageorgiou

25. Slides Graveyard
25
Lovpy by Dimitrios
Karageorgiou

26. Lovpy is available for everyone!
► Lovpy is available at PyPI:
► Also available as an open-source project on Github:
26
Lovpy by Dimitrios
Karageorgiou
python –m pip install lovpy
https://github.com/dkarageo/lovpy

27. Detecting code violations 27
 Detected two bugs in popular open-source projects:
 Django Web Framework
 Keras
Violations Detected
Threads Data Neural Math Common Total
Heuristic 3 1 2 2 6 14
MLP 1 0 1 2 3 7
MLP + Heuristic 3 1 2 2 6 14
GNN 4 1 1 1 3 12
GNN + Heuristic 5 2 3 2 6 18
► Detected common bugs of 5 different domains, in 20 erroneous python programs.
Lovpy by Dimitrios
Karageorgiou

28. Synthetic sample example 28
► Generated by Lovpy Synthetic Theorems Generator.
Lovpy by Dimitrios
Karageorgiou