keynote talk presented at the 2013 National Software Application Conference (NASAC 2013), Tianjin, China, 9 November 2013

1. NASAC 2013,Tianjin, 9 November 2013
Probability and Uncertainty
in Software Engineering
David S. Rosenblum!
Dean, School of Computing!
National University of Singapore

2. NASAC 2013,Tianjin, 9 November 2013
Software Engineering
at NUS
Hugh
Anderson
Chin
Wei Ngan
Dong
Jin Song
Aquinas
Hobor
Joxan!
Jaffar
Stan
Jarzabek
Khoo
Siau Cheng
Damith
Rajapakse
David!
Rosenblum
Abhik
Roychoudhury
Bimlesh
Wadhwa
Yap
Hock Chuan,
Roland

3. NASAC 2013,Tianjin, 9 November 2013
Certainty in
Software Engineering
Engineering of software is centered around
simplistic,“yes/no” characterizations of artifacts

4. NASAC 2013,Tianjin, 9 November 2013
Certainty in
Software Engineering
Engineering of software is centered around
simplistic,“yes/no” characterizations of artifacts
Program is correct/incorrect
Program execution finished/crashed
Compilation completed/aborted
Test suite succeeded/failed
Specification is satisfied/violated

5. NASAC 2013,Tianjin, 9 November 2013
Example!
Model Checking
! ¬p → ◊q( )∧"( )
Model
Checker
✓
✕
State Machine!
Model
Temporal
Property
Results
Counterexample!
Trace
System
Requirements

6. NASAC 2013,Tianjin, 9 November 2013
Example!
Model Checking
! ¬p → ◊q( )∧"( )
Model
Checker
✕
State Machine!
Model
Temporal
Property
Results
Counterexample!
Trace
System
Requirements

7. NASAC 2013,Tianjin, 9 November 2013
Uncertainty in
Software Engineering
✓Nondeterminism
✓Randomized Algorithms
✓“Good Enough Software”
✓Test Coverage Metrics

8. NASAC 2013,Tianjin, 9 November 2013
Uncertainty in
Software Engineering
✓Nondeterminism
✓Randomized Algorithms
✓“Good Enough Software”
✓Test Coverage Metrics
Probabilistic Modeling and Analysis

9. NASAC 2013,Tianjin, 9 November 2013
Probabilistic
Model Checking
! ¬p → ◊q( )∧"( )
Model
Checker
✓
✕
State Machine!
Model
Temporal
Property
Results
Counterexample!
Trace
System
Requirements
P≥0.95 [ ]
0.4
0.6
Probabilistic
Probabilistic

10. NASAC 2013,Tianjin, 9 November 2013
Probabilistic
Model Checking
! ¬p → ◊q( )∧"( )
Model
Checker
✓
✕
State Machine!
Model
Temporal
Property
Results
Counterexample!
Trace
System
Requirements
P=? [ ]
0.4
0.6
Quantitative Results
0.9732Probabilistic
Probabilistic

11. NASAC 2013,Tianjin, 9 November 2013
Example
Die Tossing Simulated by Coin Flipping
Knuth-Yao algorithm,
from the PRISM group
(Kwiatkowska et al.)
0
3
2
1
6
4
5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5

12. NASAC 2013,Tianjin, 9 November 2013
Example
Die Tossing Simulated by Coin Flipping
Knuth-Yao algorithm,
from the PRISM group
(Kwiatkowska et al.)
The behavior is governed by a!
theoretical probability distribution
0
3
2
1
6
4
5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5
0.5

13. NASAC 2013,Tianjin, 9 November 2013
Probabilistic
Model Checking
! ¬p → ◊q( )∧"( )
Model
Checker
✓
State Machine!
Model
Temporal
Property
Results
Counterexample!
Trace
System
Requirements
P≥0.95 [ ]
0.4
0.6
Quantitative Results
0.9732Probabilistic
Probabilistic

14. NASAC 2013,Tianjin, 9 November 2013
Probabilistic
Model Checking
! ¬p → ◊q( )∧"( )
Model
Checker
✕
State Machine!
Model
Temporal
Property
Results
Counterexample!
Trace
System
Requirements
P≥0.95 [ ]
Quantitative Results
Probabilistic
Probabilistic
0.41
0.59
0.6211

15. NASAC 2013,Tianjin, 9 November 2013
Example!
Zeroconf Protocol
s1s0 s2 s3
q
1
1
{ok} {error}
{start} s4
s5
s6
s7
s8
1
1-q
1-p
1-p
1-p
1-p
p p p
p
1
from the PRISM group
(Kwiatkowska et al.)

16. NASAC 2013,Tianjin, 9 November 2013
Example!
Zeroconf Protocol
s1s0 s2 s3
q
1
1
{ok} {error}
{start} s4
s5
s6
s7
s8
1
1-q
1-p
1-p
1-p
1-p
p p p
p
1
The behavior is governed by an!
empirically estimated probability distribution
from the PRISM group
(Kwiatkowska et al.)
packet-loss rate

17. NASAC 2013,Tianjin, 9 November 2013
Perturbed Probabilistic Systems!
(Current Research)
• Starting Points!
✓Discrete-Time Markov Chains (DTMCs)!
✓… with one or more probability parameters!
✓… verified against reachability properties:
S? ∪ S!
Guoxin Su and David S. Rosenblum,
“Asymptotic Bounds for QuantitativeVerification of Perturbed Probabilistic Systems”,
Proc. ICFEM 2013

18. NASAC 2013,Tianjin, 9 November 2013
Parametric
Markov Chains
• A distribution parameter in a DTMC is represented as a
vector x of parameters xi!
• The norm of total variance represents the amount of
perturbation:!
!
• The parameter is allowed a “sufficiently small”
perturbation with respect to ideal reference values r:!
!
• Can generalize to multiple parameters
v = vi∑
x − r ≤ Δ

19. NASAC 2013,Tianjin, 9 November 2013
Perturbation Bounds
• Perturbation Function!
!
where A is the transition probability sub-matrix for S?
and b is the vector of one-step probabilities from S? to S!
!
• Condition Numbers!
!
ρ x( )= ι? i A x
i
i b x( )− Ai
i b( )( )i=0
∞
∑
κ = lim
δ→0
sup
ρ(x − r)
δ
: x − r ≤ δ,δ > 0
⎧
⎨
⎩
⎫
⎬
⎭

20. NASAC 2013,Tianjin, 9 November 2013
Results!
Noisy Zeroconf (35000 Hosts, PRISM)
p
Actual
Collision Probability
Predicted
Collision Probability
0.095 -19.8% -21.5%
0.096 -16.9% -17.2%
0.097 -12.3% -12.9%
0.098 -8.33% -8.61%
0.099 -4.23% -4.30%
0.100 1.8567 —
0.101 +4.38% +4.30%
0.102 +8.91% +8.61%
0.103 +13.6% +12.9%
0.104 +18.4% +17.2%
0.105 +23.4% +21.5%

21. NASAC 2013,Tianjin, 9 November 2013
Additional Aspects
• Models
✓Markov Decision Processes (MDPs)!
✓Continuous-Time Markov Chains (CMTCs)
• Verification
✓LTL Model Checking!
using Deterministic Rabin Automata!
✓PCTL Model Checking!
with singular perturbations due to nested P[ ] operators!
✓Reward Properties!
✓Alternative Norms and Bounds!
Kullback-Leibler Divergence, Quadratic Bounds

22. NASAC 2013,Tianjin, 9 November 2013
Other Forms of
Uncertainty
“There are known knowns; there are things we know
we know. We also know there are known unknowns;
that is to say, we know there are some things we do
not know. But there are also unknown unknowns –
the ones we don’t know we don’t know.”!
!
— Donald Rumsfeld

23. NASAC 2013,Tianjin, 9 November 2013
Uncertainty in Testing!
(New Research)
1982: Weyuker: Non-Testable Programs!
- Impossible/too costly to efficiently check results!
- Example: mathematical software!
2010: Garlan: Intrinsic Uncertainty!
- Systems embody intrinsic uncertainty/imprecision!
- Cannot easily distinguish bugs from “features”!
- Example: ubiquitous computing

24. NASAC 2013,Tianjin, 9 November 2013
Example!
Google Latitude
~ 500m
~ 50m
~ 2m

25. NASAC 2013,Tianjin, 9 November 2013
Example!
Google Latitude
When is an
incorrect location!
a bug, and when
is it a “feature”?
~ 500m
~ 50m
~ 2m

26. NASAC 2013,Tianjin, 9 November 2013
Example!
Google Latitude
When is an
incorrect location!
a bug, and when
is it a “feature”?
And how do!
you know?
~ 500m
~ 50m
~ 2m

27. NASAC 2013,Tianjin, 9 November 2013
Example!
Affective Computing

28. NASAC 2013,Tianjin, 9 November 2013
Example!
Affective Computing
When is an!
incorrect!
classification a bug,!
and when is it a!
“feature”?

29. NASAC 2013,Tianjin, 9 November 2013
Example!
Affective Computing
When is an!
incorrect!
classification a bug,!
and when is it a!
“feature”?
And how do!
you know?

30. NASAC 2013,Tianjin, 9 November 2013
Sources of
Uncertainty
✓Output: results, characteristics of results!
✓Sensors: redundancy, reliability, resolution!
✓Context: sensing, inferring, fusing!
✓Machine learning: imprecision, user training

31. NASAC 2013,Tianjin, 9 November 2013
Sources of
Uncertainty
✓Output: results, characteristics of results!
✓Sensors: redundancy, reliability, resolution!
✓Context: sensing, inferring, fusing!
✓Machine learning: imprecision, user training
These create significant challenges for
software engineering research and practice!

32. NASAC 2013,Tianjin, 9 November 2013
Conclusion
✓Software engineering (certainly) suffers
from excessive certainty!
✓A probabilistic mindset offers greater insight!
✓But significant challenges remain for
probabilistic verification!
✓And other forms of uncertainty are equally
challenging to address

33. NASAC 2013,Tianjin, 9 November 2013
Probability and Uncertainty
in Software Engineering
David S. Rosenblum!
Dean, School of Computing!
National University of Singapore
ThankYou!