Kits to Find the Bits that Fits

1
Kits to Find
the Bits that Fits
tim.menzies@gmail.com
SAM’15, ICSE, Florence Italy, May 2015
Learning adaptive architectural
principles in the post-agile world

SOUNDS BITES
• Agile (is old)
• There are only three numbers
• Zero, one or many
• Not architecture, but architectureS,
• Architectures will change
• Rework cheaper than we thought
• Programmers (soon) will be much cheaper
• Architectural incubators
• The age of the app
• Variability modeling (feature maps)
2

Before we begin
FYI: I really like
architecture(s)

ARCHITECTURES = FENCES
Something that lets
you work alone …
• … half the time
Team members
• can be productive
in isolation
• But know when
they need to talk
to others
Good fences make
good neighbors
4

LARGE SCALE ARCHITECTURES:
BOEING 787
5

ARCHITECTURES IN
THE POST-AGILE WORLD
• The following may be old news to some of you
• But for the rest:
• Agile is old hat.
• Welcome to DevOps
• And after DevOps?
• Continuous Deployment
• standard practice by 2020
• tools like MEAN
8

9
Source: D. Reimer, A. Cho, Moving to DevOps and Beyond: https://goo.gl/25PxmH

10

11

12

QUESTION
How to make architectural reasoning flexible enough to
handle this pace of change?
13

Adaptive conclusions
and the PROMISE
Project

THE PROMISE REPO
OPENSCIENCE.US/REPO #storingYourResearchData
URL
• openscience.us/repo
• Data from 100s of projects
• E.g. EUSE:
• 250,000K+ spreadsheets
Oldest continuous
repository of SE data
• For other repos, see
Table 1 of goo.gl/UFZgnd
15
Serve all our data, on-line
The PROMISE Project: 2005… 2015

TRANSFERRING LESSONS LEARNED:
TURKISH TOASTERS TO
NASA SPACE SHIPS
16
B.Turhan,
T.Menzies, A.
Bener, J. Di
Stefano.
2009. On the
relative value
of cross-
company and
within-
company
data for
defect
prediction.
Empirical
Softw. Eng.
14(5) 2009,

17
Perspective on
Data Science
for Software
Engineering
Tim Menzies
Laurie Williams
Thomas
Zimmermann
2014 2015 2016
Lessons Learned
Our summary. And other related
books The MSR
community and others

Initial, naïve, view:
• Collect enough data …
• … and the truth will emerge
Reality:
• The more data we collected …
• … the more variance we observed
• Its like the microscope zoomed in
• to smash the slide
So now we routinely slice the data
• Find local lessons in local regions.
18
Lessons Learned:
Locality, Locality, Locality

CONUNDRUM
Q: How can I believe in architectural principles?
• That transcend individual projects
• Yet doubt the general value of any particular architecture?
• General, that is, to any other project that this current
project at this current time.
A1: Not general models
• But cost effective ways to find best local models
A2: Architectural incubators (see below)
20

“THE” ARCHITECTURE = THE
DECISIONS YOU CANNOT CHANGE
• The constants
• The base assumptions,
• made at the start of the
project.
• The decisions that you
• bless, or curse,
• every day of the project.
22

THE THING YOU’D BETTER
GET RIGHT
Since…
24

THE THING YOU’D BETTER
GET RIGHT
Since…
25
Or, if Doug Schmidt, just a few

AND IF YOU GET THE
ARCHITECTURE WRONG
Early mistakes are
most expensive
error to fix (Boehm
1981)
26

27
Architect-
ural
Evolution
of DoD
Combat
Systems.
Doug
Schmidt,
SEI blog,
Nov 15,
2013

28
Architect-
ural
Evolution
of DoD
Combat
Systems.
Doug
Schmidt,
SEI blog,
Nov 15,
2013

29
• Doug Schmidt:
• “The Department of Defense (DoD) must
move away from stove-piped solutions “
• “… towards a limited number of
technical reference frameworks … “
• “based on reusable hardware and
software components and services”
Architect-
ural
Evolution
of DoD
Combat
Systems.
Doug
Schmidt,
SEI blog,
Nov 15,
2013

30
• Tim Menzies
• “a limited number???”
• Only three numbers in the universe
• Zero
• One
• Many
• And if > 1, then expect many many more
• Need software architects: to tame the herd of architectures
Architect-
ural
Evolution
of DoD
Combat
Systems.
Doug
Schmidt,
SEI blog,
Nov 15,
2013

Some good news
1) Rework not so costly
2) Reworkers: cheaper, more plentiful

SEI TSP
DATA
33
171 projects
2005 to 2014.
Duration: (46 ,90) days (median, max)
Team size: (7,40) people (median,max)
William Curtis, Forest Shull,
Tim Menzies, Lucas Layton
FSE’15 (submitted)

SEI TSP
DATA
34
171 projects
2005 to 2014.
Duration: (46 ,90) days (median, max)
Team size: (7,40) people (median,max)

COMMENT
it is not crazy to explore extensive architectural evolution
• Continuously, during continuous deployment
Since:
• Cheaper to change code than we thought
• Programming costs are about to plummet (see next slide)
35

PROGRAMMER SALARIES:
ABOUT TO COLLAPSE
• Automatic programming
• Devanbu, ICSE NIER 2015:
• Most code repeats other code
• Large scale auto-coding?
• “The next billion”
• All of India + Africa on-line
• Many programmers,
competing for work
• Better distributed development
• Bird et al ICSE 2009,
Kocaganuli et al ICSE SEIP 2013:
• better ways to divide distributed teams
• “Micro-tasking” : LaToza and van der Hoek
ICSE NIER’15.
• Divide development into tiny (say) 5 minute
steps
• Next-gen distributed development
• Github DevOps, continuous integration
• Yang et al. ICSE NIER’13:
• Crowd sourced development 5 times cheaper
36

ARCHITECTURES =
PRODUCTIVITY
38

LARGE ASSEMBLES NEED
LARGE ARCHITECTURES
39
• Life expectancy:
• 50 years
• A backbone on which 1000s of sub-systems can be
• born,
• evolved
• replaced

LARGE ASSEMBLES NEED
LARGE ARCHITECTURES
40
• Life expectancy:
• 50 years
• A backbone on which 1000s of sub-systems can be
• born,
• evolved
• replaced
Architectural
incubator

ANOTHER LARGE SOFTWARE
ARCHITECTURE (DECADES OLD)
41
Any geeks in this audience?
Recognize this diagram?
Do you know the author?

42
Distributed packet-switching networks (On Distribute
Communications): Rand memorandum RM-3103-PR, August 1964

43
Distributed packet-switching networks (On Distribute
Communications): Rand memorandum RM-3103-PR, August 1964
Architectural
incubator

ANOTHER OLD ARCHITECTURE
(NOT QUITE AS WIDELY-USED)
The Blackboard Model of Problem Solving and the Evolution of
Blackboard Architectures, H. Penny Ni, AI Magazie, 7(2), 1986.
44

ANOTHER SOFTWARE ARCHITECTURE
(NOT QUITE AS WIDELY-USED)
The Blackboard Model of Problem Solving and the Evolution of
Blackboard Architectures, H. Penny Ni, AI Magazie, 7(2), 1986.
45
How AI handled software
engineering in the 1970s

SOME ARCHITECTURES ARE
MORE CHANGE TOLERANT
THAN OTHERS
46
More
brittle
Less
brittle

SOME ARCHITECTURES ARE
MORE CHANGE TOLERANT
THAN OTHERS
47
More
brittle
Less
brittle
Architectural
incubator

PHASING OUT INFLEXIBLE
ARCHITECTURES
48

AFTER LAMP, COMES “MEAN”
• M = MongoDB: a nonSQL DB(nested key-value pairs) ( death to SQL). No data traps
• E = Express.js : controller layer, directing application flow and marshaling data
• A = AngularJS : handles data presentation.
• N = Node.js: an extensive javascript library (look ma, no operating system)
• MEAN: one language up and down the stack (javascript).
• Faster integrated testing.
• Faster invention of new patterns
49

AFTER LAMP, COMES “MEAN”
• M = MongoDB: a nonSQL DB(nested key-value pairs) ( death to SQL). No data traps
• E = Express.js : controller layer, directing application flow and marshaling data
• A = AngularJS : handles data presentation.
• N = Node.js: an extensive javascript library (look ma, no operating system)
• MEAN: one language up and down the stack (javascript).
• Faster integrated testing.
• Faster invention of new patterns
50
Architectural
incubator

FUNCTIONAL PROGRAMMERS EAT
PATTERNS, FOR BREAKFAST
51
def visit(thing):
if isinstance(thing,list):
for x in thing:
for y in visit(x):
yield y
else:
yield thing
def all(thing):
return [x for x in visit(thing)]
map(lambda x: x**0.5, all(x)) # sqrt everything
reduce(lambda x,y: x*y, all(thing)) # mult everything
Architectural
incubator
BTW: MEAN? All Jscript,
all functional

OO PATTERNS : INCUBATOR
OR FREEZE BOX
52
Traditional view: Much like Schmidt’s limited number of technical reference frameworks.
First listed in 1992. Surprisingly few updates since.

Adaptive architecture
research
(at NCSU)

IF EVERYTHING MATTERS,
THEN CAREFULLY DESIGN
EVERYTHING
But what if only little bits matter…
Chris Theisen,
Ncstate
• 10,000,000 stack dumps
Ms Windows
• So little of the system
implicated in those errors
• Reaction of
MS engineers?
• Lets redesign the
dependencies
around those crash traces
54

THE KEY TO CHANGE: YAGNI
• You ain’t gonna need it (to be architected)
• Curiosuly: most software is never exercised enough to be buggy
• 80% bugs in 20% of the code
• Ostrand & Weyyeuker, AT&T, 2004
• Koru, IEEE Software 2009, Ericsson, IBM
• Tosun et al, IAAI’10: Turkish Software
• Hammill & Goseva IEEE TSE’09, NASA systems
55

WELCOME TO THE
AGE OF THE APP
Old school:
• MsOffice: a locked in world that users never leave
• All software from one vendor
• Large, slow to change, hard to compete
New school:
• The app
• All software from N vendors
• Small, very fast to change
56

FEATURE-ORIENTED, OR
VARIABILITY-ORIENTED, PROGRAMMING
Feature models = a
lightweight method for
defining a space of options
De facto standard for
modeling variability
57
Cross-Tree Constraints
Cross-Tree Constraints
Size ? 10 Features, 8 Rules

LARGE FEATURE MODELS =
ARCHITECTURES
58
Size: 6888 items
Features: 344,000 rules

NO SINGLE
“OPTIMUM” SOLUTION
59
Higher-level
Decision
Making
The Pareto Front
The Chosen Solution

LEARNING DESIGN CHANGE
(A “LESS” APPROACH)
Multi-objective optimization = navigating competing concerns
• Success criteria = choose features that achieve the user
preferences!
60
Suppose each feature had the following metrics:
1. Boolean USED_BEFORE?
2. Integer DEFECTS
3. Real COST
Show me the space of “best options” according to the objectives:
1. That satisfies most domain constraints (0 ≤ #violations ≤ 100%)
2. That offers most features
3. Using features with least known defects
4. Using features with least cost
5. That we have used most before
Architectural
incubator

STATE OF THE ART
61
Features
9
290
544
6888
SPLOTLinux(LVAT)
Pohl
‘11
Lopez-
Herrejo
n ‘11
Henard
‘12
Velazc
o ‘13
Sayyad &
Menzies’13b
Johanse
n ‘11
Benavide
s ‘05 Fixing model inconsistencis
27 min. for 94 features
Multi-obj configuration, IBEA
>100 hours for E-Shop, 80% HV
Single-obj, CSP
Up to 25 features
Exponential time
White ‘07, ‘08, 09a, 09b,
Shi ‘10, Guo ‘11
Test covering arrays, timed out on all
ops for Linux 6888 features (and
some ops on smaller FMs)
Multi-obj configuration, IBEA
30 configs in 30 minutes for Linux
6888 features
Objectives
Multi-objSingle-obj
Sayyad&
Menzies
’13a

Career advice
(for a plural world)

63
• Doug Schmidt: “… a limited number of technical reference frameworks …
• Tim Menzies:
• Architectures: zero or one or many
• But perhaps just a few architectural incubators?
• Supported by
• Cheaper programmers (more to them) writing more code, faster, cheaper
• Next gen inference, functional programming, feature-oriented code
Architect-
ural
Evolution
of DoD
Combat
Systems.
Doug
Schmidt,
SEI blog,
Nov 15,
2013

EXPLORE
“ARCHITECTURAL INCUBATORS”
64
Functional
programming
Feature
Models
??

67
CODERS
You want
to be
the architect
that knows how
to co-ordinate
teams that
commission new
architectureS
while rapidly
retiring
old ones
Cause the
softworld
will be
architectureS in
constant flux as
teams patch
and port old
functionality
to new
architectureS

68
CODERS
You want
to be
the architect
that knows how
to co-ordinate
teams that
commission new
architectureS
while rapidly
retiring
old ones
Cause the
softworld
will be
architectureS in
constant flux as
teams patch
and port old
functionality
to new
architectureS
plural
plural
plural

SOUNDS BITES
• Agile (is old)
69
End of my tale

SOUNDS BITES
• Agile (is old)
70
End of my tale

ARCHITECTURAL ASSESSMENT
FOR SOFTWARE SECURITY
Gary McGraw, Cigital:
72

“50% OF SECURITY FLAWS ARE
ARCHITECTURAL BUGS” -- G. MCGRAW
IMPLEMENTATION BUGS
• Buffer overflow
• String format
• One-stage attacks
• Race conditions
• TOCTOU (time of check to time of
use)
• Unsafe environment variables
• Unsafe system calls
• System()
• Untrusted input problems
ARCHITECTURAL BUGS
• Method over-riding problems
(subclass issues)
• Misuse of cryptography
• Compartmentalization problems in
design
• Privileged block protection failure
(DoPrivilege())
• Catastrophic security failure (fragility)
• Type safety confusion error
• Insecure auditing
• Broken or illogical access control (RBAC
over tiers)
• Signing too much code
73

SO YOU’D BETTER GET THE
ARCHITECTURE RIGHT
Architectural assessment methods (e.g. Mylopoulos, Soft
goals
74

ARCHITECTURAL
ASSESSMENT
Assessing options of criteria
• predictability (1), security (2), adaptability (3),
coordinability (4), cooperativity(5), availability (6), integrity
(7), modularity (8), or aggregability (9)
• Which is best? Ask the user for their value propositions
75

INDUSTRIAL RELEVANCE OF
SOFTWARE ARCHITECTURES
Source: AADL, CMU
76

Kits to Find the Bits that Fits

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (20)

Similar to Kits to Find the Bits that Fits

Similar to Kits to Find the Bits that Fits (20)

More from CS, NcState

More from CS, NcState (20)

Recently uploaded

Recently uploaded (20)

Kits to Find the Bits that Fits