1.
From Automating Software Engineering
to Empowering Developers
Margaret-Anne (Peggy) Storey
Automated Software Engineering Conference, Oct 12th 2022
mastorey@gmail.com
@margaretstorey
Special Thanks
Arty Starr
Alessandra Milanit

2.
Automation in Software Engineering - Through the Developersʼ Eyes
History of automation in software engineering
Future innovations and research
How to evaluate the impact of automation on developers

3.
Automation in Software Engineering - Through the Developersʼ Eyes
History of automation in software engineering
Future innovations and research
How to evaluate the impact of automation on developers

4.
Humans and Machines
4
@margaretstorey

5.
“while Humanity will be amusing itself, or enjoying cultivated leisure
… or making beautiful things, or reading beautiful things, or simply
contemplating the world with admiration and delight,
machinery will be doing all the necessary and unpleasant work.”
Oscar Wilde, The Soul of Man Under Socialism (1891)
@margaretstorey

6.
Automation
Automation term coined, Henry Ford, Detroit
1946
Reduce human intervention in processes, by
predetermining decision criteria, subprocess
relationships and related actions, and
embodying those predeterminations in
machines.
Groover, Mikell (2014). Fundamentals of Modern
Manufacturing: Materials, Processes, and Systems.

7.
Automation is about feedback, and faster feedback
7
@margaretstorey

8.
Advantages of automation (in general)
Faster production
Cheaper/safer
Higher quality
Predictable processes/products
Less tedious repetitive work
Simpler tasks
Faster feedback
More time for creative work
8
@margaretstorey

9.
Ironies of automation
May require different competencies and skills
Any errors may be multiplied
May not anticipate all contingencies
Retrofitting automation is complex
Semi-automated tasks rely on people
People are poor monitors
9
@margaretstorey

10.
Bystander effect
10 Ironies of automation

11.
The more efficient the automated system,
the more crucial the human contribution of the operators
The paradox of automation
11
@margaretstorey

12.
Long long ago,
in the land before
there was automated
software engineering…

13.
Taxonomy to characterize automation in
Software Engineering Research, Sharma & Sodhi,
2021 https://arxiv.org/abs/1803.09536

14.
Research trends up to 2016… (Sharma & Sodhi, Arxiv, 2021)
14

15.
Research trends up to 2016… (Sharma & Sodhi, Arxiv, 2021)
15

16.
ASE Topics (papers may have multiple topics)
https://bit.ly/ase2022papers
16

17.
2020
2010
2000
1990
1980
1970
1968
Refactoring
(Johnson)
Disruptive automations in software engineering industry @margaretstorey
Code
Authoring
AutoML

18.
2020
2010
2000
1990
1980
1970
1968
Test
Automation
(IBM)
Refactoring
(Johnson)
Disruptive automations in software engineering industry @margaretstorey
Code
Authoring
Testing,
Verification
& Analysis
AutoML

19.
2020
2010
2000
1990
1980
1970
1968
Test
Automation
(IBM)
Refactoring
(Johnson)
Disruptive automations in software engineering industry @margaretstorey
Version control
(RCS, CVS)
Code
Authoring
Testing &
Analysis
Versioning
& Review
AutoML

20.
2020
2010
2000
1990
1980
1970
1968
Test
Automation
(IBM)
Continuous
Integration
(Booch)
Refactoring
(Johnson)
Disruptive automations in software engineering industry @margaretstorey
Version control
(RCS, CVS)
Code
Authoring
Testing &
Analysis
Versioning
& Review
Build/CI
AutoML

21.
2020
2010
2000
1990
1980
1970
1968
Test
Automation
(IBM)
Continuous
Integration
(Booch)
Refactoring
(Johnson)
Disruptive automations in software engineering industry @margaretstorey
Version control
(RCS, CVS)
Code
Review
Inspiring
Practices
Code
Authoring
Testing &
Analysis
Versioning
& Review
Build/CI
AutoML
Faganʼs
Code Inspection
XP
Open source

22.
Advantages of automation in software engineering
Increased productivity
High performing teams
Maintainability
Improved quality
Predictability
Flexibility
Adaptability
Fewer errors
Faster feedback
Fail fast
Experimentation and learning
Happier developers
22
@margaretstorey

23.
Disadvantages of automation on the developers?
Misuse/abuse/overuse of automation?
Increased skill/competencies needed?
Added complexity?
Automation as technical debt?
Anything else?
23
@margaretstorey

24.
Continuous Integration
Advantages
Improves productivity speed
and quality [Vasilescu et al.]
Traceability
[Stahl et al.]
Happier developers
[Williams]
Disadvantages
Increased cognitive load
maintaining build code [Pinto et al.]
Difficulties attracting
developers [Gupta et al.]
Failing builds lower
developer morale [Souza et al.]
24
@margaretstorey

25.
Moving faster is
not always
better
unexpected
side effects

26.
Automation in Software Engineering - Through the Developersʼ Eyes
History of automation in software engineering
Future innovations and research
How to evaluate the impact of automation on developers

27.
Can we fully automate software engineering?
@margaretstorey

28.
What is software?
What it is
What it does
What people do with it
What it can be altered to do
28
@margaretstorey

29.
What is software?
“Shared theories about problem and solution domains
that are in the minds of developers” [Peter Naur, 1985]
“Every personʼs mental model is incomplete and out of date
but we need them all!” [Jessica Kerr, Explore DDD Conference, 2018]
“People are part of the code” [Jean Yang, QCon Panel, 2021]
29
@margaretstorey

30.
Software is a dynamic, living, learning system comprised of
people + code + tools + environment
and the exponential relationships between all of these parts
@margaretstorey

31.
Since we canʼt fully automate software development…
what can we do and what should we do next?
@margaretstorey

32.
Three types of innovations…
32

33.
Three types of innovations…
Incremental
innovations
Breakthrough
innovations
Experiential
innovations
33
@margaretstorey

34.
Experiential innovation and software engineering automation
Minimize Pains
Maximize Gains
Not just automate, but augment what developers do
to make them more productive and to improve their experience
34
@margaretstorey

35.
The SPACE of developer productivity:
Thereʼs more to it than you think
S – Satisfaction & Well Being
P – Performance
A – Activity
C – Collaboration & Communication
E – Efficiency & Flow
Nicole Forsgren, Margaret-Anne Storey, Chandra Maddila, Thomas Zimmermann, Brian Houck, and Jenna Butler.
2021. The SPACE of Developer Productivity: There's more to it than you think. ACM Queue. 2021 @margaretstorey

36.
Cognitive
Complexity
Three core dimensions of developer experience
36
Fast
Feedback
Flow and
Fulfillment
@margaretstorey
Michaela Greiler, Margaret-Anne Storey, Abi Noda: An Actionable Framework for Understanding
and Improving Developer Experience, TSE 2022.

37.
What is the net effect of automation on cognitive load?
How does automation impact developer knowledge and skill?
Will the automation create technical debt?
Can automation support learning for reuse?
Cognitive complexity ~ research opportunities
37
@margaretstorey

38.
Fast feedback ~ research opportunities
Do automation tools provide actionable and timely information?
How can automation improve feedback between developers?
Is faster always better?
38
@margaretstorey

39.
Developer Flow and Fulfillment
39
Flow is a state of mind, a holistic sensation, that people feel when they
act with total involvement [Mihaly Csikszentmihalyi]
Clear goals
Total sense of involvement
Loss of self-consciousness
Feeling of control and being in control
Altered sense of time
Above average skills and challenges

40.
Developer Flow and Fulfillment
40
Flow is a state of mind, a holistic sensation, that people feel when they
act with total involvement [Mihaly Csikszentmihalyi]
Clear goals
Total sense of involvement
Loss of self-consciousness
Feeling of control and being in control
Altered sense of time
Above average skills and challenges

41.
Developer Flow and Fulfillment ~ research opportunities
Can automation lead to boredom and reduce the joy in programming?
How to design automation that enhances flow experiences?
How automation can enhance flow across a team?
How can AI be an effective “member” on a software team?
41

42.
https://www.forbes.com/sites/oracle/2013/11/04/7-w
ays-americas-cup-champions-sail-like-successful-it-
teams/#7ccd8ee2b515

43.
“At the end of the day, you have to
make a judgment decision yourself,
you know. All teams rely on all sorts
of algorithms and systems, but at the
end of the day, a human has to
decide when to push a button or
when to make a move. If youʼre
asking for why the mistakes happen,
or for someone to point the finger at,
then you should point it firmly at me.”
Jimmy Spithill, Skipper of Oracle
Team USA, US Americaʼs Cup 2017

44.
Automation in Software Engineering - Through the Developersʼ Eyes
History of automation in software engineering
Future innovations and research
How to evaluate the impact of automation on developers

45.
“No matter how many technologies you implement,
the weakest link will always be the human factor”
https://www.contino.io/files/Introduction-to-DevSecOps-Best-Practices-for-Adoption.pdf

46.
Research methods - very few directly involve “humans”
https://bit.ly/ase2022papers
46
Red research methods
involve “people”
Data
Lab
Field
Survey
Theory

47.
SPACE
Defining productivity
metrics:
47

48.
https://getdx.com/

49.
Experience sampling method for flow
Flow Insight is a tool for helping developers get in and stay in flow
Flow Insight evaluates and presents flow metrics
that are associated with the code
Measuring flow
49 https://flowinsight.com/
(Arty Starr)

50.
The dimensions of developer experience may be in tension
Remove friction,
empower flow
Reduce cognitive
complexity
Improve velocity of
feedback
50

51.
“while Humanity will be amusing itself, or enjoying cultivated leisure
… or making beautiful things, or reading beautiful things, or simply
contemplating the world with admiration and delight,
machinery will be doing all the necessary and unpleasant work.”
Oscar Wilde, The Soul of Man Under Socialism (1891)
@margaretstorey

52.
Takeaways
Study developers for inspiration
Consider how your solutions can empower the
developer experience
Use research methods that study the impact on developers
mastorey@gmail.com
@margaretstorey
https://bit.ly/ase2022papers