This talk explores how Unboxed is using “Theories of Change” to guide the design of future service innovations towards desired outcomes and how this helps us evaluate the impact of our design work. Theories of change have been used since the 1990s to evaluate development programmes in many different contexts, such as philanthropy and international development. They can be used to guide the design process as well as to evaluate existing services. One of the main benefits of this approach is that it compels the design team to explicitly articulate assumptions about how a proposed concept might achieve an intended outcome and how these outcomes might lead to long-term impact. These assumptions can then be validated through prototyping and research activities. The talk will cover recent examples of applying this approach in our work with Guy’s and St Thomas’ NHS trust and with the Hackney Council housing repairs services in London. In both cases, the evidence of impact provided by this approach helped us gain buy-in from senior stakeholders and secure funding for further development of the digital services we were working on. http://unboxed.co/ http://prototypingforimpact.com/

1. Innovating for Impact
with Theories of Change
Boris Divjak
@borisdivjak
unboxed.co
2019

2. @BorisDivjak U N B O X E D
Unboxed is a … digital innovation consultancy.
service design
software development
agile collaboration

3. @BorisDivjak U N B O X E D
Our purpose is to ...

4. @BorisDivjak U N B O X E D
Our purpose is to ...

5. @BorisDivjak U N B O X E D
Our purpose is to ...

6. @BorisDivjak U N B O X E D
Our purpose is to improve lives
with digital technology

7. @BorisDivjak U N B O X E D
H E A L T H C A R E
projects
H O U S I N G
projects
E N E R G Y
projects
“Improve lives?”

8. @BorisDivjak U N B O X E D
Happier and
healthier patients?
Improved quality
of available social
housing?
Reduced
air pollution?
“Improve lives?”
H E A L T H C A R E
projects
H O U S I N G
projects
E N E R G Y
projects

9. @BorisDivjak U N B O X E D

10. @BorisDivjak U N B O X E D
Impact of innovation is difficult to demonstrate:
There are long delays between cause and effect.
There are many contributing factors.
There are often unintended consequences.

11. @BorisDivjak U N B O X E D
Zoom out with
T H E O R Y O F C H A N G E
to construct a big picture of impact
@BorisDivjak U N B O X E D

12. @BorisDivjak U N B O X E D
Desired impact
(might be affected by
other factors in addition
to our own innovations)
Long-term
outcomes

13. @BorisDivjak U N B O X E D
Service innovations
that we are proposing
to implement
Desired impact
(might be affected by
other factors in addition
to our own innovations)
New service
concepts
Long-term
outcomes

14. @BorisDivjak U N B O X E D
Service innovations
that we are proposing
to implement
Direct result of
implementing new
service concepts
Desired impact
(might be affected by
other factors in addition
to our own innovations)
New service
concepts
Intermediate
outcomes
Long-term
outcomes

15. @BorisDivjak U N B O X E D
Service innovations
that we are proposing
to implement
Direct result of
implementing new
service concepts
Desired impact
(might be affected by
other factors in addition
to our own innovations)
New service
concepts
Intermediate
outcomes
Long-term
outcomes
OUTCOMES CHAIN

16. @BorisDivjak U N B O X E D
Reduced
plastic waste in
environment
5p charge for
plastic bags
Reduced
consumption of
plastic bags
New service
concepts
Intermediate
outcomes
Long-term
outcomes

17. @BorisDivjak U N B O X E D
Reduced
plastic waste in
environment
5p charge for
plastic bags
Reduced
consumption of
plastic bags
DELA
Y
DELA
Y
Individual links are easier to measure than the
end-to-end impact
PROXY
MEASURE

18. @BorisDivjak U N B O X E D
Reduced
plastic waste in
environment
5p charge for
plastic bags
Reduced
consumption of
plastic bags
Explore other ideas that might lead
to the same impact
New materials? Recycling?

19. @BorisDivjak U N B O X E D
Reduced
plastic waste in
environment
5p charge for
plastic bags
Reduced
consumption of
plastic bags
Explore potential unintended consequences
of proposed service concepts
Plastic industry
loss?
Reusable bags
even worse?

20. @BorisDivjak U N B O X E D
In practice, this is can be a messy process

21. @BorisDivjak U N B O X E D
ASSUMPTIONS THAT REQUIRE EVIDENCE

22. @BorisDivjak U N B O X E D
‘Theory of change is an
O N G O I N G P R O C E S S O F R E F L E C T I O N
to explore change and how it happens,
and what that means for the part we play
in a particular context.’
––– James, C. (2011) “Theory of Change Review. A report commissioned by Comic Relief”

23. @BorisDivjak U N B O X E D
( R E ) D E F I N E
your Theory of Change
E V A L U A T E
outcomes and assumptions
P R O T O T Y P E
new service concepts
Zooming out
Zooming in
Innovation
for Impact
learning cycle

24. @BorisDivjak U N B O X E D
Zoom in with
R A P I D E X P E R I M E N T S
to evaluate impact of new concepts
@BorisDivjak U N B O X E D

25. @BorisDivjak U N B O X E D
Rapid experiments:
Design and test small interventions in short iterations;
Aim to learn, not to measure;
Collect just enough data to make informed decisions.

26. @BorisDivjak U N B O X E D
Example project:
GP referrals (to hospitals) are often missing key
information or test results, the absence of which
causes delays to patients receiving a diagnosis and
treatment and significant disruption to clinic schedules.

27. @BorisDivjak U N B O X E D
Better patient
information
Allocated to
correct clinic
Tests before
appointment
Increased
capacity within
existing
services
Reduced cost
per patient
New service
concepts
Faster
confirmed
diagnosis &
treatment
Fewer
unnecessary
follow-up
appointments
Happier,
healthier
patients
Intermediate
outcomes
Long-term
outcomes

28. @BorisDivjak U N B O X E D
Better patient
information
Allocated to
correct clinic
Tests before
appointment
Increased
capacity within
existing
services
Reduced cost
per patient
New service
concepts
Faster
confirmed
diagnosis &
treatment
Fewer
unnecessary
follow-up
appointments
Happier,
healthier
patients
Intermediate
outcomes
Long-term
outcomes

29. @BorisDivjak U N B O X E D
If we provide ...
online questionnaire
for patients before
their first
appointment
… then ...
patients will be
allocated to the
correct clinic
New service
concept
Intermediate
outcome

30. @BorisDivjak U N B O X E D
If we provide ...
online questionnaire
for patients before
their first
appointment
… then ...
patients will be
allocated to the
correct clinic
New service
concept
Intermediate
outcome
P R O T O T Y P E E V A L U A T E

31. @BorisDivjak U N B O X E D
P R O T O T Y P E E V A L U A T E
We started with a simple
paper questionnaire …
… and evaluated the experiment by
manually assessing each response.

32. @BorisDivjak U N B O X E D
P R O T O T Y P E E V A L U A T E
We gradually built more refined
prototypes ...
… and conducted experiments with
a variety of methods, but always with
the aim of validating our ToC.
I liked it. It
made me feel
prepared.

33. @BorisDivjak U N B O X E D
Online
questionnaire
(For patients)
Allocated to
correct clinic
Tests before
appointment
Increased capacity
within existing
services
Reduced cost per
patient
Faster confirmed
diagnosis &
treatment
Fewer
unnecessary
follow-up
appointments
Happier, healthier
patients
?
?
Refine the Theory of Change based on new learning
EXP 1
EXP 2
EXP 3
EXP 4, 5

34. @BorisDivjak U N B O X E D
Learn with the
P R O T O T Y P I N G F O R I M P A C T
toolkit for rapid experiments
with theories of change
@BorisDivjak U N B O X E D

35. @BorisDivjak U N B O X E D

36. @BorisDivjak U N B O X E D

37. @BorisDivjak U N B O X E D
P R O T O T Y P I N G
M E T H O D S
V A L I D A T I O N
M E T H O D S

38. @BorisDivjak U N B O X E D
Try it out and talk to me about it:
www.prototypingforimpact.com
Change it and make it your own under Creative Commons.

39. Thank You
Boris Divjak
boris.divjak@unboxed.co