Morning lecture given to the Social Innovation Generation graduate program from the University of Waterloo (Dr. Dan McCarthy, instructor). Slides focus on some of the key tenets of developmental evaluation and how it can apply to complex systems.

1. Developmental Evaluation
Cameron D. Norman PhD CE
Principal CENSE Research + Design
Adjunct Professor, Dalla Lana School of Public Health, University of Toronto
@cdnorman
August 28, 2013

2. Outline
• Thinking about the problem(s)
• What is Developmental Evaluation?
• Methods
• Case Examples
• Discussion & dialogue

3. Same Evaluation Tools / "
Different Thinking Tools

4. Complexity

5. Example: Organizing a children’s
party

6. Evaluation & Measurement:
Problems & Opportunities

7. What is an effective coffee date?
Is that the right question?

8. What is an effective tree?
What is the best practice for being a tree?
What is a fully optimized tree?

9. Do you know what is inside before you crack it open?

11. Confusing moving things with static things

12. The Importance of Theory

13. What is Developmental Evaluation?

14. Developmental Evaluation
• First proposed by Michael Quinn Patton (1994)
• Combines Utilization-Focused Evaluation with concepts from Complexity
Science
• Recognizes that social innovation and resiliency require a different form of
feedback to enable programs to adapt and thrive in changing conditions
• An approach to evaluation, not a specific method.
• Strategic learning / real-time evaluation

15. Developmental Evaluation
• Assists social innovators in learning strategically about how to intentionally
develop their program while acknowledging to complexity
• Supports adaptation to changing conditions
• Can identify and nurture promising interventions
• Documents innovators’ actions
• Identifies emergent processes and outcomes; enhances resiliency
• Determines if an innovation is ready for formative evaluation

16. Complicated Systems
Complex Systems
Single causality
Multiple causality
Designed & intended
Adaptive & emergent
High degree of certainty
High degree of uncertainty
Assumed predictability
Anticipation of events
Focus on ‘boxes’
Focus on ‘arrows’
Structures determine relationships
Structures & relationships interact
Linear
Non-linear
Noise, tension & flux undesirable
Noise, tension & flux embraced
Adaptation to static environment
Interaction with static environment
Reaction
Engagement
Adapted from http://www.healthandeverything.org

17. Program Improvements
Program Developments
1. Add a new topic to a training
curriculum
1. Change the scope, sequence
and delivery of curriculum
2. Provide staff training to
enhance skills
2. Change job descriptions,
reconceptualize fundamentals of the job
3. Expand recruitment effort to a
target area
3. Question and re-evaluate
recruitment strategy
4. Expand staff to serve increasing
numbers using the same model
4. Expand staff, change the model to
suit new scale and scope of practice
5. Fine-tune programming based
on participant feedback
5. Change programming to fit
ever-changing realities
6. Add more content to provide
additional information
6. Change the way the audience
interacts with content altogether

18. Design Thinking

19. Developmental Design

20. Methods & Methodology

21. Mindfulness

22. Ethnography

23. Network Analysis

25. Timelines

26. Visual Thinking

27. Gigamapping
http://www.systemsorienteddesign.net/

28. Idea Generation

29. Knowledge Mobilization

30. Strategic Learning

31. Discussion

32. Cameron D. Norman PhD
@cdnorman
censemaking.com