An overview of design-based research, design experiments, educational design research. What it is, where it came from, and how to apply it to improve teaching and learning.

1. G E N E V I È V E B E N I N G E R
8 M a r c h 2 0 1 6
Design-Based Research
A method for achieving Impact in the real world

2. An Overview: What did you call it?
 A Rose by any other name…
 Design-Based Research
 Design Experiments (Brown)
 Educational Design Research (Reeves)
 Development Research
 Design Research

3. What is Design-Based Research?
 Wang and Hannafin (2005) define DBR as:
“A systematic but flexible methodology aimed to improve educational
practices through iterative analysis, design, development, and
implementation, based on collaboration among researchers and
practitioners in real-world settings and leading to contextually-sensitive
design principles and theories” (pp. 6-7).

4. The Foundations of DBR
 Anne Brown (1992):
 "Classroom life is synergistic: Aspects of it
that are often treated independently, such
as teacher training, curriculum selection,
testing, and so forth actually form part of
a systemic whole. Just as it is impossible
to change one aspect of the system without
creating perturbations in others, so too it is
difficult to study any one aspect
independently from the whole operating
system." (pp. 141-143)

5. The Foundations of DBR
 Anne Brown (1992):
 “Theoretical advances can emerge from both the laboratory
and classroom settings. They are just that, different settings
whose features must be included in the description of the
data they produce." (p. 154)

6. The Foundations of DBR
 Barab & Squire (2004):
 "design-based research and other methods should be viewed
as complementary and supportive—allowing researchers to
understand more completely their claims. For example,
laboratory-based researchers should ask themselves how their
laboratory-based claims would benefit from further testing in
naturalistic contexts and design-based researchers should be
asking how their claims
would benefit from more
rigorous testing within
laboratory-based contexts."
(Note, p. 4)

7. An Overview: How does it work?
 McKenney and Reeves’ (2012) generic model for
design research (GMDR) describes three phases to
the design-based research process:
1. Analysis and exploration;
2. Design and construction;
3. Evaluation and reflection.

8. An Overview: How does it work?
Generic model for design research in education (McKenney & Reeves, 2012)

9. Isn’t it “Just a Project”?
 Key principles that differentiate design-based research from
other forms of participatory research (and projects):
http://www.cea-ace.ca/education-canada/article/design-based-research
 requirement for a well-defined problem
with a research-informed design
solution;
 testing of theory in real-world contexts;
 contribution to theory and practice in
addition to local impact.

10. How does it compare to other approaches?
 Why not existing research methods?
(Collins, Joseph, & Bielaczyc, 2004, pp. 20-21).
 laboratory and training settings do not account for multiple
variables, multiple participants’ expertise, and “the messy
situations that characterize real life learning”;
 ethnographic research describes in detail what and why
relationships and events occur, but it does not try to change
practice;
 large-scale studies “do not provide the kind of detailed
picture needed to guide the refinement of a design”

11. How does it compare to other approaches?
 Barab & Squire (2004):
 "The goal is not to “sterilize”
naturalistic contexts from all
confounding variables so the
generated theory is more valid and reliable.
Instead, the challenge is to develop flexibly adaptive theories
that remain useful even when applied to new local
contexts." (p. 11)

12. A Question of Alignment
 “Design-based research suggests a
pragmatic philosophical
underpinning, one in which the value of
a theory lies in its ability to produce
changes in the world." (Barab & Squire, 2004, p. 6)
 It asks “What can be done?” and “How can
we effect change?” Seeks practical
solutions
 Ontology: Reality is complex - the effects
of ideas
 Epistemology: both objective and
subjective points of view, knower seeks to
uncover and apply the known
 Methodology: Mixed Methods,
combinations of qualitative and
quantitative, whatever leads to a practical
solution!
 “‘to understand what people mean and
intend by what they say and do and to
locate those understandings within the
historical, cultural, institutional, and
immediate situational contexts that shape
them’ (Moss et al., 2009, p. 501).
 It asks “How is this understood?” and
“What is the meaning behind this?”
 Ontology: Relativist – there are multiple
realities
 Epistemology: subjective - knower and
respondent co-create understandings
 Methodology: Qualitative
(ethnographies, interviews, case studies…)
PRAGMATIC? INTERPRETIVIST?

13. Example: COAT Project
 MarylandOnline’s Certificate for Online Adjunct Teaching (COAT)
course
 "The research project explored whether the training course had any
impact on the participants’ later teaching practice. The major outcome of
this research study is the identification of design principles that can
be used by other researchers and practitioners designing online
instructor training." (Shattuck & Anderson, 2013, p. 1)
 Using a Design-Based Research Study to Identify Principles for
Training Instructors to Teach Online
http://www.irrodl.org/index.php/irrodl/article/view/1626/2710

14. Example: COAT Project Cont’d
 "This study collected data using online, asynchronous, threaded discussion
groups as focus groups…”

15. Challenges
 “Difficulties arising from the complexity of
real-world situations and their resistance to
experimental control.
 Large amounts of data arising from a need
to combine ethnographic and quantitative
analysis.
 Comparing across designs."
(Collins, Joseph, & Bielaczyc, 2004, p. 16)

16. Challenges Cont’d…
 the role of the researcher
 the time it takes to enact multiple cycles
of a design
 knowing when to stop the cycles of
iteration (Hogue, 2013).

17. Challenges…?
 “There is a fundamental challenge in
developing a design science of
education in that the enacted design
is often quite different from what
the designers intended. Brown and
Campione (1996) referred to this
problem in terms of “lethal mutations,”
where the goals and principles
underlying the design are undermined
by the way the design is enacted.” (Collins,
Joseph, & Bielaczyc, 2004, p. 17)
Q: Is this a “challenge” or should
we embrace it as reality?

18. Meeting the Challenge(s)!
 Design-Based Research Collective…
 “to refine a definition of design experimentation that is broad
enough to encompass a diversity of research perspectives, yet
rigorous enough to sustain theoretical and methodological
attacks on its robustness and cumulativity”

19. Closing thoughts…
 Combination of qualitative and quantitative “Mixed
Methods” is appropriate to the complexity of education,
learning, and context
 Exciting approach – starting to mature but needs
refinement!
 Addresses a strong need to bridge the gap between research
and real-world application
 Research must not only make sense, but must be valuable
to educators, learners, and future research!

20. “Where’s the BEEF ?”

21. References
Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in
classroom settings. The Journal of Learning Sciences, 2(2), 141–178.
Barab, S., & Squire, K. (2004). Design-based research: Putting a stake in the ground. The journal of the learning sciences,
13(1), 1-14.
Collins, A., Joseph, D., & Bielaczyc, K. (2004). Design research: Theoretical and methodological issues. Journal of the
Learning Sciences, 13(1), 15-42. doi.10.1207/s15327809jls1301_2
Hogue, R. J. (2013). Epistemological Foundations of Educational Design Research. In E-Learn: World Conference on E-
Learning in Corporate, Government, Healthcare, and Higher Education (Vol. 2013, No. 1, pp. 1915-1922).
McKenney, S. E., & Reeves, T. C. (2012). Conducting educational design research. New York, NY: Routledge
Shattuck, J., & Anderson, T. (2013). Using a design-based research study to identify principles for training instructors to
teach online. The International Review Of Research In Open And Distributed Learning, 14(5). Retrieved from
http://www.irrodl.org/index.php/irrodl/article/view/1626/2710