ABCs of Research

©2016 MarylandOnline, Inc.
Designing quality distance learning
research studies
Distance Teaching & Learning Conference ♦ August 9 – 11, 2016

Bethany Simunich
Director of Online Pedagogy and
Research
Kent State University
Presenters
Kay Shattuck
Director of Research
Quality Matters

• The Foundations of Distance Learning Research
• The Unique Considerations of Distance Learning
Research
• Break!
• Research Framework: Steps and Templates
• Reporting Out and Final Thoughts
AGENDA

Workshop Learning Outcomes
By the end of this workshop, you will be able to:
• Discuss why and how to investigate current research base.
• Describe unique DL research challenges and ways to
overcome through a targeted research plan.
• Develop a specific DL research question; identify relevant
stakeholders and inputs; plus needed resources.
• Outline a quality DL research project, including overall
methodology, that meets stakeholder needs and
contributes to our body of knowledge.

“[R]esearch will fail unless it is based on
a good foundation of knowledge of
previous research.”
Michael G. Moore
The Foundations of Distance
Learning Research

“Distance education is teaching and planned learning
that normally occurs in a different place from
teaching, requiring special techniques for course
design, special instructional techniques, special
methods of communication by electronic and other
technology, as well as special organizational and
administrative arrangements”
(Moore & Kearsley, 2012, p. 2)
Distance Education

“The future research agenda will force us to devise an
effective way to combine the old and the new research
methodologies…will involve a continuing conversation
among many constituencies, where each one of them can
make contributions to the research agenda”
(Dziuban & Picciano, 2016, pp. 187, 189)
 It’s about learning, not the technology
 It’s about replication and research, including but also
broader than teaching and learning
 Systems perspective provides foundation for nuance
and complexity
The Changing Focus of Online Research

Distance Education:
A Systems View of
Online Learning
By Michael G. Moore
& Greg Kearsley

Validated framework of research topics
Zawacki-Richter (2009)
Macro-level: Distance education systems and theories
• Access, equity, and ethics
• Globalization of education and cross-cultural aspects
• Distance teaching systems and institutions
• Theories and models
• Research methods in distance education and knowledge
transfer

Meso-level: Management, Organization and Technology
• Management and organization
• Costs and benefits
• Educational technology
• Innovation and change
• Professional development and faculty support
• Learning support services
• Quality assurance

Micro-level: Teaching and Learning in Distance
Education (over 50% of all published research)
• Instructional or learning design
• Interaction and communication in learning
communities
• Learner characteristics

Decade of QM-Focused Research: Impactful variables revealed
• Increased learner satisfaction (Aman, 2009; Finley, 2005;
Myers, 2015)
• Increased motivation and self-efficacy (Simunich et al,
2015; Myers, 2015)
• Design and teaching interact to produce outcomes
(You et al, 2014; Hall, 2010; Harkness et al, 2011; Giger et al, 2014, Swan,
2012)
• Improved learning (Runyon, 2006; Swan, 2011; Harkness, 2015;
Parscal, 2014)

• Study on impact across learning formats:
– 94% made changes in their course (objectives,
alignment, interaction) (Kearns & Mancilla, 2016)
• Course review impact:
– 95% of Course Representatives and 81% of Peer
Reviewers made or planned to make changes as a
result of the review experience
– 80% (CRs) and 65%(PRs) planned to also make
changes in their F2F course
Impact on faculty

Lessons Learned
• Isolation of input factors and implementation
influence is crucial
• Get the full picture of the “before”
• Understand the field, review the literature
• Ask the right questions of the right people

Ways to Stay Informed
Journals
• AJDE
• IRRODL
• DE
• OLC
QM db
• Search by
standard
• Search by
keyword
Other
databases
• ERIC
• JSTOR
• Academic
Search
Complete
• Google
Scholar

The Unique Considerations
of DL Research
“[D]istance education can no longer be
considered a one-dimensional
phenomenon…online distance education is a
comprehensive, many-sided process and a
multifunctional system”
Otto Peters

Challenges (Saba, 2013)
Confusing
terminology
Lack of historical
perspective
The postmodern
turn
Absence of
construct validity

Aspects that might be more difficult
Operationalization Sample Size
Isolation of
Factors
The problem of
“Before & After”

What do you want to know? Why?
Student
Satisfaction
Completion/
Retention
Instructor
Satisfaction
Student
Learning

Who wants to know? Why?
Administrators Faculty Accreditors
Students/
Parents
The Field/
Researchers

What are some barriers or obstacles?
Hard-to-get
data
Faculty
Buy-in
Level of
Implementation
and support
Control
variables

Online Quality Pie
Course Design
Course Delivery
Course Content
Institutional
Infrastructure
LMS
Faculty Readiness
Student Readiness
Each course experience is a unique, interactive combination of these factors

Example: Course Design
QM-informed design factors:
• Impact of QM Rubric when developing or redesigning course
before study
• Impact of QM informal review before study
• Pre-study assessment of course meeting QM Standards
• Details of any design changes made to meet QM Standards
Non-QM design factors:
• Impact of underlying pedagogical framework
• Impact if institutional prescribed inputs
Course Design
Course Delivery
Course Content
Institutional Infrastructure
LMS
Faculty Readiness
Student Readiness

Example: Course Delivery
• Teaching presence
• Response time
• Quality of feedback
• “Just in Time” online
teaching
Course Design
Course Delivery
Course Content
LMS
Faculty Readiness
Student Readiness

Example: Student Readiness
Course Design
Course Delivery
Course Content
LMS
Faculty Readiness
Student Readiness
• Meeting course prerequisites
• Prior online learning experience
• Technology experience
• Proactive/self-directed
• GPA
• Readiness measures

Research Framework:
Steps and Templates

Initial Steps
Conceptualize RQ
& Stakeholders
• What is it you
want to know?
Does this
question meet
stakeholder
needs? Is it
doable? Does
it contribute to
the field?
Conduct a
literature review
• Determine
guiding
assumptions
and
theoretical
foundation
Variables
• What are your
dependent
and
independent
variables?
How will they
be measured?
What will you
control for?

Next Steps
Determine
methodology
• How will you
answer your
RQ? What
sample size
will you
need? What
type of study
will you do?
Finalize RQ
• Can you get
the data you
need? Based
on this, and
your
variables,
does your RQ
need
editing?
Conduct study
& do analysis
• Carry out
your study,
analyze your
data, and
write the
final report.
Consider
how you will
use the
results.

QM Online Research Toolkit

Diving In: Using the Framework
What is your general research question?
• Why are you interested in this? Who has a stake in the answer?
• What has already been published about this topic/question?
• What theoretical framework will you use?
Has your RQ evolved based on your lit review?
• What will be your DVs and IVs?
• What is the unit of analysis you will be studying?
• How will you control for, include, or acknowledge confounding
variables? (learner, instructor, content, teaching, institutional,
LMS, etc.)
• What sample size do you need? How will you get it?
Has your RQ evolved based on your methodology?
• How will you collect and analyze your data?
• How will you use the results?

EXAMPLE: shortened online courses
• Do students learn as much in shortened/accelerated online courses as they
do in their full-term counterparts?
Why are you interested in this?
• Graduate programs are shortened courses, as are summer offerings
Who has a stake in the answer?
• Faculty, students, administrators, accreditors, professional organizations
What has already been published about this
topic/question?
• Research on accelerated face-to-face courses, student motivation, rigor,
teaching strategies in shortened courses
What theoretical framework will you use?
• Equivalency theory, motivation theory (MSLQ)

• Additional RQ: Do levels of motivation differ?
What will be your DVs and IVs?
• DV: student learning, student motivation; IV: term length
What is the unit of analysis you will be studying?
• Online students
How will you control for, include, or acknowledge confounding
variables?
• Discipline: use courses from more than one area
• Teaching: same instructor for both terms; both instructors have experience teaching
online and have taken online teaching professional development
• Course: both courses met QM standards in an informal review; courses contained
exact same materials and same amount of work
• Questions included for: LMS experience, online course experience, shortened
course experience, GPA
What sample size do you need? How will you get it?
• 162 total: 98 fall, 64 summer

• No
How will you collect and analyze your data?
• Create survey for courses that includes demographic
questions, MSLQ questions; download final grades,
project grade, and knowledge test grade
How will you use the results?
• Share with NCCSS, publish in Summer Academe, use
with faculty developing shortened courses to discuss
rigor, motivation factors, use when discussing
learning in shortened courses with faculty and
administrators

Reporting Out and
Final Thoughts

Reporting out
• Why are you interested in this? Who has a stake in the answer?
• What has already been published about this topic/question?
• What theoretical framework will you use?
• What will be your DVs and IVs?
• What is the unit of analysis you will be studying?
• How will you control for, include, or acknowledge confounding
variables? (learner, instructor, content, teaching, institutional,
LMS, etc.)
• What sample size do you need? How will you get it?
• How will you collect and analyze your data?

• What do you want to know and why?
• Who else cares about this?
• What has been done before on this
topic/question?
• It it doable? Who needs to be included? Who can
help?
Big Takeaways

Find your motivation

ABCs of Research

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à ABCs of Research

Similaire à ABCs of Research (20)

Dernier

Dernier (20)

ABCs of Research

Notes de l'éditeur