Expert Perceptions of the Feasibility of MOOCs

Expert Perceptions of the
Feasibility of MOOCs
Reporting on a Modified E-Delphi Study
Shalin Hai-Jew
Paper Forum
2014 Big XII Teaching and Learning Conference
Oklahoma State University
Stillwater, Oklahoma
Aug. 4 – 5, 2014

Presentation description
 For any innovation to take, experts and other decision-makers in a field
have to buy into the endeavor. This is likely so for the adoption of MOOCs.
To determine the feasibility of MOOC adoption, a modified e-Delphi study
was conducted recently to capture the thinking of an expert panel of
individuals involved in e-learning. This presentation describes the processes
of setting up the modified e-Delphi study, beginning with the extensive
literature review undertaken for the development of the survey instrument
(and the use of NVivo 10 to code the sources). The major findings from this
qualitative and mixed methods research suggests critical design, policy,
and other structural issues that need to be addressed individually and
collectively in order to roll out successful MOOCs.
 Caveats: This is a quick view, with various types of data reduction from the
content analysis of the responses in the modified e-Delphi study.
Expert Perceptions of the Feasibility of MOOCs
2

Welcome!
 What are your interests re: this presentation?
 Any experiences with modified e-Delphi studies?
 What are your experiences with MOOCs? (Designing? Creating?
Teaching? Administering? Supporting? Taking?)
3

A modified e-Delphi study
 A classic Delphi study (a structured group procedure):
 Delphi study as a research method originating in the 1950s by the RAND
Corporation (“Research ANd Development”); first described in research in
Dalkey & Helmer, 1965
 Seating an expert panel from the same field or widely disparate ones
 Working with experts in a field (or in unrelated fields) on a difficult problem
 Eliciting insights individually (and anonymously) in an unstructured way in an
early exploration phase and then moving to structured elicitation of the
experts
4

A modified e-Delphi study (cont.)
 A classic Delphi study (a structured group procedure) (cont.):
 Sharing the expert insights and tacit knowledge; examining whether there is
a move toward consensus or convergence on a difficult or complex
problem
 Evolving the process based on new information
 Forecasting (~ Delphic oracle) and enhancing long-range human policy-
making and decision-making
5

A modified e-Delphi study (cont.)
 A modified e-Delphi study:
 May be one-stage or multi-stage; may manifest differently over time
 Brings in various sampling technologies including through information and
communication technology (ICT)
 Enables geographically distributed expert panels
 Uses a range of survey techniques and technologies (with a focus on
experts and on forecasting and on policymaking)
 Does not strive for consensus per se (researchers expect a wide range of
responses, including dissensus)
6

An overview of the research workflow
7

The extant literature on MOOCs
(in 2013)
 Impetuses for MOOCs
 Types of MOOCs
 Teaching and learning platforms with digital content delivery, comment
ecosystems, mass assessments, back-end tracking, and augmentation by
social media
 Teaching strategies
 Application of automation
 Research on MOOC learning experiences and learner “types” (such as
completing, auditing, disengaging, and sampling)
 Funding
8

The extant literature on MOOCs (cont.)
 Challenges with learner participation, retention, and completion
 Big data research potentials
 Types of courses and course contents
 Credentialing and badging
9

The survey instrument
For the modified e-Delphi study
10

The open-ended survey instrument
 About You
 Part 1: Impressions of MOOCs (open-ended questions with text-based
responses for all sections with a few exceptions)
 Experiences
 Impressions
 Sentiments
 Part 2: Local Adoptions of MOOCs as Course Offerings
 Benefits of MOOCs (for stakeholder groups, for reputation, for budgets, for
resources)
 Pre-requisites for adoption
11

The open-ended survey instrument (cont.)
 Part 3: Ensuring MOOC Quality
 Strategies for managing learner “massiveness”
 Definition of “quality learning” in MOOCs
 Student identity in MOOCs
 Criteria for course topics selection for MOOCs
 Preferred instructor characteristics
 Part 4: MOOCs in the Future
 MOOC platforms as disruptive technologies or not
 Open-courseware movement? Open-source movement? As critical parts of MOOCs?
 Informal learning and certification or badging? Formal learning?
 Long-term potentials?
12

13 Sliders
indicating
future
likelihood for
the
“Potential
Scenarios”
(last)
question

The open-ended survey instrument (cont.)
 Potential Scenarios
 Near-term and midterm scenario for MOOCs
 Popularity
 Use by universities for formal learning and college credit? Use by corporations for
training?
 Star professors or common professors
 Learner-taught MOOCs
 Used for development in developing countries
 Cobbled MOOC platforms using Web 2.0 technologies
 Copyright or open-source
 Big data advancements and insights
14

Seating the e-Delphi expert panel
 Announcements through professional electronic mailing lists;
announcements in the microblogging site Twitter (social network
connectivity), and others
 Self-identified and self-selected experts
 Anonymous respondents
 A fairly small dataset (may range from a few to a thousand or so
traditionally)
 Some 103 total respondents but not responses on all questions
15

Panel: Years spent in online learning in
any capacity
16

Panel: Direct experiences with MOOCs18

Panel: Years in the current professional
position
 9.5 years average
 Median: 10 years
 Range from 1 – 30 years in the current professional position
20

Panel: Direct experiences w/ MOOC
courses
21

Panel: Ways respondents learned
about MOOCs (check all that apply)
22

Panel: Familiarity measures with
MOOCs
23

Panel: Familiarity measures with
MOOCs (on a bell curve)
24

Some findings…
25

Potential positive social implications of
MOOCs in a globalist context
26

Potential negative social implications
of MOOCs in a globalist context
27

Possible benefits of MOOCs to the local
university
28

Anticipated pre-requisites for
launching MOOCs
29

Understood strategies for managing
massive numbers of students
30

A Wordle™ of text responses for the
definition of quality in MOOCs
31

Identity verification strategies for
MOOC learners
32

Strategies for the selection of MOOC
course topics
 Current areas of research to spark innovations
 General areas
 University specialists
 Learning groups of non-traditional and mid-career professionals to meet
employer needs
 Long-tail niche courses
 Topics that other MOOCs haven’t covered
 Future focused topics
 “Why not anything?”
33

Criteria for the selection of faculty to
lead MOOCs
34

MOOC platforms disruptive or
evolutionary?
35

Intellectual property (IP) and MOOCs36

MOOC suitability for various types of
learning
37

Long-term potentials for MOOCs38

MOOC scenarios
Near- and midterm
39

Summary of MOOC predictions
 Little chance MOOCs will disappear in the near-term because of a lack of
clear funding models
 A fairly high probability MOOCs will be popularized beyond well-funded
universities
 A fairly high likelihood that MOOCs would be used for training by
corporations
 A low likelihood that MOOCs would be used as formal college courses
 A high likelihood that MOOCs would be used as short courses, tutorials, and
automated trainings
 A low probability that star professors will be the main ones teaching MOOCs
40

Summary of MOOC predictions (cont.)
 A better than chance likelihood that non-star professors will lead MOOCs
 A 50% chance likelihood that MOOCs will be taught by the learners in that
MOOC
 A low probability that MOOCs will only be developed in advanced or well
developed countries (a high probability that MOOCs will be adopted and
developed in developing countries)
 A high likelihood that MOOC platforms will be developed and maintained
commercially
 A high likelihood that MOOC platforms will be created by cobbling Web 2.0
technologies
41

Summary of MOOC predictions (cont.)
 A high likelihood that MOOC learners will be separated into an elite tuition-
paying core and a non-paying general public periphery
 An equal chance that learners will be treated equally (without a paying
elite vs. a general non-paying public)
 A fairly high probability that MOOC courses would fall under copyright
protection (and other intellectual property laws); also a fairly high
probability that MOOCs will be released in an open-source way
 A high probability of new insights of human learners and their needs from
“big data” resulting from MOOCs; respondents also suggested that there is
a low probability of not finding out new insights from the big data related to
MOOCs (in a conversely-phrased “checking” question)
42

Iff and other conditionals
 Actual and documented learning
 Effective design
 Respect for the individual learner and his /her personhood
 Faculty voice and choices
 Egalitarianism in terms of faculty leading
 A clear (and fair) funding model
 Useful research from the big data
 Proper identification of learners
 Proper crediting of learning (in the various contexts)…
43

Real-world test of the findings?
 In the year since this study was conducted (June 2013), what predictions
do you see as having been borne out? (Is this a valid test of the
“predictions” of the modified e-Delphi study results? Are there extant
trends that were not addressed here?
 Any core changes to the status quo in regards to MOOCs?
44

Implications? Questions? Comments?
 Are some of these findings intuitive? Others counter-intuitive? Elaborate.
 How would you answer some of these survey questions? Why?
 What other questions would you have asked in this instrument? Why?
 In terms of variations on MOOCs (MCs, MOCs, MOCs, OOCs, OCs, OCs),
which do you see as having the most potential? Why? (A recent
suggestion involved the creation of LOOCs or “local open online courses”.)
 What do you think is next?
45

References
 Hai-Jew, S. (2014). Iff and other conditionals: Expert perceptions of the
feasibility of massive open online courses (MOOCs)—A modified e-Delphi
study. Ch. 13. In Hai-Jew’s Remote Workforce Training: Effective
Technologies and Strategies. IGI-Global.
46

An addendum re: the research
method
47

Some thoughts on modified e-Delphi
studies
Overall
 Serves as an effective research methodology that uses survey system
technologies well (particularly those that incorporate multimedia elements
like videos, audio files, slideshows, and images)
 Works well as an open-ended method with plenty of designed open-ended
questions (without any forced consensus or forced dissensus)
Seating Expert Panels
 May be benefitted by the far-distance and near-in access to various
experts
 May be weakened by self-assertions of expertise by anonymous
participants
48

studies (cont.)
Creating Related Survey Instruments
 Requires in-depth and thorough secondary research about the topic to
create depth and breadth
 Requires plenty of vetting of test instruments
Single or Multi-Staging?
 May be more effective as a multi-stage process particularly for more
complex issues
49

studies (cont.)
Light Use
 May work as a near-term and mid-term forecasting method (a kind of
predictive analytics), if held lightly
 If something has to be put at risk based on the results of the modified e-Delphi
study, it would be good to conduct much more research
Additional Research Necessary about Effective Modified e-Delphi Study
Methods
 Would benefit from more research in terms of efficacy
50

The benefits of maintaining a research
journal
 Structured reflective practice
around the research
 A documented venue for
questioning at every stage of the
research work
 Not released with the work at
publication
51

Conclusion and contact
 Dr. Shalin Hai-Jew
 Instructional Designer
 Information Technology Assistance Center (iTAC)
 Kansas State University
 212 Hale Library
 Manhattan, KS 66506-1200
 785-532-5262 (work phone)
 shalin@k-state.edu
 A special thanks to the attendees at the SIDLIT Special Interest Group (SIG) lunch
session on MOOCs who helped catch a few problematic charts, which have since
been re-drawn and labeled. :P It’s always good to have many eyes on a work,
especially before going public!
52

Expert Perceptions of the Feasibility of MOOCs

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (20)

Similaire à Expert Perceptions of the Feasibility of MOOCs

Similaire à Expert Perceptions of the Feasibility of MOOCs (20)

Plus de Shalin Hai-Jew

Plus de Shalin Hai-Jew (20)

Dernier

Dernier (20)

Expert Perceptions of the Feasibility of MOOCs