Contenu connexe Similaire à Pedagogical and Assessment Design for Online Learning (20) Pedagogical and Assessment Design for Online Learning1. Pedagogical and Assessment
Designs for Online Learning
Barbara Means
Center for Technology in Learning
SRI International
Keynote presentation for the CITE Research Symposium
E-Learning Designs & Designs for Learning
March 6, 2010
Hong Kong
© 2007 SRI International
2. Outline of Talk
Emergence of online learning at secondary level in
U.S.
Variations in online learning designs
Research synthesis on the effectiveness of online
learning
Role of formative assessment in enhancing
learning outcomes for blended instruction
© 2007 SRI International
2
3. Growing Importance of Online Learning
Long a part of corporate and military training
Online learning has become a big part of higher
education and of self-directed adult learning
Now it is becoming part of K-12 education (mostly
at the secondary level)
© 2007 SRI International
4. K-12 adoption of online learning rising in U.S.
Enrollments up from 45,000 in 2000 to 1,000,000 in 2007
Online Learning Enrollment
0.08
0.07
0.06
0.05
0.04 Online Learning
0.03 Enrollment
0.02
0.01
0
1998 2000 2002 2004 2006 2008
Source: Estimates of secondary school online enrollments from Picciano & Seaman, 2009
Copyright Clayton M. Christensen
5. K-12 adoption of online learning rising in U.S.
Enrollments up from 45,000 in 2000 to 1,000,000 in 2007
0.10
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01
Online Learning
Enrollment
0
1998 2000 2002 2004 2006 2008 2010 2012 2014
Source: Estimates of secondary school online enrollments from Picciano & Seaman, 2009
Copyright Clayton M. Christensen
6. K-12 adoption of online learning rising in U.S.
0.10
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01
Online Learning
Enrollment
0
1998 2000 2002 2004 2006 2008 2010 2012 2014
Clayton Christensen predicts that by 2019 half of all
high school courses will be taken online.
Copyright Clayton M. Christensen
7. U.S. districts and states are starting to
encourage online learning
Most school districts (75% in 2007) provide online
learning options for at least some of their students
Several state-funded programs offer online high school
courses (Florida, Illinois, Michigan, Kentucky)
24 of 50 states now have statewide full-time online
schools (“cyberschools”)
Two states (Michigan and Alabama) require students to
take an online course in order to earn a high school
diploma
One state (Florida) requires all of its districts to make
online learning options available to their students
© 2007 SRI International
7
8. Typical Reasons for Offering Online Course
Options
Provide a course not otherwise available
Provide more college-level (Advanced Placement)
course options
Give students a second chance through “credit
recovery” programs
© 2007 SRI International
8
9. Different Forms of Online Learning
Fully online v. hybrid or “blended”
Module within a course, supplemental formal course,
degree program
Scheduled course (“class-linked”) v. self-paced
Instructor-developed v. professionally developed
© 2007 SRI International
10. Instructional Design Elements in Online
Learning
Communication type
- Synchronous
- Asynchronous
Learner control
Instructional approach
- Expository
- Active
- Interactive
© 2007 SRI International
12. Active Learning Designs
Support the practice of
skills and elicit student
responses
Carnegie Learning’s Algebra I Tutor
© 2007 SRI International
12
13. Interactive Learning Designs
Involve collaboration with
other learners or an
instructor with the focus
of engagement emerging
over time
Political Science 105 from University of North Carolina Greensboro
© 2007 SRI International
13
14. SRI’s Meta-Analysis of Online Learning
Research
Systematic search for studies
from 1996 through July 2008 on the
effectiveness of online learning
Limited to studies that
- compared an e-learning or blended
condition to face-to-face instruction
-measured learning objectively with the
same measure for treatment and control
groups
-used an experimental or quasi-
experimental design with control for any
pre-existing differences between groups
-provided the statistical information
needed to compute an effect size
Out of 1,132 articles reviewed, 51
study effects met the requirements
for meta-analysis
© 2007 SRI International
15. Meta-Analysis Findings
Average effect size of +0.24
favoring the online condition
Advantage over face-to-face
instruction was larger for studies
using blended approaches (E.S. =
+0.35) than for studies using pure
e-learning (E.S. = +0.14)
Advantage over face-to-face
instruction was larger for studies
where
- online students spent more time
learning than did those in the face-to-
face class
- the online and face-to-face conditions
varied in terms of content and
instructional approach
Taken as a whole, the findings suggest that the observed advantage of online learning is a
product of redesigning the learning experience, not of the medium per se.
© 2007 SRI International
16. SRI’s Review of Studies Comparing Different
Online Learning Designs
Online learning usually more
effective when
- it stimulates more active engagement
with the content
- it includes prompts for learner
reflection
-learners have an element of control
over their interactions with the software
Some common practices usually
had no effect
-adding additional media not related to
the content to be learned
- adding multiple-choice quizzes
Results were inconclusive with
respect to different instructional
approaches
www2.ed.gov/rschstat/eval/tech/evidence-based-practices/finalreport.pdf
© 2007 SRI International
17. Conjectures on the Blended Advantage
More learning time
More interactivity
Selective re-teaching of difficult concepts and
skills
© 2007 SRI International
18. The Concept of Formative Assessment
Assessment for learning rather than
assessment of learning
Occurs during the course of learning while there
is still time to improve outcomes
A quality of the way in which assessment
information is used, not inherent in the test itself
One of the most powerful levers for improving
learning outcomes (Black & Wiliam, 1998)
© 2007 SRI International
19. e-Learning Offers New Options for Formative
Assessment
We can do much more than just interspersing sets of
multiple-choice questions into instructional materials
Technology enables automated capture of complex
learner behaviors
Assessment can be embedded in the learning in ways
that feel natural rather than forced
© 2007 SRI International
19
20. Learning in a Multi-User Virtual Environment
Chris Dede’s River City MUVE (Science, 2009)
Enlargement of Microscope
© 2007 SRI International
20
21. River City
Teaches concepts from biology, ecology and
epidemiology
Students work in teams of three, moving through the
city to run tests in response to the mayors’ challenge
Teams keep online journals, analyze data, form
hypotheses, and write up their research in a report for
the mayor
Researchers developed measures of science concept
knowledge, science inquiry skills, and sense of
efficacy as a scientist based on student actions within
the River City environment
© 2007 SRI International
21
22. Such online learning environments can be
designed to capture . . .
Where a student went
With whom the student communicated and what was said
The artifacts the student activated
Databases the student used
Data that the student gathered using virtual scientific
instruments
Screenshots and notations the student entered in a virtual
notebook
Hints that the learner requested
All of this information can be used for assessment.
© 2007 SRI International
22
26. Examples from Carnegie-Mellon’s Online
Learning Initiative (OLI)
Others incorporate intelligent tutoring and carefully
designed exercises
© 2007 SRI International
26
27. Examples from Carnegie-Mellon’s Online
Learning Initiative (OLI)
Others incorporate intelligent tutoring and carefully
designed exercises
© 2007 SRI International
27
28. The Boxplot
The boxplot graphically represents the distribution of a quantitative variable by visually displaying
The five number summary and any observation that was classified as a suspected outlier using the
15(IQR) criterion.
Here is how a boxplot is constructed (this is for the “Best Actress” dataset – to see the dataset click here.)
To see a static version of this movie, click here. Use the pull-down menu to label the various points on the boxplot.
X That is not quite right. I think you might be
Source: Carnegie-Mellon University, OLI confusing Q1 and Q3.
Recall that Q1, the first quartile of the distribution,
Course in Statistics is represented
By the bottom edge of the box. Q3, the third
quartile of the 28
© 2007 SRI International
30. Formative Assessment Is Woven into the
Online Learning
Interactive simulations that students can manipulate are
woven into the courseware. Students interact with the
simulation and then respond to probes that get at their
understanding of how the simulation works.
“Did I Get This?” quizzes follow the presentation of new
material so that students can check their understanding
without fear of hurting their course grade.
Short essay questions that call on students to make
connections between different concepts are embedded
throughout the online course material.
“Muddiest Point” requests ask students what they thought
was confusing.
© 2007 SRI International
30
31. In Blended OLI Courses Instructors Take
Advantage of eLearning Assessments
Each student works with the
online course materials
outside of class.
© 2007 SRI International
31
32. In Blended OLI Courses Instructors Take
Advantage of eLearning Assessments
Source: Heffernan & Thille, Carnegie Mellon Online Learning Initiative web presentation
The e-learning system aggregates information from all the students who used it.
© 2007 SRI International
32
33. In Blended OLI Courses Instructors Take
Advantage of eLearning Assessments
The instructor sees what concepts
students had trouble with.
© 2007 SRI International
33
34. In Blended OLI Courses Instructors Take
Advantage of eLearning Assessments
In class, the instructor focuses on concepts that students
struggled with when they worked on line.
© 2007 SRI International
34
35. The OLI Statistics Course Study
Students were assigned at random to take college statistics
on line or in a conventional class.
Students in the online version of the class finished the
material in half a semester or half the time taken by
students in the conventional class.
The two classes took the same final examination, and the
online students’ scores were significantly better than those
of the students in the conventional class.
No difference between the two groups on a long-term
retention test.
Study has been replicated in a community college and a
large public university.
Source: Lovett, Meyer, & Thille, 2008
© 2007 SRI International
35
36. Principles for e-Learning Design
Incorporate a range of online activities that engage learners
with content.
Give learners options for different ways to learn content and
with options for review and support (scaffolding).
Embed assessment within the learning activities and use
multiple assessment approaches.
Make assessment feedback available to the student as well
as to the instructor.
Use information from students’ online activity to shape the
content of face-to-face instruction.
© 2007 SRI International
36
Notes de l'éditeur The Open Learning Initiative is a project started at Carnegie Mellon University with funding from the William and Flora Hewlett Foundation.