"Adventures in Emerging Media: An Updated Look at Engagement Methods in Online Courses" Presentation given at 8am November 2, 2012 at AECT by Emily Johnson.
1. Adventures in Emerging Media:
An Updated Look at
Engagement Methods
in Online Courses
Emily K. Johnson, Reuben Rogak,
Rudy McDaniel, Robb Lindgren,
and Jon Friskics
University of Central Florida
Texts & Technology Ph.D. Program
ekj@knights.ucf.edu
2. • Online
• Survey Course
• Funded by a two-year grant
• Elective
• School of Visual Arts and Design
• Fall 2010, Fall 2011, and Fall 2012
3. Introduction
• Continuation of research on student
preferences in online course design
• Novel aspects of a unique, online course
taught at UCF in the Fall semester of
2011 entitled Adventures in Emerging
Media (AEM) were analyzed
4. Introduction
Unique course features that were studied:
• An unfolding narrative released to
students throughout the semester
• Student choice in which course modules
they wished to complete each week
• Assignments that were tied directly to
students’ potential careers and job
interviews
• A game-based achievement system
(added in Year 2)
5. Background
Prior research suggests that these elements
increase student motivation and learning:
•Relevant Tasks
•Student Agency
•Student Collaboration
•Narrative Elements
6. Unique Course Features
• Assignments
tied directly to
students’ potential
careers and job
interviews
10. Badges
“A 'badge' is a symbol or indicator of an
accomplishment, skill, quality or interest. From
the Boy and Girl Scouts…to the more recently
popular geo-location
game, Foursquare, badges have been
successfully used to set goals, motivate
behaviors, represent achievements and
communicate success in many contexts.”
-The Mozilla Foundation and Peer 2 Peer University. “Open Badges for
Lifelong Learning” (2012). Retrieved from:
https://wiki.mozilla.org/images/5/59/OpenBadges-Working-
Paper_012312.pdf
11. Achievements
Achievements, or earned tokens of
accomplishment, often encourage players
to spend more time within digital systems
(esp. videogames) and to alter their
playing habits in order to unlock particular
types of challenges (e.g., find every coin
in a given area or unlock a particular
puzzle within a certain amount of time).
12. Rationale for Achievements
Motivate students to engage in behavior linked
to success in online courses such as:
• Completing assignments early
• Answering a peer’s question
• Offering constructive feedback to peers
• Completing more than one module per week
(only one per week required)
--Constituted a small percent of students’
participation grades.
15. 5 Visible Achievements
Awarded for:
• Posting a required introduction on the
course discussion forum
• Successfully completing the first three
weeks of the course
• Successfully completing two-thirds of the
modules
• Successfully completing all required
modules
• Turning in the Week Twelve Milestone for
the Final Project
16. 6 Hidden Achievements
Awarded for:
• Being the first person to post a project
each week
• Answering a peer’s course-related
question on the discussion forum
• Completing three modules for one week
• Completing four modules for one week
• Watching at least some of each video
contained in each module for a given
week
• Turning in a project with exceptional
detail or technical skill
19. Design
• Non-experimental
• Descriptive
• Mixed methods
• Likert-scale options yielding quantitative
data
• Open-ended questions added a qualitative
approach
• Focus group after semester ended
20. Survey Participants
• 138 undergraduate students at UCF
enrolled in Adventures in Emerging Media
in Fall 2011
• 84 undergraduate students at UCF
enrolled in a total of four
other, “traditional” online courses in the
sciences and humanities completed a
modified survey
21. Sample questions for both groups:
This course has affected the way I think about my
career goals.
Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree
I feel that I benefited from interactions and
collaborations with other students in this course.
Strongly Disagree 1 2 3 4 5 6 7 Strongly Agree
In the last 8 weeks of this course, how often have you
done the following?
Commented on another student's work
Very Often – Often – Sometimes – Never
22. AEM Specific Question Example
What aspects of this course caused you to put more
effort into these assignments, as compared to a
“regular” online course? (Choose all that apply)
• The ability to choose my learning modules each week.
• The ability to choose one learning module for grading, AND
go back to take additional modules with "no pressure."
• A story that unfolds in pieces over the period of the course
(the "dream job" scenario).
• The relevancy of assignments designed to help me get a
job after graduation.
• Having my work recognized in some way (like on the
"Achieve" page).
• The ability to share my work with peers.
• The ability to critique the work of my peers.
• The ability to have my work critiqued by peers.
23. Design
• Responses of students from the AEM
course and the four other courses were
compared to analyze student perception
and behavior between the two types of
courses.
24. Achievements
Pearson
Variable 1 Variable 2 Correlation Significance (p)
(r)
Comfort with Motivated to Receive
.228 .010*
Technology (1 to 7) Achievements (1 to 7)
Number of Gaming Motivated to Receive
.172 .057
Hours per Week Achievements (1 to 7)
• Pre-Survey: Badge system motivating particularly
for those who were comfortable with technology and
games
25. Achievements
Felt Achievement System Had Motivated by Other Students
Positive Impact on Course Receiving Achievements
7 = Strongly Agree 7 = Strongly Agree
7 7
6 6
5 5
4 4
3 3
2 2
1 1
0 0
Males Females Males Females
26. Achievements
Pearson
Variable 1 Variable 2 Significance (p)
Correlation (r)
Felt Achievement System Discussed Ideas Outside of Class
.175 .040*
Was Positive (1 to 7) (1 to 7)
Worked Harder To Receive
Commented On Other Students’
Achievements .217 .010*
Work (1 to 7)
(1 to 7)
Worked Harder To Receive Amount Of Time Spent
Achievements Collaborating With Other .242 .004**
(1 to 7) Students
Seeing Others Get
Total Number Hours Spend On
Achievements Was .158 .064
Course Per Week
Motivating (1 to 7)
• Post-Survey: Positive feelings about achievement system
were correlated with other positive feelings in the course
27. Student Attitudes
• AEM Favorite Aspect: Course Design
• Other Positive Responses:
Choice in learning modules
Ability to take additional modules
without being tested on them
Relevancy of assignments
30. Results: Likert Statements
AEM Other
Challenging Course 5.25 4.87
Learned Interesting Things 5.52 6.45
Meaningful Assignments 4.95 6.24
Benefitted from Student Collaboration 4.45 5.52
Tried to do well 5.93 6.42
Proud of work 5.70 5.37
Affected career goals 4.62 5.10
Recommend to a friend 5.05 6.31
Interest in similar course design 5.07 6.00
• No notable differences
31. Results
Aspects you would like to see in other courses:
• Choice of learning modules each week: 79%
• The ability to take additional modules without
being tested on them: 64%
• Relevancy of assignments: 55%
• Have work critiqued by peers: 53%
• Share work with peers: 49%
• Ability to critique the work of peers: 44%
• Have work recognized: 38%
• Unfolding story: 28%
32. Results
Aspects that caused you to put forth more effort:
• Choice of learning modules each week: 54%
• Relevancy of assignments: 39%
• Peers critiquing your work: 41%
• Sharing work with peers: 37%
• Ability to take additional modules without
being tested on them: 36%
• Ability to critique the work of peers: 32%
• Having my work recognized: 30%
• Unfolding story: 14%
33. Discussion
• AEM students- higher frequency of
collaboration
• Research suggests that collaboration is
important
• Worthwhile to incorporate aspects of this
course lending themselves to collaboration
into other online courses.
34. Discussion
• Little difference was seen in Likert-scale
averages between AEM & other courses
• This could be due to the nature of
education at this level
• Course selection
• Intrinsically motivated
• Understand importance of
assignments
35. Discussion
• Interesting gender patterns in achievement
opinions – Are badges more motivating to
females? Was the design similar to social
games, which females tend to prefer?
• AEM students indicated a desire to see
many of the unique features implemented
in future online courses
• Some factors to consider when designing a
course
36. Discussion: Future Research
• Similar course in different fields of study
• Determine most effective features
• Investigate effects on skill mastery
37. Intended Improvements
• Improving narrative coherence;
including student input
• Updating digital creation guidelines
• New module creation and improved
test creation system
• Updating the look-and-feel of the site
38. References
Beldarrain, Y. (2006). Distance education trends: Integrating new
technologies to foster student interaction and collaboration.
Distance Education, 27(2), 139-153.
Caine, R. N., & Caine, G. (1991). Making Connections: Teaching
and the human brain. Alexandria, VA: Association for
Supervision and Curriculum Development.
Cameron, J. a. P., W. D. (2002). Rewards and intrinsic motivation:
Resolving the controversy. Westport, Connecticut: Bergin &
Garvey.
Carr, D. (1986). Narrative and the real world: An argument for
continuity. History and Theory, 25(2), 117-131.
Deci, E. L. & Ryan, R. M. (1990). Intrinsic motivation and self-
determination in human behavior. New York: Plenum.
Keller, J., & Suzuki, K. (2004). Learner motivation and e-learning
design: A multinationally validated process. Journal of
Educational Media, 29(3), 229-239.
39. References
Lauritzen, C. a. J. M. (1997). Integrating learning through story:
The narrative curriculum. Albany, N.Y: Delmar Publishers.
Lindgren, R. & McDaniel, R. (in press). Transforming online
learning through narrative and student agency.
Educational Technology & Society.
Liu, N. F., & Carless, D. (2006). Peer feedback: The learning
element of peer assessment. Teaching in Higher
Education, 11(3), 279-290.
McCombs, B. L., Whisler, Jo Sue. (1997). The learner-centered
classroom and school: Strategies for increasing student
motivation and achievement. San Francisco, CA: Jossey-
Bass.
Palloff, R., & Pratt, K. (2000). Making the transition: Helping
teachers to teach online. EDUCAUSE: Thinking it through.
Nashville, Tennessee.
40. References
Passe, J. (1996). When students choose content: A guide to
increasing motivation, autonomy, and achievement.
Thousand Oaks, CA: Corwin Press.
Patall, E. A., Cooper, H., & Wynn, S. R. (2010). The effectiveness
and relative importance of choice in the classroom. Journal
of Educational Psychology, 102(4), 896.
Radovan, M. (2011). The relation between distance students'
motivation, their use of learning strategies, and academic
success. The Turkish Online Journal of Educational
Technology, 10(1), 216-222.
Varma, S., McCandliss, B. D., & Schwartz, D. L. (2008). Scientific
and pragmatic challenges for bridging education and
neuroscience. Educational Researcher, 37(3), 140-152.
ekj@knights.ucf.edu
