This document discusses strategies for measuring the impact of mobile tools on teaching and learning. It begins with an agenda that includes discussing mobility and teaching/learning as well as strategies for measuring mobile tools' impact. The document then provides examples of different frameworks for mobile learning, including behaviorist, constructivist, situated, collaborative, and learning support approaches. It also discusses challenges in mobile research and evaluating mobile applications. Case studies are presented on measuring the effectiveness of mobile learning in blended courses and its use in a medical school setting. Recommendations are made for building successful mobile learning frameworks and factors to consider in initiatives.

1. Mobile
Teaching
and
Learning:
E n g a g i n g
S t u d e n t s
a n d
M e a s u r i n g
I m p a c t
Veronica
Diaz,
PhD
Associate
Director
EDUCAUSE
Learning
IniBaBve

3. Agenda
• Mobility
and
teaching
and
learning
• Strategies
and
methodologies
to
help
measure
the
impact
of
mobile
tools
on
teaching
and
learning

4. Resources
hGp://Bnyurl.com/mecar
hGps://sites.google.com/site/nercomp12/

5. Got
mobility?

9. Discipline-­‐specific
customized
mLearning
Generic
mobile
instrucBonal
funcBons
(clickers,
twiGer,
LMS)
Pushing
mobile
content
(registrar,
library,
tram)

10. BEHAVIORIST
acBviBes
that
promote
learning
as
a
change
in
observable
acBons
Source:
hGp://www2.futurelab.org.uk/resources/documents/lit_reviews/Mobile_Review.pdf.

11. CONSTRUCTIVIST
learning
as
an
acBve
process
in
which
learners
construct
new
ideas
or
concepts
based
on
both
their
current
and
past
knowledge
Source:
hGp://www2.futurelab.org.uk/resources/documents/lit_reviews/Mobile_Review.pdf.

12. SITUATED
acBviBes
that
promote
learning
within
an
authenBc
context
and
culture
Source:
hGp://www2.futurelab.org.uk/resources/documents/lit_reviews/Mobile_Review.pdf.

13. COLLABORATIVE
acBviBes
that
promote
learning
through
social
interacBon
Source:
hGp://www2.futurelab.org.uk/resources/documents/lit_reviews/Mobile_Review.pdf.

14. LEARNING
SUPPORT
acBviBes
that
assist
in
the
coordinaBon
of
learners
and
resources
for
learning
acBviBes
Source:
hGp://www2.futurelab.org.uk/resources/documents/lit_reviews/Mobile_Review.pdf.

15. Behaviorist
(e.g.,
clickers)
ConstrucBvist
(e.g.,
simulaBons)
Situated
(e.g.,
PBL)
CollaboraBve
(e.g.,
TBL)
Learning
Support
(e.g.,
PLE)

16. The
mobile
device
(tablet,
smart
phone,
etc.)
The
mobile
applicaBons
(apps,
cloud,
etc.)
The
goals
(engagement,
content
delivery,
etc.)
…………
YOUR
MOBILE
INITIATIVE
ACTIVITY,
PART
1

17. The
Thing(s)
To
Measure
vs
How
To
Measure

18. Measuring
the
I mpact:
5
S tudies

19. CASE
STUDY:
MOBILE
LEARNING
IN
A
BLENDED
COURSE
Source:
hGp://www.docstoc.com/docs/38964563/Assessing-­‐the-­‐EffecBveness-­‐of-­‐Mobile-­‐Learning-­‐in-­‐Large-­‐
HybridBlended

20. Focus
Areas
if
student
enjoyed
the
learning
experience
how
students
feel
about
interacBng
with
fellow
learners
how
students
felt
about
their
relaBonship
to
their
instructors
in
the
mobile
blended
learning
environment
mobile
blended
classroom’s
effects
on
students’
study
habits

22. Overall
SaBsfacBon
Course
OrganizaBon

23. Course
AcBviBes
Student
InteracBon

24. Instructor
InteracBon
RelaBonship
to
Content

25. Sustainability

26. MOBILE
RESEARCH:
CHALLENGES
AND
GOALS
Source:
hGps://lra.le.ac.uk/bitstream/2381/8162/3/%5B14%5DVavoulaSharples-­‐
mlearn2008%5B1%5D.pdf

27. Capturing
learning
context
and
learning
across
contexts
Challenge
1

28. Has
anyone
learned
anything
yet?
Challenge
2

29. An
ethical
quesBon
Challenge
3

30. The
technology
itself
Challenge
4

31. The
big
picture
Challenge
5

32. Formal
vs
informal
Challenge
6

33. 5
recommendaBons
• Capture
and
analyze
learning
in
context
with
consideraBon
of
learner
privacy
• Assess
the
usability
of
the
technology
and
how
it
affects
the
learning
experience
(PLE)
• Look
beyond
measurable
cogniBve
gains
into
changes
in
the
learning
process
and
pracBce
• Consider
organizaBonal
issues
in
the
adopBon
of
mobile
learning
pracBce
and
its
integraBon
with
exisBng
pracBces
• Span
the
lifecycle
of
the
mobile
learning
innovaBon
that
is
evaluated,
from
concepBon
to
full
deployment
and
beyond

34. BUILDING
A
SUCCESSFUL
MLEARNING
FRAMEWORK
Source:
hGp://www.irrodl.org/index.php/irrodl/arBcle/view/346/875

35. ProporBonate

36. Appropriate
to
the
specific
learning
technologies,
to
the
learners,
and
to
the
ethos
of
the
learning
–
ideally
built
in,
not
bolted
on

37. Alignment
between
learning
goals
and
technology

38. Unintended
results

39. How
and
what
will
you
measure?
What
would
success
look
like?
…………
YOUR
MOBILE
INITIATIVE
ACTIVITY,
PART
2

40. CASE
STUDY:
MOBILE
LEARNING
IN
MEDICAL
SCHOOL
Source:
hGp://www.biomedcentral.com/1472-­‐6920/10/57

41. Context
• 57
students
in
cohort
• 4-­‐year
– Bachelor
of
Medicine
– Bachelor
of
Surgery
• 2
online
tools
– Blackboard
– Interlearn
• Access
to
– Internet
– Customized
sokware
– Info
repositories
• Sharing
info
within
and
between
cohorts
• 2
RQs
– In
what
ways
does
mobility
support
learning?
– What
areas
need
further
development?

43. Student
Improvement
SuggesBons
• BeGer
and
more
electronic
learning
resources
– more
materials,
especially
audiovisual
resources
– more
flexibility
in
accessing
materials
• Guidelines/support
on
managing
the
VLE
• Improvements
to
VLE
– Streamlining
organizaBon
of
informaBon
– Reduced
the
number
of
clicks
to
access
resources
– OrientaBon
for
students

44. Johns
Hopkins
University
10/18/2010
EVALUATING
MOBILE
APPLICATIONS

45. Student
MoBvaBon
AuthenBcity
Curriculum
ConnecBon
User
Friendliness
DifferenBaBon
Feedback

47. Review
your
plan
and
ask…
• What
would
be
the
ramificaBons
and
opportuniBes
for
learning
if
this
technology
were
adopted?
• What
kinds
of
teaching
and
learning
engagements
might
this
technology:
make
beGer
or
enable?
• If
we
decide
to
do
a
pilot,
what
kind
of
evaluaBon
methodology
can
we
overlay
on
the
project
to
assess
outcomes?
• What
kind
of
addiBonal
research
needs
to
be
done
concerning
this
technology
(student)?

48. The
Thing(s)
To
Measure
vs
How
To
Measure

49. How
do
I
ask?
• QuesBonnaires
– Paper
– Online
• Interviews
– Individual
– Focus
Group
• ObservaBons
– Classroom
– Online
•
•
•
•
•
•
Focus
groups
Surveys
Tracking
tools
Journaling
Pre
post
tests
Debriefs

50. Types
of
QuesBonnaires
Paper
distributed
in
class
Mail
E-­‐mail
Online,
e.g.,
SurveyMonkey
TwiGer
poll
Facebook

51. Types
of
Interviews
Face-­‐to-­‐face
Telephone
or
Skype
Teleconference
Synchronous
Chat
Asynchronous
forum
Focus
group
TwiGer
poll
Facebook

52. ObservaBon
OpBons
Onlooker
Overt
Participant
Covert
Fully-explained
Unexplained
Once or limited
On-going
Narrow focus
Broad focus
Source:
hGp://www.amazon.com/ConducBng-­‐EducaBonal-­‐Design-­‐Research-­‐McKenney/dp/0415618045/ref=sr_1_1?s=books&ie=UTF8&qid=1329938445&sr=1-­‐1

53. Triangulate
to
Tell
the
Full
Story
Q
I
Full
Story
O
Source:
hGp://www.amazon.com/ConducBng-­‐EducaBonal-­‐Design-­‐Research-­‐McKenney/dp/0415618045/ref=sr_1_1?s=books&ie=UTF8&qid=1329938445&sr=1-­‐1

54. QUESTIONS

55. Lunch

56. Measuring
the
iniBaBve
…………
YOUR
MOBILE
INITIATIVE
ACTIVITY,
PART
3

57. Challenges
and
unintended
consequences.
…………
YOUR
MOBILE
INITIATIVE
ACTIVITY,
PART
4

58. QUESTIONS

59. Empowering
Students
and
Instructors:
ReflecBons
on
the
EffecBveness
of
iPads
for
Teaching
and
Learning,
Morrone,
Anastasia
S.
hGp://www.educause.edu/Resources/
EmpoweringStudentsandInstructo/250105

60. Device
Apps/Sokware
Goals
Methods
Outcomes
Empowering
Students
and
Instructors:
ReflecBons
on
the
EffecBveness
of
iPads
for
Teaching
and
Learning,
Morrone,
Anastasia
S.
hGp://www.educause.edu/Resources/
EmpoweringStudentsandInstructo/250105

61. Launching
a
University
Tablet
IniBaBve:
RecommendaBons
from
Stanford
University’s
iPad
ImplementaBon,
Brian
Tobin.
hGp://www.educause.edu/Resources/
LaunchingaUniversityTabletInit/250812

62. Device
IniBaBve:
Launching
a
University
Tablet
RecommendaBons
from
Stanford
University’s
Apps/Sokware
iPad
ImplementaBon,
Brian
Tobin.
hGp://www.educause.edu/Resources/
Goals
LaunchingaUniversityTabletInit/250812
Methods
Outcomes

63. Closing
in
on
Vocabulary
AcquisiBon
TesBng
the
Use
of
iPods
and
Flashcard
Sokware
to
Eliminate
Performance
Gaps,
Andrea
L.
Nixon.
hGp://www.educause.edu/Resources/
ClosinginonVocabularyAcquisiB/248520

64. Device
Closing
in
on
Vocabulary
AcquisiBon
TesBng
the
Use
oApps/Sokware
to
f
iPods
and
Flashcard
Sokware
Eliminate
Performance
Gaps,
Andrea
L.
Nixon.
hGp://www.educause.edu/Resources/
Goals
ClosinginonVocabularyAcquisiB/248520
Methods
Outcomes

65. DISCUSSION/QUESTIONS

66. Veronica
Diaz,
PhD
Associate
Director
EDUCAUSE
Learning
Initiative
vdiaz@educause.edu
http://www.educause.edu/eli
Resources
http://tinyurl.com/mecar