ECER 2013, 21. Emerging Researchers' Group (for presentation at Emerging Researchers' Conference), Session: ERG SES C 02

1. A
Research
Synthesis
of
Social
Media
in
Higher
Educa8on
Challenges
and
Limita8ons
Franco
Rau,
Faculty
of
Human
Sciences
Technische
Universität
Darmstadt,
Germany
ECER
2013,
ERG
SES
C
02:
ICT
in
EducaEon,
09.09.2013

2. “It
would
be
foolish
to
ignore
the
tremendous
opportuni8es
the
Social
Web
offers
to
educa8on.”
(Wheeler
2009,
p.
4)

3.
“…
how
is
social
media
actually
being
used
beyond
the
world
of
academic
ed-­‐
tech
conferences,
journals
and
discussions
forums?“
(Selwyn
2010,
p.
3)

4. “Before
stepping
into
the
piVall
of
discussing
the
educaEonal
potenEal
of
the
next
‘big
thing’
in
educaEonal
technology,
[…]
,it
is
necessary
to
thoroughly
and
rigorously
evaluate
and
assess
the
implementa8on
of
Web
2.0
tools
so
far.”
(GouseE
2010,
p.
354)

5.
“Before
stepping
into
the
piVall
of
discussing
the
educaEonal
potenEal
of
the
next
‘big
thing’
in
educaEonal
technology,
[…]
,it
is
necessary
to
thoroughly
and
rigorously
evaluate
and
assess
the
implementa8on
of
Web
2.0
tools
so
far.”
(GouseE
2010,
p.
354)
Objective: carry out a
systematic review
Today: Focus on
methodical challenges
and limitations

6. Agenda
clarifying
the
objecEve
methods
and
limitaEons
preliminary
results
1
2
3

7. clarifying
the
objec8ve
1

8. creaEng
a
clear
picture
...

9. creaEng
a
clear
picture
...
...
of
the
actual
use
of
social
media
in
higher
educaEon
seZngs
focussing
on
aspects
of
interacEon
and
parEcipaEon
by
conducEng
a
systema8c
review.

10. „A
class
of
web
applica8ons
that
supports
group
interacta8ons,
especially
produc8ve
and
(semi-­‐)public
interacta8ons.“
„social
media“
(Rau
2012,
p.
13)

11. „systemaEc
review“
(Cooper
&
Hedges
2009,
p.
6)
„[...]
primary
focus
and
goal:
research
synthesis
aKempt
to
integrate
empirical
research
for
the
purpose
of
crea8ng
generaliza8ons.“

12. „systemaEc
reviews“
„[...]
have
specific
characteris8cs:
an
explicit
study
protocol,
addressing
a
pre-­‐
specified,
highly
focused
quesEon(s);
explicit
methods
for
searching
for
studies;
appraisal
of
studies
to
determine
their
scienEfic
quality;
and
explicit
methods
for
combining
the
findings“
(Dixon-­‐Woods,
Booth
&
Sueon
2007,
p.
375-­‐376)

13.
“The
study
is
categorized
into
six
secEons:
[1]
Facebook
users;
[2]
reasons
people
use
Facebook;
[3]
harmful
effects
of
Facebook;
[4]
Facebook
as
an
educaEonal
environment;
[5]
Facebook‘
effect
on
culture,
language
and
educaEon;
and
[5]
the
relaEonship
between
Facebook
und
subject
variables.“
(Tess
2013,
p.
A60)
Example:
Aydin (2012): A review of research
on Facebook as an educational
environment
Educational Technology Research and Development
Volume 60, Issue 6 , 1093-1106

14. “The
study
is
categorized
into
six
secEons:
[1]
Facebook
users;
[2]
reasons
people
use
Facebook;
[3]
harmful
effects
of
Facebook;
[4]
Facebook
as
an
educaEonal
environment;
[5]
Facebook‘
effect
on
culture,
language
and
educaEon;
and
[6]
the
relaEonship
between
Facebook
und
subject
variables.“
(Aydin
2012,
p.
1093)
What‘s
missing?

15. methods
and
limita8ons
2

16. key
characterisEcs/stages
searching
strategy
data
appraisal/assessment
data
analysis
A
B
C
Cooper
&
Hedges
(2009,
p.
9),
Dixon-­‐Woods
et
al.
(2007,
p.
376-­‐377),
Hannes
&
MacaiEs
(2012,
p.
403-­‐405)

17.
“A
review
of
literature
was
undertaken
in
September
and
October
of
2012.
I
system-­‐
a8cally
searched
the
following
specialized
database
sources:
Web
of
Science,
EBSCOhost
and
ERIC.
AddiEonally
Google
Scholar
searches
were
performed.
Search
words
and
phrases
included
Facebook,
Higher
EducaEon,
Social
Media,
[…]”
(Tess
2013,
p.
A60)
searching
strategy
A
Example #A:
Tess (2013): The Role of social
media in higher education classes
(real and virtual) – A literature
review
Computers in Human Behavior, Volume 29, Issue 5, A60-A68

18. “A
review
of
literature
was
undertaken
in
September
and
October
of
2012.
I
system-­‐
a8cally
searched
the
following
specialized
database
sources:
Web
of
Science,
EBSCOhost
and
ERIC.
AddiEonally
Google
Scholar
searches
were
performed.
Search
words
and
phrases
included
Facebook,
Higher
EducaEon,
Social
Media,
[…]”
(Tess
2013,
p.
A60)
transparent
strategy?
A

19.
reproducible
strategy?
Screenshot
-­‐
hep://eric.ed.gov
(29.08.2013)
A

20. A
„some”
databases
len?
Screenshots
-­‐
hep://eric.ed.gov,
hep://www.tandfonline.com,
hep://www.sciencedirect.com,
hep://www.apa.org/pubs/databases/psycinfo/index.aspx
(12.08.2013)

21. A
open
issues
How
much
research?
Which
databases
to
choose?
Screenshots
-­‐
hep://eric.ed.gov,
hep://www.sciencedirect.com,
hep://www.apa.org/pubs/databases/psycinfo/index.aspx
(12.08.2013)

22.
my
approach
• Focus
on
relevant
journals
– relevant
scope
&
peer
reviewed
– at
least
>
10
ar8cles
that
match
the
following
search
criteria
• PublicaEon
date:
2008-­‐2012
• Fulltextsearch
via
“Taylor
&
Francis
Online”
and
“ScienceDirect”
(with
the
aim
/
neccessity
to
limit
the
data
base)
• Use
a
combina8on
of
3
Keywords:
– “web”
AND
“learning”
AND
“higher
educaEon”
– “social”
AND
“learning”
AND
“higher
educaEon”
A
Screenshots
-­‐
hep://www.tandfonline.com,
hep://www.sciencedirect.com
(12.08.2013)

23.
my
approach
• relevant
journals
– ScienceDirect:
16
journals
(out
of
54
journals
found
in
EducaEon,
Media
and
Technology,
EducaEonal
Psychology)
– Taylor
&
Francis:
38
journals
(out
of
235
journals
found
in
EducaEon)
• poten8ally
eligible
studies
(including
duplicates
between
keyword
combinaEons)
– “web”
AND
“learning”
AND
“higher
educaEon”
• ScienceDirect:
1396
ar8cles
(out
of
16
journals)
• Taylor
&
Francis:
872
ar8cles
(out
of
38
journals)
– “social”
AND
“learning”
AND
“higher
educaEon”
• ScienceDirect:
2352
ar8cles
(out
of
16
journals)
• Taylor
&
Francis:
3263
ar8cles
(out
of
38
journals)
A
Screenshots
-­‐
hep://www.tandfonline.com,
hep://www.sciencedirect.com
(12.08.2013)

24.
“First,
they
must
be
experiment
in
design
including
two
groups
with
random
selecEon,
one-­‐group
pretest
and
poseest,
or
two
groups
without
random
selecEon.”
(Hew
&
Cheung
2013,
p.
50)
data
appraisal/assessment
B
Example #B:
Hew & Cheung (2013): Use of Web
2.0 technologies in K-12 and
higher education: The search for
evidence-based practice
Educational Research Review, Volume 9, 47-64

25. “First,
they
must
be
experiment
in
design
including
two
groups
with
random
selecEon,
one-­‐group
pretest
and
poseest,
or
two
groups
without
random
selecEon.”
(Hew
&
Cheung
2013,
p.
50)
inclusion
criteria
B

26. “First,
they
must
be
experiment
in
design
including
two
groups
with
random
selecEon,
one-­‐group
pretest
and
poseest,
or
two
groups
without
random
selecEon.”
“[…]
studies
that
relied
on
student
self-­‐report
data
such
as
student
quesEonnaire
survey
and
interview
were
excluded.”
(Hew
&
Cheung
2013,
p.
50)
excluding
qualitaEve
data?
B

27.
“First,
they
must
be
experiment
in
design
including
two
groups
with
random
selecEon,
one-­‐group
pretest
and
poseest,
or
two
groups
without
random
selecEon.”
“[…]
studies
that
relied
on
student
self-­‐report
data
such
as
student
quesEonnaire
survey
and
interview
were
excluded.”
(Hew
&
Cheung
2013,
p.
50)
open
issues
B
Which
inclusion
and
exclusion
criteria
to
choose?
How
to
judge
the
quality
of
a
publicaEon?

28.
my
approach
• reading
abstracts
(full-­‐text
if
needed)
• inclusion
criteria:
– empirical
study
(case
study,
experimental,
etc.)
– study
focus
on
a
natural
seZng
in
higher
educaEon
– use
of
social
media
to
support
learning
• exclusion
criteria:
– online
course/distance
educaEon
B
Screenshots
-­‐
hep://www.tandfonline.com,
hep://www.sciencedirect.com
(12.08.2013)

29.
my
approach
Screenshots
-­‐
hep://www.tandfonline.com,
hep://www.sciencedirect.com
(12.08.2013)
• InteracEve
Learning
Environments:
– 9
ar8cles
(out
of
55)
• Studies
in
Higher
EducaEon
– 5
ar8cles
(out
of
54)
B

30. “The
basic
unit
of
analysis
was
each
individual
empirical
ar8cle.
Using
the
constant-­‐compara8ve
method
espoused
by
Lincoln
and
Guba
(1985),
the
coding
scheme
was
not
predetermined
prior
to
our
analysis
but
emerged
inducEvely
from
the
data.
[…]
“
(Sim
&
Hew
2010,
p.
153)
data
analysis
C
Example #C:
Sim & Hew (2010): The use of
weblogs in higher education
settings: A review of empirical
research
Educational Research Review, Volume 5, Issue 2, 151-163

31. “The
basic
unit
of
analysis
was
each
individual
empirical
ar8cle.
Using
the
constant-­‐compara8ve
method
espoused
by
Lincoln
and
Guba
(1985),
the
coding
scheme
was
not
predetermined
prior
to
our
analysis
but
emerged
inducEvely
from
the
data.
[…]
“
(Sim
&
Hew
2010,
p.
153)
basic
unit
of
analysis
C

32. • constant-­‐compara8ve
method
– wriEng
memos
– started
with
„open
coding“
my
approach
C

33. preliminary
results
3

34. • ...
using
social
media
for
peer
feedback
&
assessment
preliminary
codes

35. • ...
using
social
media
for
peer
feedback
&
assessment
– “To
foster
the
social
learning
culture,
we
also
asked
students
to
check
into
their
classmates’
blogs
to
see
what
other
people
have
wrieen.
[…]
students
are
encouraged
to
read
others’
posEngs
and
then
write
comments
to
some
of
those
posEngs.”
(Chang
and
Chang
2011,
p.
4)
– “InstrucEonal
acEvity:
The
class
acEvity
was
essenEally
comprised
of
one
student
(author)
providing
a
proposal
to
a
second
student
(peer
reviewer)
who
would
then
review
the
proposal
and
criEque
the
proposal
that
was
then
returned
back
to
the
author
to
review
the
criEque.”
(Mendenhall
&
Johnson
2010,
p.
268)
preliminary
codes

36. • ...
using
social
media
to
increase
the
transparency
of
students
workflow/collabora8on
preliminary
codes

37. • ...
using
social
media
to
increase
the
transparency
of
students
workflow/collabora8on
– In
addiEon,
the
facilitator
regularly
viewed
every
student’s
space
and
provided
feedback
on
student
work.
(Lu
and
Churchill
2012,
pp.
5)
– Content
evoluEon
Day
of
week
acEvity
in
by
...
WikiHaskell
(Palomo-­‐Duarte
et
al.
2012,
pp.
10-­‐12)
preliminary
codes

38.
Franco
Rau,
M.Ed.
f.rau@apaed.tu-­‐darmstadt.de
@FrancoRau
hep://de.slideshare.net/FrancoRau/
Thank
you!

39. [1]
Aydin,
S.
(2012):
A
Review
of
Research
on
Facebook
as
an
EducaEonal
Environment.
Educa&onal
Technology
Research
and
Development.
60
(6):
1093–1106
[2]
Chang,
Y.J.
&
Chang,
Y.S.
(2011):
Assessing
peer
support
and
usability
of
blogging
in
hybrid
learning
environments.
Interac&ve
Learning
Environments;
hep://dx.doi.org/10.1080/10494820.2011.619889
[3]
Cooper,
H.
&
Hedges,
L.V.
(2009):
Research
Synthesis
as
a
ScienEfic
Process.
In:
Cooper,
Hedges
&
ValenEn
(eds.):
The
Handbook
of
Research
Synthesis
and
Meta-­‐Analysis.
Russel
Sage
FoundaEon,
New
York,
3—18
[4]
Dixon-­‐Woods,
M.
et
al.
(2007):
Synthesizing
qualitaEve
research:
a
review
of
published
reports.
Qualita&ve
Research
2007;
7;
hep://dx.doi.org/10.1177/1468794107078517
[5]
GouseE,
A.
(2010):
Web
2.0
and
educaEon:
not
just
another
case
of
hype,
hope
and
disappointment?
Learning,
Media
and
Technology
35(3):
351—356
[6]
Hannes,
K.
&
MacaiEs,
K.
(2012):
A
move
to
more
systemaEc
and
transparent
approaches
in
qualitaEve
evidence
synthesis:
update
on
a
review
of
published
papers.
Qualita&ve
Research
2012;
12;
hep://dx.doi.org/10.1177/1468794111432992
[7]
Hew,
K.F.
&
Cheung,
W.S.
(2013):
Use
of
Web
2.0
technologies
in
K-­‐12
and
higher
educaEon:
The
search
for
evidence-­‐based
pracEce.
Educa&onal
Research
Review;
9:
47–64;
hep://dx.doi.org/10.1016/j.edurev.2012.08.001
[8]
Lu,
J.
&
Churchill,
D.
(2012):
The
effect
of
social
interacEon
on
learning
engagement
in
a
social
networking
environment,
Interac&ve
Learning
Environments,
hep://dx.doi.org/10.1080/10494820.2012.680966
[9]
Mendenhall,
A
.
&
Johnson,
T.
E.
(2010):
Fostering
the
development
of
criEcal
thinking
skills,
and
reading
comprehension
of
undergraduates
using
a
Web
2.0
tool
coupled
with
a
learning
system,
Interac&ve
Learning
Environments,
18:3,
263—276;
hep://dx.doi.org/10.1080/10494820.2010.500537
[10]
Palomo-­‐Duarte
et
al.
(2012):
Assessment
of
collaboraEve
learning
experiences
by
graphical
analysis
of
wiki
contribuEons,
Interac&ve
Learning
Environments,
hep://dx.doi.org/10.1080/10494820.2012.680969
[11]
Peled,
Y.
et
al.
(2012):
CharacterisaEon
of
pre-­‐
service
teachers'
aZtude
to
feedback
in
a
wiki-­‐environment
framework,
Interac&ve
Learning
Environments,
hep://dx.doi.org/10.1080/10494820.2012.731002
[12]
Wheeler,
S.
(2009).
Learning
Space
Mashups:
Combining
Web
2.0
Tools
to
Create
CollaboraEve
and
ReflecEve
Learning
Spaces.
Future
Internet,
1(1),
3-­‐13.
[13]
Rau,
F.
(2012):
Social
Sonware
in
der
Hochschullehre.
KriEsche
Analyse
didakEscher
Szenarien.
Master
Thesis,
urn:nbn:de:kobv:517-­‐
opus-­‐63145
[14]
Selwyn,
N.
(2010).
The
educaEonal
significance
of
social
media
–
a
criEcal
perspecEve.
Keynote
at
Ed-­‐Media
Conference
2010;
hep://
www.scribd.com/doc/33693537/
The-­‐educaEonal-­‐significance-­‐of-­‐social-­‐
media-­‐a-­‐criEcal-­‐perspecEve
[15]
Sim,
J.W.S.
&
Hew,
K.F.
(2010):
The
use
of
weblogs
in
higher
educaEon
seZngs:
A
review
of
empirical
research.
Educa&onal
Research
Review,
5(2):
151—163;
hep://dx.doi.org/10.1016/j.edurev.2010.01.001
[16]
Tess,
P.A.
(2013):
The
Role
of
social
media
in
higher
educaEon
classes
(real
and
virtual)
–
A
literature
review.
Computers
in
Human
Behavior,
29
(5):
A60-­‐A68
references