Food Chain and Food Web (Ecosystem) EVS, B. Pharmacy 1st Year, Sem-II
Massive Open Online Courses in Higher Education – Performance Assessment in Open Learning Arrangements
1. Massive Open Online Courses
in Higher Education –
Performance Assessment in Open Learning
Arrangements
8th InternationalTechnology,
Education and Development Conference,
10th-12th March 2014,Valencia
Presenting author: Daniela Pscheida,TU Dresden, Germany
Contact: daniela.pscheida@tu-dresden.de
Twitter: @danielapscheida
2. cMOOCs in Higher Education
(c = connectivist)
Connectivist MOOCs are Open Online Courses based on the
online activity and thematic exchange of learners
instead of broadcasting video lectures.
That‘s why this format has high potential for higher education –
as it supports the training of important soft skills for the 21st century:
media literacy
experiences in personal learning
creation of personal knowledge networks
3. Challenges of cMOOCs in
Higher Education
The implementation of a cMOOC learning scenario is
connected to certain challenges:
curricular integration, e.g. if there are participants from different
universities and faculties
technological and media capabilities, e.g. if there are participants
with different levels of experience
supervision and motivation of students
documentation and assessment of learning performance
4. PerformanceAssessment in cMOOCs
Two incompabilities between connectivist learning and
higher education:
The context of higher education demands a complete collection
and transparent assessment of learning outcomes along
predefined learning objectives to award Credit Points – while
connectivism encourages learners to focus on their own learning
interests and needs (openness of learning process).
The core idea of connectivism assumes intrinsically motivated
learners – while student learners are used to receive continuous
feedback and orientation (guidance) from lecturers.
5. The SOOC13
SOOC = Saxon Open Online Course
learning innovation project by three German universities
(Dresden, Chemnitz, Siegen)
nine-week course in summer term 2013, 242 registered participants, 122
students
topic: „Learning 2.0 – Individual learning and knowledge management
with social media“
course organization via central weblog (www.sooc13.de), materials and
impulses
blog aggregator to collect course-related blog posts from participants
and tweets with hashtag #sooc13
expert talks and live sessions via Adobe Connect
6. Tasks and requirements
The SOOC13
regular participation in live
sessions (expert talks)
writing and publishing at
least one blog post in each
thematic section
e-portfolio (collection of
the results of research and
reflection tasks and
smaller experiments)
7. Performance recording and feedback
The SOOC13
first: RSS feed aggregator
later we additionally introduced: e-portfolio-form
support by two student tutors
likes and individual comments on participants‘ contributions –
to stimulate further discussions and debates
no feedback for assessment purposes during the course run
8. Performance assessment
The SOOC13
no exam
assessment based on different course activities, especially the
collected e-portfolio contributions
general criteria: spelling, grammar, language, comprehension,
compliance with scientific standards
quality of content: understanding and correct application of
technical terms and theories, etc.
‚soft‘ criteria: aspects arising from self-directed learning and
connectivism, e.g. reflectivity and referring back
9. Evaluation of the SOOC
Survey conducted as online questionnaires at the beginning, in the
middle and at the end of the course and
virtual evaluation workshop after the course in the form of an online
written discussion
Results presented here summarize
the interim and final evaluation of the sooc13 as well as
the interim evaluation of the sooc1314.
10. „Do you think you complied with the requirements of
the course?“
Evaluation Results from SOOC13
60%
23%
17%
0% 10% 20% 30% 40% 50% 60% 70%
Yes
No
I cannot say
N = 30
more than one third of
participants were unsure
about or did not think so
the main reason for
uncertainty regarding the
own performance was a
lack of feedback (60%)
11. „Are you satisfied with the assessment system of the
SOOC13?“
Evaluation Results from SOOC13
With 41% of participants being
uncertain about that and
another 21% saying „no“,
there‘s a good chance for
improvement.
38%
21%
41%
0% 10% 20% 30% 40% 50%
Yes
No
I cannot say
N = 29
12. Improvement of cMOOC assessment system by:
an instrument, that allows continuous feedback for contributions
increase the transparency in terms of the assessment criteria
... to avoid uncertainty among students
... to raise the quality of discussions
introduce digital badges for the second course run
Lessons Learned
13. Digital badges in SOOC1314
Digital Badges
as a formative measurement instrument
„small“ badges for each reported contribution of the e-portfolio
form in states:
bronze
silver
gold
each badge was combined with a brief verbal judgement
to pass the course at least a bronze badge had to be received at
any given task
14. Evaluation Results from SOOC1314
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
The portfolio-form is easy to use and well structured.
I understand the significance of the badges.
For me the literal assessments clarify the significance of the
badges.
For me the Badges and the literal assessments are transparent
methods of evaluation.
N = 20
I fully agree
I somewhat agree
I cannot say
I rather disagree
I do not agree at all
15. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
The evaluation with badges helps me, to appreciate my own
achievements.
The volume of the literal assessment is appropriate.
The evalution method with badges and literal assessments is fair.
The literal assessment show me possibilities, where I can improve next
time.
N = 20
always
often
sometimes
rarely
never
Evaluation Results from SOOC1314
16. Evaluation results from the SOOC1314 survey prove, that the
new assessment system with badges and verbal judgements is
much more transparent and understandable – which also led to
an increased satisfaction of participants.
Conclusion
Further innovations at the horizon:
using methods of learning analytics to improve the overview
and objectivity of a cMOOC in higher education.