Delivered as part of the ICOPER Webinar Series

1. Webinar Series – Session 2
Outcome-based Learning
Opportunities
Michael Derntl1, Jad Najjar2 and Susanne Neumann1
1 University of Vienna, Austria
2 WU Vienna, Austria

2. Webinar overview
Background and motivation for outcome-based education
Formulating learning outcomes
Learning outcome based teaching methods and learning
designs
Models and standards for learning outcomes and
opportunities
Learning outcome motivated assessment

3. Using Adobe Connect

4. Using Adobe Connect

5. Background and motivation
Why outcome-oriented teaching and learning?
The Bologna Process
quality assurance in higher education
a unified educational system (mobility, transparency and mutual
recognition of qualification)
Revise curricula integrate student centeredness and
learning outcome orientation
Pedagogical issues became central: Alignment of teaching
and assessment methods in accordance with learning
outcomes

6. Background and motivation
What is your background?
Please shortly state in the chat:
1. Your background.
(e.g. higher education, PhD student, vocational training, consulting,
instructional design, other…)
2. Your position or relation to teaching (if any).
3. Subject(s) you teach / PhD topic.

7. Formulating learning outcomes
Learning outcomes
A learning outcome is a statement of what a learner is
expected to know, understand or be able to do as a result of
a learning process ―European Qualification Framework for
Lifelong Learning (EQF)
Teaching
methods
Learning Workload
ALIGNMENT
outcomes (ECTS)
Assessment

8. Formulating learning outcomes
Types of learning outcomes
Learning outcomes
Competence (personal and professional)
The proven ability to use knowledge, skills and
personal, social and/or methodological abilities.
proven ability
to use Skills
The ability to apply knowledge and use know-
how to complete tasks and solve problems.
Personal, social ability
and/or methodological to apply
abilities
Knowledge
The outcome of the assimilation
of information through learning.

9. Formulating learning outcomes
Intended vs. achieved learning outcomes
The teacher defines the intended learning outcomes for a
learning opportunity
The teacher assesses and learners demonstrate the
achievement of intended learning outcomes

10. Formulating learning outcomes
Sentence structure of intended learning outcomes
An intended learning outcome is formulated as a sentence
that follows a basic structure:
Learners are able to Verb Object
Describes intended Describes what students
(cognitive) process are expected to acquire
or construct
Example:
Learners are able to enumerate the most influential politicians of the 20th century
What type of learning outcome is this?

11. Formulating learning outcomes
Examples of learning outcomes and their type
Students are able to …
… explain the seven layers of the ISO/OSI Reference
Model Knowledge
… enumerate the most influential politicians of the 20th
century
… design an interactive website that complies with Web
Accessibility Guidelines Skills
… filter out relevant literature for a given research
problem.
… guide their team mates in improving the architectural
model of a system. Competence
… moderate a virtual task-force meeting

12. Formulating learning outcomes
Guidelines
1. Students are taught about software design flaws
→ Focus on the learning rather than the teaching!
2. Students are able to identify critical design flaws in a given software system
3. Students complete Unit 1 on basics of academic writing
→ Focus on the outcome, not the process!
4. Students are able to explain the basic principles of academic writing
5. Students are able to give a good presentation about their semester project
→ Make sure that achievement can be assessed!
6. Students are able to convey the essential outcomes of their semester project
in a five-minute slideshow presentation

13. Formulating learning outcomes
Activity
Formulate and write in the chat:
If you’re a (PhD) student: a learning outcome of the program or
course you‘re taking.
If you’re a teacher/tutor: a learning outcome of a course you’re
teaching.
Indicate in parentheses whether the outcome is related to:
knowledge (K)
skills (S) or
competence (C)
Example:
Students are able to explain the basic principles of academic writing (K)

14. Teaching methods and learning designs
Problem outline
Alignment of outcomes, methods, and workload
Here: focus on aligning teaching methods with learning outcomes
Goal: facilitating learners in achieving the intended outcomes
Teaching
methods
Learning Workload
ALIGNMENT
outcomes (ECTS)
Assessment

15. Key concepts
ICOPER concept model
Sharable Educational Resources
is a
Assessment TEACHING
Method METHOD
is a
intended defines Learning
Learning result of INSTRUCTIONAL roles Facilitator
imple- imple-
ments Outcome ments MODEL
Learner
is a is a
Assessment LEARNING
Design DESIGN involves
uses
uses instantiation
of
Learning
Content
Learning
Opportunity
instantiation
of is a includes
Stored in Assessment
the OICS Opportunity

16. Teaching methods and learning designs
Examples of Teaching Methods 1/5
Lecture: The teacher is the primary delivery channel for
instructional content – often through presentation.

17. Teaching methods and learning designs
Examples of Teaching Methods 2/5
One-Minute Paper: Learners quickly write thoughts to make
their level of understanding explicit.

18. Teaching methods and learning designs
Examples of Teaching Methods 3/5
Feedback: Reinforce or correct actions performed by
learners.

19. Teaching methods and learning designs
Examples of Teaching Methods 4/5
Think-Pair-Share: To break long presentations, learners first
think about a question individually, then discuss their
thoughts with a partner, and lastly report back to the teacher.

20. Teaching methods and learning designs
Examples of Teaching Methods 5/5
Role Play: is a “dramatized case-study”. Its purpose is to
have members of a group act out roles surrounding a
situation, condition, or circumstance to make different
perspectives on the issue visible.

21. Teaching methods and learning designs
Description of instructional models
ICOPER researched description elements that practitioners
require for browsing, selecting and implementing
instructional models.
General description: title, summary, rationale, subject, learning
outcomes, learner characteristics, group size, duration, setting,
author/licensing
Detailed description: graphical representation, sequence of activities,
roles, assessment method, resources, variations, implementations,
references
Comments: teacher reflections, student feedback, peer review
This allows for a structured human-readable description

22. Teaching methods and learning designs
Description of instructional models
To describe instructional models in an interoperable
machine-readable format, there are several existing
standards and specifications, e.g.:
IMS Learning Design (LD)
SCORM
IMS Simple Sequencing (SS)
In ICOPER:
Instructional models described with IMS LD (other formats allowed),
with LOM metadata record(s) on top
Storage in the Open ICOPER Content Space (OICS)
for OICS/LOM introduction see the replay of Webinar 1; for IMS LD
the upcoming Webinar 3; for application demos Webinar 5.

23. Teaching methods and learning designs
Alignment of Learning Outcomes and Assessment
Teaching
methods
Learning Workload
ALIGNMENT
outcomes (ECTS)
Assessment

24. Teaching methods and learning designs
Alignment of Learning Outcomes and Assessment
Learning outcome:
Students are able to moderate a virtual task-force meeting.
Aligned assessment:
A tutor observes a task-force meeting where a student moderates,
and gives feedback to the student based on pre-defined performance
criteria.
Misaligned assessment:
Students take a multiple-choice test on how to moderate meetings.

25. Teaching methods and learning designs
Alignment of Learning Outcomes and Teaching
Methods: General
General alignment: What do we mean by “alignment” when
talking about teaching methods?
Teaching
methods
Learning Workload
ALIGNMENT
outcomes (ECTS)
Assessment

26. Teaching methods and learning designs
Alignment of Learning Outcomes and Teaching
Methods: General
Learning outcome:
Students are able to moderate a virtual task-force meeting.
Teaching methods:
Lecture
One-minute paper
Feedback
Think-pair-share
Role play

27. Teaching methods and learning designs
Voting Activity
Take a vote:
Which teaching method(s) do you consider a good/best fit for
this learning outcome?
“Students are able to moderate a virtual task-force meeting”
Teaching methods:
Lecture
One-minute paper
Feedback
Think-pair-share
Role play

28. Teaching methods and learning designs
Alignment of Learning Outcomes and
Teaching Methods: Specific
Use taxonomy for alignment, for instance,
Anderson & Krathwohl taxonomy of educational objectives
Students are able to moderate a virtual task-force meeting.
LO:
moderation
Source: Anderson, L. W., & Krathwohl, D. R. (Eds.). (2001). A Taxonomy For Learning, Teaching, And Assessing: A Revision of Bloom's Taxonomy of Educational Objectives
(Complete ed.). New York: Addison Wesley Longman.

29. Teaching methods and learning designs
Alignment of Learning Outcomes and
Teaching Methods: Specific
TM: TM: One- TM: TM: Think- TM:
Lecture Minute-Paper Feedback Pair-Share Role Play
LO:
Moderation
Assessment:
Moderation
Source: Anderson, L. W., & Krathwohl, D. R. (Eds.). (2001). A Taxonomy For Learning, Teaching, And Assessing: A Revision of Bloom's Taxonomy of Educational Objectives
(Complete ed.). New York: Addison Wesley Longman.

30. Teaching methods and learning designs
Teaching methods
What to look after besides learning outcome alignment when
choosing teaching methods?
Allow learners to practice and do things themselves; don’t just “talk”
about it
Plan for social interactions between learners
Support learning process (feedback, scaffolding, coaching)

31. Teaching methods and learning designs
Learning Design
Building a learning design: Adding content and assessment
to the teaching method according to learning outcome.
Generic teaching method
Specific learning design integrating
the teaching method
Integration of a whiteboard
tool to collect reasons

32. Models and standards
Specifications in Focus
Learning Outcomes (IEEE RCD)
Describe and manage Learning Outcome Definitions
ICOPER LOD AP
Learning Opportunities (CEN MLO)
Publish metadata of learning opportunities
ICOPER MLO APP
Personal Achievements (new spec.)
Describe and share information on learners/teacher achieved learning
outcomes and associated evidence, level and context data
Personal Achieved Learning Outcomes (ICOPER PALO)
Backbone for a new CEN specification

33. Models and standards
Metadata for Learning Outcomes
Describes the characteristics of a learning outcomes
Enable storage, referencing, sharing Learning outcomes data

34. Indexed large set of learning outcomes
http://oics.icoper.org/LOD/

35. Models and standards
Sources Learning Outcomes
OpenLearn (OUUK)
All courses have defined learning outcomes
Stored in IEEE LOM metadata classification category
IEEE/ACM Computer Science Curricula (Task Force)
All course descriptions have learning outcomes
PDF file
Used worldwide by many universities
21st Century Engineer (NSF project)
36 generic learning outcomes
Engineering domain
.ppt file
European Computer Science Driving License (ECDL)
Full list of learning outcomes per topic
Excel sheets
WU Vienna – Business Education study programme
Local XML into LOD XML

36. Models and standards
Learning Opportunities
Example:
Course catalog

37. Models and standards
Metadata for Learning Opportunities
Describe an opportunity a learner can follow
Enables publishing metadata across universities/countries

38. Generate MLO Instances
Course catalogs
Local XML format
Transformation into
ICOPER MLO XML
1000s instances
Want to produce MLO
instances from your
course catalogs?…Talk
to us ;-)
http://oics.icoper.org/MLO/

39. Models and standards
Metadata for Personal Achievements
Represent information on achieved/required/desired learning outcomes of a learner. Also teachers
(taught outcomes)
Share data with LMSs, recruitment systems, social applications

40. Models and standards PALO Example
Najjar et al. (2010) A data model for
describing and exchanging Personal
Achieved Learning Outcomes (PALO).
International Journal of IT Standards
and Standardization Research, vol. 8,
pp. 87-104.

41. Models and standards
Generated PALO Data
Data collected across
systems
ATOM/RSS feeds
Shared in
iGoogle
Facebook
LMSs
etc.

42. Learning outcome motivated assessment
Evidence of achievement
Successful completion of learning
opportunity
Evidence that a learning outcome
was achieved/attained
Each learner achievement has an
assessment record
Assessment record is a summary
result of one or more learning
assessments

43. Learning outcome motivated assessment
Approaches
Automatic
System generates assessment records
from assessment results in LMS
Submit data to learner profile when
course is closed
Semi-automatic
Student claimes achievement of
outcomes submit assessment records
to learner profile
Teacher approves assessment records
data submitted to learner profile

44. Closing
More Resources & Feedback
ICOPER Training Infrastructure:
Learning Outcome Course:
http://training.icoper.org/moodle/mod/resource/view.php?id=269
Instructional Modeling Course:
http://training.icoper.org/moodle/mod/resource/view.php?id=270
Your feedback:
What do you take home from this webinar?
Please share via the chat window or raise your hand to
speak.
Feedback form:
http://tinyurl.com/28gjny2

45. ICOPER Webinar Series
Upcoming Webinars
Webinar 3: “eContent for designing and delivering
outcome-based learning opportunities”
February 3, 2011 (next Thursday)
16.00-17.30
Webinar 4: “Learning outcome-based assessment and
applications”
February 10, 2011
16.00-17.30
Webinar 5: “Applications for outcome-based learning”
February 17, 2011
16.00-17.30