Presentation at the ICDE2017 World Conference on Online Learning

1. Recent Developments in
Assessment
Stephen Murgatroyd, PhD
Chief Innovation Officer
Contact North | Contact Nord

2. This
Presentation..
Based on a paper in this book, just published:
 Anshuman Khare and Deborah Hurst (Eds): On the
Line - Business Education in the Digital Age. New
York: Springer, 2017.
 Murgatroyd, S. (2017) New Approaches to the
Assessment of Learning - New Possibilities for Business
Education.

3. 1.OnDemand
Assessment
 Assessment used to be available at the discretion of the
institution - fixed dates.
 Growing number of examples of assessment on demand
- anytime, anywhere - based on competencies and
capabilities.

4. 2.Automated
Assessment
Generation
 Varafy’s assessment generating platform is able to
generate over 400,000 different versions of the same
assessment activity in minutes, using examples and
rubrics - enabling 24x7 assessment of the same skills,
knowledge or capability.
 Free to use platforms, such as TAO, enable:
 Enable them to develop a much wider range of item types
(including technology-enhanced items), and to embed
graphics and videos. Ideally, this would expand their ability
to measure a range of skills as well as providing an authentic
way for students to respond.
 Support the measurement of higher-order skills required by
the Common Core standards.
 Be certified to open standards, such as QTI, allowing ease of
item export into the existing UMass item bank.
 Support item rendering across a wide range of test-taking
devices, e.g., desktops, laptops, and tablets.

5. 3.Automated
Markingof
Assessments
 Automated marking of multiple choice assessments not
new.
 But now AI/machine intelligent systems can mark a
variety of forms of assessment, including:
 Free form essays, including creative writing
 Short answer questions.
 Computer coding and similar tasks
 Sample tests marked by humans are fed into an
algorithm that learns how to recognise quality
responses by reverse-engineering scoring decision

6. 4.VideoBased
Assessmentof
Competencies
 Valid-8 - part of the Vamatric suite of offerings - is able
to measure, record and confirm the exact levels of
achievement of each individual, apprentice or student.
 IBM Watson permits students and their instructors to
identify the exact moments in a video when a
particular body of knowledge, capability or skill is
demonstrated automatically.
 All of these video moments can be validated against
competency frameworks and protocols - validation is
the key component here.

7. 5.PeertoPeer
Assessment
 Use of peer assessment is growing, especially in
relation to MOOC’s for credit.
 A recent study by Strang (2015) shows:
 Consistency of peer assessment across a number of peers
 No significant difference between peer assessment and
instructor assessment

8. 6.Adaptive
Assessment
 Assessment for learning (as opposed to the assessment
of learning) which provides both instant feedback and
is capable of adjusting subsequent learning tasks is
also embedded in all major LMS systems.
 The next stage of this field is to accelerate learning
through adaptive algorithms - accurate
multidimensional assessment linked to varied routes to
the same learning outcome.

9. 7.Growthofe-
portfolios
 Institutions and students gather evidence of
capabilities and go “beyond the transcript”.
 E.g RIIPEN enables faculty + industry colleagues to co-
create assessments, offer feedback and grades to
students and capture learning on RIIPEN’s e-Portfolio.

10. 8.Assessment
Based
Qualifications
 University of Wisconsin Flex Degrees - competency
based assessment (no courses, no teaching, on demand
assessment) - BS Business, BS Nursing, BS
Information Science and Technology, Associate of Arts
and Science.
 Many other institutions also following.

11. 9.TQF‘s
 Trans-National Qualification Frameworks (TQF’s)
permit the acceptance of credit / assessments between
nations - e.g. Virtual University of the Small States of
the Commonwealth (VUSSC).

12. 10.Security
 Significant developments with respect to the security
of online assessments:
 Finger-print recognition
 Facial recognition software
 Pattern recognition systems - is the style and approach
used similar to that normally used by this registered
user?

13. ARenaissancein
Assessment
 The only competitive advantage a University or college
has is the credibility of their assessments of capability
and competence.
 These (and related) developments are the launch-
points for a renaissance in assessment - see
 Hill, P. and Barber, M. (2014) Preparing for a
Renaissance in Assessment. Pearson.