1. Open Standards and Specifications for Outcome-based Assessment Israel Gutiérrez Universidad Carlos III de Madrid Madrid, April 12th 2010

2. Standards and Specifications

3. Computer-based assessment Reduce effort and costs: permit continuous assessment Assessment re-use: interoperability needed Variety of assessment formats Proprietary: MoodleXML, HotPotatoes, Blackboard Ad hoc: OpenMark IMS QTI Open Standards and Specifications for Assessment

4. Outcome-based Assessment Universities developing an outcome-based learning strategy Adaptation to European Space for Higher Education Improve course structure for accreditation (ABET) Creation of new courses Learning outcomes Assessment as a key process

5. Outline Analysis of assessment standards and specifications Usage of assessment formats in Europe Interoperability issues Next generation assessment scenarios Best practices

6. Methodology Analysis of Assessment Standards and Specifications Metamodels (UML diagrams) from MDA Understand better studied formats Abstraction from syntax (XML binding) Focus on content

7. Studied Assessment Formats Assessment purpose General purpose IMS QTI FML OpenMark SuML QAML

8. Qualitative Comparison Key features Multiple response allowance Correct response indication Metadata capabilities Hybrid question management Question types (from FREMA) Constructed: essay, short answer, FIB, etc. Constrained: MCQ, MRQ, TF, matching, etc. Miscellaneous: simulations, practical sessions

9. Assessment Formats Comparison

11. Personal interview

12. In-depthreport

13. Qualitativeanalysis

14. Samplecharacteristics

15. Highereducation

16. Multicultural

17. MultilingualSample size

18. Geographical Distribution

20. Scarce acceptance of IMS QTI specification

21. High technological diversity

22. LMS, ad-hoc tools, etc.

23. Diversity of assessment methodologies

25. Assessment Interoperability Moodle QTIlrn QTIclix QTIexport QTIMoodle QTI 1.2.1 QTI 1.2.1 QTI 1.2.1 .LRN Clix

27. Adaptation

30. Gathering of recommendations

31. Open ICOPER Content Space

33. Outcome-based Assessment App Use cases (cont.) Annotate assessment resources Peer review Teacher reflection Student feedback Update learner’s profile Stakeholders Faculty members Course developers Online demo: http://mozart.gast.it.uc3m.es:9000/

34. Outcome-based Assessment App Learningoutcome-based search Implementation for dotLRN based on ICOPER LOM profile Learningoutcome profile update

35. Outcome-based Assessment App: Evaluation Engineering evaluation Implementers Tool developers End user evaluation Faculty members Course developers Epistemological evaluation Researchers Standardization bodies

36. ICOPER Assessment app Assessment resources should include information about Intended learning outcomes Recommended assessment methods Annotations about resources Usage of integrated environment for outcome-based assessment

37. Best Practices for Assessment

38. Thank you for your attention! http://www.it.uc3m.es/igrojas/index.html igutierrez@inv.it.uc3m.es