Extending Computerized Adaptive Testing to Multiple Objectives: Envisioned on a Case from the Health Care.
•Télécharger en tant que PPTX, PDF•
2 j'aime•632 vues
In the age of information, only personalized learning and education enables the adaption to always changing requirements. In health care, labor forces are especially in a continuous need of improvement and adaption. Maintaining an optimal care sets a dual goal in the education of health professionals: adapt to scientific developments while retaining the compliance with changing laws and regulations. Computerized adaptive testing (CAT), as a dynamic approach to education, is providing here the right adaptivity by optimizing time and precision of learning and scaling to the ability of the learner. This paper will provide an overview on the current state of CAT and connected methodologies and shed light on the potentials for the next generation of adaptive testing, which can support the emergence of novel ways of education. Its strength is shown on a scenario from the health care sector.
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (18)
En vedette
En vedette (18)
Similaire à Extending Computerized Adaptive Testing to Multiple Objectives: Envisioned on a Case from the Health Care.
Similaire à Extending Computerized Adaptive Testing to Multiple Objectives: Envisioned on a Case from the Health Care. (20)
Plus de Eduworks Network
Plus de Eduworks Network (14)
Dernier
Dernier (20)
Extending Computerized Adaptive Testing to Multiple Objectives: Envisioned on a Case from the Health Care.
- 1. - 24 Extending Computerized Adaptive Testing to Multiple Objectives: Envisioned on a Case from the Health Care. Christian Weber, Réka Vas Egovis 2014
- 2. - 24 Background: The eduworks network University of Salamanca 2014.10.03. Corvinno Technology Transfer Center 2
- 3. - 24 For a better educational testing Know-ledge Culture Cognitive Type Matching Labour Education Human profile 2014.10.03. Corvinno Technology Transfer Center 3
- 4. - 24 The health perspective 2014.10.03. Corvinno Technology Transfer Center 4
- 5. - 24 Need for ongoing training in the health care • Immense pressure on doctors within their daily practice and as well in terms of an (expected) ongoing training E.g. obligation for training in Germany since 2005 • More and more tasks are transferred to nurses or other caretaker E.g. they have a major role in Alzheimer early detection (World Alzheimer Report 2011) • But .. a focus for ongoing training for nurses is missing need for support in education But the need comes with special demands.. 2014.10.03. Corvinno Technology Transfer Center 5
- 6. - 24 The demands of medical training • Time is restricted For education (but also for the support of domain experts) • The learning has an educational context • Must-know knowledge regulations & mandatory knowledge • Should-know knowledge new insights & specializations • The learning has a cultural context Care taker and nurses have different cultural backgrounds with different perception to learning. patients 2014.10.03. Corvinno Technology Transfer Center 6
- 7. Time to learn Time to test - 24 Designing a new test Know-ledge Restricted time Culture Influence Learning goal Nurses Labour Education Human profile 2014.10.03. Corvinno Technology Transfer Center 7
- 8. - 24 Computerized Adaptive Testing - CAT pass fail pass pass pass pass fail fail Ab i l i t y L e v e l pass Te s t I t ems 100 I t em Di f f i c u l t y 2014.10.03. Corvinno Technology Transfer Center 8
- 9. Item response function (3PL) as the probability that a person with ability “theta” answers an item • b – difficulty, (half way between c(min) and 1) • a – discrimination, (slope at b, differentiating the users) • c – chance for guessing (building on a logistic function) 2014.10.03. Corvinno Technology Transfer Center 9 - 24
- 10. - 24 Limitations of question organization Know-ledge Culture Cognitive Type Q70 Q40 Q60 Q60 Q60 Q60 Q40 Q65 Q70 Q50 2014.10.03. Corvinno Technology Transfer Center 10
- 11. - 24 Limitations of question organization Know-ledge Add Meta Information • Cultural context • Learning type • Obligation yes/no •... Culture Cognitive Type Q40 Q60 Q60 Q60 Q70 Q60 Q40 Q65 Q70 Q50 2014.10.03. Corvinno Technology Transfer Center 11
- 12. - 24 Limitations of question organization Know-ledge Exploring the user on the education Add Meta Information • Cultural context • Learning type • Obligation yes/no •... Culture Cognitive Type Q40 Q60 Q60 Q60 Q70 Q60 Q40 Q65 Q70 Q50 User domain Educational domain 2014.10.03. Corvinno Technology Transfer Center 12
- 13. - 24 Overcoming limitations • How to grasp the user knowledge in the best possible way? Framing question with no final answer. • How to add more context to it? Extending the structure and indicators. • How to detect also his/her understanding of an area? Using knowledge, structured and in relation. 2014.10.03. Corvinno Technology Transfer Center 13
- 14. - 24 The demands of medical training - revisited • Time is restricted For education (but also for the support of domain experts) • The learning has an educational context • Must-know knowledge as meta data regulations & mandatory knowledge • Should-know knowledge as meta data new insights & specializations • The learning has a cultural context Care taker and nurses have different cultural backgrounds with different perception to learning. through modelling 2014.10.03. Corvinno Technology Transfer Center 14
- 15. - 24 Differential Item Functioning - DIF • Occurs when people of different groups have a different probability to answer correct, independent of the same underlying ability. Groups in terms of culture, educational character or learning style • Detection normally through statistical analysis of the performance on the test. The designer view: Detection only if a grouping context is known.. 2014.10.03. Corvinno Technology Transfer Center 15
- 16. - 24 DIF context through virtualization • Instead of removing DIF items, redundant item modelling (Makransky and Glas) • DIF biasing Harder or easier (difficult level) • Alternatively: Redundant question and real structure But how could a structure look like?.. 2014.10.03. Corvinno Technology Transfer Center 16
- 17. - 24 Structure - A new concept of layers Q12 Q13 Q21 Q11 Q22 Q32 Q33 Q31 Qxy Qxy+ 1 Q42 Q41 1 2 3 4 y e.g. item difficulty, cognitive type.. educational domain complexity / relations difficulty new ?= f(diff Q xy , compl N x) ++ meta-information 2014.10.03. Corvinno Technology Transfer Center 17
- 18. - 24 Testing strategy: Thinking about a new frame with (Complex) Event Processing (CEP) Event Agent Agent Consumer Consumer Event Consumer the system behavior as a stream of data Pattern Detection Trans-for-mation Filter Agent Data Data Event producer 2014.10.03. Corvinno Technology Transfer Center 18
- 19. e.g learning type compatible to job x fast tester - 24 A new frame for CAT with context Domain Ontology change/modify Testing Strategies Event Consumer Meta- Strategies Q 3 2 Q 3 3 Q xy+ 1 Q 4 1 Q 4 2 3 4 y Data Context Agent Context Agent Context Agent the system behavior as a stream of data Intelligent Test Engine Q x y Event producer e.g right/wrong ~perform well time for answer Event Q 1 2 Q Q 2 1 Q 2 2 Q 3 1 2 Data + access to the ontology Q 1 1 1 3 1 2014.10.03. Corvinno Technology Transfer Center 19
- 20. - 24 Exploration, yes - but where (how) to go.. Q Q Q Q ? ? ? Testing Strategies Meta- Strategies Intelligent Test Engine 2014.10.03. Corvinno Technology Transfer Center 20
- 21. - 24 Next level questions.. • How are the rules to switch and modify the testing strategy and how to manage it on the fly? ..depth/breadth first, related concepts, maintain main test • What is the best choice to model a context parameters for the strategic exploration? • What is the best fusion of parameters? .. psychometrics and graph/ontology based complexity measures 2014.10.03. Corvinno Technology Transfer Center 21
- 22. - 24 Data Sources • Now: • Educational knowledge within the domain ontology as classes and relations • Logging feedback from the actual testing • Future: • Target-oriented performance feedback from the user To: • grouping of users after special criteria • Feedback on the content – e.g. Improving meta-data on the nodes 2014.10.03. Corvinno Technology Transfer Center 22
- 23. - 24 Summary • Need for a crisp testing to detect training potential in knowledge intense professions. Especially in the health care sector • CAT is a suitable solution for educational testing.. ..but there is room for improvement for a context aware testing • A new frame and thinking could enable a next generation of context sensitive tests, with.. • structured backing knowledge (ontology) • a novel testing behaviour (modifying test behaviour online) 2014.10.03. Corvinno Technology Transfer Center 23
- 24. - 24 Thank you for your attention! Corvinno Technology Transfer Center H-1093 Budapest, Közraktár utca 12/a. Tel: +36 1 210 80 62 http://www.corvinno.com
Notes de l'éditeur
- In the UK nurses also have the obligation to attend to an ongoing vocational training
- CAT has the pros – shorter test time, higher precision, a random test assesment Cons – items need to be calibrated, time not fixed and depends on the user, tests are not directly comparable (assessment tests)