Presented by Dan Thompson, Coordinator of Instructional Design, Assessment, and Educational Technology, Oklahoma State University Center for Health Sciences
In order to satisfy accreditation requirements and appropriately map the curriculum, Oklahoma State University College of Osteopathic Medicine is using ExamSoft’s category feature to map assessment items to the appropriate learning objective for each lesson. These items are then mapped with the instructional method using the Bloom's level of the assessment item. Mapping assessment items with the objectives and instructional method allows the instructional design team and faculty to link these items back to the course objectives, course goals, program goals, and school mission. This mapping effort was part of a recent accreditation visit and proved to be a vital tool in showing our accrediting body how we "close the loop" from the beginning of the program to the end of each course assessment. This was well received during the accreditation visit and has prompted growth of this mapping effort throughout the curriculum.
Satisfying Accreditation Requirements Through Assessment Item Categorization
1. Dan Thompson
Coordinator of Instructional Design, Assessment, and
Educational Technology
Oklahoma State University Center for Health Sciences
2. Session Objectives
At the completion of this session,
participants will be able to:
Describe category mapping and how to
appropriately use this feature in curriculum
mapping for accreditation
Utilize category mapping to appropriately align
assessment items with necessary accreditation
requirements
Create a simple categorization process that is
conducive for curricular mapping to assist in
meeting accreditation needs
3. What are Categories?
Each item in ExamSoft can be tagged to
categories that will allow us to identify the
exact subject matter, difficulty level (ex.
Bloom’s), author, instructional method
implemented, style of question, learning
objective(s) being assessed, etc.
Items can be placed into an unlimited number of
categories
Stats can be acquired based on each
category to learn the desired information.
4. How to Categorize
Questions
Questions can be easily categorized when the question
is being created in ExamSoft
Questions that have already been created and used in
assessments can be categorized as well
Statistical analyses of these questions can be done
retroactively
Categories should be pre-made, which will allow
question creators to simply click on the categories and
add them for each question
Categories work as a folder structure if needed
Very helpful in staying organized in curricular mapping and
preparing for accreditation!
5. Faculty/Staff Roles for
Curricular Mapping
Primary item categorization in the system is
completed by instructional design staff
Faculty and academic support staff contribute
to this effort as well
**Recommendation: create a process first and
limit the number of individuals that work with
this process.
○ Improves consistency and program organization
6. Faculty/Staff Roles for Curricular
Mapping
Content experts are needed for this process
as assessment items need to be matched to
learning objectives
Assessment items can be mapped to more than
one learning objective if applicable
Category statistics are monitored and
reviewed to assure program effectiveness and
to identify areas for improvement
7. Categorization Structure
Methodology
Systems-based curriculum
For curricular mapping and in
preparation for accreditation, items
are categorized by:
Discipline
○ Example: pharmacology, physiology, histology
○ Case-based clinical lectures/sessions and
clinical problem solving group sessions
○ Indicates that we address the content (by
quantity) in our curriculum appropriately
8.
9. Categorization Structure
Methodology
Items are also mapped to the learning
objectives they address
Functions on two levels:
Small scale:
○ Indicates that we are assessing our learning objectives
○ The mapped items can come from quizzes, exams, and/or
other assignments
○ Shows student outcomes by learning objective – are students
appropriately meeting the learning objectives and therefore
reaching our course goals? This allows us to prove that
statistically
10. Categorization Structure
Methodology
Large scale:
Completes the back end portion of the
curricular mapping for accreditation
Throughout the curriculum mapping for
accreditation process, the focus is to
completely map the curriculum from the
beginning (program mission) to the end
(course assessments)
This ties the process together and closes the
loop on curricular mapping for accreditation
11.
12. Moving Forward
Our categorizing of assessment items
with learning objectives is currently
growing
Once completed, an overall view of how
we are meeting learning objectives (and
therefore course goals, program goals,
mission, etc) will be available
This initiative is being continued after
receiving positive feedback
13. Tips/Tricks
Begin this process as early as possible
Establish the process before categorizing any
exam items
Identify individual roles in the mapping process
Create the folder structure in advance
○ *The folder structure can be altered, so this process can be
flexible to meet an evolving curriculum
Be proactive – ask/require faculty to identify which
learning objectives their assessment items map to when
questions are entered into ExamSoft
This was not done at our institution initially
Retroactively categorizing items is an option, but is a more
difficult task
14. Unintended (Positive!)
Outcomes
Created an opportunity to improve and
streamline our learning objectives to
improve communication with students
Improved faculty knowledge in the
educational process – built in faculty
development!