This document outlines quality assurance processes for online course development at Ashford University and the University of Rockies. It discusses: - Institution overviews and curriculum team structures - Curriculum and course development tools/templates/guidance including outcomes mapping, style guides, development checklists - Assessment development including rubrics, outcomes mapping, and electronic grading rubrics - Internal quality assurance reviews at both the content and instructional design levels - External quality assurance including Quality Matters reviews - Faculty training models including QM certification - Research on the impact of quality assurance processes such as the effect of a course development template on alignment

1. ASSURING QUALITY IN LARGE-
SCALE ONLINE COURSE
DEVELOPMENT
Tina Parscal, PhD
Provost
tina.parscal@rockies.edu
Deborah Riemer
VP of Assessment & Academic Quality
deborah.riemer@ashford.edu

2. Presentation Outline
Institution Overviews
Curriculum & Course Development (Tools & Templates)
Assessment Development (Tools & Templates)
Internal QA Reviews
External QA Reviews
Development Models
Faculty Training
Research and Evaluation

3. Institution Overviews

4. Ashford University
Founded in 1918
Online Figures Curriculum Team
 88,000+ students  Faculty (SMEs)
 2200+ faculty  4 Executive Deans
 VP of Asmt & Quality
 606 courses
 Assessment Manager
 58 programs
 Quality Manager
 500-750 course sections starts
 4 Program Managers
per week
 5 Instructional Designers
 7 Curriculum Coordinators
 2 Learning Tech Specialists
 2 Doc Quality Coordinators
 4 Assessment Analysts

5. University of Rockies
Founded in 1998 – became UOR in 2007
Online Figures Curriculum Team
 2000+ Students  Director of Curriculum
Development
 275 Faculty
 Director of Academic Quality
 214 courses
 Assessment Faculty
 5 programs  3 Instructional Designers
 15 course sections starts per  2 Curriculum Coordinators
week  1 Faculty/SME/Course
Developer (per course)
 1 External QA Assurance (per
course)
 14 Faculty Curriculum Area
Coordinators

6. Quality is Intentional
Course Development
Assessment Development
Internal QA Reviews
External QA Reviews
We weave it into the process . . .
Development Models
Faculty Development
Research & Evaluation

7. Curriculum & Course Development
Guidance, Templates, Tools, Model

8. Curriculum Development
Faculty Driven
Centrally
Developed

9. Curriculum Development
Outcomes Based
Curriculum &
Course Alignment

10. Course Development Training
Guidance, Templates, and Tools
Prepping the
Team

11. Course Development
Mapping and Organizing
Guidance, Templates, and Tools
Learning
Outcome
Alignment
Ensuring
Quality
Design

12. Course Development Tools
Development Template and Blog Guidance

13. Course Development Tools
Wiki Checklist

14. Course Development
Style Guide Blog & eCollege Template
Guidance, Templates, and Tools
Consistent
Presentation
Style Guide
and
Documented
Processes

15. Assessment Development

16. Assessment Development
Waypoint Assessment Tool
Linking to
• Richer feedback to Outcomes • Data will be used to
students impacting inform change in
achievement and success • Alignment of rubric curriculum and processes
• Enhanced experience for criteria to learning resulting in improved
faculty and students outcomes and experience for students
competencies
Measures of
Grading Rubric
Student Learning

17. Assessment Development
Grading Rubric Development Checklist and Blog
Guidance, Templates, and Tools
Grading Rubric
Development
Guidance

18. Assessment Development
Waypoint Electronic Grading Rubric
Guidance, Templates, and Tools

19. Assessment Management
TaskStream Outcomes Mapping
Guidance, Templates, and Tools

20. Assessment Management
TaskStream Assessment Plan

21. Internal QA Reviews

22. UoR Internal QA Reviews
Content Area
Coordinator reviews for:
Course Plan
Course Design & Topical
Organizer
Learning Resources
Outcomes
Assessment Strategy
Discussion Prompts

23. AU Internal QA Reviews

24. External QA Reviews

25. External QA Reviews
Quality Matters

26. External QA Reviews
UoR Weekly Internal Reports for Stakeholders

27. External QA Reviews
Quality Matters Blog
Positive
Feedback
Communicating
with the Review
Team

28. External QA Reviews
Organizing QM with Access

29. External QA Reviews
AU Weekly Report Emailed to Stakeholders

30. Development Models

31. Ashford University
Course Development Model
Development Model
Course Development Approval -
SMEs Selected Approval – ID
Identified Period Dean
• Pre-Dev • Pre-Dev • Week 1-4 • Week 5 • Week 6
Final
Rubrics Course Built Final
Course QA Adjustments
Developed in LMS Approval
Made
• Weeks 7-8 • Weeks 9-10 • Week 11 • Week 12 • Week 13
Course Ready Quality Matters
for Production Review

32. University of the Rockies
Course Development Model
Course Development Model
Development
Developers Learning
New Course 1st Draft Final Draft
(SMEs) Resources
Approved Approved by Review by ID
Selected Selected
ID/CAC
• Pre-Dev • Pre-Dev • Week 1-2 • Week 3-4 • Week 3-5
Course Course Quality Final
Approved by Loaded into Assurance Adjustments Final Approval
CAC LMS Review (QA) Made
• Weeks 5-6 • Weeks 6-7 • Week 7 • Week 8 • Week 8-9
Selected for
Course Ready for Quality
Online Teaching Matters
Review

33. Faculty Training

34. Faculty Training
Integrating QM awareness into faculty
and staff responsibilities . . .
Faculty QM peer reviewer certified Assessment Analysts
Faculty QM master reviewer certified Instructional Designers
Faculty Trainers Curriculum Coordinators

35. Research & Evaluation

36. Template Survey
Guidance, Templates, and Tools

37. The University of Rockies has implemented a course
development template that supports the Quality Matters
principle of alignment. The template requires course
developers to explicitly state how the
discussions, activities, and written assignments align
with the weekly and course outcomes. This template is
an improved version of the original template where
explicit alignment was not required. As a quality
assurance reviewer I have noted the positive impact this
template has had on course quality and alignment of
essential course elements. As a result the courses will
provide a rich learning experience.
- Carole Hruskocy, Ph.D.

38. Quality Matters Grant
Action-Based Research Grant on Teaching and Learning
Research Questions
1. What is the extent to
which QM certification
impact student
perceptions of a course?
2. Which standards have
the greatest impact on
student perceptions of a
course?
3. What do instructors
perceive as the positive
and negative aspects of
teaching the same
course before and after
the QM course review
process?

39. Emergent Research
Quality Matters Trends
QM Standard
Peer Review
Feedback

40. Questions
Tina.Parscal@Rockies.edu
Deborah.Riemer@Ashford.edu