WCET 2008 Conference presentation

1. Quality? Who Says?
WCET Annual Conference
November 9, 2007

2. Objectives
 Walk away with multiple views on rubrics for
assessing quality of online instruction
 Take ideas from three different quality assurance
methods
 Recognize the impact and value of a quality
assurance program

3. Our Panel
Kay Kane, Quality Matters, MarylandOnline
Allison Peterson, Texas Woman’s University
David Curtis, Park University
Michael Anderson, UT TeleCampus

4. Agenda
Introduce QA programs
Address Key Issues:
Impact on participation
Study outcomes
Value of the process
Lessons Learned
Q&A
Final Thoughts

5. Quality Assurance
Programs
What’s it all about?
How do we do it?

6. Texas Woman’s
University
A Brief Introduction to the Quality
Enhancement Plan

7. Texas Woman’s University
 Texas Higher Education Coordinating Board
(THECB)
 Principles of Good Practice (PGP)
 Institutional Context & Commitment
 Quality Enhancement Plan (QEP)
 Quality Matters (QM)

8. Quality Matters

9. Course
Course Meets
Quality Expectations Peer Review
Course Feedback
Revision

10. The Rubric
 Eight standards:
 Course Overview and Introduction
 Learning Objectives
 Assessment and Measurement Key components
 Resources and Materials must align.
 Learner Interaction
 Course Technology
 Learner Support
 ADA Compliance

11. QM Strengths
Grounded in:
 research literature
 national standards of best practice
 instructional design principles
www.QualityMatters.org

12. Park University
By the Numbers!
 Online Students = 52,000+
 This counts all students taking all classes, so a student taking
four online class is counted as four (4) students.
 Online Courses = 322 Discrete Courses
 A total of 524 Sections
 Online Faculty = 455 +

13. Quality Starts…
…and doesn’t stop
 Training
 Developing
 Reviewing

14. Key Issues
 Impact on participation
 Study outcomes
 Value of the process
 Lessons Learned

15. Impact of
Participation on
Faculty Peer
Reviewers

16. Year One – Spring 2007
 January 2007 – August 2007
 10 Faculty
 6 Staff
 QM Reviewer Training
 Revise 1 Course – 14 Essential Standards
 $400 Stipend

17. Year One – Results
 3 Course Reviews
 1 Review and Recognized
 1 in Review
 1 Preparing for Review
 2 Course Reviewers
 1 Summer
 1 Fall
 3 Potential Reviewers

18. Year Two – Fall 2007
 September 2007 – August 2008
 25 Faculty
 1 Staff
 QM Reviewer Training
 Revise 1 Course – All 40 Standards
 Review 1 Course
 $400 Stipend

19. Year Three – Fall 2008
 September 2008 – August 2009
 35 Faculty
 QM Reviewer Training
 Revise 1 Course – All 40 Standards
 Review 1 Course
 $400 Stipend

20. Training Changes
 Online Educator Symposium
 Online Course
 Presentations
 One-to-one consultations

21. Quality Matters
Study Outcomes

22. FY07 Course Reviews
85 courses reviewed FY07
 23 different institutions
 45 QM Managed
 40 Independent

23. FY07 Course Reviews
Upon initial review:
 38% met expectations
 62% do not yet meet expectations
 30% of these were revised and met expectations
Currently Recognized by QM: 56%

24. Common Themes
 Common areas for course improvement
 These are potential targets for:
 faculty training
 special attention in initial course development

25. Common Areas for Improvement
Module objectives measurable/consistent with course objectives 45%
"Self-check“/practice assignments for timely feedback 42%
Instructions on how to meet the learning objectives 40%
Equivalent alternatives to auditory and visual content 39%
Instructor response and availability 38%
Requirements/skills/prerequisites clearly stated 35%
Navigational instructions 32%
Criteria for the evaluation of students' work and participation 31%

26. Impact on Faculty
and Reviewers during Grant
Faculty said as a result of the review:
91% made changes in the course
89% felt quality of course design improved
Reviewers said as a result of the review:
73% made changes to own online course
100% said valuable professional development activity

27. Student Learning & Interaction
College of Southern MD
 General education IT course (100+ students)
 Revisions made in all learning modules:
 created Learning Guides (explicit roadmap)
 reorganized presentation and design
 added classroom assessment techniques (CATs)
 Increase in:
 the frequency of content access by students
 % of A grades awarded in the course

28. Impact of Navigation Directions
Prince George’s CC, MD
 Standard I.1 – course navigation
 After revision of navigation bar:
 Students asked fewer questions: course
navigation, locating information, course
requirements
 Less student concern about what they needed
to do to succeed

29. Student Perceptions & Priorities
SUNY Canton
 3 standards most often noted as not being met:
 Course navigation directions
 Assessments & measurement provide feedback to
students
 Clear standards for instructor response & availability

30. Park University
Value of the Process

31. Value of the Process
 The Administration has always been concerned
with maintaining academic integrity in all
programs, on-ground and online.
 The goal was never to be “the biggest”, just the
best. And by being the best, we might be the
biggest 

32. Value of the Process
 The adoption of the Quality Matters program and
ideology assures that the design and development of
online classes will meet the highest quality standards of
academic excellence.
 The “master course” structure for online courses,
combined with using the QM rubric for online reviews,
assures that a course AND all sections of that course
will met QM Standards and Expectations.

33. Value of the Process
 SO 141 – Introduction to Sociology example

34. Quality (at) Training
…get off on the right foot
Training workshop required one year out
 Course examples
 Content design
 Interactivity (communication)
 Assessment
 Multiple online courses
 Principles of Good Practice Self-Study
 Instructional Design Tutorial

35. Quality Development
…from the first word
Developing in stages
 Checkpoint #1
 Syllabus, sample lesson
 Checkpoint #2
 ½ of the course, Mac
 Content, interaction, assessment
 Copy editing

36. Quality Reviews
…before the course runs
Reviewing from multiple perspectives
 Technical review (external)
 Content
 Communication
 Assessments
 Services
 Links
 Final Check (internal)

37. Quality Evolves
…or becomes extinct
Critical factors
 Faculty buy-in
 Student satisfaction
 Objective
 Pre-tech reviews
 Tech reviews
 Subjective
 Checkpoints
 Rubrics

38. Technical calls &
student evaluations
200
14.00%
180 Technical issues
Courses
12.00%
1
160 in courses
10.00%
140
120
8.00%
100
6.00%
80
60
4.00%
40 Student design
2.00%
20 complaints2
0.00%
0
SP00 FA00 SP01FA01 SP02 FA02 SP03 FA03 SP04 FA04 SP05 FA05 SP06
SP00 FA00 SP01 FA01 SP02 FA02 SP03 FA03 SP04 FA04 SP05 FA05 SP06 FA06
FA06 SP07
SP07
1 Help Desk tickets as 2 Studentnegative
a percent of courses comments on design as a
percent of evaluations

39. Quality Looks Ahead
…or falls behind
Planned changes
 Training
 Best practices
 Advanced workshops
 Development
 Community of Practice
 Template/technique sharing
 Reviewing
 cQual application

40. Wrap Up / Q&A

41. Final Thoughts…

42. Thank you!