2. What is Quality Matters?
Quality Matters is an inter-institutional
peer review process that is dedicated to
the continuous improvement of online
course design.
3. The Evolution of Quality Matters
• Rapid growth of online education
• Persistent questions regarding quality of online and
hybrid/blended courses
• U.S. Department of Education Fund for the
Improvement of Post-Secondary Education (FIPSE)
awarded a grant to MarylandOnline, Inc.
4. The Evolution of Quality Matters
• The rubric and materials
evolved from that grant
• Quality Matters is now a
self-supporting non-profit
organization dedicated to
quality in online education
6. Principles of Quality Matters
Continuous
• The process is designed to ensure that all reviewed
courses will eventually meet expectations.
• The rubric-based review is integral to a continuous
quality improvement process.
Centered
Collegial
Collaborative
7. Principles of Quality Matters
Continuous
Centered
• On research: The development of the rubric is
based on national standards of best practice, the
research literature, and instructional design
principles.
Collegial
Collaborative
8. Principles of Quality Matters
Continuous
Centered
Collegial
• A Quality Matters review is part of a faculty-driven,
peer review process.
• The review process is intended to be diagnostic
and collegial, not evaluative and judgmental
Collaborative
9. Principles of Quality Matters
Continuous
Centered
Collegial
Collaborative
• The review is based on collaboratively identified
evidence found in the course.
• The review is flexible and not prescriptive.
• The review team consists of three experienced online
instructors as reviewers along with the course faculty
developer.
12. Design vs. Delivery
Example Design Delivery
Example 1: You prepare for The "delivery" phase
Real Life ("design") a party by begins when the guest
(Party) sending invitations, rings the
shopping, cleaning, doorbell. Your plans
preparing the pool or may or may not work
the grill, etc. out (a guest spills
wine, food runs out,
etc.).
13. Design vs. Delivery
Example Design Delivery
Example 2: You plan your discussion Delivery begins when
Online board by telling the students arrive:
Classroom students how they how and when your
should participate and actually participate in
how they can expect the discussion; the
you to participate; how quality and timeliness
grades or points will be of student responses,
assigned. etc.
15. QM’s Standard of Quality – 85%
•Peer reviewers are
experienced online faculty
trained to apply the rubric
•QM recommends
reviewing “mature” courses
that have been taught at
least two semesters
16. QM’s Standard of Quality – 85%
Two 85% factors to consider:
1. Course must earn 81 out
of 95 possible points
(about 85%)
2. Reviewers use the 85%
threshold to determine if
the specific standard is
met (not 100% for the
yes/no decision)
17. QM’s Standard of Quality – 85%
• Chemistry class
• 15 course-level
objectives
• Only 13 are measurable
• Still meets the 85%
standard
18. QM Process
Tools include:
• Extensive annotations
• Training
• Three experienced online faculty
members serve on the peer
review team
• QM Process relies on the
collective experience
19. Quality Matters
Overview and Principles
Please close this window to return
to your workshop in Moodle.
Notes de l'éditeur
This presentation focuses on the main elements of the Quality Matters program. This Module 1 Lesson introduces those elements that will be covered again in more detail later in the course.
Quality Matters is an inter-institutional peer review process that is dedicated to the continuous improvement of online course design. This overview focuses on the basics of the rubric, the process, and the principles behind it.
The rapid growth of online education has forced institutions, faculty, and students to quickly adopt and adapt to this new mode of education. Given its recent evolution, there are people who continue to question the quality of education achieved through the online format.In an effort to address the concerns about quality online education, the U.S. Department of Education's Fund for the Improvement of Post-Secondary Education (FIPSE) awarded a three year grant in 2003 to MarylandOnline, Inc., Maryland's higher education distance learning consortium, as a means to begin to address the issue of online course quality assurance.
The Rubric and materials you are using in this course evolved from that grant, originally titled Quality Matters: Inter-Institutional Quality Assurance in Online Learning. Over time, Quality Matters has become a self-supporting, non-profit organization.
Quality Matters is based on four primary principles – it is continuous, centered, collegial, and collaborative.
It is continuous. The process is designed to ensure that all reviewed courses will eventually meet expectations.The rubric-based review is integral to a continuous quality improvement process.
It is centeredon research: The development of the rubric is based on national standards of best practice, the research literature, and instructional design principles.
It is collegial. A Quality Matters review is part of a faculty-driven, peer review process.The review process is intended to be diagnostic and collegial, not evaluative and judgmental
It is collaborative.The review is based on collaboratively identified evidence found in the course rather than the personal preference of an individual reviewer.The review is flexible and not prescriptive (many ways to meet each standard).The review team consists of three experienced online instructors as reviewers along with the course faculty developer.
Integral to the QM process is the belief that there are many factors that affect the quality of an online course. The many elements that can impact the quality of an online course includes course design, course delivery, course content, institutional infrastructure, the Learning Management System, Faculty Readiness and Student Readiness.Of all of the factors listed, QM's work addresses only one aspect of online course quality -- course design. The QM program does not address in detail any of the other factors. Most specifically, it does not address anything related to course delivery (how the instructor actually teaches the course). The evaluation of course delivery resides within each institution itself.
One of the key QM concepts is the distinction between course design and course delivery. QM recognizes that this is a fine line, but course design is the primary emphasis during a review.
Let’s look at two examples to illustrate this – one from “Real Life”, and one from the online classroom. In the “real life” example, you prepare – or design – the party by sending invitations, shopping, cleaning, preparing the pool or the grill. The delivery phase is when the guest rings the bell. Plans may or may not work out.
Our second example is in the online classroom. You plan your discussion board by telling students how they should participate and how they can expect you to participate; how grades or points will be assigned. The delivery begins when the students arrive: how and when your actually participate in the discussion; the quality and timeliness of student responses, etc.
During a course review, Quality Matters (QM) is not looking for "just good enough." Rather, QM is looking for above average -- approximately 85% or B+. Although this is somewhat subjective, the basis for the decision is based on the evidence found in the course compared to the standard and its annotation. Standards are based in the research literature and widely accepted standards about effective online learning.
Peer reviewers are all experienced online faculty who have attended QM training and learned to apply the rubric. QM relies on the experience, expertise and common sense of its faculty reviewers to conduct reviews fairly and consistently and to judge whether the course meets expectations at the "85%" standard.The emphasis on above average is also the reason that QM primarily recommends reviewing "mature" courses -- those that have been taught at least two semesters. If the course has been taught over several semesters, the faculty developer has had time to "fine tune" the course and to make management and content improvements.
New QM reviewers often have many questions about the 85% standard and often find it confusing. There are actually two 85% factors to consider during a QM review:The first is that a course must earn at least 81 out of 95 points (or about 85%). Each of the 41 specific review standards receives a "yes" or "no" vote regardless of point value. The points are added up and must be at the 85% or greater percentage (AND meet all 21 essential standards) to be QM recognized.The second is that when you are conducting your independent review of the course and making your determination if the specific standard is met, use the 85% rule for yourself in making that judgment. The standard does not have to be 100% to be a "yes". The "85% rule" is a guide for you as a reviewer to gauge whether you will choose "yes" or "no" for that particular standard
Here's an example: A Chemistry course might have 15 course-level learning objectives. Of these, two are not measurable (the other 13 are measurable). Most reviewers would decide YES, this meets Standard 2.1. They would then write strong recommendations for how to improve the two weak objectives.
QM is admittedly a subjective process and making decisions about whether a standard is or is not met can be difficult. QM provides extensive annotations and training for its reviewers, but there are likely to be many instances of differing opinions. We know that everyone wants a "right/wrong" answer, but frequently, that's just not possible. Online courses are very complex and likely to be more shades of gray than absolute. And that's the reason QM "hires" three experienced online faculty members to serve on the peer review team. QM relies on the collective experience, knowledge and common sense of the team to arrive at the best possible feedback for the course instructor.
Thank you for watching this presentation on the Overview and Principles of Quality Matters. Please close this window to return to your workshop in Moodle.
