Adelstein, D., & Barbour, M. K. (2017, April). Examining the International Association for K–12 Online Learning (iNACOL) standards for K–12 online course design. A poster presentation at the annual meeting of the American Educational Research Association, San Antonio, TX.

1. Introduc)on
Online education has evolved at a frantic pace
over the past two decades. As it quickly became
common place in the K-12 environment, there
was increased attention paid to design, delivery,
and support specific to K-12 online learning.
While a handful of private and public institutions
created design standards, the non-proprietary
National Standards for Quality Online Courses
put forth by the International Association for K-12
Online Learning (iNACOL) have become widely
accepted and used throughout the United States.
However, the standards were not developed
using traditional methods that are normally used
to determine reliability and/or validity of
standards. This project sought to begin the
process of addressing this oversight.
Methodology
To review and test the reliability and validity of the
iNACOL (2011) course design standards, a
common multi-step approach found in a variety of
other studies was used. This process consisted of
three distinct phases.
Phase one was a review to determine support
and the content validity for individual elements of
the iNACOL standards using both K-12 online
and general online education literature.
Phase two used two panels of experts to review
the standards and results of phase one to
continue the testing of the content validity. Over
three rounds of review, the experts helped to
revise the standards and the rubric.
Phase three had four groups of reviewers utilizing
the rubric on current K-12 online courses to test
the new instrument for inter-rater reliability.
Results
Conclusions
Over the past 20 years there have been a wide
range of studies into K-12 online learning.
However, that number shrinks considerably when
focused on K-12 online course design. The
research conducted in phases one and two of this
project have the benefit of giving more credence
to the iNACOL standards. This not only provides
more confidence in the standards, but gives the
designers a more nuanced and focused approach
to the creation process. Phase three provided a
basic blueprint and starting point for future
research into this set of course design standards,
which should be expanded upon (Adelstein &
Barbour, 2016c).
Future research should examine other widely
accepted standards (e.g., VHS, QM, etc.) using
the same methodology and procedure, with
results compared between all sets of standards.
Further, the revised iNACOL rubric from this
project should continue to be tested in the real
world by building new online courses, and not just
measuring it against existing courses.
David Adelstein – Jingsong Schools
Michael K. Barbour – Touro University, California
References
Adelstein, D., & Barbour, M. K. (2016a). Building better courses: Examining the content
validity of the iNACOL national standards for quality online courses. Journal of Online
Learning Research, 2(1), 41-73. Retrieved from http://www.editlib.org/p/171515
Adelstein, D., & Barbour, M. K. (2016b). Redesigning design: Field testing a revised
design rubric based of iNACOL quality course standards. International Journal of E-
Learning & Distance Education, 31(2). Retrieved from http://www.ijede.ca/index.php/
jde/article/view/976
Adelstein, D., & Barbour, M. K. (2016c). Redesigning design: Streamlining K-12 online
course creation. MACUL Journal, 37(1), 20-21.
Adelstein, D., & Barbour, M. K. (in press). Improving the K-12 online course design review
process: Experts weigh in on iNACOL national standards for quality online courses.
International Review of Research in Open and Distance Learning, 18(3).
International Association for K-12 Online Learning. (2011). National standards for quality
online courses version 2. Vienna, VA: Author. Retrieved from
http://www.inacol.org/wp-content/uploads/2015/02/national-standards-for-quality-
online-courses-v2.pdf
Redesigning the iNACOL Standards for K-12 Online Course Design
Phase One
This phase reviewed the current iNACOL
standards against current K-12 online
education and other relevant literature with a
focus on virtual learning (Adelstein & Barbour
2016a). The results showed:
• all standards were aligned with current
online education literature, and
• it was difficult to align specifically with K-12
online learning literature due to limited
availability research.
Table 1. Types of courses used for phase three
review.
Table 2. Comparing all reviewers across Section
C elements as an example of data collection from
phase three.
Phase Two
This phase consisted of an expert panel
reviewing and revising the standards based on
phase one (Adelstein & Barbour, in press). The
results showed:
• the standards were vital as a whole,
• not all of the elements are appropriate
specifically for course design, and
• elimination and revision of standards allowed
for a streamlined rubric.
Phase Three
The final and most challenging phase, due to the limited number of reviewers and courses,
tested the revised rubric against current K-12 online courses (Adelstein & Barbour 2016b). The
results showed:
• the revised rubric did not meet the reliability threshold for percentages, and
• numerous individual elements were found to be reliable.