2. What is CLRN?
California Learning Resource Network
http://clrn.org
State-funded Education Technology Service
Established as part of larger ed. Technology legislation in 1999
Initial charter: Review supplemental
electronic learning resources to CA
content standards
Expansion to online courses, OER and
data assessment tools
3. What is CLRN?
California eLearning Census
http://www.clrn.org/census/
eLearning Strategies Symposium
http://elearns.org
@elearns
5. Review Process: How &
What
Literature review & initial criteria
SREB, iNACOL, TxVSN, QM, and WA Digital Learning
CLRN Stakeholder meetings
Criteria rewrite: California
Collaboration with Texas
Criteria rewrite: national
6. How CLRN Reviews
Utilized existing CLRN review sites
Five physical locations around CA
Subject specific
Established reviewer cohorts (15-25 educators)
Three-person review teams
Edited & normed at CLRN Central
http://brianbridges.org
7. CLRN Review Process
Database driven
Publishers complete submission online
Attach standards correlation
document
Noting where each standard may be found
Both teacher and student
accounts/passwords are required
8. CLRN Course Review
Process
Eight-step process
1. Team assignment
2. Content (CCSS) Standards review
Develop, Practice, Assessment
3. Minimum Requirements
Reading levels, spelling/grammar, content accuracy, etc.
4. Social Content Review
12 areas, including male/female roles, ethnic/cultural groups, &
advertising.
9. CLRN Review Process
5. Online Standards
Met, Partially Met, or Not Met
Comments for all Partially Met standards
Reviewers utilize CLRN comment sheet
6. Features
Course Teacher
Instructional Design
Universal Access
Assessment
Technology
Support Materials
10. CLRN Review Process
7. Profile
Publisher Info
Course length and type
Accreditation (UC, NCAA, AP, CLRN-Certified)
Languages, etc
8. Abstract
Full course summary
Course objectives and outcomes
Relevant Research
11. CLRN Central Review
Validation & Norming
Work the course
Review/update all 52 course standards
Standardize review comments
Notes inform reviewer retraining
12. CLRN Central Review
Editing & Proofreading
Review, modify, and standardize content standards rating &
comments
Publisher Feedback
Seven-day window
New evidence required for Re-Review
One Re-Review permitted per course.
Publishing
Reviews valid for three years
13. CLRN-Certified
University of California Partnership
A-G Approved Courses
CLRN-Certified
80% content standards
80% online course standards
15 power standards
14. What’s Next?
World Language Reviews
Math Common Core Standards
Next Generation Science Standards
Adding Grade 6-8 Reviews
Career Technical Education Standards
Professional Development
15. Content
A2. The course content and
assignments are aligned with the
state’s content standards…
Is the course teaching, providing practice, and assessing each
standard?
17. A Textbook is
not a course
If it were, you could just throw the
kid a book and tell them to read it.
18. Content
A3: The course content and
assignments are of sufficient rigor,
depth and breadth to teach the
standards being addressed. *
Develop, Practice, Assess
19. Engaging
B3: The course instruction and
activities engage students in
active learning.*
Reading and watching are
not active.
20. B3 Consideration
The course provides multiple
opportunities for students to be actively
engaged in the content that includes
meaningful and authentic learning
experiences such as collaborative
learning groups, student‐led review
sessions, games, analysis or reactions to
videos, discussions, concept mapping,
analyzing case studies, etc
21. It’s about Pedagogy
How will you engage students?
What authentic projects will you
use for practice and assessment?
How will students collaborate and
participate in discussions?
22. Formative Assessments
Should Inform Instruction
B4. The course and course
instructor provide students with
multiple learning paths, based on
student needs that engage
students in a variety of ways.*
23. Lower Order Thinking
Skills
B5. The course provides
opportunities for students to
engage in higher-order thinking,
critical reasoning activities and
thinking in increasingly complex
ways. *
24. B5 Consideration
Assignments, activities, and assessments
provide opportunities for student to elevate
their thinking beyond knowledge and
comprehension into the realm of analyzing
situations, synthesizing information, or
evaluating an argument. Activities should
include open‐ended questions, and
encourage students to categorize and
classify information.
25. Media Rich
B11. Students have access to
resources that enrich the course
content.
D4. Rich media are provided in
multiple formats for ease of use
and access in order to address
diverse student needs.*
26. Assessments
C2. The course structure includes
adequate and appropriate
methods and procedures to assess
students’ mastery of content. *
27. C2 Consideration
C2. Assessment types are
matched to the level of
knowledge being tested. Both
formative assessments (that inform
and support learning) and
summative assessments (that
demonstrate mastery) are a part
of the course structure
30. Student Progress
C4. Assessment strategies and
tools make the student
continuously aware of his/her
progress in class and mastery
of the content. *
31. Professional
Development
E7. Course instructors… have been
provided professional development in
the behavioral, social, and when
necessary, emotional, aspects of the
learning environment.
E8. Course instructors…receive
instructor professional development,
which includes the support and use of
a variety of communication modes to
stimulate student engagement online.
38. Analysis
First 114 course reviews
31% CLRN-Certified (36 total)
32 courses only missing D10 (28%)
Most common problem
Content standards alignment
47 courses (41%)< 80% content
standards
Range from 29% met to 78% met
47. Online and Blended
Population
Actual Count Projected Count
Virtual: 19,820 210,000
Blended: 86,257 3.4% of the student
population
Total: 106,077
48. Blended Model
Breakdown
Self-Blend 60%
Hybrid-Virtual 36%
31% of districts & charters
Rotation use more than one model
29%
Flex
17%
0% 10% 20% 30% 40% 50% 60% 70%
Standards-aligned, engaging content can be purchased from a publisher, found in open source repositories, or created in-house.
With iNACOL standard A2 stating, “The course content and assignments are aligned with the state’s content standards..”, you want to make sure that the content you provide students not only teaches (demonstrates) a skill, but also provides students opportunities to practice and assess each skill or standard. CLRN’s reviews include these three components of each standard identified for a course.
Your textbook is not a course though. While textbooks are aligned with the standards and may include practice activities and assessments, placing your book online, be it commercial or open source, is amateurish at best. Quality courses will include text though, but not entire chapters printed screen after screen. The better courses CLRN have reviewed include portions of text mixed with video lecture clips, streaming video, simulations, games, and short formative assessments. Creating quality online lessons is a much more time-consuming task than creating face-to-face lessons. Provide ample lead-time to create online lessons.
Online course standard B3 states that course instruction and activities must engage students in active learning, including authentic projects and activities that challenge students beyond knowledge and comprehension. Rather than focus primarily on multiple-choice tests for assessments, it’s best to provide students knowledge work where they create, evaluate, and analyze. Students should regularly participate in online discussion groups, be they synchronous or asynchronous.
That there are a variety of assessments too. Find the right assessment for what is being taught.
All teaching and learning materials must be accessible to all students. Period. If you’re creating video lectures, streaming video clips, or providing narrated presentations, each must either have closed-captions or a transcript. Online standard D10, and the Department of Justice, expects it and your students deserve it. Sites like Universal Subtitles http://www.universalsubtitles.org/ are easy to use and allow your captioned videos to play from their site, or you may download the time codes to upload to YouTube.
This is a See Learn presentation about the California eLearning Census.
See learn conducted the California elearning census between March and May 2012. The data in the following slides are from 481respondents representing 30% of all districts and direct funded charters.
We foundthat 45% of all districts and charters indicated they were using some form of elearning. This data appears to say that we have either passed the tipping point or a point of no return.
Of the districts that were NOT using online learning, 32% shared they were currently in the planning stages. When added to our prior group, we may find next year that 63% of all districts are utilizing online or blended learning.
the data indicates that much higher percentages of students are enrolled in online and blended learning in the upper levels with the majority at the high school level.
Of the four blended learning models, the most popular is the Self Blend, followed by Hybrid Virtual School, a model used by Independent Study schools in california. This seems to indicate that non-consumers, students who are using eLearning to supplement their transcript or schools that provide online courses not offered in the classroom, are a driving force.Census data reflects 31% of districts and direct-funded charters reported they are utilizing more than one blended learning model.Self Blend: 60%Enriched-Virtual: 36%Rotation: 29%Flex: 17%
Of the four blended learning models, the most popular is the Self Blend, followed by Hybrid Virtual School, a model used by Independent Study schools in california. This seems to indicate that non-consumers, students who are using eLearning to supplement their transcript or schools that provide online courses not offered in the classroom, are a driving force.Census data reflects 31% of districts and direct-funded charters reported they are utilizing more than one blended learning model.Self Blend: 60%Enriched-Virtual: 36%Rotation: 29%Flex: 17%
OER resources, digital materials available for free or a very low cost from sources including Khan Academy and the National Repository of Online Courses (NROC), are helping districts add a digital component without investing in developing or acquiring content. Districts and direct-funded charters also indicated they are utilizing supplemental software/Internet resources or open source materials for online and blended learning in addition to or instead of licensed courses; 14% indicated they utilize two or more OER providers.
