While more than two-thirds of states already conduct some kind of statewide online assessments, the new assessments of the Common Core State Standards from PARCC and Smarter Balanced are increasing both interest and anxiety in school districts. To help relieve the anxiety, SETDA has created a series of case studies Implementing Online Assessments: Pathways to Success (assessmentstudies.setda.org), with a narrative and resources. Each narrative provides an in-depth look at what it took a district to implement the assessments as well as key factors in each state’s approach to infrastructure and training and communication. There also are downloadable resources that range from training agendas to checklists for technology readiness to sample letters from the state to school districts. Mike Nelson, Director of Curriculum and assessment of Coeur d"Alene School District 271, ID, Patches Hill, Technology Systems Manager of Indian River School District, DE and Scott Smith, Chief Technology Officer of Mooresville Graded School District, NC will explain what they did to get ready for their online assessments and answer your questions.

1. Ready for Online Assessments? Help is Here.
Call-in number: 571-392-7703 | PIN: 635 397 980 649
October 2013

2. Poll #1
How did you hear about this event?
A. SETDA/Assess4ed Promotion
B. Connected Educators Month Promotion
C. Twitter/Facebook
D. A Colleague

3. About SETDA
• Ten-year old DC-based, national, non-profit member
association
• Serve, support, and represent U.S. state and territorial
directors (SEA leadership) for educational technology
• Forum for:
– Research and best practices
– Inter-state collaboration
– Professional development
– Public-private partnerships
– State-federal relations

4. SETDA Resources for Online
Assessment Readiness
• assess4ed.net
• Technology Requirements for Large-scale
Computer-based and Online Assessment
• Technology Readiness for College and Career
Ready Teaching, Learning and Assessment
• SEPC
• Implementing Online Assessments – Pathways
to Success
For access to SETDA’s resources to go http://www.setda.org/web/guest/assessment

5. Agenda
• Welcome
–
Geoff Fletcher, Deputy Executive Director, SETDA
• Implementing Online Assessment Website
Overview District Sharing
–
–
–
Mike Nelson, Director of Curriculum and assessment of Coeur d"Alene
School District 271, ID
Patches Hill, Technology Systems Manager of Indian River School
District, DE
Scott Smith, Chief Technology Officer of Mooresville Graded School
District, NC
• Q&A, Geoff Fletcher, SETDA

6. Poll #2
Chose the statement that best describes your
role in helping your state/district prepare for
online assessments.
A. Technology contact
B. Assessment contact
C. I am driving the whole show
D. I am doing nothing to help prepare – yet
E. I will implement it when it happens

11. Presenters
Mike Nelson
Director of Curriculum and
Assessment
Coeur d’Alene School District
271, ID
Patches Hill
Technology Systems
Manager
Indian River School District,
DE
Scott Smith
Chief Technology Officer
Mooresville Graded School
District, NC

12. Ef fective Use
Online
Platforms for
Formative &
Summative
Assessment
Michael S. Nelson
Director of Curriculum and Assessment
Coeur d’Alene School District (Idaho)
msnelson@cdaschools.org

13. District Back ground
 Coeur d’Alene School District 271
http://www.cdaschools.org





Public school district of 10,300 students in Idaho
530 teachers in 17 schools (7 of last 14 State teachers of the year)

11 elementary; 3 middle schools; 2 traditional high schools

6 magnet or charter schools
Part of the Smarter Balanced Assessment Consortium
Highest performing large district in Idaho
90GB of bandwidth coming into our district

17 mobile laptop/iPad labs available if needed

Blended learning for several courses

14. Professional Devel.
Approach
We empower faculty through:
•Job embedded staff collaboration
– Weekly on Mondays
– 1 day per month district-driven
•“Top 40” Professional Devel. Team
– Stiggins’ Assessment Training Institute
•Building Leadership opportunities

15. Our Assessments are
used for…
• Diagnostic
to determine the student’s pre-knowledge and
identify strengths & weaknesses
• Formative and Interim
to provide the student with feedback on their
progress during a course
• Summative
to estimate performance at the end of a course
and ‘grade’ the student’s work

16. Smar ter Balanced
Questioning Types

17.
Our Assessment
Quandar y
Discovery
Education
Paper
Scannable
OMR
Schoolnet
Skyward
Clicker Delivery
XX
Online Delivery
XX
XX
XX
Paper Delivery
XX
XX
XX
XX
XX
XX
XX
XX
XX
XX
XX
XX
XX
XX
Item Analysis
XX
XX
Reporting
Features
XX
XX
XX
Import to Grades
XX
XX
XX
XX
XX
XX
XX
Delivery Options
Question Types
Selected
Response
Constructed
Response
Technology
Enhanced
Performance
Tasks
End of Exam
Costs
Ease of Use
Administration
Ease of Use
Annual Maintain
and/or Upgrades
Other Considerations
Ties to Standards

18. We Chose Skyward…
After evaluating options, we found that our student
management system did the best in:
•test factual knowledge effectively
•include web-based resources, media objects
•track student attempts
•provide detailed, specific feedback
•mark automatically, quickly, consistently
•provide question banks, randomized ordering
•produce statistical information for test items
•Export to gradebook and statistical packages

19. Areas of Note
• construction of good objective tests still depends on
instructors
• building up question pools is time-consuming
• specialized mathematics and scientific notation still
cumbersome to incorporate
• provision of secured test environment
• review or grading essays and project-based
assessment/performance tasks
• detect plagiarism in essays
• robust infrastructure is required to avoid failure
during exams

20. Students
Taking
Assessments

21. Getting to the Exam
Click on “Open Student Access” on the upper left corner of the screen.
Student “assignment” will be listed at the top of the screen
along with their current grades and academic calendar.
Click on “Take Assignment”

22. Ending the Exam
When choosing “Save
and Complete Exam” a
summary screen will
appear asking you to
review your work and
ask whether you want to
complete the
assessment.
Click “Yes” to end.

23. Essay Questions
•
•
Read the prompt carefully.
Create a well-crafted essay that fully answers the prompt.
•
You can expand the response box by dragging down on
the bottom-right corner of the window.

24. Selected Response Stems
X
X
X
• These are NOT traditional multiple choice.
• Each question may have multiple correct
answers.
• Read the question carefully AS WELL AS
EACH answer choice.
• Place an X in each of the boxes that correctly
answer the question correctly.
• There may be an image, document or other
information available for answering your
question.
• Look for the
button

25. Post- Assessment
Analysis
Post-test, we have the ability to analyze each assessment by
question stem or by teacher distribution.
•Report-based or chart form

26. Final
Thoughts

27. Organizational Suppor t
Structure
Phase-In
• Work with provider
• Pilot work at different levels
• Constantly communicate
Provide Supports in a variety • How to handouts
of methods
• PowerPoint presentations
• Short instructional videos
• Top 40 Team
• Technology Leaders
• Hand holding may be
necessary
Provide Timely Results and
Successes
• Empower faculty to
manipulate and use results
• All assessment should
inform instruction
• Triangulate results with
other works

28. Thank
You.
Michael S. Nelson
Director of Curriculum and Assessment
Coeur d’Alene School District (Idaho)
msnelson@cdaschools.org

29. Delaware, IRSD
and Online
Assessments

30. Summary of levels
A multi-step process at multiple levels
◦ State infrastructure implementation
◦ State assessment implementation – summative standardization
◦ Personalized learning and formative standardization

31. History and Background
Delaware has only three counties that encompass 19 school districts (16 traditional
districts and 3 county-wide vocational/technical districts).
The move to adopt and fully implement Common Core State Standards (CCSS)aligned assessments in 2014 builds on Delaware’s nearly twenty-year history of
planning toward better information sharing across the state.
Faced with standardized testing contract expiration after the 2009-2010 school year,
the state formed a taskforce with two priorities for the next iteration of tests:
immediate scoring, and the ability to track data over time. These priorities led to
the group’s ultimate recommendation for a shift to online assessment.
Delaware wanted to emphasize testing flexibility in how the test is administered
and within the assessment. The Delaware Comprehensive Assessment System
(DCAS) provides multiple opportunities for students to demonstrate proficiency and
provides academic achievement including a measure of fall-to-spring and year-toyear individual student growth.
Delaware has a law requiring all state-mandated and high-stakes assessments to be
administered digitally.

32. Key Success Factors
Centralized statewide network that provides Internet connectivity to
each school.
Statewide pupil accounting system that tracks accommodations used by
every district already in place.
Annual school technology survey to assist in determining readiness.
Stakeholders involved starting very early in the field testing process and
focused on technology capacity and hardware issues, particularly the
logistics of setting up schools.

34. What districts did
Prior to field testing in 2010, districts and schools were using technology for
instruction, but administering paper-and-pencil assessments.
Assessment no longer meant entire buildings would have to shut down for
several days while all students inside completed the exams because of the
move to broad assessment windows.
Technology Readiness – Including network, bandwidth and computers
readiness.
Teacher/Staff Assessment Readiness – Including training of district
technology coordinators, as well as training of school and district test
administrators.
Student Readiness – All students viewed a presentation that introduced the
DCAS field test and completed both a computer skills assessment and a
training test.
Scheduling – Districts developed careful schedules to complete in time
allowed.

35. What districts did
Displaced some instruction because there aren’t enough laptops with
particular features to meet all demand scenarios.
Establishing Wi-Fi in every school to assure secure, reliable, redundant
connections. Currently, all but two buildings are wireless at 100% coverage
of facility.
PLCs provided a structure for decentralized decision making about the
timing and configurations for online assessment that work best.
Despite the considerable flexibility offered by broad testing windows, most
schools continue to follow the more traditional approach of holding shorter
testing windows comprised of a few days during which nearly all students
are tested.
Four schools at IRSD (2 high schools, 1 middle school and 1 elementary
school) are participating in the spring 2013 SMARTER Balanced pilot test.

36. Key Factors for Future
Systems Adoption at IRSD
As we move to more personalized learning it will prove key to build
systems that follow open standards.
Joined BRINC(State) and IMS GLC (International).
IMS Global and its members have helped us significantly in
understanding the total interoperability of our data and how they
should integrate with other state and local systems.
Only choosing systems that follow QTI and APIP at district level to keep
impact of assessment database longevity and interoperability from
being a concern. Also focusing on Common Cartridge (CC) and Learning
Tools Interoperability (LTI).

37. IMS Standards Prove Key
@ IRSD
QTI - Question and Test Interoperability is a technical specification
that describes formats & services for exchange of digital assessment
items and tests among assessment system software components.
◦
◦
◦
◦
XHTML based
Permits inclusion of other mark-ups, e.g., MathML, ChemML
Supports alternative accessibility representations
Future version will merge with HTML5 & digital book formats
APIP Accessible Portable Item Protocol is a technical specification
that describes formats & services for exchange of accessible digital
assessment items and tests among assessment system software
components.
◦ APIP v1.0 uses a precisely defined subset of QTI v2.1
◦ Expands the accessibility content feature in QTI
◦ Uses accessibility definitions from IMS AccessForAll specifications (Metadata and Personal
Needs and Preferences - PNP)

38. How does APIP relate to
QTI in terms of origin?
QTI has been implemented and evolved over 10+ years of experience of
leading assessment, publisher and learning platform providers
worldwide.
APIP arose out of seed funding from the U.S. Department of Education
to a project led by a consortium of U.S. states.

39. Conformance Profiles

40. Additional Links
District Website
http://www.irsd.net/
Materials Used
http://assessmentstudies.setda.org/resources/
IMS Overview
http://www.youtube.com/watch?v=corV3-WsIro
Open Infrastructure
http://www.imsglobal.org/IMSSupportsK12.pdf

41. Mooresville Graded School District
Digital Conversion
Scott S. Smith
Ed.D

42. Mooresville Graded School District:
Demographics
Number of Schools
8
Certified Staff
~ 400
Non Certified Staff
~ 300
Total Students
K-12
5900
Students receiving Free/Reduced Lunch
has increased in MGSD...
Asian
2%
Multi Racial
3%
Hispanic
8%
African American
17%
Caucasian
70%

43. Why a Digital
Conversion?
•
Close the digital divide
•
Relevant instruction
•
21st century readiness
•
Real world experience
•
Instructional practice
•
•
Improved academic
achievement
Moral imperative

44. Ingredients
•
•
•
Building the culture
21st century tools and
online resources
Data rich & intensive – with
immediate feedback and
results
•
Capacity building
•
All in!!!

45. The Tech Specs
• Over 5500 MacBook Airs for every teacher and student grades 3-12
• Online content available virtually 24 / 7
• Ubiquitous wireless
• 802.11n Access Point in every classroom (at minimum)
• High availability connectivity
• 500 MB Pipe for the district
• GB Fiber between locations
• Cultural shift / personalized instruction
• Support for online assessments

46. Online Assessments
• Daily online assessments in Angel (Learning Management System)
• Implementation of Lock Down Browser in conjunction with Angel
• Quarterly Assessments with ThinkGate
• End of Grade and end of Course Exams online

47. Meeting Bandwidth Needs
• Prior to Digital Conversion
• 40MB Connection
• Year 1 (2008 – 2009)of Digital Conversion (~2200 laptops)
• 100 MB Connection
• Year 2 (2009 – 2010) of Digital Conversion (~4000 laptops)
• 250 MB Connection
• Year 4 (2011-12)of Digital Conversion (~5000 laptops)
• 500 MB Connection
• Year 6 (2013-14) of Digital Conversion (~5500 laptops)
• 500 MB Connection (watching closely)

48. Online Assessments
• Average week – 274 MB of a 500 MB pipe (Sept 16 – Sept 20, 2013)

49. June 3, 2013 – First day of Online Exams
• Average 173 MB of a 500 MB pipe

50. Mooresville High School
Grades 9 - 12
•
Honor School of Excellence
•
High Growth
•
Met 100% AMO Targets
Overall Composite for Students Testing Proficient
on State End-of-Course Exams
Composite increase of 23
percentage points over 6 years

51. Mooresville Graded School District
Getting students across the line!
4-Year Cohort Graduation Rate
MGSD is 3rd
in the state
out of 115
districts for
its 2012
graduation
rate!

52. Mooresville Graded School District
District Achievement
16%
16%
Growth!
Growth!

53. Mooresville Graded School District
District Achievement
District
Percent Proficient using
EOG Reading, Math,
Science; EOC
Per Pupil
Expenditure
(state ranking)
District
Enrollment
Numbers
1
Camden County
90%
7989 (73)
1891
2
Mooresville Graded
School District
89%
7463 (100)
5622
Polk County
89%
9604 (22)
2363
Chapel Hill Carrboro
88%
8979 (39)
11,504
3
MGSD Ranked
2nd in the
State
2011-12
Formula: Number of Proficient Test Scores on Reading Grades 3-8, Math Grades 3-8, and EOCs divided by the total number of tests taken in school year 2011-2012.

54. Questions?
http://assessmentstudies.setda.org
Webinar Archive & Resources
http://www.setda.org/web/guest/connectededucators
More information: Geoff Fletcher, gfletcher@setda.org

55. Claim your CEM Digital Badge
•
•
•
•
Go to http://badges.connectededucators.org
Click the “Have claim code?” button
Enter the code: rere124
Click “Claim”
NOTE: Participants will have to create a Mozilla
account and backpack to claim a badge if they
do not have an account already.

56. CEM Activities for State Leadership
http://connectededucators.org/events/#/
audience=249
http://connectededucators.org/cem-for-states/

57. Webinar Resources
http://assessmentstudies.setda.org
Webinar Archive & Resources
http://www.setda.org/web/guest/connectededucators
More information: Geoff Fletcher, gfletcher@setda.org