COVID impacts and results of extension programs

1. EDUCATION POLICY INITIATIVE AT CAROLINA
Year 1 Findings from NCDPI’s IES-funded study: Assessing the
Long-Term Impacts of School Extension Programs on Student Re-
Engagement and Learning Recovery
Grant Program: Using Longitudinal Data to Support State Education
Recovery Policymaking (84.305S)
Sarah Fuller, Douglas Lauen, Kevin Bastian, Tom
Swiderski, Rosie Miesner, Camille Mikkelsen, John
McDonnell, Samantha Capers
University of North Carolina at Chapel Hill
Michael Maher, Jeni Corn, Calen Clifton, Erin Manuel
North Carolina Department of Public Instruction

2. EDUCATION POLICY INITIATIVE AT CAROLINA 2
Overview of the IES Research Grant
● Funding: State Longitudinal Data Systems Grant from the Institute
for Education Sciences (IES)
● Research Partner: Education Policy Initiative at Carolina (EPIC) at
UNC-Chapel Hill
● Approach: Uses descriptive and regression analysis to look at the
impacts of the COVID-19 pandemic and recovery efforts statewide
● Aim: To provide evidence to support student re-engagement and
academic recovery.
Assessing the Long-Term Impacts of School Extension Programs on Student Re-Engagement and Learning Recovery

3. EDUCATION POLICY INITIATIVE AT CAROLINA
Primary
Research
Questions of
the IES
Grant
How has the COVID-19 pandemic
affected student learning, student
engagement, and progress
through school?
How are the school extension
programs implemented?
What are the impacts of school
extension programs on student
learning, school engagement, and
progress through school?
3

4. EDUCATION POLICY INITIATIVE AT CAROLINA
Overview of Data and Methods
Assessing the Long-Term Impacts of School Extension Programs on Student Re-Engagement and Learning Recovery
Student Outcome Data
• Student absences
• Course grades and
passing
• Grade Retention
Summer Program Data
• LEA program plans
• LEA post-program
surveys
• Summer program
enrollment and
attendance
• iReady scores
Methods
Descriptive comparisons:
• Over time - pre- and
post-pandemic
• Between student
subgroups
• Between summer
program enrollees and
non-enrollees
4

5. EDUCATION POLICY INITIATIVE AT CAROLINA 5
Summary of Key Findings
● In 2020-21, students had more absences and lower course grades, were
more likely to be chronically absent, to fail at least one course, and to be
retained in grade.
● Student enrolled in summer 2021 programs had lower test score proficiency,
more failed courses, and more chronic absences.
● Across districts, slightly more than half of elementary and middle school
enrollees attended at least 75% of summer program days, about a quarter of
elementary schoolers and about 40% of middle schoolers increased at least a
grade level of iReady.
● Summer program enrollees were less likely to repeat previously failed
courses.
Assessing the Long-Term Impacts of School Extension Programs on Student Re-Engagement and Learning Recovery

6. EDUCATION POLICY INITIATIVE AT CAROLINA
EDUCATION POLICY INITIATIVE AT CAROLINA
Statewide Impacts of the COVID-19 Pandemic on
Absences, Course Grades, and Grade Retention in 2020-21
Assessing the Long-Term Impacts of School Extension Programs on Student Re-Engagement and Learning Recovery

7. EDUCATION POLICY INITIATIVE AT CAROLINA
Impact of COVID-19 on Student Absences, 2020-21
7
Assessing the Long-Term Impacts of School Extension Programs on Student Re-Engagement and Learning Recovery
8.3
11.6
9.1
16.3
10.9
15.3
0
5
10
15
20
25
30
Pre-pandemic
2020-21
Pre-pandemic
2020-21
Pre-pandemic
2020-21
Elementary Middle High
Average
days
absent
Average Days Absent
10.1
19.7
13.1
27.2
18.0
24.1
0
5
10
15
20
25
30
Pre-pandemic
2020-21
Pre-pandemic
2020-21
Pre-pandemic
2020-21
Elementary Middle High
%
Chronically
absent
Percent Chronically Absent

8. EDUCATION POLICY INITIATIVE AT CAROLINA
Impact of COVID-19 on Student Grades, 2020-21
8
Assessing the Long-Term Impacts of School Extension Programs on Student Re-Engagement and Learning Recovery
3.1
2.4
2.8
2.5
0
1
2
3
4
Pre-pandemic 2020-21 Pre-pandemic 2020-21
Middle High
Mean
quality
points
earned
Average Quality Points
15.5
39.6
21.7
34.9
0
10
20
30
40
50
Pre-pandemic 2020-21 Pre-pandemic 2020-21
Middle High
%
failed
any
course
Percent Failing At Least One Course

9. EDUCATION POLICY INITIATIVE AT CAROLINA
Impact of COVID-19 on Grade Retention, 2020-21
9
Assessing the Long-Term Impacts of School Extension Programs on Student Re-Engagement and Learning Recovery
3.9 3.7
0.9 1.2
0.5 0.7
5.9
13.7
3.4
9.1
1.7
5.5
0
5
10
15
20
25
30
35
40
0
2
4
6
8
10
12
14
16
Pre-pandemic
2020-21
Pre-pandemic
2020-21
Pre-pandemic
2020-21
Pre-pandemic
2020-21
Pre-pandemic
2020-21
Pre-pandemic
2020-21
Grade K Grade 3 Grade 8 Grade 9 Grade 10 Grade 11
%
Grade
Retained

10. EDUCATION POLICY INITIATIVE AT CAROLINA
EDUCATION POLICY INITIATIVE AT CAROLINA
Impacts on Outcomes by Student Subgroups
Assessing the Long-Term Impacts of School Extension Programs on Student Re-Engagement and Learning Recovery

11. EDUCATION POLICY INITIATIVE AT CAROLINA
Impact of COVID-19 on Chronic Absences by Subgroup
11
Assessing the Long-Term Impacts of School Extension Programs on Student Re-Engagement and Learning Recovery
22.0
32.9
21.7
29.2
10.5
19.9
29.8
42.4
12.4
35.6
14.9
29.5
33.3
8.7
13.3
12.8
12.2
19.2
7.8
12.4
17.4
24.9
8.2
19.3
11.5
13.7
16.5
5.9
0
5
10
15
20
25
30
35
40
45
Not EL
EL
Not SWD
SWD
Highest
Quartile
Q3
Q2
Lowest
Quartile
Non-ED
ED
White
Hispanic
Black
Asian
English Learner
Students with
Disabilities
Prior achievement
Economic
Disadvantage status
Race/ethnicity
%
Chronically
Absent
2020-21 Pre-pandemic

12. EDUCATION POLICY INITIATIVE AT CAROLINA
Impact of COVID-19 on Course Failures by Subgroup
12
Assessing the Long-Term Impacts of School Extension Programs on Student Re-Engagement and Learning Recovery
35.5
59.8
35.4
46.1
15.3
29.8
44.7
60.6
24.0
53.7
26.2
50.0
48.7
17.0 18.3
35.2
17.2
29.7
5.4
11.7
21.3
38.6
12.0
28.4
12.9
24.5
28.5
7.0
0
10
20
30
40
50
60
70
Not EL
EL
Not SWD
SWD
Highest
Quartile
Q3
Q2
Lowest
Quartile
Non-ED
ED
White
Hispanic
Black
Asian
English Learner
Students with
Disabilities
Prior achievement
Economic
Disadvantage status
Race/ethnicity
%
Failing
Any
Course
2020-21 Pre-pandemic

13. EDUCATION POLICY INITIATIVE AT CAROLINA
Impact of COVID-19 on Grade Retention by Subgroup
13
Assessing the Long-Term Impacts of School Extension Programs on Student Re-Engagement and Learning Recovery
9.1
22.0
9.0
14.4
1.9
5.4
11.2
21.0
5.0
16.2
5.4
15.8
14.1
4.0
3.2
10.0
3.1
7.7
0.5
1.3
3.4
9.6
1.9
6.2
2.1
5.4
6.0
1.3
0
5
10
15
20
25
Not EL
EL
Not SWD
SWD
Highest
Quartile
Q3
Q2
Lowest
Quartile
Non-ED
ED
White
Hispanic
Black
Asian
English Learner
Students with
Disabilities
Prior achievement
Economic
Disadvantage status
Race/ethnicity
%
Retained
in
Grade
2020-21 Pre-pandemic

14. EDUCATION POLICY INITIATIVE AT CAROLINA
EDUCATION POLICY INITIATIVE AT CAROLINA
Student Enrollment in Summer 2021 School Extension
Programs
Assessing the Long-Term Impacts of School Extension Programs on Student Re-Engagement and Learning Recovery

15. EDUCATION POLICY INITIATIVE AT CAROLINA
Summer 2021 Program Enrollment by District Characteristics
15
Assessing the Long-Term Impacts of School Extension Programs on Student Re-Engagement and Learning Recovery
20%
28%
14%
29%
27%
19%
13%
20%
21%
27%
28%
33%
5%
14%
23%
7%
22%
18%
14%
9%
13%
16%
21%
20%
23%
4%
9%
11%
4%
15%
11%
10%
6%
8%
11%
15%
14%
13%
4%
0%
5%
10%
15%
20%
25%
30%
35%
40%
Total
Am
Indian
Asian
Black
Hispanic
Multi
White
Female
Male
Economic
Disadvantage
Students
with
Disabilities
English
Learners
Academic
and
Intellectually
Gifted
Percent
of
Students
Enrolled
in
Summer
Program
K-5 6-8 9-12

16. EDUCATION POLICY INITIATIVE AT CAROLINA
2020-21 Proficiency Rates by Summer 2021 Program Enrollment
16
Assessing the Long-Term Impacts of School Extension Programs on Student Re-Engagement and Learning Recovery
15% 16%
24%
10%
21%
46%
8%
31%
18%
46%
50%
58%
38%
51%
73%
29%
63%
50%
0%
10%
20%
30%
40%
50%
60%
70%
80%
Math Reading Science Math Reading Science Math 1 English 2 Biology
K-5 6-8 9-12
Percent
Proficient
Enrolled Not Enrolled

17. EDUCATION POLICY INITIATIVE AT CAROLINA
EDUCATION POLICY INITIATIVE AT CAROLINA
Short Term Outcomes of Summer 2021 School Extension
Programs
Assessing the Long-Term Impacts of School Extension Programs on Student Re-Engagement and Learning Recovery

18. EDUCATION POLICY INITIATIVE AT CAROLINA
Percent of Summer 2021 Enrollees Gaining at Least 1 Grade Level
on iReady Scores
18
Assessing the Long-Term Impacts of School Extension Programs on Student Re-Engagement and Learning Recovery
24%
43%
26%
44%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
K-5 6-8
Percnet
of
Students
Increasing
by
at
Least
1
Grade
Level
Reading Math

19. EDUCATION POLICY INITIATIVE AT CAROLINA
Percent of Summer 2021 Enrollees Repeating Courses Failed in
2020-21
19
Assessing the Long-Term Impacts of School Extension Programs on Student Re-Engagement and Learning Recovery
40%
25%
33%
30% 31%
65%
57%
61%
56%
49%
0%
10%
20%
30%
40%
50%
60%
70%
Any English Math Science Social Studies
Percent
Repeating
Course
Enrolled Not Enrolled

20. EDUCATION POLICY INITIATIVE AT CAROLINA
EDUCATION POLICY INITIATIVE AT CAROLINA
Summary of Key Findings and Next Steps
Assessing the Long-Term Impacts of School Extension Programs on Student Re-Engagement and Learning Recovery

21. EDUCATION POLICY INITIATIVE AT CAROLINA 21
Next Steps
● Examine changes in student outcomes in the 2021-22 school year.
● Measure longer term impacts of summer 2021 school extension
programs on student outcomes in the 2021-22 school year.
● Describe characteristics of summer 2022 programs and summer
2022 program participants.
Assessing the Long-Term Impacts of School Extension Programs on Student Re-Engagement and Learning Recovery

22. EDUCATION POLICY INITIATIVE AT CAROLINATHE UNIVERSITY OF NORTH CAROLINA AT CHAPEL HILL
For questions, contact:
Sarah Crittenden Fuller
Sarah.fuller@unc.edu