The document summarizes findings from the Young Lives 2016-17 School Survey in India on educational effectiveness and equity. Some key findings include: - Private unaided schools tended to add more student learning value, even after accounting for student backgrounds, while government schools showed more variability. - Larger schools with more sections added more student learning value than smaller, single-section schools across school types. - Wealthier, male students and those with more educated mothers tended to attend schools that added more student learning value. - Starting learning gaps in 9th grade widened over the school year as disadvantaged students attended less effective schools.

1. @yloxford
@younglivesindia
Young Lives 2016-17 School Survey
Value-added analysis and school effectiveness
Caine Rolleston (Institute of Education, UCL)
Rhiannon Moore (Young Lives, University of Oxford)
@caine_rolleston
@rhi_moore

2. WHY EDUCATIONAL ‘EFFECTIVENESS’?
• During lives of YL cohorts large
increases in enrolments towards
universal access
• Much interest in system expansion &
reform, dilemmas of access, quality,
equity, efficiency
• YL survey data showed wide disparities
in learning levels between countries
suggesting differences of system
quality/equity (quality for whom?)
• Within country gaps e.g. urban/rural,
public/private
• Unique opportunity to study educational
effectiveness comparatively across 4
developing countries

3. • Two major themes (1) educational effectiveness (2) equity
• Began with question of ‘what makes for an effective
system/school’
• Designed value-added school surveys & assessments (repeated
measures)
• AIM to identify more and less effective schools in context
• BUT this evidence is highly contingent, v limited generalisability
• i.e. not deterministic, many paths dependent on stage/route
• BUT important lessons are possible across systems, though not
usually about ‘inputs’
• Four YL countries represent very different approaches to the
distribution of educational opportunities, key questions:
– Who goes to more effective schools and why?
– Do all pupils benefit equally from school effectiveness?
SCHOOL EFFECTIVENESS RESEARCH

4. • Value-added is a measure of student
progress over a defined period of time
– Aim of ‘controlling’ as much as possible for
differences in student outcomes which are
outside the control of the school
• Progress relative to students in other
schools with a similar starting score
– Anticipate all students will make some
progress over one year.
– Value-added measures if students in a
particular school make more or less
progress than others.
• Designed to compare ‘like for like’
• Provides a summary measure of school
quality
WHAT DO WE MEAN BY ‘VALUE-ADDED’?

5. • Unconditional value-added estimated
using student attainment scores at the
beginning and end of a defined period
of time
– Possible critique – sets lower
expectations for schools serving
disadvantaged children?
• Conditional value-added estimates
include student background factors
– Recognises that students are not
randomly allocated to schools.
– Takes into account that it may be
harder for students from disadvantaged
backgrounds to make progress.
CONDITIONAL OR UNCONDITIONAL?

6. Young Lives 2016-17 School Survey: India

7. SCHOOL SURVEY RESEARCH DESIGN
School effectiveness design:
➢ Student performance in terms of
progress (rather than cross-sectional
measure)
➢ The teaching and learning processes
that affect student progress
➢ The ‘value-added’ of one year of
school
To do this, we administered:
➢ Cognitive tests at beginning and end
of one school year
➢ Background instruments and
psychosocial measures to
contextualise learning progress

8. YOUNG LIVES SCHOOL SURVEYS: INDIA
India Secondary School Survey (2016-17)
• Class 9 students
• Different types of school
• Progress in Maths and English in Class 9
• Tests at the beginning & end of the school year

9. SECONDARY SCHOOL SURVEY: SAMPLE
Sample design to explore school
choice available in each of the 20
Young Lives sites
Sample stratified by 4 school types:
•State government schools
•Tribal/Social Welfare schools
•Private Aided schools
•Private Unaided schools
Number of schools sampled in each
site proportional to the total
number of schools in that site:
Total number of schools in a
site Proportion sampled
> 80 schools 10% sampled
21-80 20% sampled
8-20 schools 50% sampled
<8 schools 100% sampled
(exception: less prevalent school types
are oversampled)

10. INDIA SCHOOL SURVEY: SCHOOLS
➢ 205 schools
➢ School type:
• 14% Private Aided;
• 27% Private Unaided;
• 41% State Government;
• 18% Tribal/Social Welfare
➢ 64% located in rural areas
➢ 59% have one Class 9 section
➢ Medium of instruction:
• 41% Telugu
• 38% English
• 18% Telugu & English
• 2% Urdu

11. Findings: Variation in school value-added

12. UNCONDITIONAL VALUE-ADDED - MATHS
-100
-50
0
50
100
0 50 100 150 200
School by Value-Added rank
Private Aided Private Unaided State Govt TSW
• Clear patterns when we look at unconditional value-added – private
unaided schools add more value, govt schools more varied

13. CONDITIONAL VALUE-ADDED - MATHS
-100
-50
0
50
100
0 50 100 150 200
School by Value-Added rank
Private Aided Private Unaided State Govt TSW
• Pattern becomes less clear when look at conditional value-added – govt
and TSW schools move up the distribution

14. GROWING INEQUALITIES – STARTING SCORES AND VA
-100
-50
0
50
100
300 400 500 600 700
Wave 1 maths test score (school average)
School Fitted line
• Schools with higher scores at the start of Class 9 add more value over
the school year – suggests inequalities will continue to widen over time

15. Findings: Which schools add more value?

16. DIFFERENCES BY SCHOOL MANAGEMENT
• Private unaided schools appear to add considerably more value on
average, even when we control for the more advantaged background
of their students.
-20-10
0
102030
Private Aided Private Unaided State Govt Tribal Social Welfare
Mean School VA (uncon) Mean School VA (con)

17. LARGER SCHOOLS ADD MORE VALUE…
• Are there too many small schools? Findings suggest that smaller
schools (those with fewer sections) add less value than larger schools
-5
05
1015
1 section 2 sections 3 or more sections
Mean school VA (uncon) Mean school VA (con)

18. AND THIS IS TRUE ACROSS SCHOOL TYPES
• The same pattern is seen across govt and private
schools
• Those a larger number of sections in Class 9 add more
value than those with just one section
• Efficiency, competition, location?
0
1020304050
1 section 2 sections 3 or more sections
Private Unaided schools only
Mean school VA (uncon) Mean school VA (con)
-5
05
MeanMathsvalue-added
1 section 2 sections 3 or more sections
State Govt schools only
Mean school VA (uncon) Mean school VA (con)

19. MINIMAL URBAN – RURAL GAP IN MATHS
• Findings suggest an urban-rural gap in value-added – but this decreases
considerably when we control for differences in student background
(for maths – gap remains large for English)
-5
05
10
Rural Urban
Mean school VA (uncon) Mean school VA (con)

20. Findings: Who attends schools adding more value?

21. WEALTHIER STUDENTS ATTEND BETTER SCHOOLS
-10
0
1020
Q 1 (poorest) Q 2 Q 3 Q 4 Q 5 (least poor)
Mean school VA (uncon) Mean school VA (con)
• Considerable evidence of ‘sorting’ by student background -
students from wealthier households attend schools which add a lot
more value

22. AND SO DO BOYS…
• On average boys are attending better schools than girls
(although the gap is relatively small)
012345
Female Male
Mean school VA (uncon) Mean school VA (con)

23. AND THOSE WITH MORE EDUCATED MOTHERS
• Children with more educated mothers attend schools
which add more value than those whose mothers have not
been to school
0
10203040
Never been to schoolPrimary Secondary Upper Secondary Higher ed.
Mean school VA (uncon) Mean school VA (con)

24. WE ALSO SEE SORTING WITHIN SCHOOLS
➢ Children from less wealthy households are ‘sorted’ into
less effective sections within schools as well.
-5
05
1015
Q 1 (poorest) Q 2 Q 3 Q 4 Q 5 (least poor)
Mean school VA (uncon) Mean school VA (con)
Mean class VA (uncon) Mean class VA (con)

25. Discussion and implications

26. SUMMARY OF FINDINGS
• Schools in this sample are relatively homogeneous in
intake, with larger differences found between schools
• Suggests ‘sorting’ of children into schools – characteristic of a
school system with extensive range of ‘school choices’
• More advantaged students appear to be ‘sorted’ into more
effective schools
• This has implications for equality of opportunities during and after
secondary school
• There are considerable gaps at the start of Class 9, and
these widen over time as schools with higher starting
schools also add more value
• Part of differences between schools and school types comes
from differences in student background – but a sizable gap in
effectiveness remains when this is controlled for

27. POLICY IMPLICATIONS & DISCUSSION
• Need for govt policy to mitigate negative effects of ‘sorting’
for children from disadvantaged backgrounds
• Starting scores are already much lower for these children – need for
action much earlier than Class 9
• Also need remedial action to counter lower progress made in Class 9
• Findings indicate that very small schools face particular
challenges in effectiveness
• There is a real need for policy to address this, given the large
number of very small schools
• Although on average private schools add more value, our
findings show that many government schools are equally or
more effective – but the sector is much more heterogeneous
• Need to improve consistency of govt school sector by ‘raising the
floor’ of achievement and progress through setting minimum
standards and increasing quality assurance measures

28. @yloxford
@younglivesindia
Young Lives 2017-18
Classroom Observation Sub-study
Teacher-student interactions, equity and
learning
Caine Rolleston (Institute of Education, UCL)
Rhiannon Moore & Ana Grijalva
(Young Lives, University of Oxford)
@caine_rolleston
@rhi_moore

29. WHY CLASSROOM OBSERVATION?
• From the school survey, we can identify certain
teachers as more or less ‘effective’
• But we don’t know what is happening in those classrooms to
explain this
• Aim to unlock the ‘black box’ of the classroom
• The Classroom Observation study aimed to collect data
which can be used to answer RQs such as:
• To what extent do teacher-student classroom interactions
explain differences in student learning attainment in secondary
classrooms?
• What in terms of observed interactions in the classroom
explains higher and lower effectiveness (value-added)?
• What are the characteristics of classroom environments where
students learn more?
• How do teacher-student interactions vary between different
types of schools, and between schools in different localities?

30. Classroom Observation Study 2017-18

31. CLASSROOM OBSERVATION STUDY DESIGN
• 3 components of the study:
• Classroom observation using CLASS
• Semi-structured teacher questionnaires
• Classroom videos, coded using CLASS
• 45 maths/English teachers in 23 schools
• In 4 districts (2 in AP, 2 in Telangana)
• Teachers purposively sampled using
data from 2016-17 school effectiveness
survey, based on the following criteria:
• Mixture of teachers with high / low /
average VA
• Different school management types
• Urban / rural areas

32. OBSERVATION USING CLASS METHOD
• Classroom Observation was undertaken using the
Classroom Assessment Scoring System-Secondary (CLASS-S)
method
• Designed by Robert Pianta at University of Virginia
• CLASS aims to measure teacher-student interactions
identified as being important to how students learn
• Several studies have found that higher scores on CLASS are
positively associated with student academic performance
and positive academic attitudes
• The CLASS method was designed for use in the USA, but
has also been used in many other countries in South
America, Europe, Africa, and East Asia
• But it has never been used in India before this study

33. CLASS-S DIMENSIONS
• CLASS-S is based on 3 domains of teacher-student
interaction, split into 11 dimensions
Domain Dimension
Emotional Support
Positive climate
Teacher sensitivity
Regard for student perspectives
Classroom
organisation
Behaviour management
Productivity
Negative climate
Instructional Support
Instructional learning formats
Content understanding
Analysis and inquiry
Quality of feedback
Instructional dialogue
Student engagement

34. OBSERVING TEACHERS USING CLASS
• Observers trained and certified in use of the CLASS
method
• Observers work in pairs to observe each teacher
• This ‘double assessment’ increases score validity
• Teachers are observed for 4 cycles:
• 1 cycle = 30 mins (15 mins observing, 15 mins coding)
• So each teacher was observed for 2 lessons
• Each teacher is giving a score for each dimension using
the CLASS rubric
• Scores range between 1-7
• 1-2 is a ‘low score’, 6-7 is a ‘high score’
• These dimension scores are then averaged to give a
score for each domain

35. Classroom Observation Findings

36. VARIATION IN CLASSROOM PRACTICES
• Considerable variation in teacher CLASS scores
• Scores are highest in the ‘Classroom Organisation’
domain
• Similar pattern seen across all school types, districts and
rural / urban areas
Subject
Emotional Support
Classroom
Organisation
Instructional Support
Mean
score
SD Range
Mean
score
SD Range
Mean
score
SD Range
Maths 4.5 0.84 2.5 – 5.75 5.6 0.61 4.33 - 6.33 4.3 0.88 3 - 5.95
English 4.2 0.92 2.42 – 6.33 5.4 0.71 3.75 – 6.75 3.8 1.11
2.05 -
6.5

37. CLASSIFYING TEACHERS BY CLASS SCORE
• Teachers can then be classified by their CLASS score into
low / medium / close to high
• No teachers scored 6-7 so we have classified any teachers
achieving over 4.55 as being ‘close to high scoring’
Maths English
Number of
teachers
Range of
scores
Number of
teachers
Range of
scores
Close to high
CLASS score
6 5.25 - 5.75 2 5.92 – 6.33
Medium CLASS
score
16 3.33 – 5.08 13 3.17 – 5.17
Low CLASS score
1 2.5-2.5 7 2.42 – 4.17

38. WHO IS TAUGHT BY HIGH SCORING TEACHERS?
• Patterns in the characteristics of students taught by high / average /
low ranked teachers
• Students from more disadvantaged groups (e.g. SES, parental
background) are more likely to be taught by lower ranked teachers
0
20406080
100
Close to High Scoring teacher Medium Scoring teacher Low Scoring teacher
Never been to school Primary Secondary
Upper Secondary Higher ed.

39. DO HIGH SCORING TEACHERS ADD MORE VALUE?
• There is a positive association between CLASS ranking and teacher
contextual value-added score
• Stronger relationship for English than for maths
-40-20
0
2040
Close to high CLASS score Medium CLASS score Low CLASS score
Mean English VA Mean maths VA

40. CLASS SCORE & VALUE-ADDED
• English and maths teachers both score more highly in
Classroom Organisation than any other domain
• This is true for teachers who have above average VA and below
average VA
• Classroom Organisation scores also more consistent – less variation
234567
Above average VA Below average VA
Emotional Support Classroom Organisation Instructional Support
23456
MeanCLASSscore(maths)
Above average VA Below average VA
Emotional Support Classroom Organisation Instructional Support

41. Classroom Observation Video Clips

42. CLASSROOM OBSERVATION VIDEO: AIMS
• In addition to live observation we also filmed the lessons
of 6 teachers
• Teachers were selected based on high CLASS scores
• Also wanted a mixture of maths/English teachers and teachers
from government and private schools
• These videos were then coded using CLASS
• These videos provide some examples of the types of
teacher-student interactions taking place in the
classrooms where our classroom observation study took
place

43. Discussion

44. DISCUSSION
• Data from the Classroom Observation study provides
further evidence that children from disadvantaged
backgrounds in India are sorted into less effective schools
• Subject to a ‘double disadvantage’ in terms of home
background and school quality
• High scores in Classroom Organisation domain compared
to other domains – positive?
• No single route to good teaching – need for structured
ways for teachers to improve
• Findings suggest that CLASS is predictive of teacher
effectiveness in the Indian context