The yardstick for success is no longer improvement by national standards alone but the best performing education systems

1. OECD EMPLOYER
BRAND
Playbook
1
PISA 2012
Evaluating systems
to improve education
The yardstick for success is no longer
improvement by national standards
alone but the best performing
education systems
Bett 2014, School Leaders Summit
23 January 2014
Michael Davidson
Head of Early Childhood and
Schools Division, OECD Directorate
for Education and Skills

2. 2 PISA in brief
• Over half a million students…
– representing 28 million 15-year-olds in 65 countries/economies
… took an internationally agreed 2-hour test…
– Goes beyond testing whether students can
reproduce what they were taught…
… to assess students’ capacity to extrapolate from what they know and c
reatively apply their knowledge in novel situations
– Mathematics, reading, science, problem-solving, financial literacy
– Total of 390 minutes of assessment material
… and responded to questions on…
– their personal background, their schools
and their engagement with learning and school
• Parents, principals and system leaders provided data on…
– school policies, practices, resources and institutional factors that help
explain performance differences .

3. 3
Helen the Cyclist
Helen has just got a new bike. It has a speedometer which
sits on the handlebar. The speedometer can tell Helen the
distance she travels and her average speed for a trip.
Helen rode 6 km to her aunt’s house. Her speedometer
showed that she had averaged 18 km/h for the whole trip.
Which one of the following statements is correct?
A. It took Helen 20 minutes to get to her aunt’s house.
B. It took Helen 30 minutes to get to her aunt’s house.
C. It took Helen 3 hours to get to her aunt’s house.
D. It is not possible to tell how long it took Helen
to get to her aunt’s house.
PISA 2012 Sample Question 2

4. 4
Correct Answer: A. It took Helen 20 minutes to get to her aunt’s house.
This item belongs to the change and relationships category. This involves understanding
fundamental types of change and recognising when they occur in order to use suitable
mathematical models to describe and predict change.
SCORING:
Description: Calculate time travelled given average speed and distance
travelled
Mathematical
content area:
Change and relationships
Context: Personal
Process: Employ
Helen the Cyclist
PISA 2012 Sample Question 2

5. 5
Percent of 15-year-olds who scored Level 3 or Above
Shanghai-China
Singapore
HongKong-China
Korea
ChineseTaipei
Macao-China
Japan
Liechtenstein
Switzerland
Estonia
Netherlands
Finland
Canada
Poland
Vietnam
Germany
Belgium
Austria
Ireland
Denmark
Australia
CzechRepublic
Slovenia
NewZealand
France
UnitedKingdom
Iceland
OECDaverage
Latvia
Norway
Luxembourg
Portugal
Spain
Italy
RussianFederation
SlovakRepublic
Sweden
Lithuania
UnitedStates
Hungary
Israel
Croatia
Greece
Serbia
Turkey
Bulgaria
Romania
UnitedArabEmirates
Kazakhstan
Chile
Thailand
Malaysia
Uruguay
Montenegro
Mexico
Albania
Qatar
CostaRica
Brazil
Argentina
Tunisia
Jordan
Peru
Colombia
Indonesia
0
10
20
30
40
50
60
70
80
90
100
PISA 2012 Sample Question 2

6. What do 15-year-olds know…
…and what can they do with what they know?
Mathematics (2012)
6
Each year OECD countries spend 200bn$ on maths education in school

7. Singapore
Hong Kong-ChinaChinese Taipei
Korea
Macao-China
Japan Liechtenstein
Switzerland
Netherlands
Estonia Finland
Canada
Poland
Belgium
Germany Viet Nam
Austria Australia
IrelandSlovenia
DenmarkNew Zealand
Czech Republic France
United Kingdom
Iceland
LatviaLuxembourg Norway
Portugal ItalySpain
Russian Fed.Slovak Republic United States
LithuaniaSwedenHungary
Croatia
Israel
Greece
SerbiaTurkey
Romania
Bulgaria
U.A.E.
Kazakhstan
Thailand
Chile Malaysia
Mexico
410
420
430
440
450
460
470
480
490
500
510
520
530
540
550
560
570
580
Mean score
High mathematics performance
Low mathematics performance
… Shanghai-China performs above this line (613)
… 12 countries perform below this line
Average performance
of 15-year-olds in
Mathematics
Fig I.2.13

8. Socially equitable
distribution of learning
opportunities
High mathematics performance
Low mathematics performance
Average performance
of 15-year-olds in
mathematics
Strong socio-economic
impact on student
performance
Singapore
Hong Kong-ChinaChinese Taipei
Korea
Macao-China
Japan Liechtenstein
Switzerland
Netherlands
Estonia Finland
Canada
Poland
Belgium
Germany Viet Nam
Austria Australia
IrelandSlovenia
DenmarkNew Zealand
Czech Republic France
United Kingdom
Iceland
LatviaLuxembourg Norway
Portugal ItalySpain
Russian Fed.Slovak Republic United States
LithuaniaSwedenHungary
Croatia
Israel
Greece
SerbiaTurkey
Romania
Bulgaria
U.A.E.
Kazakhstan
Thailand
Chile Malaysia
Mexico

9. AustraliaAustria
Belgium Canada
Chile
Czech Rep.
Denmark
Estonia
Finland
France
Germany
Greece
Hungary
Iceland
Ireland
Israel
Italy
Japan
Korea
Luxembourg
Mexico
Netherlands
New Zealand
Norway
Poland
Portugal
Slovak Rep.
Slovenia
Spain Sweden
Switzerland
Turkey
UK
US
Singapore
Hong Kong-ChinaChinese Taipei
Macao-China
Liechtenstein
Viet Nam
Latvia
Russian Fed.
Lithuania
Croatia
Serbia
Romania
Bulgaria United Arab Emirates
Kazakhstan
Thailand
Malaysia
02468101214161820222426
2012
Shanghai-China
Socially equitable
distribution of learning
opportunities
Strong
socio-economic
impact on student
performance

10. 200
494
-3 -2 -1 0 1 2 3
School performance and socio-economic background:
United Kingdom10
AdvantagePISA Index of socio-economic backgroundDisadvantage
Student performance and students’ socio-economic background
School performance and schools’ socio-economic background
Student performance and students’ socio-economic background within schools
Studentperformance
700
Private school
Public school in rural area
Public school in urban area

11. 0
2
4
6
8
10
12
14
16
18
20 Shanghai-China
HongKong-…
Macao-China
Singapore
Korea
ChineseTaipei
Japan
Netherlands
Poland
Portugal
Germany
Italy
Spain
UnitedKingdom
France
Norway
Iceland
NewZealand
RussianFed.
UnitedStates
Denmark
Sweden
Hungary
SlovakRepublic
Mexico
Greece
Israel
Indonesia
Brazil
Chile
Argentina
%
Percentage of resilient students
More than 10
% resilient
Between 5%-10% of resilient
students Less than 5%
Fig II.2.4
11

12. AustraliaAustria
Belgium Canada
Chile
Czech Rep.
Denmark
Estonia
Finland
France
Germany
Greece
Hungary
Iceland
Ireland
Israel
Italy
Japan
Korea
Luxembourg
Mexico
Netherlands
New Zealand
Norway
Poland
Portugal
Slovak Rep.
Slovenia
Spain Sweden
Switzerland
Turkey
UK
US
Singapore
Hong Kong-ChinaChinese Taipei
Macao-China
Liechtenstein
Viet Nam
Latvia
Russian Fed.
Lithuania
Croatia
Serbia
Romania
Bulgaria United Arab Emirates
Kazakhstan
Thailand
Malaysia
02468101214161820222426
2012
Shanghai-China
Socially equitable
distribution of learning
opportunities
Strong socio-economic
impact on student
performance

13. AustraliaAustria
Belgium Canada
Chile
Czech Rep.
Denmark
Estonia
Finland
France
Germany
Greece
Hungary
Iceland
Ireland
Israel
Italy
Japan
Korea
Luxembourg
Mexico
Netherlands
New Zealand
Norway
Poland
Portugal
Slovak Rep.
Slovenia
Spain Sweden
Switzerland
Turkey
UK
US
Australia
Austria
Belgium
Canada
Chile
Czech Rep.
Denmark
Estonia
Finland
France
Germany
Greece
Hungary
Iceland
Ireland
Israel
Italy
Japan
Korea
Luxembourg
Mexico
Netherlands
New Zealand
Norway
Poland
Portugal
Slovak Rep.
Slovenia
Spain
Sweden
Switzerland
Turkey
UK
US
Singapore
2003 - 2012

14. AustraliaAustria
Belgium Canada
Chile
Czech Rep.
Denmark
Estonia
Finland
France
Germany
Greece
Hungary
Iceland
Ireland
Israel
Italy
Japan
Korea
Luxembourg
Mexico
Netherlands
New Zealand
Norway
Poland
Portugal
Slovak Rep.
Slovenia
Spain Sweden
Switzerland
Turkey
UK
US
Australia
Austria
Belgium
Canada
Chile
Czech Rep.
Denmark
Estonia
Finland
France
Germany
Greece
Hungary
Iceland
Ireland
Israel
Italy
Japan
Korea
Luxembourg
Mexico
Netherlands
New Zealand
Norway
Poland
Portugal
Slovak Rep.
Slovenia
Spain
Sweden
Switzerland
Turkey
UK
US
Singapore
2003 - 2012
Brazil, Italy, Macao-
China, Poland, Portugal,
Russian Federation,
Thailand and Tunisia
saw significant
improvements in math
performance between
2003 and 2012
(adding countries with more recent
trends results in 25 countries with
improvements in math)

15. AustraliaAustria
Belgium Canada
Chile
Czech Rep.
Denmark
Estonia
Finland
France
Germany
Greece
Hungary
Iceland
Ireland
Israel
Italy
Japan
Korea
Luxembourg
Mexico
Netherlands
New Zealand
Norway
Poland
Portugal
Slovak Rep.
Slovenia
Spain Sweden
Switzerland
Turkey
UK
US
Australia
Austria
Belgium
Canada
Chile
Czech Rep.
Denmark
Estonia
Finland
France
Germany
Greece
Hungary
Iceland
Ireland
Israel
Italy
Japan
Korea
Luxembourg
Mexico
Netherlands
New Zealand
Norway
Poland
Portugal
Slovak Rep.
Slovenia
Spain
Sweden
Switzerland
Turkey
UK
US
Singapore
2003 - 2012
Norway, the United States and
Switzerland improved equity
between 2003 and 2012

16. Of the 65 countries…
…40 improved at least in one subject
16

17. 17 Change in performance between PISA 2003 and 2012
Indonesia
Thailand
Russian Fed.
United States
Latvia
Spain
Norway
Luxembourg
Ireland
Austria
Switzerland
Japan
Liechtenstein
Korea
Brazil
Tunisia
Mexico
Uruguay
Turkey
Greece
Italy
Portugal
Hungary
Poland
Slovak Republic
OECD average
Germany
Sweden
France
Denmark
Iceland
Czech Republic
New Zealand
Australia
Macao-China
Belgium
Canada
Netherlands
Finland
Hong Kong-China
-4
-3
-2
-1
0
1
2
3
4
5
350 400 450 500 550 600
Averageannualmathematicsscorechange
Average mathematics performance in PISA 2003
ImprovingperformanceDeterioratingperformance
PISA 2003 performance below the OECD average
PISA 2003 performance
above the OECD average
Fig I.2.18

18. But underachievement persists
…
18

19. 0
10
20
30
40
50
60
70
80
90
100 Shanghai-China
Singapore
HongKong-China
Korea
Estonia
Macao-China
Japan
Finland
Switzerland
ChineseTaipei
Canada
Poland
Netherlands
Denmark
Ireland
Germany
Austria
Belgium
Australia
Latvia
Slovenia
CzechRepublic
Iceland
UnitedKingdom
Norway
France
NewZealand
OECDaverage
Spain
RussianFederation
Luxembourg
Italy
Portugal
UnitedStates
Sweden
SlovakRepublic
Hungary
Israel
Greece
Turkey
Chile
Mexico
Argentina
Brazil
Indonesia
%
Percentage of low-performing students
in mathematics
Tab I.2.1a
19
Across OECD, 23% of students are low
performers (Below Level 2). They can extract
relevant information from a single source
and can use basic algorithms, formulae,
procedures or conventions to solve
problems involving whole numbers.

20. Gender differences remain
20

21. Gender differences in mathematics performance Fig I.2.25
-50
-40
-30
-20
-10
0
10
20
30
Jordan
Qatar
Thailand
Malaysia
Iceland
U.A.E.
Latvia
Singapore
Finland
Sweden
Bulgaria
RussianFed.
Albania
Montenegro
Lithuania
Kazakhstan
Norway
Macao-China
Slovenia
Romania
Poland
Indonesia
UnitedStates
Estonia
ChineseTaipei
Shanghai-China
Belgium
Turkey
Greece
France
Hungary
Serbia
SlovakRepublic
Vietnam
Canada
Netherlands
OECDaverage
Portugal
Uruguay
Croatia
Israel
CzechRepublic
Australia
UnitedKingdom
Switzerland
Germany
Argentina
Denmark
Mexico
NewZealand
Tunisia
Ireland
HongKong-China
Spain
Brazil
Japan
Korea
Italy
Peru
Austria
Liechtenstein
CostaRica
Chile
Luxembourg
Colombia
Score-pointdifference(boys-girls)
Boys perform better than girls
Girls perform better than boys
21

22. Gender differences in reading performance
-80
-70
-60
-50
-40
-30
-20
-10
0
Jordan
Qatar
Bulgaria
Montenegro
Finland
Slovenia
U.A.E.
Lithuania
Thailand
Latvia
Sweden
Iceland
Greece
Croatia
Norway
Serbia
Turkey
Germany
Israel
France
Estonia
Poland
Romania
Malaysia
RussianFed.
Hungary
SlovakRepublic
Portugal
Italy
CzechRepublic
Argentina
OECDaverage
Austria
Kazakhstan
Switzerland
Macao-China
Uruguay
Canada
Australia
NewZealand
ChineseTaipei
Singapore
Belgium
VietNam
UnitedStates
Denmark
Tunisia
Brazil
Luxembourg
Spain
Ireland
Indonesia
Netherlands
HongKong-China
CostaRica
UnitedKingdom
Liechtenstein
Japan
Shanghai-China
Mexico
Korea
Chile
Peru
Colombia
Albania
Score-pointdifference(boys-girls)
In all countries and economies
girls perform better than boys
Fig I.4.12
22

23. The share of immigrant students in OECD countries
increased from 9% in 2003 to 12% in 2012…
…while the performance disadvantage of immigrant students
reduced by 11 score points during the same period (after
accounting for socio-economic factors)
23

24. -100
-50
0
50
100
150
Australia
Israel
UnitedStates
Hungary
ongKong-China
Singapore
SlovakRepublic
Canada
NewZealand
ChineseTaipei
Ireland
Turkey
UnitedKingdom
Luxembourg
Latvia
Chile
Argentina
RussianFed.
CzechRepublic
OECDaverage
Germany
Greece
Norway
Estonia
Iceland
Italy
Austria
Netherlands
Spain
France
Portugal
Sweden
Denmark
Switzerland
Belgium
Brazil
Finland
Mexico
Shanghai-China
Scorepointdifference
before accounting for students' socio-economic status after accounting for students' socio-economic status
Differences in mathematics performance between
students without and with an immigrant background
Students without an immigrant
background perform better
Students with an immigrant
background perform better
Fig II.3.4
24

25. Disciplinary climate improved
Teacher-student relations improved between 2003 and 2012 in all but
one country; and disciplinary climate also improved during the period,
on average across OECD countries and in 27 individual countries
25

26. Hong Kong-China
Brazil
Uruguay
Albania
Latvia
Lithuania
Chinese Taipei
Thailand
Bulgaria
Jordan
UAE Argentina
Indonesia
Kazakhstan
Peru
Costa Rica
Montenegro
Tunisia
Qatar
Singapore
Colombia
Malaysia
Russian Fed.
Romania
Viet Nam
Shanghai-China
USA
Poland
New Zealand
Greece
UK
Estonia
Finland
Luxembourg
Germany
Austria
Czech Rep.
France
Japan
TurkeySweden
Hungary
Australia
Canada
Chile
Belgium
Netherlands
Spain
Switzerland
Slovenia
Portugal
Norway
Mexico
Korea
Italy
R² = 0.16
300
350
400
450
500
550
600
650
0 10 20 30 40 50 60 70
Mathematicsperformance(scorepoints)
Percentage of students in schools who skipped at least one day of school in the two weeks prior to
the PISA test
Countries with large proportions of truants
perform worse in mathematics
Fig IV.1.22

27. Social and emotional dimensions matter too
Students’ Engagement, Drive and Self-Beliefs
are all related to their performance
28

28. 0 20 40 60 80 100
Agree: I feel like I belong at school
Disagree: I feel lonely at school
Agree: I feel happy at school
Agree: Things are ideal in my school
Agree: I am satisfied with my school
%
Korea OECD average United Kingdom
Students' sense of belonging
Percentage of students who agree/disagree with the following statements:
Fig III.2.12
29

29. Students’ mathematics self-efficacy
Percentage of students who feel very confident or confident about having to do the foll
owing tasks in mathematics:
30 40 50 60 70 80 90 100
Using a <train timetable> to work out how
long it would take to get from one place…
Calculating how much cheaper a TV would
be after a 30% discount
Calculating how many square metres of
tiles you need to cover a floor
Understanding graphs presented in
newspapers
Solving an equation like 3x+5=17
Finding the actual distance between two
places on a map with a 1:10 000 scale
Solving an equation like 2(x+3)=(x+3)(x-3)
Calculating the petrol-consumption rate of
a car
%
Korea OECD average United Kingdom
Fig III.4.2
30

30. Money makes a difference…
…but only up to a point
31

31. Spending per student from the age of 6 to 15 and
mathematics performance in PISA 2012
Slovak Republic
Czech Republic
Estonia
Israel
Poland
Korea
Portugal
New Zealand
Canada
Germany
Spain
France
Italy
Singapore
Finland
Japan
SloveniaIreland
Iceland
Netherlands
Sweden
Belgium
UK
Australia
Denmark
United States
Austria
Norway
Switzerland
Luxembourg
Viet Nam
Jordan
Peru
Thailand
Malaysia
Uruguay
Turkey
Colombia
Tunisia
Mexico
Montenegro
Brazil
Bulgaria
Chile
Croatia
Lithuania
Latvia
Hungary
Shanghai-China
R² = 0.01
R² = 0.37
300
350
400
450
500
550
600
650
0 20 000 40 000 60 000 80 000 100 000 120 000 140 000 160 000 180 000 200 000
Mathematicsperformance(scorepoints)
Average spending per student from the age of 6 to 15 (USD, PPPs)
Cumulative expenditure per student less than USD 50 000
Cumulative expenditure per student USD 50 000 or more
Fig IV.1.8

32. Governance matters
Schools with more autonomy over curricula and assessments tend to
perform better than schools with less autonomy where they are part of
school systems with more accountability arrangements and greater
teacher-principal collaboration in school management
33

33. Schools with more autonomy perform better than schools with
less autonomy in systems with more collaboration
Teachers don't participate in
management
Teachers participate in
management455
460
465
470
475
480
485
Less school autonomy
More school autonomy
Score points
School autonomy for resource allocation x System's level of teachers
participating in school management
Across all participating countries and economies
Fig IV.1.17

34. Lessons from PISA
on successful
education systems

35. Thank you !
Find out more about PISA at www.pisa.oecd.org
• All national and international publications
• The complete micro-level database
Email: Michael.Davidson@OECD.org

36. Do you have an idea on how to use this data
to improve education in your country?
Would you like to work with
us to develop that idea?
Apply to the
Thomas J. Alexander
fellowship programme!
http://www.oecd.org/edu/thomasjalexanderfellowship.htm