Conferencia de Alfonso Echazarra, analista de la OCDE, sobre los resultados de PISA 2015 y el futuro de esta evaluación presentada dentro del Simposio Ciencias e Inglés en la evaluación internacional. La cultura de la evaluación en Ciencias e Inglés.
Simposio “Ciencias e Inglés en la evaluación internacional”: PISA 2015 results and future developments
1. Name of Speaker
PISA 2015 results and
future developments
Alfonso Echazarra
Analista PISA-OCDE
2. PISA 2015: A summary
• Approximately 540 000 students…
representing about 29 million 15-year-olds in the schools of the 72 participating
countries and economies
… took a computer-based test lasting a total of 2 hours…
Not only evaluates if students can reproduce what they have learned at school…
…assesses students’ capacity to apply creatively their knowledge and skills in a
variety of situations
…and answered questions about their schools, personal context and attitudes towards
learning
• Parents, principals, teachers and policy-makers provided information about …
School policies, practices, resources and institutional factors that can explain the
differences in performance
3. PISA 2015: A summary
• New features
Science as core subject
Computer-based assessment in most education systems
Log-file data
Collaborative problem-solving
Teacher questionnaire
• A tool to learn and improve
Collaboration between countries, experts and social agents sharing experiences,
policies and best practices
Data triangulation combining the perspectives from students, teachers,
principals, parents and policy-makers
Evidence-based and constructive dialogue
5. PISA 2015 reports
• OECD Reports
Volumen I: Excellence and Equity in Education
Volumen II: Policies and Practices for Successful Schools
Volumen III (Spring 2017): Students’ Well-Being
Volumen IV (Spring 2017): Students’ Financial Literacy
Volumen V (Autumn 2017): Collaborative Problem Solving
Thematic reports (2018/2019): To be decided
Options: Teaching and learning in science, rural and urban schools,
out-of-school learning, …
• National reports.
PISA 2015. Programa para la evaluación internacional de alumnos:
Informe español
…
6. Performance in science
Figure I.4.2
300
350
400
450
500
550
Singapore
Japan
Estonia
ChineseTaipei
Finland
Macao(China)
Canada
VietNam
HongKong(China)
B-S-J-G(China)
Korea
NewZealand
Slovenia
Australia
UnitedKingdom
Germany
Netherlands
Switzerland
Ireland
Belgium
Denmark
Poland
Portugal
Norway
UnitedStates
Austria
France
Sweden
OECDaverage-35
CzechRepublic
Spain
Latvia
Russia
Luxembourg
Italy
Hungary
Lithuania
Croatia
CABA(Argentina)
Iceland
Israel
Malta
SlovakRepublic
Greece
Chile
Bulgaria
UnitedArabEmirates
Uruguay
Romania
Moldova
Albania
Turkey
TrinidadandTobago
Thailand
CostaRica
Qatar
Colombia
Mexico
Montenegro
Georgia
Jordan
Indonesia
Brazil
Peru
Lebanon
Tunisia
FYROM
Kosovo
Algeria
DominicanRepublic
Score points Mean science performance Confidence interval (95%)
8. Mean performance in science, by international deciles of the
PISA index of economic, social and cultural status (ESCS)
250
300
350
400
450
500
550
600
650
VietNam76
Macao(China)22
B-S-J-G(China)52
Japan8
Singapore11
HongKong(China)26
ChineseTaipei12
Estonia5
Finland2
Korea6
Portugal28
Germany7
Canada2
Poland16
Spain31
UnitedKingdom5
Latvia25
Slovenia5
Switzerland8
Australia4
NewZealand5
Ireland5
CzechRepublic9
Denmark3
Hungary16
OECDaverage12
Netherlands4
France9
Italy15
Belgium7
Norway1
Sweden3
Austria5
Russia5
UnitedStates11
Croatia10
Lithuania12
CABA(Argentina)18
SlovakRepublic8
Chile27
Luxembourg14
Iceland1
Malta13
Uruguay39
Greece13
Romania20
Israel6
Turkey59
Indonesia74
Moldova28
Mexico53
Thailand55
Bulgaria13
Colombia43
CostaRica38
TrinidadandTobago14
Peru50
Jordan21
Montenegro11
UnitedArabEmirates3
Brazil43
Georgia19
Tunisia39
Lebanon27
FYROM13
Algeria52
Qatar3
Kosovo10
DominicanRepublic40
Scorepoints
Bottom decile Second decile Middle decile Ninth decile Top decile
Figure I.6.7
% of
students in
the bottom
international
deciles of
ESCS
OECD median student
9. Percentage of low-achievers in science
Table II.2.2a
0
10
20
30
40
50
60
70
80
90
VietNam
Macao(China)
Estonia
HongKong(China)
Singapore
Japan
Canada
Finland
ChineseTaipei
Korea
Slovenia
Ireland
Denmark
B-S-J-G(China)
Poland
Germany
Latvia
Portugal
UnitedKingdom
NewZealand
Australia
Russia
Spain
Switzerland
Netherlands
Norway
Belgium
UnitedStates
CzechRepublic
Austria
OECDaverage-35
Sweden
France
CABA(Argentina)
Italy
Croatia
Lithuania
Iceland
Luxembourg
Hungary
SlovakRepublic
Israel
Malta
Greece
Chile
Bulgaria
Romania
Uruguay
Albania
UnitedArabEmirates
Moldova
Turkey
TrinidadandTobago
CostaRica
Thailand
Mexico
Colombia
Jordan
Qatar
Georgia
Montenegro
Indonesia
Brazil
Peru
Lebanon
FYROM
Tunisia
Kosovo
Algeria
DominicanRepublic
%
Percentage of students below proficiency level 2
10. Percentage of top performers in science
Table II.2.2a
0
5
10
15
20
25
Singapore
ChineseTaipei
Japan
Finland
B-S-J-G(China)
Estonia
NewZealand
Canada
Australia
Netherlands
UnitedKingdom
Korea
Slovenia
Germany
Switzerland
Macao(China)
Belgium
UnitedStates
Sweden
VietNam
France
Norway
OECDaverage-35
Austria
Malta
Portugal
HongKong(China)
Poland
CzechRepublic
Ireland
Denmark
Luxembourg
Israel
Spain
Hungary
Lithuania
Italy
Croatia
Latvia
Iceland
Russia
SlovakRepublic
Bulgaria
UnitedArabEmirates
CABA(Argentina)
Greece
Qatar
TrinidadandTobago
Uruguay
Chile
Georgia
Moldova
Romania
Brazil
Montenegro
Thailand
Lebanon
Albania
Colombia
Turkey
FYROM
Jordan
CostaRica
Peru
Mexico
Indonesia
Tunisia
Algeria
DominicanRepublic
Kosovo
%
Percentage of students at proficiency levels 5 and 6
11. The global pool of top performers: A PISA perspective
Figure I.2.18
United States;
21,7%
B-S-J-G (China);
13,1%
Japan; 12,6%
Germany; 5,7%
Viet Nam; 5,2%
United Kingdom; 4,9%
Korea; 4,4%
France; 4,3%
Russia; 3%
Canada; 3%
Chinese Taipei;
2,8%
Australia; 2,1%
Poland; 1,8%
Netherlands; 1,5%
Italy; 1,5%
Spain; 1,4%
Brazil; 1,2%
Singapore; 0,8%
Belgium; 0,7% Finland; 0,6%
Switzerland; 0,6%
Sweden; 0,6%Portugal; 0,5% New Zealand;
0,5%
Israel; 0,5% Others; 4,9%
12. Performance in reading
Figure I.4.2
300
350
400
450
500
550
Singapore
HongKong(China)
Canada
Finland
Ireland
Estonia
Korea
Japan
Norway
NewZealand
Germany
Macao(China)
Poland
Slovenia
Netherlands
Australia
Sweden
Denmark
France
Belgium
Portugal
UnitedKingdom
ChineseTaipei
UnitedStates
Spain
Russia
B-S-J-G(China)
OECDaverage-35
Switzerland
Latvia
CzechRepublic
Croatia
VietNam
Austria
Italy
Iceland
Luxembourg
Israel
CABA(Argentina)
Lithuania
Hungary
Greece
Chile
SlovakRepublic
Malta
Uruguay
Romania
UnitedArabEmirates
Bulgaria
Turkey
CostaRica
TrinidadandTobago
Montenegro
Colombia
Mexico
Moldova
Thailand
Jordan
Brazil
Albania
Qatar
Georgia
Peru
Indonesia
Tunisia
DominicanRepublic
FYROM
Algeria
Kosovo
Lebanon
Score points Mean reading performance Confidence interval (95%)
13. Performance in mathematics
Figure I.5.2
300
350
400
450
500
550
Singapore
HongKong(China)
Macao(China)
ChineseTaipei
Japan
B-S-J-G(China)
Korea
Switzerland
Estonia
Canada
Netherlands
Denmark
Finland
Slovenia
Belgium
Germany
Poland
Ireland
Norway
Austria
NewZealand
VietNam
Russia
Sweden
Australia
France
UnitedKingdom
CzechRepublic
Portugal
OECDaverage-35
Italy
Iceland
Spain
Luxembourg
Latvia
Malta
Lithuania
Hungary
SlovakRepublic
Israel
UnitedStates
Croatia
CABA(Argentina)
Greece
Romania
Bulgaria
UnitedArabEmirates
Chile
Turkey
Moldova
Uruguay
Montenegro
TrinidadandTobago
Thailand
Albania
Mexico
Georgia
Qatar
CostaRica
Lebanon
Colombia
Peru
Indonesia
Jordan
Brazil
FYROM
Tunisia
Kosovo
Algeria
Lebanon
DominicanRepublic
Score points Mean mathematics performance Confidence interval (95%)
14. Students’ career expectations
Figure I.3.2
0
5
10
15
20
25
30
35
40
45
50
DominicanRep.12
CostaRica11
Jordan6
UnitedArabEm.11
Mexico6
Colombia8
Lebanon15
Brazil19
Peru7
Qatar19
UnitedStates13
Chile18
Tunisia19
Canada21
Slovenia16
Turkey6
Australia15
UnitedKingdom17
Malaysia4
Kazakhstan14
Spain11
Norway21
Uruguay17
Singapore14
TrinidadandT.13
Israel25
CABA(Arg.)19
Portugal18
Bulgaria25
Ireland13
Kosovo7
Algeria12
Malta11
Greece12
NewZealand24
Albania29
Estonia15
OECDaverage19
Belgium16
Croatia17
FYROM20
Lithuania21
Iceland22
Russia19
HKG(China)20
Romania20
Italy17
Austria23
Moldova7
Latvia19
Montenegro18
France21
Luxembourg18
Poland13
Macao(China)10
ChineseTaipei21
Sweden21
Thailand27
VietNam13
Switzerland22
Korea7
Hungary22
SlovakRepublic24
Japan18
Finland24
Georgia27
CzechRepublic22
B-S-J-G(China)31
Netherlands19
Germany33
Indonesia19
Denmark48
%
Percentage of students who expect to work in science-related professional and
technical occupations when they are 30
Science-related technicians and associate professionals
Information and communication technology professionals
Health professionals
Science and engineering professionals
%ofstudentswith
vagueormissing
expectations
15. Boys and girls’ expectations of a science career, OECD average
Figure I.3.5
0 5 10 15 20
...science and engineering
professionals
...health professionals
...information and
communication technology
(ICT) professionals
...science-related technicians
or associate professionals
%
Girls Boys
Students who expect to work as...
16. Student performance in science, by immigrant background
Figure I.7.4
300
350
400
450
500
550
600
650
Greece
CostaRica
Jordan
CABA(Argentina)
Israel
Sweden
France
Slovenia
Austria
Germany
Netherlands
Denmark
Italy
Norway
Belgium
OECDaverage
Spain
Croatia
UnitedStates
Luxembourg
Switzerland
Qatar
Portugal
Russia
UnitedArabEmirates
UnitedKingdom
Ireland
Australia
Estonia
HongKong(China)
NewZealand
Canada
Macao(China)
Singapore
Score points Non-immigrant students Second-generation immigrant students First-generation immigrant students
Only countries where the
immigrant student population
>6.25% are shown
18. 10%
28%
62%
OECD countries
Variation in science performance
between systems, schools and students
22%
26%
53%
All countries and
economies
Figure II.7.1
Education systems
Schools
Students
19. Higher-performing education systems in science-related outcomes
ABOVE-AVERAGE
SCIENCE
PERFORMANCE
STRONGER
THAN
AVERAGE
EPISTEMIC
BELIEFS
ABOVE-AVERAGE
PERCENTAGE OF
STUDENTS EXPECTING TO
WORK IN A SCIENCE-
RELATED OCCUPATION
Norway
Belgium
B-S-J-G (China)
Estonia
Finland
Germany
Japan
Korea
Macao (China)
Netherlands
Poland
Switzerland
Viet Nam
CABA (Argentina)
Israel
Spain
United Arab Emirates
United States
Croatia
Georgia
Iceland
Lithuania
Malta
Sweden
Brazil
Bulgaria
Chile
Colombia
Costa Rica
Dominican Republic
Jordan
Kosovo
Lebanon
Denmark
Hong Kong
(China)
New Zealand
Chinese Taipei
Australia
Canada
Ireland
Portugal
Singapore
Slovenia
United Kingdom
Mexico
Peru
Qatar
Trinidad and Tobago
Tunisia
Turkey
Uruguay
Figure II.2.2
20. Science-related extracurricular activities offered at school
Figure II.2.9
0
10
20
30
40
50
60
70
80
90
100
HongKong(China)
Korea
B-S-J-G(China)
Thailand
Qatar
UnitedArabEmirates
ChineseTaipei
Poland
UnitedKingdom
Russia
Montenegro
UnitedStates
Macao(China)
Romania
Malta
Algeria
Bulgaria
SlovakRepublic
Japan
Tunisia
Indonesia
Israel
Portugal
Canada
Slovenia
Croatia
Hungary
Kosovo
Jordan
DominicanRepublic
CABA(Argentina)
NewZealand
Germany
Albania
CzechRepublic
Italy
Latvia
VietNam
Lebanon
Estonia
Turkey
Singapore
OECDaverage
Georgia
FYROM
TrinidadandTobago
Australia
Switzerland
Chile
Uruguay
Colombia
Ireland
Lithuania
Luxembourg
Mexico
Peru
France
CostaRica
Greece
Netherlands
Moldova
Spain
Finland
Brazil
Iceland
Denmark
Sweden
Belgium
Austria
Norway
%
Percentage of students in schools offering:
Science club Science competitions
32. CABA (Argentina)
Costa Rica
Sweden
Bulgaria Romania
Viet
Nam
Uruguay
United States
Norway
Chile
Hungary
B-S-J-G
(China)
Turkey
Mexico
Portugal
Iceland
Korea
Albania
Japan
Trinidad and Tobago
UAE
Algeria Ireland
Indonesia
New
Zealand
Colombia
Peru
Macao (China) Spain
Switzerland
Lebanon
Netherlands
Slovak
Republic
UK
Slovenia
Brazil
Kosovo
Finland
Thailand
Latvia
R² = 0.20
40
50
60
70
80
90
100
40 50 60 70 80 90 100
Socio-economicinclusionacrossschools
Academic inclusion across schools (%)
OECD average
OECD
average
High academic inclusion
Low socio-economic inclusion
Low academic inclusion
High socio-economic inclusion
Academic and social inclusion across schools
Figure II.5.12
35. Learning time and science performance
Figure II.6.23
6
7
8
9
10
11
12
13
14
15
16
0
10
20
30
40
50
60
70
Finland
Germany
Switzerland
Japan
Estonia
Sweden
Netherlands
NewZealand
Australia
CzechRepublic
Macao(China)
UnitedKingdom
Canada
Belgium
France
Norway
Slovenia
Iceland
Luxembourg
Ireland
Latvia
HongKong(China)
OECDaverage
ChineseTaipei
Austria
Portugal
Uruguay
Lithuania
Singapore
Denmark
Hungary
Poland
SlovakRepublic
Spain
Croatia
UnitedStates
Israel
Bulgaria
Korea
Russia
Italy
Greece
B-S-J-G(China)
Colombia
Chile
Mexico
Brazil
CostaRica
Turkey
Montenegro
Peru
Qatar
Thailand
UnitedArabEmirates
Tunisia
DominicanRepublic
Scorepointsinscienceperhouroftotallearningtime
Hours Intended learning time at school (hours) Study time after school (hours) Score points in science per hour of total learning time
36. Science performance and learning time
Figure II.6.23
Finland
Germany Switzerland
Japan Estonia
Sweden
Netherlands
New Zealand
Macao
(China)
Iceland
Hong Kong
(China) Chinese Taipei
Uruguay
Singapore
Poland
Spain
United States
Israel
Bulgaria
Korea
Russia Italy
Greece
B-S-J-G (China)
Colombia
Chile
Mexico
Brazil
Costa
Rica
Turkey
Montenegro
Peru
Qatar
Thailand
United
Arab
Emirates
Tunisia
Dominican
Republic
R² = 0.21
300
350
400
450
500
550
600
35 40 45 50 55 60
PISAsciencescore
Total learning time in and outside of school
OECD average
OECD average
OECDaverage
37. • PISA 2018
Global competences: assessment and questionnaire
Students’ well-being questionnaire: initiative linked to the Better
Life Initiative looking to offer a holistic view about the well-being
of students (economic situation, housing, health, education,
physical safety, sense of well-being, …)
Overlap with TALIS survey about teachers
Adaptive testing
• PISA 2021
Creativity and critical thinking?
• PISA 2024
Skills in foreign languages?
Entrepreneurship?
PISA 2018 and beyond: New developments and proposals
38. Learning strategies (2000)
Problem-solving (2003)
Embedding of attitudinal aspects in assessment (2006)
Digital literacy (2009)
Creative problem-solving (2012)
Collaborative problem-solving (2015)
Global competences (2018)
Creativity and critical thinking (2021)?
Developmental domains
39. Global competence is the capacity to analyse global and
intercultural issues critically and from multiple perspectives,
to understand how differences affect perceptions,
judgments, and ideas of self and others, and to engage in
open, appropriate and effective interactions with others
from different backgrounds on the basis of a shared respect
for human dignity.
PISA definition of Global Competence
40. How well are students prepared for life and employment in culturally diverse
societies and in a globalised world?
How much are students exposed to global news and how do they understand
and critically analyse intercultural and global issues?
What approaches to multicultural, intercultural and global education are used
at school?
What approaches are used to educate culturally diverse students and how are
schools leveraging this diversity to develop students’ global competence?
What approaches are used to stimulate peer-to-peer learning between
students from different cultures?
How well are schools contesting cultural and gender biases and stereotypes,
including their own?
Some questions PISA seeks to answer
41. Schools can:
provide opportunities for young people to learn about global developments
that affect the world and their lives
teach students how they can develop a fact-based and critical worldview of
today
equip students with the means to access and analyse a broad range of
cultural practices and meanings
engage students in experiences that facilitate international and intercultural
relations
promote the value of diversity, which in turn encourages sensitivity, respect
and appreciation.
Schools can make a difference
42. · Openness towards people from
other cultures
· Respect for other cultures
· Global-mindedness
· Knowledge of global issues
· Intercultural knowledge
·Analytical and critical thinking
·Perspective taking
· Respectful communication
· Adaptability
Components
Skills Knowledge Attitudes
Values
· Valuing Human Dignity
· Valuing Cultural Diversity
GLOBAL COMPETENCE
“Skills” are the capacities for carrying out a complex pattern of either thinking (in
the case of a cognitive skill) or behaviour (in the case of a behavioural skill) in
order to achieve a particular goal.
Global Competence requires numerous skills, including the ability to:
communicate in more than one language; communicate appropriately and
effectively with people from other cultures or countries; comprehend other
people’s thoughts, beliefs and feelings, and see the world from their perspectives;
adjust one’s thoughts, feelings or behaviours to fit new contexts and situations;
and analyse and think critically in order to scrutinise and appraise information and
meanings
An individual may have a large range of knowledge,
understanding and skills, but lack the disposition to use them. An
“attitude” may be defined as the overall mind-set which an
individual adopts and typically consists of four components: a
belief or opinion about the object, an emotion or feeling towards
the object, an evaluation (either positive or negative) of the
object, and a tendency to behave in a particular way towards that
object.
The dimensions of Global Competence
43. The cognitive test – from information to critical understanding of
global and intercultural issues
Analytical, critical and
perspective taking skills
· Select information
· Assess claims
· Explain issues
· Recognize contexts and
perspectives
·Understand implications
Knowledge
·Knowledge of global
issues
· Intercultural knowledge
Global and
Intercultural
Understanding
Contexts
Personal, Local, Global
Attitudes
·Interest in other cultures
·Interest in global issues
·Global mindedness
·Respect
The PISA test will assess how students can use their
knowledge and critical thinking skills to understand
issues of critical importance to the world (global
issues) and issues that affect open and respectful
interactions across cultures (intercultural issues).
Note: Belgium refers only to French and German-speaking communities
Note: Belgium refers only to French and German-speaking communities
Rural: fewer than 3 000 people; Town: 3 000 to 100 000 people; City: over 100 000 people
Countries and economies that are not above-average in any of the three variables: Albania, Algeria, Austria, Czech Republic, France, FYROM, Greece, Hungary, Indonesia, Italy, Latvia, Luxembourg, Moldova, Montenegro, Romania, Russia, Slovak Republic and Thailand.
Repeated a grade at least once in primary, lower secondary or upper secondary
Cross-country analysis. All variables included in the same regression model. R2: 44%.