To facilitate maximally open access to engineering higher education without compromising standards or unfairly exposing unequipped students.

1. Enhance the Attractiveness of Studies in Science and Technology
WP 6: Formal Barriers
Kevin Kelly
Trinity College Dublin
WP 6 Co-ordinator

2. WP 6: Formal Barriers
Aim of the Work Package:
• To examine the formal barriers to engineering
education at third-level
• To document & recommend measures to
facilitate maximally open access to
engineering higher education without
compromising standards or unfairly exposing
unequipped students

3. WP6 Key Deliverables
Three phases of work:
• Survey of education systems in partner countries
• Comparison Framework
• Report on formal barriers to engineering higher
education

4. Status of Deliverables
1: Survey of Education Systems
Current Status: Completed
• Extensive questionnaire developed and distributed to
all partners
• Questions investigated national education systems
from primary level to university entry, regarding
provision of science, engineering & technology (SET)
subjects
• Survey data gathered was used to inform other WP 6
deliverables

5. Status of Deliverables
2: Comparison Framework
Current Status: Completed (pending final contributions)
• Aim is to provide ‘at a glance’ information to compare
partner countries under key headings, relevant to all
work packages
• Provides necessary context to enable conclusions to
be drawn between national structures with significant
variations
• Combination of graphs, tables and textual information
used

6. Status of Deliverables
3: Report on Formal Barriers
Current Status: Finalising report
The report documents:
• main factors restricting access to engineering higher
education
• research illustrating the impact of these barriers on
access to engineering
• data analysis of the relationship between entry
barriers and subsequent student progression
14.4.2011
6

7. Report on Formal Barriers
Main categories of barriers identified:
• Entry requirements for engineering courses
• Structures within the school system
– such as streams which force students to choose at an early age
between academic or vocational pathways
• Socio-economic factors
– engineering programmes in several countries appear to be less
diverse in socio-economic terms than other programmes

8. 1. Entry Requirements
Key points:
• No subject requirements set for entry in Italy and
Belgium
• Mathematics required in all other countries, plus
Physics and Chemistry in most
• Limited students eligible for entry as a result
– max 12% eligible in Sweden and Ireland

9. 1. Entry Requirements (cont’d)
Do entry requirements serve the intended purpose?
• Evidence found in some partner countries of a link
between achievement in the subjects required for entry,
and subsequent performance in engineering university
programmes
– e.g. in IST students who have grades lower than those now
required have significant difficulty in successfully completing the
programme

10. 1. Entry Requirements (cont’d)
Alternatives to standard entry requirements
• Alternative entry routes exist in most partner countries
• Function: To facilitate access to university for non-
traditional students & those not meeting standard entry
requirements
• Proportion of students entering via these means varies
from 0% (Italy) to 29% (Sweden)

11. 2. High School System
Structural Factors:
• Separation between academic and vocational
branches of high school in most countries
• This impacts on options for higher education 
vocational students may not be eligible for academic
study at university
• Subject specialisation creates further restrictions
– students who don’t specialise in science/technology may be
ineligible for engineering programmes

12. 2. High School System (cont’d)
Gender and subject specialisation:
• Persistent gender differences in uptake of
engineering-relevant subjects, especially Physics
• In Ireland some technology-related subjects are not
taught in all-girls schools
• Research suggests causal relationship between lower
% of girls specialising in Physics, Chemistry and
Mathematics in high school and low numbers of
female students entering engineering programmes at
university

13. 3. Socio-economic Factors
• Engineering courses in many countries are less
accessible to students from lower socio-economic
backgrounds than other fields (Eurostudent IV)
• This is reflected in several ATTRACT countries
– in Ireland, Sweden and Italy engineering programmes tend to
be less socio-economically diverse than other programmes
• In Ireland, PISA mathematics scores were significantly
higher among students designated as ‘high socio-
economic status’ than among others

14. Aalto Trinity
KU Leuven PoliTo IST KTH Uppsala
University College
Country Belgium Finland Ireland Italy Portugal Sweden Sweden
Multi- Multi-
University Type General General Technical Technical Technical
disciplinary disciplinary
National Ranking #1 n/a #1 #2 #2 #4 #3
Government: Government: Government: Government: Government:
Government: Governmen
75% 71% 66% 45% 41%
80% t: 80%
Student fees:
10%
Core Funding
Private Student fees: Student fees: Private
Sources Private Public
sources/contrac 24% 9% sources:
donations: 29% funds: 8%
t research: 15% Research 13%
income: 43%
Private
Other (own
Other: 10% Other: 10% Other: 2% Other: 7% sources:
income): 50%
12%
# of students
studying to
36,820 17,020 11,290 18,792 9,445 14,000 20,000
degree/accredited
professional level
9% 25% 6% 75% 94% 100% 12%
· % studying
(4,124 (4,289 (14,053 (8,832 (14,000 (2,300
engineering (700 students)
students) students) students) students) students) students)
# of advanced or
4,454 2,496 3,335 1,135 1,500 2,000
doctoral students
21% 26% 14% n/a 69% 100% 5%
· % studying
(1,500 (100
engineering (964 students) (657 students) (460 students) (779 students)
students) students)

15. Comparison of high school systems
Belgium Ireland Portugal Sweden Finland Italy
Tracks
(e.g.
N N Y Y Y Y
literature,
science)
Higher
and lower N Y N N N N
levels
Core
Y N Y Y Y Y
subjects
Student
choice
Y N N Y Y N
over
quantity

16. Structure of Education Systems

17. Structure of Education Systems

18. Contact Time in Schools

19. Exposure to Mathematics

20. Finland Ireland Italy Portugal Sweden
School certificate Yes Yes Yes Yes No
exams
and/or - - and/or -
Ongoing Yes No No Yes Yes
performance at
second-level
General
and/or - - and/or -
admission
requirements
Entrance exams Yes No No No No
(Managed by
institution)
- - - and/or and
Other n/a n/a n/a Yes† Yes‡

21. Portuga
Finland Ireland Italy Sweden
l
Yes* Yes – No Yesˠ Yes
Maths
55% +
Physics Yes* No Yesˠ Yes
Approx.
Additional admission
Yes* 10% of No Yesˠ Yes
requirements for Chemistry
courses
Engineering courses
No require one No No Required
Biology science in certain
subject courses
Advanced 12% n/a 38% 11%
mathematics:
% of students who meet STEM 42%
requirements Physics/
Chemistry:
17%

22. Purpose of Barriers
1. Identification of student ability
2. Pre-requisite knowledge (i.e. university
does not need to teach this!)

23. Appropriateness & Effectiveness
• Reasons (historical) for design and implementation of
barriers
• Evidence of whether (appropriate) and how well
(effective) these barriers work
• Pre-requisite knowledge – by definition it is effective. Appropriate is
more difficult to say!
• Students who pass barriers should do better than those who don’t.
– Those who don’t pass barriers aren’t let in!
– Use excess of performance over barriers to measure how well these
metrics capture ability to progress

24. Barrier Effectiveness – Italy
Number of students who took a low note at the entry
250
200
150
test
100
50
0
0 8 10 16 18 20 26 28 30 36 38 46
number of credits passed the first year

25. Barrier Effectiveness – Italy
Number of students who took a high note at the entry
200
180
160
140
120
100
test
80
60
40
20
0
0 8 10 16 18 20 26 28 30 36 38 46
number of credits passed the first year

26. Barrier Effectiveness – Portugal
• the variables of parental education level and stage of
admission were not significant;
• the variables regarding the academic background
proved all significant, in particular the impact of the
grade obtained in the secondary education in the
academic achievement (40% performance
improvement for every 10 points) and the frequency
of Physics in Secondary (72% increase in the
probability of success, compared to those who had
not);

27. Barrier Effectiveness – Portugal
• in the socio-economic status and family income dimension, the
variables gender and household income level were significant,
showing that women have a higher probability of success than
men (+10%) and the level of household income below the
national average increases by 8% school performance;
• regarding the motivations and expectations, it appears that if the
student doesn’t access his/her first choice of programme (-16%)
and if he/she student did not anticipate to engage in all subjects
expecting a good average (-9%) there is a negative impact on
academic achievement. The early choice of the course (prior to
the year of admission) has a positive impact on the success rate
(+22%);

28. Barrier Effectiveness – Portugal
• academic achievement. The early choice of the
course (prior to the year of admission) has a positive
impact on the success rate (+22%);
• contextually it is observed that the fact that a student
is away from his/her usual residence exerts a
negative effect on their academic performance (there
is a 17% decrease in academic performance
demonstrated by students who are away from
residence) and that the travel time is also reflected
negatively (a student who takes more than 1h in each
travel to IST decreases 10% in academic
performance).

29. Factors Analysed - Ireland
• Inputs
– Whether a student took a particular subject (binary)
– Mark achieved in each subject (0-100)
– Degree (one of two available) programme chosen (binary)
– Gender (binary)
– Year (have things changed over 10 year period) (1-10)
– CAO mark (cumulative grade in best 6 subjects) (0-600)
– Living at home (binary)
• Outputs
– Had to take second exam sessions (Binary)
– Progressed to 2nd year (Binary)

30. Key questions
• Can we identify those entrants less likely to progress
• How accurately can we do this – i.e. what is the
balance between correctly identifying those with
difficulties and incorrectly identifying those who won’t

31. 128 Students

32. 15 Students on average will fail their exams (~12%) – 113 passed

33. We can select these using various strategies – e.g. random, or
targeting certain individuals

34. Here for example we are selecting 24 students (we’ll assume we
have some strategy!)
In terms of effective detection/identification, we could measure it
as:
4/15 failing students were correctly identified. This is a True
Positive Rate of 27%
20/113 passing students were incorrectly identified. This is a False
Positive Rate of 17%

35. A system with perfect discrimination could identify all those failing
students correctly, AND have no students incorrectly identified
Most practical systems will involve some compromise between
incorrectly identifying some students, versus correctly identifying
those who will fail
Graphically this tradeoff can be plotted on something called a
receiver operating characteristic

36. Receiver Operating Characteristic - Progression
1
0.9
0.8
0.7
True Positive Rate
0.6
All Factors
0.5
Random
0.4 CAO
CAO + Maths
0.3
0.2
0.1
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
False Positive Rate

37. Receiver Operating Characteristic - First Sitting
1
0.9
0.8
0.7
True Positive Rate
0.6
All Factors
0.5
Random
0.4 CAO
CAO + Maths
0.3
0.2
0.1
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
False Positive Rate

38. Exam Mark versus Effect Size
0.2
0.1
0
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100
-0.1
Accounting
-0.2
-0.3
-0.4

41. Logistic Analysis of Performance
Input Weighted Logistic
Probability
Variables Sum Function
[0 / 1] [- ∞, + ∞] [0 - 1]
[0 - 100]

42. Probability of succesful completion of exams
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
-4 -3 -2 -1 0 1 2 3 4
Weighted sum of input factors

43. Previous work in Ireland
CAO Maths English Student
A
High
B
High
C
Low
D
High
E
High
F
Low
G
Low
H
I
High
J
High
K
Low
L
Low
M
High
N
Low
O
Low
P

44. Previous work in Ireland
0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0%
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P

45. 100.00%
Gender Uptake of Various Subjects
90.00%
80.00%
70.00%
60.00%
Female
Male
50.00%
40.00%
30.00%
20.00%
10.00%
0.00%

46. 10.00%
Relative Subject Difficulty
5.00%
0.00%
-5.00%
-10.00%
Male
Female
-15.00%

47. PISA – maths, science & GDP
Finland
Ireland
USA
Sweden Belgium
Italy
Portugal

48. PISA – top students in maths & science
“the number of students reaching level 5 or 6 in
mathematics and science will be particularly
important for countries wishing to create a pool of
workers able to advance the frontiers of scientific
and technological knowledge in the future and
compete in the global economy”
– OECD Report – PISA 2009 Results Vol. 1

49. Percentages of Students at Level 5(6) in
Maths and Science
Country Maths Science
Shanghai 24(26) 20(4)
Finland 16(6) 15(4)
Belgium 14(5) 8(2)
OECD average 9(3) 7(2)
Sweden 8(3) 7(1)
Portugal 7(3) 3(1)
Italy 7(2) 5(1)
Ireland 5(1) 8(1)

50. Recommendations
Three categories:
a. Changes to admission requirements
b. Structural Changes to Education System
c. Socio-economic and cultural issues
Relative difficulty of implementation
• Easy, moderate or difficult

51. Recommendations (a)
Changes to admission requirements
Mathematics
Physics and/or Chemistry

52. Recommendations (b)
Structural changes to education system
Higher level of preparation in STEM
Later ‘tracking’ of students
Higher core STEM content for all students

53. Recommendations (c)
Socio-economic and cultural issues
Need to show relevance of STEM to real life
Encourage more girls
Increase participation from marginal socio-economic
groups