Presentation for XХXII International Conference Problems of Decision Making under Uncertainties (PDMU-2018), August 27-31, 2018, Prague, Czech Republic
DOI: 10.13140/RG.2.2.27143.44966
Decision-making on assessment of higher education institutions under uncertainty
1. Decision-making on assessment of
higher education institutions under
uncertainty
Vladimir Bakhrushin, Prof., D.Sc.
Zaporizhia National Technical University,
Institute of Educational Analytics, MESU,
HERE team
PDMU-2018
XXXII International Conference
August 27-31, 2018
Prague, Czech Republic
2. Some problems and applications of
assessment
Budget funding of HEI on education and
research activity
University ratings
Staff and departments ratings
Accreditation of HEI
Status of HEI (Research, National)
3. Budget funding of higher education
Today the main mechanism of budget funding is the state order based on
forecasting of needs in specialists of different professions and norms of
expenses for training.
Until 2016, the distribution of the state order was carried out by the Ministry
of Education and Science based on the distribution of previous years and its
own preferences, which often were subjective.
Since 2016, the algorithm of a broad competition is used. It is based on the
Gail-Shepley algorithm and uses the priorities of entrants and their
competitive scores calculated on the basis of external independent evaluation.
4. The main problems of the existing model
Inconsistency of the predictions to reality under conditions of
rapid changes in society, economy and technology;
Inconsistency of orientation on profession under conditions of
massification of higher education;
Mismatch between the financial norms and needs;
Increasing of disparities in the distribution of budget financing
between regions and fields of knowledge
5. Budget code
State budget expenditures for higher education are distributed between
higher education institutions based on the formula, which should take
into account, in particular, the following parameters:
number of students according to the levels of education and
specialties and the ratio of the cost of training;
the level of results of external independent evaluation of entrants;
indicators of the quality of educational and scientific activity of higher
educational institutions
6. Apportionment formula
FS – stability funding (80-85% of previous year
total funding)
FD – performance-based funding in support of
development (10-15%)
FR – reserve (5%)
FRFDFSF ++=
7. Stability funding
Kcont – correction on changing of the contingent
of students, which takes into account their
distribution by levels of education and specialties
Kinf – correction on inflation
( ) ( ) ?*K*KtFStFS
FSFS
infcontii
i
i
∗−=
= ∑
1
8. Development funding
KDi – summary score of i-th HEI for performance-based
indicators;
h – minimum funding per student;
Ni – number of students at i-th university.
∑
=
−−=
i
i
i
i
KD
KD
FDFD
FRFSFFD
( )iiii N*h;FDFSmaxF +=
9. Optimization problem
The apportionment formula in such cases actually sets the objective function in the
multidimensional optimization problem under uncertainty. Some features:
there are no clearly defined criteria of optimality;
the influence of indicators on the criterion is not well known;
the data used for the assessment may contain statistical and other errors;
admissibility of using common indicators and assessment methods for different
fields of knowledge and different sciences has not been proved.
10. Uncertainty of objectives
More efficient use of budget funds - what is the
efficiency?
Reduction of HEI quantity – for what?
Improving the quality of higher education – but what is
the quality?
Increasing the competitiveness of HEI and HE system as
a whole - Why will it grow?
There is no reliable information about the effect of the
selected parameters on the objectives
11. Efficiency
Result / Cost
Those who say about efficiency in education often forget
about costs
Result: Gross Enrollment Ratio, Gross Domestic Product,
Human Development Index, …?
What are the optimal values for GER and some other possible
results?
Do GER, GDP, HDI, … depend only on budget funding for HE?
13. Budget funding (% GDP)
Built using the data from: http://data.uis.unesco.org
14. Budget funding
(% of government spending)
Built accor using the data from: http://data.uis.unesco.org
15. Conclusion 1
Ukraine needs to substantially increase spending
per student, but does not have the necessary
funds.
Therefore, today the main tasks are:
to increase the efficiency of the use of available
funds;
to create favorable conditions for attracting
extrabudgetary funds.
20. Conclusion 2
In the context of global massification of higher education and
global competition, Ukraine can not significantly reduce the
overall involvement of young people in higher education.
Therefore, it is necessary to distribute budgetary financing in such
a way as to maximize the growth of indicators of economic and
social development of Ukraine. But not always the state support
should be in the form of budgetary funding, sometimes it is
enough not to interfere
21. Reduction of HEIs quantity
Are more large HEI more qualitative?
Are more large HEI more effective?
Are more large HEI more competitive?
Is the optimal size the same for different
profiles – classic, polytechnic, pedagogical,
medical, arts, …?
22. University Size vs ARWU position
Built using the data from: http://www.shanghairanking.com/ARWU2017.html
23. HEI size for different profiles in Switzerland
Federal Statistical Office
https://www.bfs.admin.ch/bfs/en/home/statistics/education-science/pupils-
students.gnpdetail.2018-0165.html
24. Quality of Education
Standards and Guidelines for Quality
Assurance in the European Higher Education
Area (ESG-2015)
Standards ISO-9000-2015
Ukrainian Laws of education and higher
education
Various university ratings
25. Forbs vs ARWU for USA
Harvard University: 1 / 1
Stanford University: 2 / 2
Princeton University, MIT, Caltech: 4-6 / 4-6
…
Brown University: 9 / 100 – 150
Dartmouth College 12 / 201 - 300
Pomona College, Claremont McKenna College,
Williams College: 10, 11, 13 / -Built using the data from: https://www.forbes.com/top-colleges/list/#tab:rank;
http://www.shanghairanking.com/ARWU2017.html
26. Competitiveness
Is effect of selected parameters the same for
different institutions?
There are different possibilities to attract the
extrabudgetary funds in different fields of
education and different regions
There is responsibility of the government for
providing the needs in specialists of certain
professions
27. Uncertainty of criteria
How many criteria must be?
What exactly criteria can be taken?
What indicators can be taken?
What normalization procedures should be
taken for the indicators?
What should be the structure of the integral
criterion?
30. Cost indicators
Number of students studying for budget funds
(is non constant during financial year)
Normative of budget funding per student for
field of knowledge, levels and forms of
education (how to calculate, how to take into
account the differences between HEI)
31. Quality of the contingent
All students or only those who study for budget funds
Entrants or enrolled
All together or with consideration of fields of study
Independent external evaluation scores or competitive score
Indicators – mean, median, discrimination coefficient, the
share of those whose score is not less than the given value, …
33. Distributions of IEE scores
http://testportal.gov.ua/wp-content/uploads/2017/08/ZVIT_ZNO_2017_Tom_2.pdf
34. Results of scientific activity
The share of academic staffs who have 5 or
more publications indexed in Scopus or Web
of Science Core Collection
Share of funds received from scientific and
research activity
35. Overall quality indicators
Ranking of institution in one of the ratings
ARWU, QS, THE, taking into account position
in the rating (by the highest rank in case of
ranking in several ratings)
36. Employment of graduates
The average or median taxes paid by
graduates of certain years
Due to inaccessibility of the necessary data, such
indicator can be introduced only in 2-3 years
37. Some other indicators
Several other indicators are yet discussed now
Some indicators were rejected because they are not valid in
real conditions:
share of doctors and candidates of science;
number of scientific publications and patents;
number of foreign students;
share of postgraduate students who received a previous
higher education degree in another HEI
38. Thanks
The report uses the ideas and comments of
participants of the MESU working groups
expressed during discussions.
The author is especially grateful to V. Kovtunets,
Yu.Rashkevitch, O. Sharov, Ye. Stadny
Thanks for Your attention