Presentation by Marian Lemke, SIGMA (ENG) Second SIGMA Regional ENP East Conference on Public Procurement, Kyiv 29-30 May 2018

1. © OECD
Principles of Public Administration as
a tool to assess the public
procurement reform
Marian Lemke, OECD/SIGMA
29 May 2018, Kyiv

2. 1
Six core areas of a comprehensive
framework
Strategic framework of public administration reform
Policy
development
and
co-ordination
Public service
and human
resource
management
Accountability
Service
delivery
Public
financial
management,
including
public
procurement

3. 2
Principles of Public Administration
Derived from EU acquis, international standards and
requirements, and good practice in EU/OECD
countries, SIGMA developed the Principles of Public
Administration
The Principles define what good public administration
looks like in practice and outline its main
requirements as a guide for countries in their reform
processes, but also enable benchmarking of
performance
The Principles comprise only key requirements and
criteria, not “nice to haves”

4. Evidence-based measurement of
public administration reform
3
Valid and consistent conclusions, based on a “good
enough” body of evidence, measured reliably, and with
clear justifications.

5. 4

6. 5

7. 6
Purpose of the Methodological
Framework
• Without knowing the starting point it is hard to
decide what needs to be changed.
• Without analysing progress over time it is
impossible to say what has been achieved.
• A tool which allows for evaluation of the current
state of affairs and progress over time and
comparison with other countries.

8. 7
Aims for the analysis and
monitoring exercise
• Accurate, balanced, and economic
measure of each Principle
• Good measure of implementation
• Good measure of outcomes
• Benchmarking and comparisons
• Robust conclusions based on clear
criteria, that are replicable
• Actionable recommendations

9. Indicators that actively mix methods
and triangulate data for good analysis
Mixed methods:
 Desk review
 Interviews
 Test of practice
 On-site verification
 Review of cases
 Survey methodology
 Administrative data
 Official statistics
8

10. Purpose of the Methodological
Annex to the Principles
• Without knowing the starting point it is hard to
decide what needs to be changed
• Without analysing progress over time it is
impossible to say what has been achieved
• Therefore, the purpose is to provide a
methodological tool which allows countries to
evaluate their current state of affairs and
progress over time
9

11. What the Methodological Annex includes
• A list of indicators that can be used to support
analysis and monitoring:
All indicators include descriptions that guide
their application
There are both qualitative and quantitative
indicators
• An overview of potential sources of information
• A methodological guide for analysis
10

12. Public Procurement
Principle 8: Public procurement regulations (including
public-private partnerships and concessions) are aligned
with internationally recognised principles of economy,
efficiency, transparency, openness and accountability, and
are duly enforced; there is central institutional and
administrative capacity to develop, implement and monitor
procurement policy effectively and efficiently.
11

13. Indicator 6.8.1: Quality of legislative framework for
public procurement and PPPs/concessions
This indicator measures the quality of the legislative
framework for public procurement and public-private
partnerships (PPPs)/concessions, above and below World
Trade Organization (WTO) Government Procurement
Agreement (GPA) thresholds. Opportunities for participation
of small and medium-sized enterprises (SMEs) in public
procurement are assessed, as well as whether practical
measures are taken to allow proper implementation of the
legislation. The other indicators in the public procurement
area analyse the actual implementation of laws and
regulations and the results thereof.
12

14. Indicator 6.8.2: Central institutional and
administrative capacity to develop, implement and
monitor public procurement policy effectively and
efficiently
This indicator measures to what extent public
procurement policy is systematically developed,
implemented and monitored, how central public
procurement functions are distributed and
regulated, and to what extent the preparation
and implementation of policies is open and
transparent.
13

15. Public Procurement (2)
Principle 9: The remedies system is aligned with
applicable agreements and international regulations and
with internationally recognised good practice of
independence, probity and transparency and provides
for rapid and competent handling of complaints and
sanctions.
14

16. Indicator 6.9.1: Independence, timeliness and
competence of the complaints handling system
This indicator measures the effectiveness of the
system for handling complaints on public
procurement. First, the quality of the legislative and
regulatory framework is assessed. Then, the
strength of the institutional set-up for handling
complaints is analysed. Next, the actual
performance of the review system is measured.
Finally, the performance of the remedies system for
PPPs/concessions is evaluated.
15

17. Sub-indicators (1)
Legislative mechanisms for handling complaints
1) Right to challenge public procurement decisions
5
2) Time limit for challenging decisions taken by procuring entities
2
3) Available remedies
4
4) Mechanisms to ensure implementation of the review body’s
resolutions
2
5) Right to challenge decisions of the review body
3
16

18. Sub-indicators (2)
Institutional set-up for handling complaints
6) Legal provisions ensure the independence of the review body and its
members
7
7) Adequacy of the organisational set-up and procedures of the review
body
4
8) Public availability and timeliness of data on the review system
4
17

19. Sub-indicators (3)
Performance of the review system
9) Fairness of fee rates for initiating review procedures
3
10) Actual processing time of complaints
3
11) Complaint submission in practice
4
12) Quality of decision making by the review body
4
13) Cases changed or returned after verification by the court (%)
2
18

20. Sub-indicators (4)
Performance of the remedies system in PPPs/concessions
14) Right to challenge lawfulness of actions/omissions in PPP/concessions
procedures
5
15) Legal provisions ensure independence of the review body for
PPPs/concessions and its members
5
16) Timeliness and effectiveness of complaints handling system for
PPPs/concessions
5
19
Total
points
0-8 9-19 20-30 31-41 42-52 53-62
Final
indicator
value
0 1 2 3 4 5

21. Public Procurement (3)
Principle 10: Public procurement operations comply
with basic principles of equal treatment, non-
discrimination, proportionality and transparency, and
ensure the most efficient use of public funds; procuring
entities have appropriate capacities and use modern
procurement techniques.
20

22. Indicator 6.10.1: Efficiency, non-discrimination,
transparency and equal treatment practiced in
public procurement operations
This indicator measures the extent to which public
procurement operations comply with basic principles of
equal treatment, non-discrimination, proportionality and
transparency, while ensuring most efficient use of public
funds. It measures performance in the planning and
preparation of public procurement, the transparency and
competitiveness of the procedures used, the extent to which
modern approaches and tools are applied, and how the
contracts are managed once they have been concluded.
21

23. Indicator 6.10.2: Availability and quality of support
to procuring entities and economic operators to
strengthen professionalisation of procurement
operations
• This indicator measures the availability and quality of support given to
procuring entities and economic operators to develop and improve
the knowledge and professional skills of procurement officers and to
advise them in preparing, conducting and managing public
procurement operations. This support is usually provided by a central
procurement institution.
• This indicator does not directly measure the capacity of procuring
entities. The assessment is of the scope of the support (whether all
important stages of the procurement cycle are covered), its extent,
and its quality and relevance for practitioners (whether it provides
useful, practical guidance and examples). Surveys of procuring entities
and economic operators are used to gauge the relevance and practical
applicability of the support.
22

24. Illustrative examples

25. +11
+5
-13
+10
-1
+7
100 160
Progress against baseline by country (2015-2017)
Albania
Bosnia and
Herzegovina
The former
Yugoslav
Republic of
Macedonia
Kosovo
Montenegro
Serbia
0 294
142
102
139
130
155
142

26. Indicators

27. Contracts are based only on acquisition price
98.0% 99.7% 99.38%
98.0%
87.4%
64.6%
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
80.0%
90.0%
100.0%
Albania The former
Yugoslav Republic
of Macedonia
Kosovo Montenegro Serbia Turkey
Regional average

28. Many procurement procedures are cancelled
8.10%
14.49%
24%
19.7%
9.7% 9.94%
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
Albania The former
Yugoslav Republic
of Macedonia
Kosovo Montenegro Serbia Turkey
Regional average

29. Thank you for your attention!
More information can be found at:
www.sigmaweb.org
28