This document provides findings from applying a quality checklist to annual monitoring reports on public administration reform strategies. It summarizes the key quality issues identified and provides data on how well recent reports addressed various quality criteria. The document concludes by emphasizing areas that could be improved in 2021 reports and encouraging a commitment to meeting higher quality standards. It also includes discussion questions for a virtual roundtable on challenges in preparing such reports and monitoring reform implementation.

1. 23/02/2021
1
© OECD
Findings of application of the
Quality Checklist for
Annual Monitoring Reports
Martins Krievins
OECD/SIGMA Senior Policy Advisor
Zoom, 19.02.2021.
Welcome!
1
Albania Georgia Kosovo*
North
Macedonia Montenegro Serbia Ukraine
* This designation is without prejudice to positions on status, and is in line with United Nations Security Council Resolution 1244/99 and the Advisory Opinion of the International Court of Justice on Kosovo’s declaration of independence.

2. 23/02/2021
2
What & Why?
2
Annotated structure of contents for reports
monitoring implementation of
specific planning documents
Advice
Checklist for evaluating the quality of reports
monitoring implementation of
specific planning documents
Assess
Quality checklists are not a
discipline in PAR olympic games…
3
Sets out key quality criteria for
annual monitoring reports
Identifies basic issues that
any report should follow
Place emphasis on use of
data and evidence
Can be used as internal
self-assessment tool

3. 23/02/2021
3
Overall quality issues
4
85%
80%
57%
77%
69%
Provide precise information on actions and indicators
Have separate chapter on risk mitigation
Comply with national-level regulations
Presents information in visually attractive form
Provides financial information
ALL DATA stems from SIGMA Analytical Report “Annual Monitoring Reports on Implementation of Public
Administration Reform (PAR) related Strategies: Key Lessons from 2020 and Suggestions for 2021”
* Please, note that all figures had been rounded up to ensure easiness of perception.
Specific quality questions:
Executive Summary
5
visual
info
54
%
“success
stories”
39
%
basic
statistics
46
%
performanc
e indicators
23
%
financial
info
31
%
77%
have an
Executive
Summary

4. 23/02/2021
4
Specific quality questions:
Aggregate Overview (85%)
6
provide aggregate overview on implementation
progress achieved under specific policy objectives
provide information on overall budget execution
describe involvement of civil society organisations
have information on implementation of previous recommendations
• have information on achieving vision / “bigger picture”
• discuss reasons behind worse than expected results
• have overview of outcome-level indicators quantitative
data
discuss trends / dynamics of performance indicators
46%
38%
8%
15%
0%
Specific quality questions:
Implementation of objectives
7
46%
discuss failures
faced during
implementation
process and set
performance
targets
31%
use performance
indicators to
discuss
achievement of
policy objectives
23%
clearly spell out
key conclusions
on challenges
and
recommendation
for decision
makers how to
overcome those
15%
provide
information on
financing
implementation
of reform
objectives

5. 23/02/2021
5
Specific quality questions:
Risk management
8
43%
of reports on
implementation of
strategies that
have a chapter on
risks, actually
report on
materialisation of
those and
describe
mitigation steps
taken.
Specific quality questions:
Communication activities
9
80% 20%
of analysed
reports
of analysed reports
have dedicated chapter of sub-chapter
describing internal and external
communication activities carried out in
regard to reform implementation.
do not address issues related to
communication regarding reform
objectives, implementation successes
and other awareness building
measures to build support for the
reform.

6. 23/02/2021
6
Specific quality questions:
Reform actions
10
still use the
“partially
implemented”
status to report
on
implementation of
actions
Preciseness
dedicated annex
is attached to
explain
implementation
status of action
plan
Annex
provide
information on
ALL actions of the
action plan and
not just current
year
Details
85%
69% 62%
Towards better PAR reports in 2021
11
We all know what happened in 2020, and it had a major
impact, therefore:
 despite implementation problems faced, reports should
still be prepared on time and clearly document all
different issues faced during pandemics;
 it is important to clearly identify negative and positive
impacts that the pandemics has had on reforms;
 causal link between the Covid-19 and challenges faced
would allow to target those during 2021.
Focus should be paid to address identified weaknesses:
 dynamics of performance indicators in relation to
objectives;
 challenges faced and decisions needed to overcome
those;
 financing of reform implementation;
 implementation success per institution;
 involvement of civil society in reform monitoring;
 internal and external communications.

7. 23/02/2021
7
Commitment for 2021?
12
85%
80%
57%
77%
69%
Provide precise information on actions and indicators
Have separate chapter on risk mitigation
Comply with national-level regulations
Presents information in visually attractive form
Provides financial information
100%
100%
80%
100%
80%
2020 2021 ???
Any questions or comments
before we move to discussion?
13

8. 23/02/2021
8
Virtual roundtable
What are the two most
difficult issues you face
during preparation of
annual monitoring reports
on PAR-related strategies?
Why these? Any solutions?
Is it challenging to ensure
adequate monitoring and
reporting during the transition
period between the two
strategies?
Have you actually used
the findings from the
monitoring reports in the
process of elaboration of
the new strategy?
Do you use any IT
solutions for monitoring
reform implementation
and elaboration of
monitoring reports?
Any good and bad experiences?
Limitations faced?
Did you find the SIGMA annexes
to Strategy toolkit on quality
checklist of monitoring reports
and the annotated table of
contents of monitoring reports
useful?
Any ideas for
improvement of Toolkit?
14
Concluding remarks
15

9. 23/02/2021
9
Thanks for participation!
16