Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Integrated Development Performance Monitoring and Evaluation System in Indonesia to Support National Development Planning Process
1. INTEGRATED DEVELOPMENT PERFORMANCE MONITORING AND EVALUATION SYSTEM
Arief Wiroyudo
INTRODUCTION
Since 2000, Indonesia has been undergoing a massive transformation from centralized to decentralized
government and regional autonomy. This new governmental system is fundamentally reshaping the
political, economic and social structure of the country. The big-bang decentralization leads to the less
role of central government in the regional development. Therefore, there are new tasks and
responsibilities in terms of function assignment and financing to local government as consequences of
decentralization.
Underlying the national policy on performance monitoring and evaluation (MONEV) is accomplishment
of government efforts in implementing development programs that improve the capability of
departments and agencies to allocate limited resources appropriately and to manage them efficiently.
The successful implementation of performance evaluation is seen as an institutional development that is
needed to provide better information as an input for the next planning process.
The importance of the MONEV quality awareness implies that those suppliers (line-ministries and/or
government officials in local levels) of evaluation reports are responsible for the provision of good
MONEV information demanded by decision-makers. However, the activities of MONEV in Indonesia still
face some problems, such as many rules that are required by sector and local government to make
reports to central government, there was less impact of MONEV activities to the next plan, lack of
synchronization and overlapping issue, and many evaluation activities are very costly in terms of money
and time. Integrated MONEV of wholly national development performance becomes a big challenge for
Indonesia. It is essential to link and synchronize both information on monitoring and evaluation
activities between central and regional government in order to formulate the next national
development plan.
The purpose of this paper is to emphasis the need of Integrated Development Performance MONEV
System in Indonesia to support national and regional development planning process efficiently.
This paper has some points as main discussions to introduce that can be shown as follows:
1. To give a theoretical framework of MONEV development system
2. To Analyze the national development MONEV system in Indonesia case
3. To recommend some key points in order to develop an integrated development MONEV system in
Indonesia
MONITORING AND EVALUATION DEVELOPMENT SYSTEM OVERVIEW: A THEORETICAL FRAMEWORK
From time to time, there has been a global change in public sector management as consequences of
public forces to make government to be more accountable to their stakeholders. These are some
concerns and questions being raised by internal and external stakeholders, and many governments are
struggling in addressing and answering them. The public citizens are really caring about the services
delivered by the government. For examples questions are, Have policies, programs, and projects led to
the desired results and outcomes? How do we know we are on the right track? How do we know if there
are problems along the way? How can we correct them at any given point in time? How do we measure
Page 1
2. progress? How can we tell success from failure? The public services production can be illustrated on the
following figure.
Production of Public Services
Source: Hideaki Tanaka, 2009
Monitoring and Evaluation: What Is It All About?
Evaluation is very important phase in policy analysis. As one of the procedures of the policy analysis
integrated framework, it cannot be separated with other procedures. Monitoring as the previous
procedure provides the fact and information that will be assessed by evaluation action. The policy
performance information produced through evaluation will be used as a feedback for next procedure in
policy analysis framework. According to Dunn (1994), the relationships of monitoring and evaluation
activities on a cycle of an integrated framework of policy analysis can be described as following figure.
An Integrated Framework of Policy Analysis
Source: William Dunn, 1994
Source: William Dunn, 1994
Page 2
3. The OECD (2002) defines monitoring and evaluation as follows:
Monitoring is a continuous function that uses the systematic collection of data on specified indicators to
provide management and the main stakeholders of an ongoing development intervention with
indications of the extent of progress and achievement of objectives and progress in the use of allocated
funds.
Evaluation is the systematic and objective assessment of an ongoing or completed project, program, or
policy, including its design, implementation, and results. The aim is to determine the relevance and
fulfillment of objectives, development efficiency, effectiveness, impact, and sustainability. An evaluation
should provide information that is credible and useful, enabling the incorporation of lessons learned into
the decision making process of both recipients and donors.
According to these two definitions, it is confirmed there are some different activities but monitoring and
evaluation are complementary each other on a planning and budgeting cycle process. Monitoring gives
information on where a policy, program, or project is at any given time (and over time) relative to
respective targets and outcomes, while evaluation gives evidence of why targets and outcomes are or
are not being achieved.
Why We Do Evaluation?
Evaluation is a complement to monitoring in a condition when a monitoring system sends information
that the efforts are not in the track, and then good evaluative information can help clarifying the
realities and trends noted with that monitoring system.Patton&Sawicki (1986) explained that the policy
analysis process does not stop with the implementation of the apparently superior policy. Even after a
policy has been implemented, doubt may remain as to whether the proper problem was identified,
whether an important aspect of the problem was ignored, or whether the policy conclusion or
recommendation might been wrong. These concerns require us to monitor and evaluate policies and
programs to see that the correct alternative is implemented, to assure that it does not irregularly
change form, and to determine whether it is having the preferred impact, whether it should be
redesigned or modified, or whether it should be terminated.
Mandated by certain regulations, the government (central and local governments) is responsible for
programs and activities which contribute as a basis for budget establishment. Each program and activity
produces outputs (immediate results) and outcomes (medium term results). Over the longer terms
outcomes lead to impact (long term results). Logical framework of development process can be
described as following figure.
Page 3
4. Logical Framework of Development Process
Source: Framework for Managing Programme Performance Information, National Treasury, Republic of South Africa, May 2007
Furthermore, focusing on evaluation system, it contributes to three basic functions (Mackay, 1998):
•
Accountability: making sure that public institutions, and their staff, are held accountable for
their performance.
•
Allocation: making sure that resources are allocated to those activities which contribute most
effectively to achieving the basic objectives of the institution.
•
Learning: making sure we learn from our successes and failures, to do things better in future.
ANALYSIS OF THE NATIONAL DEVELOPMENT EVALUATION SYSTEMIN INDONESIA CASE
The economic crisis in the late 1990’s became to catalyst that prompted the Government of Indonesian
to give high efforts to reform the public sector starting in the early 2000. Furthermore, in this era,
Indonesia has been undergoing a massive transformation from centralized to decentralized government
and regional autonomy. One of the main focuses of the reforms was in the area of public financial
management, with the goal of promoting good governance, greater transparency and accountability in
the conduct of government affairs.
Despite having traditionally a strong statistical system, monitoring and evaluation is a major weakness of
the Indonesian government planning system. Feedback of progress and results achieved is rarely
provided, despite the numerous requirements for filling out forms and lodging them. Most of those
forms are for recording purposes only, not for monitoring progress and evaluating results. At the
present time, there are so many regulations related with MONEV activities, therefore this condition
Page 4
5. increases the working units workload due to the multiplicity of evaluation. Moreover this condition also
made the ambiguity of responsible on evaluation result.
MONEV plays a key role in the existing integrated planning, budgeting and implementation development
activities. This circumstance has reflected that state ministries must evaluate performance of programs
according to the regulation standards. However, it is generally recognized that MONEV mechanism and
reporting systems in Indonesia are not very efficient and effective as can be seen from following
problems:
Legal Aspect Problem
Legal aspects for MONEV of development activities in Indonesia mainly are derived from 3 laws. Firstly,
Law No. 25/2004 on National Development Planning System which related with national development
plan from all sectors as a whole. Secondly, Law No. 17/2003 on National Finance which related with
budget disbursement, and thirdly, Law No. 32/2004 on Local Government which related with regional or
sub national development plan. These laws provide a new framework for financial management in the
public sector, and establish the basic principles of transparency and accountability, as well as regulate
the planning, budgeting and the accounting systems. The relationships of those development activities
can be seen as follows.
Development Budget and Planning Process in
Central and Regional Level
20 years
5 years
Annual
National
Longterm
Plan
Ministries
’Workplan
Ministries’
Budget
Work Plan
Detailed
State
Budget
National
Midterm
Plan
Gov’t
WorkPlan
Draft
State
Budget
State
Budget
Central
Government
Ministries
Strategic
Plan
Synchronized in the National Forum on Dev’t Plan
Regional
Mid Term
Plan
Regional
WorkPlan
Draft Reg
Budget
Regional
Regional
Work Unit
Strategic
Plan
Regional
WorkUnit
WorkPlan
Reg
Budget
Work Plan
Detailed
Regional
Budget
Law of Dev’t Plan
Budget
Local
Government
Regional
Longterm
Plan
Law of Finance
Source: Directorate Allocation of Development Funding, Bappenas
According to the Law No. 25/2004 on National Development Planning System, it states that ministries,
institutions and regional government work units are responsible for conducting MONEV activities of
their development plans of the previous period. This law also stipulated that the more detailed
procedures for MONEV of the implementation development plans will be issued through Government
Regulations. The prominent government regulation of those regarding MONEV activities of national
development plan implementation was issued in 2006. This regulation No. 39 Year 2006 provides
Page 5
6. specific table formats for the line-ministries, institutions and regional government work units to report
on program implementation, and formats for the consolidation of such information.
The second government regulation, relating to MONEV of regional development plan implementation is
Government Regulation No.8 Year 2008. Interestingly, this regulation does not refer to any article of Law
25/2004 as the legal basis for issuing the regulation, but it takes Article No. 154 of another Law (Law
No.32/2004) as its legal basis. The linkage between national and regional development planning is
acknowledged in article 150 of Law 32/2004, which states that regional planning is part of the national
development planning. This government regulation refers to the need for issuance of several Ministerial
Regulations (Minister of Home Affairs) to regulate the National Development Plan Forum/Consultation
process, integrated planning and budgeting, and MONEV of regional development plans.
In sort, according to those examples above, we can see that there are many regulations on MONEV
activities. Each regulation asked for the specific forms as mandates, however many of them seem not
correlated each other on an integrated MONEV development system.
Lack of synchronization and Overlapping issue
Initially, there were many definitions of MONEV activities of national and regional development in
Indonesia. Many government institutions had evaluation specified in their mandates, but often meaning
different things. This circumstance also made the ambiguity condition about who would be responsible
for MONEV supervision or coordinator. Furthermore, this MONEV activity often occur overlaps condition
in implementation. In fact, MONEV reporting is an additional burden to already busy of responsiblestaffs. In some experiences regarding MONEV activities, there are so many reports that should be
prepared by the MONEV providers (planning bureau), so they cannot do anything but just fill-up the
MONEV tables rather than make strategic plans.
Lack of feedback from MONEV reports
According to the theories and concepts explained earlier, MONEV results feedback actually used as a
valuable input for the next planning process. However, in Indonesia case this MONEV report is not fully
used as input for the next planning. The poor quality of the MONEV documents, un-measurable
information, and there were no institutions that capable to develop MONEV reports with integrating
cross-sector issues are some critical problems for creating a good MONEV report as an input for decision
makers at the top level for the next strategic development plan activities.
In many chances most of the reports are only focus on aid disbursement and physical progress, instead
of program or activity performance indicators. As a consequence, the impact of development, especially
for social aspects, was often lacking in monitoring and evaluation. So, it is difficult condition for
Bappenas as a planning institution to come up with such good development planning with those limited
information provided by line ministries.
Lack of resources to carry out MONEV
In many cases, monitoring activities of development performance in Indonesia are carried out by junior
staffs of government officials. Typically, the activities of data input, table formatting, and recording are
some main activities within this monitoring phase. However, for the evaluation, this advanced activity
still carried out by the lower staffs rather than more advanced and skilled staffs. This circumstance made
Page 6
7. the decision makers reluctant to use current MONEV information because monitoring and evaluation
results and reports are often not well compiled and poorly information. For example, project
implementers still have to report progress using a wide range of formats, which indicates a lack of
streamlining and rationalization in reporting. Up to now, many top level decision makers have realized
the importance of MONEV. They want to have staffs whose capability to carry out the MONEV activity
and can provide them useful information to develop policies. Unfortunately, the capacity development
for MONEV training is still limited. There are no certain trainings and courses to carry on development
MONEV activity in local level or even central level. The activity mostly is derived from senior to junior
staff without knowing the essential meaning and importance of MONEV activity itself.
RECOMMENDATION
According to some existing problems, there are some future expectations for improvement of MONEV
activities. Hence, there are some recommendations as follows:
Establishment of National Coordination Group. In Indonesia, practically departments and agencies are
responsible for conducting MONEV activities. Departments were not prepared to hand over their control
of departmental evaluation to Bappenas. According to the respected laws, the concept/framework of
having Bappenas as functionally responsible to the MONEV system and the departments to responsible
for doing the actual evaluation work was the key to the acceptance of an overall approach for
performance evaluation in Indonesia.
As the current situation, Bappenas with its limited staff would never have been able to do the
monitoring and evaluation work as needed. Retaining functional responsibility, prescribing how
evaluation work was to be carried out and reported, and having departments do the evaluation work,
was the only viable solution. Departments are closest to the development work and they have access to
the information needed for the MONEV activities. Unlike developed countries where there may be many
alternative sources of information possible for evaluation, in developing countries there is only one real
source of information which is the departments. It confirmed that evaluation work can only be
performed well if the departments are involved. However, without central guidance from Bappenas,
departments would do poorly evaluation or would do evaluation work using a wide range of different
approaches that make the evaluation result cannot use to fulfill the requirement as a basic input need
for the next planning process. Bappenas would then be unable to fulfill its national task regarding
performance evaluation.
Considering that condition above and also many regulations related to the MONEV activities, the
existence of so called “National Coordinator Group” will be very essential part. This board would
essentially more focus on coordination and collaboration activities in the development and
implementation of MONEV processes that are used by ministries at the present time. By Giving
Bappenas a role as “policy director’ of sector ministries is a best solution to have comprehensive
consideration of National program priorities. The main task of this national coordination group is to
provide documents or reports as a mandate to direct the development of performance management
and indicators of national development as a whole, and to report to the Cabinet through a designated
Minister on direction, progress, issues and problems.
Establishment of Indicator Resource Group. Bappenas now has the functional responsibility for
performance evaluation in overall while departments actually carry out the evaluation work. Bappenas
works closely with departments to develop and use performance indicators for GOI developmental
projects (donor-assisted or not), as part of the budgeting system. The proposed members of this group
are the staffs in Bappenas who have responsibilities in MONEV activities (MONEV working unit).
Page 7
8. Recently, Bappenas has new working unit which have specific responsibilities on MONEV activities at
Deputy Level (same with Directorate General Level). Together with the staffs from ministries/agencies
who have similar responsibilities in MONEV of their each ministry/agency and internal organization
which perform as ODA providers, this group is expected to develop a good quality of KPIs (Key
Performance Indicators).
Rational selection of indicators will help to address the classic problem of too many indicators. On the
other hand, the use of hierarchy of indicators at different levels will help to focus on the national
strategic outcomes through bottom-up filtration until monitoring and evaluation information needs are
met at the district, sectoral, provincial and national levels. Quantitative indicators need to be
supplemented by informed judgment and common sense.Furthermore, it is important to build a review
mechanism into the system. The various sets of indicators for all levels should be reviewed and
challenged in terms of their effectiveness, cost implications, data quality and source of data collection,
and linkages among performance indicators.
The use of these performance indicators for development projects brings Bappenas and the
departments closer together through both parties using performance indicators as a common language.
With this arrangement, ODA providers now can support the national policy on performance evaluation
by cooperating with related national organizations in the development and using performance
indicators for development project inputs, outputs, results and impacts. The use of the same set of
structured performance indicators for a given development project by both ODA providers and
Indonesia as recipient country should strengthen MONEV capacity development. Parallel reporting
systems for recording the achievements of ODA provider projects should be discouraged. Instead, as
much as possible, ODA providers should adapt to the evaluation systems which developed by recipient
country or in this case Indonesia.
Establishment of Integrated MONEV System Data Base. At this time, the availability of data are
scattered in each ministry/institution, meanwhile there is no coordination among data providers. It is
difficult to have cross-sector and cross-region analysis and prepare a consolidated report. Not all
indicators data are available regularly, especially for data that should be obtained from sector side (nonsurvey data). There are many data have been collected by sector institutions but some of them are not
processed properly even there should be guidance forms based on some mandated regulations.
Furthermore, there is no synchronized perception in filling the form and data processing of sector data.
Therefore, the integrated MONEV system will serve as a multi-stakeholder platform. It will foster and
enhance partnerships, collaboration, and programmatic synergies among stakeholders. It is an
opportunity to have a common agreement on indicator metadata, data collection and data processing
issues, data flow management, system support, and management of database development and
maintenance. Under Bappenas supervision, Central Statistical Board (BPS) can handle those activities as
a source of data provision, data collection, data sharing, and data management of documented all
MONEV activities. This BPS will produce a kind of MONEV system database that can be accessed by all
stakeholders.
Development of implementation schedules for new MONEV processes. The existing condition of
MONEV activities has given the increasing number and complexity of the MONEV functions that
required from the working unit staffs, along with increasingly complex reporting requirements, it may
not adequately power capacity to provide the necessary quality of the reports. Furthermore, as the
degree of preparation needed increases with complexity, it becomes necessary to ensure that the extra
time, costs, and resources are efficiently used. So, the integrated schedule for all MONEV activities
should be developed. Starting from monitoring until ex-post evaluation should be established with
clearly agenda, who, what, where, when, and how the MONEV activities be conducted are critical
Page 8
9. matter. All activities must be documented and posted to the integrated MONEV database system.
Improving the role of Indonesia Development Evaluation Community (InDEC). Knowledge from the
same organization abroad that brings new ideas and contributes in improving the MONEV policies can
be expanded from the presence of establishing community evaluator in Indonesia. It should be admitted
that the MONEV policies have not started in its optimal at this point in time. Generally, this concern is
caused by the lack of skills of the people responsible in the evaluation of the tasks towards the
implementation, the limited information about the procedure in conducting work scheme evaluation,
segregation of system, and evaluation reports as well as the minimum coordination among the
facilitators.
As many countries practices according to the involvement of evaluation community beside the
government side, it is also believed that intensive interaction and exchange of information as well as the
thought among MONEV actors spread to professional, academics, government officers, and NGOs both
locally and internationally are urgent to create added value and high advantage in creating better
climate in the MONEV system in Indonesia.
--o0o--
Arief Wiroyudo adalah perencana pertama pada Direktorat Perkotaan dan Perdesaan, Bappenas
REFERENCES
Barberie, A. (1998). Indonesia’s National Evaluation System. Evaluation Capacity Development Working
Paper Series No. 3. World Bank. Washington DC
Dunn, W. (1994). Public Policy Analysis: An Introduction. Pearson Education, Inc. New Jersey.
Kusek, J. Z. & Rist, R. C. (2004).Ten Steps to a Results-Based Monitoring and Evaluation System.World
Bank.Washington, DC.
Mackay, K. (1998). Public Sector Performance-The Critical Role of Evaluation. World Bank. Washington
DC.
McNamara, C.(2007). Field Guide to Nonprofit Program Design, Marketing and Evaluation. Authenticity
Consulting, LLC
Patton, C. and Sawicki D.S.,(1986). Basic Methods of Policy Analysis and Planning. Prentice Hall,
Englewood Cliffs
Solihin, D. (2008). Setting Up the Development Performance Evaluation System: Overview on Current
Practices and Future. Directorate for Regional Development
Performance Evaluation, Bappenas.
Tanaka, H. (2009). Performance Measurement and Evaluation. Presentation of Public Expenditure
Management: Theory and Practice. November, 2009
OECD (Organisation for Economic Co-operation and Development). (2002). Glossary of Key Terms in
Evaluation and Results-Based Management. OECD/DAC, Paris
Bappenas. (2004). Mid Term Development Plan Document. Bappenas, Jakarta
Bappenas and Direktorat Perekonomian Daerah. (2006). Pengolahan dan Analisis Laporan Monitoring
Pelaksanaan Dana Alokasi Khusus (DAK), Jakarta, Bappenas and Direktorat Perekonomian Daerah
DepartemenKeuangan.(2007). Sistem Informasi Keuangan Daerah Direktorat Jenderal Perimbangan
Keuangan Pusat dan Daerah [Regional Fiscal Information System of the Directorate General of
Central and Regional Fiscal Balance] (http://www.sikd.djpk.depkeu.go.id)
Page 9