IM has reflected on its strategy and performance over the past 5 years. There are areas of strong performance, such as generating enthusiasm for education, but also questions around sustainability and impact on learning outcomes. Three potential areas for improvement are: 1) focusing more on improved learning outcomes; 2) thinking more systemically and working to address political barriers; and 3) revising the strategy for promoting school-based management. Other suggestions include developing PMs to think more systemically, profiling Galuh officers, improving knowledge management, and conducting more strategic evaluations. Learning from organizations like Kinerja and Pratham could provide lessons in areas like community-led school management and measuring learning outcomes.

1. Education Partnership –
Performance Oversight and Monitoring (POM)
Performance Oversight & Monitoring (POM)
“Aide Memoire” – IM Performance Reflection
Mark Fiorello - SOLIDARITAS
Australian Aid – managed by the Palladium Group on behalf of the Australian Government

2. Background & Context
• DFAT has expressed interest to support IM as part of its
emerging strategy to support “local solutions to local
problems.”
• IM “Phase 1” (2010-2015): IM has compiled a significant
amount of knowledge and experience, which it has used to
iteratively adapt and improve its work over time
• IM leadership feels that there are a series of ‘unanswered’
questions about the organization’s overall strategy and
performance.
2

3. Goals
• Review, clarify, and (as necessary) further develop IM’s
‘theory of change’,
• Reflect on and – to the extent possible – conduct a more
structured assessment of various aspects of IM’s strategy to
date
• Identify other organizations which may be relevant for IM to
learn from, and – to the extent possible – identify key lessons
from those organizations that IM can use to further improve
its effectiveness going forward
3

4. Overall Approach
• Collaborative Development of “Program Logic”
– 2 x participative workshop
• Compiling Information about IM’s performance
– Document review
– Discussions with Evi Trisna, Rahmi Yunita, Hikmat Hardono
– Series of discussions/interactions with Galuh officers
– Field visit to Bima
• Identifying Areas of Potential Improvement
• Identifying Other Relevant Organizations & Lessons from those
Organizations
• Providing “Items for Consideration”
4

5. Australian Aid – managed by XXXXX on behalf of AusAID
“Theory of Change” / Logika Perubahan
 Hard Copy & Prezi

6. Australian Aid – managed by XXXXX on behalf of AusAID
General Conclusions from
Performance Reflection

7. Understanding IM’s Performance Requires
Understanding IM as an “Adhocracy”
• ‘Adhocracy’ is the opposite of bureaucracy:
– A flexible, adaptable and informal form of organization that is defined by
a lack formal structure/hierarchy.
– Characterized by non-routine tasks and adaptive, creative behavior
– Flexible & non-permanent, adhocracies can respond faster to emerging
challenges & frequent change
– “Support Staff” play a key role
• IM works in a very non-centralized way, where it is largely left to
PM’s, Penggerak, & volunteers to “figure it out” (with some
support from Galuh)
• Can create tensions / discomfort
– When direction is wanted or needed
– Because it is different from many people’s perceptions of what an
organization “should be”
– Because information (including about performance) is dispersed
7

8. Positives – Areas of Strong Performance
• PM’s ability to generate interest, enthusiasm, & momentum
around education
– Starts in the classroom: change in enthusiasm & performance of students
generates attention & respect, starting with parents and teachers
• With active encouragement from PMs, some parents & teachers
begin to change behavior
• Emergence of local initiatives, which starts from the
identification & activation of penggerak
– IM stimulating new ideas , providing space and encouragement for
creativity, and connecting actors
– Many initiatives are filling a gap left by lack of govt performance
• 1 year of experience leaves a very strong impression on PMs
– Setahun mengajar, seumur hidup terinspirasi
8

9. Question Marks – Areas of Uncertain Performance
• Sustainability after IM is no longer physically present
– Will positive behavioral change remain sustained, even if local
ecosystems are not yet conducive (due to negative influence from
government)?
– Will local actors to continue to promote change even when IM is no
longer physically present?
• For new local initiatives:
– Will they be able to obtain the necessary resources from partners?
– Will the initiatives themselves be able to continue to learn and adapt?
• How can IM provide effective support from afar?
• Can IM more effectively manage/leverage its alumni network?
• Are PMs and/or local initiatives actually contributing to a
positive change in learning outcomes?
9

10. Australian Aid – managed by XXXXX on behalf of AusAID
Potential Areas for Improvement
3 areas related to strategies for change
5 areas related to organizational practices

11. 1. Focus on the goal of improved learning & abilities
among primary school children
• IM is clearly generating enthusiasm, introducing and encouraging
new ideas, and increasing community and volunteer participation
around education
• Big question: can children read, write, count and think better?
• Actors are generally aware of problems with calistung, but lack:
– a common language to talk about these problems
– a way to assess the strategic value or contribution of initiatives in
addressing these problems
• Risk: without focus, positive energy is just ‘euphoria’ and is
wasted (and people lose interest)
• Suggestion: being more explicit about learning outcomes, and
creating/contributing to a discourse around education in terms of
learning outcomes
11

12. 2. Thinking systemically & working politically
• Due to its philosophy and use of OM, IM is very focused on
positive changes among individual actors
• However, the behavior of actors (their likelihood for positive
deviance) is significantly influenced by their environment.
• Little evidence to suggest IM is helping to address – or even
focused on or willing to criticize – systemic problems in education.
Reasons include:
– Local decision makers not explicitly targeted in intentional design
– IM’s status as “non-partisan, non-political”
– IM’s desire to identify and work through local penggerak
– IM’s culture of optimism
– Capacity among Galuh and PMs
12

13. 2. Thinking systemically & working politically (2)
• The case of Bima is not an isolated one: many root causes of
problems in education are systemic
– Corruption (including of BOS & BSM)
– Lack of support to & oversight of principals & teachers
– Politicization of the bureaucracy & poor human resource management
– Lack of quality in teacher higher education for would-be teachers
• SDN Tambora. IM will leave, and there is a high risk things will
revert back to the way they were before.
• Gerakan Bima Mengajar. Penggerak & PMs had to lobby hard
behind the scenes to ‘save’ their idea from the government.
• Suggestion:
– Need to clarifying concept of “dynamic & conducive ecosystem”
– IM should explicitly target a change attitude / behavior of key decision
makers related to systemic problems
13

14. 3. Revising the Strategy for Promoting School-Based
Management
• Area of theory of change which is generally acknowledged as not
changing is related to school principals & SBM
• Case of SDN Tambora. The requirement for sustained
improvement is SBM (or private education).
• IM cannot apply the same strategy for principals as it does for
teachers. Instead, it needs to work through:
– Community pressure (bottom up)
– District or sub-district oversight (top down)
• Alternatively, the potential for improvement in SBM could be
considered as a criteria for selection and/or continued support.
• Suggestion:
– Need to revise / develop the strategy for influencing principals.
Intermediate outcomes can include working through: school committee,
parents & community leaders, & district education officers.
14

15. 4. Development & Ongoing Support for PM’s, especially
related to “thinking systemically & working politically”
• The role of PMs covers both “Pengajar Muda” and “Penggerak
Muda”
• PM training is predominantly focused on their role as Pengajar
Muda, with comparatively little focus on how to better
understand or work within the education and political system
– Skills are focused on teaching, facilitation, and coaching
• Input from PMX Bima:
– Need for additional topics outside pedagogy, including related to school
management, relevant policies, advocacy, and capacity building strategies
– Provision of supplemental material (bahan pengayaan)
– An honest (but still constructive) handover from the previous PMs
– Appropriate support from Galuh, in terms of coaching, assistance, and
encouragement
• Galuh Officers need to be at least at the same level as PMs
15

16. 5. Profiling, Selection & Development for Officers &
Managers in Galuh
• The success of an adhocracy depends on “support staff” having
sufficient competency to function independently
– Managers: translate strategic vision into more concrete plans within
specific contexts; provide support and guidance to team that encourages
flexibility and creativity
– Officers: ability to understand context, identify what needs to happen, and
then make it happen – all without intensive direction
• Galuh has not yet devoted the same attention to selecting &
developing Officers & Managers as it has for PMs
• Need to clarify the core profiles of Galuh Managers & Officers,
including in terms of their ability to function in an adhocracy
(flexibility, initiative, comfort with uncertainty, management style)
• Previous versions of Officer Development Program & Manager
Development Program considered not very effective
16

17. 6. Knowledge Management & Intermediation
• IM is undoubtedly a learning organization
• Much of IM’s knowledge remains tacit – there is no culture of
documenting key lessons learned
• Barrier to documenting or sharing negative experiences, even as a
basis for internal learning
• Lack of “organizational memory” is a problem when there is staff
turnover (this is currently being felt)
• IM’s knowledge is mostly based on its own experience and the
experience of network members
• Huge opportunities for IM to play a role in “knowledge
intermediation”, both for PMs and for penggerak
– Introducing new ideas is a strategic intervention
– Lots of concepts/ideas readily available, both in terms of education and
community mobilization
– High demand for new ideas
17

18. 7. Evaluation & Reflection
• IM leadership has genuine questions about whether the overall
strategy is effective or not
• IM’s M&E is mostly at the operational level, and is not focused
around strategic questions
– This is the first time IM is taking a more structured look at which of its
outcomes have been achieved, and to what extent.
– No documentation of the process / results of key initiatives (links to
problems with knowledge management & focus on learning outcomes)
• Challenges:
– Adhocracy means information is dispersed & strategies are constantly
changing
– Limited internal capacity to conduct M&E
– (until now) lack of theory of change
• Active reflection is a key component of success for IM, and
presumably also for local initiatives. But reflection may need to
be more actively facilitated for local initiatives.
18

19. 8. Communicating “What is IM”
• Anecdotally, there are many questions and even misperceptions
around “what is IM”…
– General public (outside the IM network) and even applicants to IM:
sending teachers to teach
– Local government in Bima: ‘representatives’ from KemDikBud
– Alumni: what is IM’s strategy? Is it shifting from placing PMs to promoting
local initiatives?
• Fits with IM’s nature as an adhocracy: information (and even
identity) is not centralized, and constantly evolving
• Underscores the importance of alignment and clear overall goals
(for example: on learning outcomes)
19

20. Australian Aid – managed by XXXXX on behalf of AusAID
See Notes & Proposed Revisions to IM’s
“Theory of Change” / Logika Perubahan

21. Australian Aid – managed by XXXXX on behalf of AusAID
Potential Lessons Learned from Other
Relevant Organizations

22. What are we looking for? Organizations that are…
• focusing attention on learning outcomes…
• organizing communities to address systemic problems, including
by working politically (but in a positive, non-partisan way)…
• working to promote school-based management…
• providing development & support to community organizers…
• are running effective officer & manager development programs…
• effectively managing knowledge and making internal and external
knowledge available and accessible to potential users…
• conducting useful, strategic evaluations of their own work…
But…we are also looking for adhocracies, since they will be most
relevant for the IM context (for some of the points above, more
bureaucratic organizations might not be appropriate sources of
learning)
22

23. Some candidates:
• Kinerja (USAID-funded education program in Indonesia):
– Promotion of community-led school based management (by creating space
for parents, community members to identify problems in schools)
• Pratham (Indian education NGO)
– Very focused on improving (and measuring) learning outcomes
– Working across many regions in India
– Developing a variety of innovative interventions, including outside
government systems
– Focused on measuring results through (a) its own research & evaluation
unit and (b) partnerships with external researchers
• Communication with & references to a variety of KM networks &
organizations
• Idea: crowd-sourcing the previous list.
23

24. Australian Aid – managed by XXXXX on behalf of AusAID
Items for Consideration

25. 1. Define progress markers (penanda kemajuan) in
relation to time over the projected 5 year cycle of
change in both schools and districts.
• IM should attempt to explicitly acknowledge and communicate
the shift in focus of PMs from “Pengajar Muda” to “Penggerak
Muda” over the course of 5 years.
• In certain circumstances, IM should consider placing a PM at the
district level (for example, in Year 5 or Year 6)
KABUPATEN Memetakan aktor
+ membangun
kepercayaan
Masuk diskursus +
menggagas &
mengembangkan inisiatif
Memperkuat
inisiatif & handover
ke aktor local
Memberikan
dukungan sesuai
kebutuhan
SEKOLAH Membangun
antusiasme & goodwill
+ memberikan contoh
Mendorong perubahan
perilaku terkait pengajaran,
manajemen, & komunikasi
Memperkuat
sistem & handover
ke aktor local
Memberikan
dukungan sesuai
kebutuhan
T=0 T=1 T=2 T=3 T=4 T=5 T=6
25

26. 2. Identify and potentially incorporate simple / rapid
assessments of reading & math skills into IM’s
interventions at the school and/or district level.
• Such tools will provide concrete data that can be used to focus
attention, rally community support, and measure change
• Examples:
– EGRA & EGMA (already available for Indonesia)
– Pratham-type tools/surveys
• “It is possible to use simple, reliable, and scientific methods of sampling and
assessment on a large scale for high impact at a low cost. [Pratham’s ASER
Survey] has also been an excellent example of building local participation at a
national level and has allowed ordinary citizens to understand the current
status of elementary education.” - Pratham
26

27. 3. Adopt a more structured approach to promoting
school-based management
• Use key ideas from the Kinerja approach to SBM to create a
‘communication space’ for the Kepala Sekolah, Komite Sekolah,
and parents/community members to discuss problems, agree on
actions, and monitor progress:
– Compile complaints and suggestions from students, parents, community
members, and teachers in a structured and transparent manner
– Bring the complaint and suggestion data to a forum for discussion
– Agree on improvements and a participatory monitoring mechanism, and
document the agreement in a service charter. Refer to minimum service
standards as relevant.
– Use the service charter as a basis for school planning and budgeting.
• IM should be upfront with schools and districts about
expectations around SBM implementation. Where schools
demonstrate no improvement on clear criteria (principal
attendance, BOS transparency), IM should consider withdrawing.
27

28. 4. Establish an internal evaluation and reflection unit, or
develop organizational partnerships to ensure that
evaluation occurs
• An internal evaluation and reflection unit could:
– Conduct and/or manage strategic evaluation work, including using learning
assessments
– Support the ongoing development and review of IM’s program logic and/or
the program logic for specific initiatives
– Contribute to the discourse around educational performance and
education evaluation in Indonesia
– Develop and manage a network of external evaluators (e.g. Indonesian and
international researchers) to conduct evaluations
– Facilitate active learning and reflection for local initiatives
• Example: Pratham established a research & evaluation unit in
2005 (10 years after the NGO was established)
28

29. 5. Experiment with Conducting Exit Interviews and After
Action Reviews on Video
• Exit interviews (for PMs and Galuh Staff) and After Action Reviews
(reflections on key activities) are vital to documenting lessons
learned and creating an ‘organizational memory’
• If recorded on video, tagged appropriately, and placed online (e.g.
youtube), these could form a valuable resource for Galuh Staff,
PMs, and Penggerak to search for and learn from other
experiences
• Note: an ‘organizational historian’ may be necessary to manage
this information and coordinate processes.
More information: http://www.kstoolkit.org/Exit+Interviews
29

30. 6. Expand the Concept of Training and Support for PMs
• In terms of their ability to ‘think systemically and work politically’,
PM training should cover the following:
– Basic concepts of “iterative adaptation”: root causes of problems, positive
deviance, authorizing environment, etc
– The use of basic tools for understanding problems, e.g. 5 whys and
fishbone diagrams (for problem identification)
– The use of basic tools for planning for stakeholder engagement, e.g. force
field analysis and awareness & interest mapping
• Compile and provide a set of additional references (pengayaan)
for PM on a variety of topics identified as relevant by current and
previous PMs
• Use IM’s professional volunteer network to providing post-
training mentoring/coaching to PMs related to specific topics or
skills, for example related to:
– Particular facilitation techniques
– School management or school finance
– Persuasive writing
30

31. 7. Encourage Galuh Officers (and other IM actors) to
review and summarize relevant content for
consumption by other officers, PMs, and Penggerak
• Over time, this information could form a ‘knowledge bank’ that is
available for use in a variety of initiatives
• Information could be grouped by a variety of themes, for
example:
– Pedagogy
– Community development
– Education management
– Curriculum
– Capacity Development
• This would also expose Galuh Officers to key conceptual
knowledge
31

32. 8. Develop competency profiles for Galuh Officers and
Managers that reflect their suitability to the
organizational culture and required competencies
• Selection processes should ensure that Galuh Officers and
Managers are well-suited to the ‘adhocratic’ nature of IM: they
should be flexible, creative, learning-oriented, comfortable with
uncertainty, etc.
• In general, Officers and Managers should follow the same
rigorous recruitment and selection procedures as PMs
• Officers should be at least on the same level as PMs, but also
need additional competency in terms of working within an
organization
• The Officer Development and Manager Development programs
could be reinstated but need to be adjusted to reflect the needs
and realities of the organization.
32

33. 9. Keep the organizational culture, but change the
organizational culture
• IM’s optimism and “can-do” spirit is key to the success of the
organization and the movement
• Many parties feel that more honesty and openness is needed.
• Actively cultivate an honest and positive culture
– Acknowledging a problem is the first step to solving it
– Problems seen as opportunities rather than barriers
– “Yes, and” rather than “Yes, but…”
33

34. 10. Explore collaboration with the INOVASI program
• INOVASI has an explicit focus on learning outcomes, and expertise
and a mandate to evaluate innovative education practices that
could contribute to significant changes in learning outcomes
• INOVASI can also learn from IM about building relationships,
energy, and positive momentum at the school level
• IM could propose an evaluation of one or more initiatives (e.g.
Gerakan Bima Mengajar or RuBI BIMA) to INOVASI
34

35. Thank You
Australian Aid – managed by XXXXX on behalf of AusAID