Human Rights Project Monitoring and Evaluation Guide
•Télécharger en tant que PPTX, PDF•
4 j'aime•3,598 vues
Monitoring a evaluace lidskoprávních projektů - Prezentace z workshopu Demas 3. 12. 2013
Recommandé
Contenu connexe
Tendances
Tendances (20)
En vedette
En vedette (20)
Similaire à Human Rights Project Monitoring and Evaluation Guide
Similaire à Human Rights Project Monitoring and Evaluation Guide (20)
Plus de Inka Pibilova
Plus de Inka Pibilova (19)
Dernier
Dernier (20)
Human Rights Project Monitoring and Evaluation Guide
- 1. Workshop Monitoring a evaluace lidskoprávních projektů Monitoring and evaluation of human rights projects Inka Píbilová inka@evaluace.com 3. prosince 2013 1
- 2. Monitoring x evaluation x audit Evaluation • Assessment of project efficiency, effectiveness, impact, relevance and sustainability for the purpose of learning and accountability to stakeholders Monitoring • Ongoing analysis of project progress towards achieving planned results with the purpose of improving management decision making Audit • Assessment of (i) the legality and regularity of project expenditure and income i.e. compliance with laws and regulations and with applicable contractual rules and criteria; (ii) whether project funds have been used efficiently and economically i.e. in accordance with sound financial management;; and (iii) whether project funds have been used effectively i.e. for purposes intended. • Primarily a financial and financial management focus, with the focus of effectiveness being on project results. http://www.europa.eu.int/comm/europeaid/qsm/index_en.htm 2
- 3. Why to monitor and evaluate projects? • Accountability • Learning 3
- 5. Stakeholder Analysis Matrix Example: River Pollution http://www.europa.eu.int/comm/europeaid/qsm/index_en.htm 5
- 6. Influence /Power Stakeholder Analysis – Power / Interest / Attitude KEEP SATISFIED INFLUENCE & ENGAGE Attitude Policy makers CSOs MONITOR Activists KEEP INFORMED Interest 6
- 7. Problem Tree Example: River Pollution http://www.europa.eu.int/comm/europeaid/qsm/index_en.htm 7
- 8. Objectives and strategy selection Example: River Pollution http://www.europa.eu.int/comm/europeaid/qsm/index_en.htm 8
- 9. Inputs – Activities - Outputs – Outcomes – Impacts Direct influence of implementers: Inputs I need for my work… (funds, people, time) Activities I do…. (research, conference) Outputs I produce … (research paper, 100 participants to conference) Indirect influence of implementers: Outcomes I strive to… (Results) (raise awareness on X) Impacts I aim to… (reduce human rights violations) 9 http://www.europa.eu.int/comm/europeaid/qsm/index_en.htm
- 11. Logical framework Issues identified during project evaluations: • Logical frameworks too complicated, too many objectives / indicators • Too ambitious and/or too vague objectives • Indicators difficult to measure / data not collected or difficult to collect • Logframe not used for internal monitoring / evaluation and decision making Tips: • Numbering Objectives (1,2), Results (1.1., 1.2., ...), Activities (1.1.1., 1.1.2.) • Check source of verification – how will data be collected, by whom, when/how regularly? http://www.europa.eu.int/comm/europeaid/qsm/index_en.htm 11
- 12. Use both quantitative and qualitative indicators Compare using trends (increase), thresholds (min. 30%), targets (strategy by 12/Y1) Quantitative - SMARTER Specific / Simple (to understand, collect) Measurable Attainable/Available at cceptable costs Relevant to project / stakeholders Time-bound Evaluate/Engaging Reevaluate/Recordable Qualitative - SPICED Subjective Participatory Interpreted and communicable Cross-checked and compared Empowering Diverse / disaggregated (by gender) Min. 30 % of participants initiate a project aiming to address a local issue. Reasons why participants have (not) implemented a project to address a local issue http://www.smarttoolkit.net/?q=node/391 http://www.europa.eu.int/comm/europeaid/qsm/index_en.htm 12
- 13. Risks Analysis Assumptions in logframe http://www.europa.eu.int/comm/europeaid/qsm/index_en.htm 13
- 14. Monitoring: Why, for whom, what, how Why? • Timely identification of successes and problems during project implementation • Informed and timely decision making by project managers to support implementation • Accountability for the resources used and results achieved • Stakeholder awareness and participation • The evaluation of project achievements and audit of activities and finances 14
- 15. Use logframe to prepare Monitoring Plans Year 1 Indicator / Country 15.000 copies of publications distributed (5.000 each) 120 teachers trained (40 each) Min. 50% of teachers implement the new tools within 6 months • • • • CZ PL SK 90% 110% 100% 100% Achieved 45 60% 40 40% 120 40% 35 30% Total Note Achieved Not achieved Use indicators from the logframe Split by year and partner Use plan vs. actual, % or trends over time Develop adequate sources of verification • Distribution lists • Attendance sheets with contacts! • On-line feedback forms after 6 months (combined with supervision of teachers) • Media monitoring, public surveys... 15
- 16. Monitoring: Activity schedule with tasks and responsibilities 16
- 17. Monitoring Progress Report (Narrative) Monitoring Visit Checklist http://www.europa.eu.int/comm/europeaid/qsm/index_en.htm 17
- 18. Other monitoring tools 18 Forms at docs.google.com Geolocation at www.googlemaps.com, www.ushahidi.com
- 19. Monitoring tips • Keep the users of information clearly in mind (who needs what information?) • Build on local information systems and sources - costs and sustainability of new systems • Collect only the minimum amount of information required – keep it simple and practical, more information is not better information • Use Inception reports when team differs from the proposal writters, or when time passed by between identification and grants approval... usually 3 months after project start, revises original project proposal - updates logframe, sets annual action plan and monitoring plan
- 20. Evaluation: Why, for whom, what, how 20
- 21. Evaluator Selection Interviews Surveys Focus groups Case studies Final debriefing of all partners Draft evaluation report commented by all partners Final evaluation report Desk study Preliminary findings & conclusions Initial briefing and inception Terms of Reference – Objectives, scope, stakeholders, questions, budget, schedule, outputs, use. Plan external evaluation well ahead Communication with the Project Partners Inception phase Field research Reporting phase 1-3 months 1-3 months 1-2 months 21
- 24. How to measure learning outcomes? http://leanlearning.wikispaces.com/learning_analytics 24
- 26. What does participation mean? 26
- 27. Reporting: Why, for whom, what, how 27
- 28. Where to learn more Road to Results https://openknowledge.worldbank.org/bitstrea m/handle/10986/2699/52678.pdf?sequence=1 EPDET - annual training in Evaluations by World Bank consultants http://www.dww.cz/english.php?page=epdet1 Evaluations of Czech development and humanitarian projects http://www.mzv.cz/jnp/cz/zahranicni_vztahy/rozvojova_spolupr ace/dvoustranna_zrs_cr/evaluace/index.html Different evaluation methods and approaches http://betterevaluation.org/ EC Project Cycle Management Manual http://www.europa.eu.int/comm/europeaid/qsm/index_en.htm 28
Notes de l'éditeur
- When to use logframe?ProblemAnalysis and ObjectiveSettingAnalysisofcauses and effects / prioritisationShows relevance to policies / contextChecksrisks and accountabilityforresultsEstablishesmeasurement systém and data collectionImplementationProvidesframeworkforResultsOriented monitoringEvaluationProvidesframeworkforevaluation as per OECD/DAC criteriaObservedIssuesToocomplicated, too many objectives / indicatorsTooambitiousobjectivesIndicatorsdifficult to measure / data not collectedordifficult to collectLogframe not usedforinternal monitoring / evaluation and decisionmaking
- When to use logframe?ProblemAnalysis and ObjectiveSettingAnalysisofcauses and effects / prioritisationShows relevance to policies / contextChecksrisks and accountabilityforresultsEstablishesmeasurement systém and data collectionImplementationProvidesframeworkforResultsOriented monitoringEvaluationProvidesframeworkforevaluation as per OECD/DAC criteriaObservedIssuesToocomplicated, too many objectives / indicatorsTooambitiousobjectivesIndicatorsdifficult to measure / data not collectedordifficult to collectLogframe not usedforinternal monitoring / evaluation and decisionmaking
- When to use logframe?ProblemAnalysis and ObjectiveSettingAnalysisofcauses and effects / prioritisationShows relevance to policies / contextChecksrisks and accountabilityforresultsEstablishesmeasurement systém and data collectionImplementationProvidesframeworkforResultsOriented monitoringEvaluationProvidesframeworkforevaluation as per OECD/DAC criteriaObservedIssuesToocomplicated, too many objectives / indicatorsTooambitiousobjectivesIndicatorsdifficult to measure / data not collectedordifficult to collectLogframe not usedforinternal monitoring / evaluation and decisionmaking
- Examples of qualitative and quantitative indicatorshttp://www.smarttoolkit.net/?q=node/391Quantitative indicators for the ‘accessibility’ of an information service’:- Distance of rural communities from a low-cost telecentre with a helpdesk- Percentage change in the number of visitors to the telecentre over a defined periodQualitative indicators for the ‘improved knowledge’ resulting from an information service:- The telecentre is close enough to a range of communities for them to use it when they need to- Positive and negative experiences of the users of the information in solving the problem for which information was sought- Ability of the users to discuss the problem in questionReference points for comparison1. Trends (e.g., a consistent increase or decrease in requests for support; increasing feedback from readers of a publication)2. Thresholds (e.g., at least three districts covered by a database; the minimum number of students on a course)3. Targets (e.g., the number of documents distributed by a stipulated deadline; the proceedings of a conference available in printed form by a given date)
- FromAssumptions – createRisksWeassumethat- Assumptionwillhappen – results/objectiveswillbeachievedRisk mighthappen, itthat case wehavepreparedmitigationmeasures so thatrisks do not happerprojectresults / objectives