Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
Design and implementation
of operational components
of Result-Based Financing
schemes
The case of the verification in Beni...
Introduction
• Few evaluations have been published and many Randomized Controlled Trials
(RCTs) are currently ongoing to a...
Introduction
Role of verification in RBF’s Theory of Change
Verification
- Calculate rewards  pay bonus in a transparent,...
Context
The RBF project pilot
• Started in October 2011 with WB
funds – in 8 districts (“zones de
santé”)
• Population cov...
Context
Key roles in the scheme’s management
World Bank Funding
A Project Coordination Unit at MoH in charge of
signing co...
Context
Design of the RBF scheme
• Contracts signed, and verification performed, with facilities in
both control and inter...
Context
Design of the verification of results
Quantity verification
Monthly basis
Technical quality evaluation
Quarterly b...
Research methods
Secondary data
Project documents
(reports, budgets) 2012-2015
TA activities daily timesheets
(n=20)
RBF s...
Findings
Results of verification
Indicators Timeframe Results
Difference between declared
and verified data on num. of
ser...
Findings
Implementation issues in verification processes
Planned Observed
Provide data, analyzed for feedback
and coaching...
Findings
Little time for data analysis, feedback and coaching
Proportion of time spent on different activities for imp. ag...
Findings
verification implementation issues
Planned Observed
Provide data, analyzed for feedback
and coaching to facilitie...
Findings
Verification implementation issues
Planned Observed
Provide data, analyzed for feedback
and coaching to facilitie...
Findings
Payment delays
Quarter Quarter End
Invoice
Transmission
Delay
due to verif.
Transfer
Delay
due to transfer
Total ...
Findings
Verification implementation issues
Planned Observed
Provide data, analyzed for
feedback and coaching to facilitie...
Findings
Verification implementation issues
Planned Observed
Provide data, analyzed for feedback
and coaching to facilitie...
Findings
Verification implementation issues
Planned Observed
Provide data, analyzed for feedback
and coaching to facilitie...
Findings
Verification processes are costly
Funds to implem
agency for verification
activities (only)
Funds to CBOs
0.33
0
...
Discussion & conclusions
• Methodologically
What are Impact Evaluation really testing, if the theory of change is
modified...
Acknowledgment
• Thanks to all the Technical Assistants of the Implementation Agency
who provided time, insight and expert...
Prochain SlideShare
Chargement dans…5
×

Why should we pay more attention to the operational components of RbF schemes ? A case study on the design and implementation of the "verification" in Benin

216 vues

Publié le

Matthieu antony

Publié dans : Santé & Médecine
  • Soyez le premier à commenter

  • Soyez le premier à aimer ceci

Why should we pay more attention to the operational components of RbF schemes ? A case study on the design and implementation of the "verification" in Benin

  1. 1. Design and implementation of operational components of Result-Based Financing schemes The case of the verification in Benin Matthieu Antony Agence Européenne pour le Développement et la Santé European Agency for Development and Health mantony@aedes.be Dar es Salaam / 25.11.2015
  2. 2. Introduction • Few evaluations have been published and many Randomized Controlled Trials (RCTs) are currently ongoing to assess the effects of RBF - Basinga and al, 2011; Falisse and al, 2014; Bonfrer and al, 2014; Huillery and al, 2015 • Quantitative methods are often silent on the paths and processes through which results are achieved, and on wider health system effects - Witter and al, 2013 • Our study aims at analyzing how implementation challenges can modify the original design, affect RBF’s theory of change and therefore the scheme’s potential for results – We focus on the ‘verification of results’ in Benin – We also draw some lessons on the design and implementation of key operational components of RBF (such as verification), and on verification itself
  3. 3. Introduction Role of verification in RBF’s Theory of Change Verification - Calculate rewards  pay bonus in a transparent, timely and regular manner - Detect fraud & signal a threat of sanctions to providers - Channel the “voice” of communities and patients, and improve providers’ accountability - Improve governance & stewardship through DHMTs’ involvement - Provide reliable data  data analysis  feedback and “coaching” * Should be financially viable Performance (quantity + quality of outputs) Payment
  4. 4. Context The RBF project pilot • Started in October 2011 with WB funds – in 8 districts (“zones de santé”) • Population covered: 2,377,559 • Focus on health facilities’ productivity and quality of health services • On-going RCT • Scale up to 21 districts with the support of GF and GAVI (April 2015)
  5. 5. Context Key roles in the scheme’s management World Bank Funding A Project Coordination Unit at MoH in charge of signing contracts, purchasing services and payment transfers. An implementation agency in charge of technical assistance, coaching and verification
  6. 6. Context Design of the RBF scheme • Contracts signed, and verification performed, with facilities in both control and intervention arm (n = 188) • Quantity indicators – 8 indicators at community level – 28 indicators at health centre level – 14 indicators at hospital level • Quality checklist – 400 items • “Results validation meeting” at central level on quarterly basis
  7. 7. Context Design of the verification of results Quantity verification Monthly basis Technical quality evaluation Quarterly basis Community verification Quarterly basis • Counting health services produced from facilities’ registries – performed by implem. agency • Assessing quality against checklist – carried out by DHMTs for health centres & by peers for hospitals – under the supervision of implem. agency • Tracing patients in communities • Patients’ satisfaction survey – carried out by contracted Community Based Organizations (CBOs) – implem. agency in charge of sampling of patients to track, CBOs supervision, reports’ validation
  8. 8. Research methods Secondary data Project documents (reports, budgets) 2012-2015 TA activities daily timesheets (n=20) RBF specific data series 2012-2015 (n=188) Focus Groups with CBOs (n=5) Document review Participant observation
  9. 9. Findings Results of verification Indicators Timeframe Results Difference between declared and verified data on num. of services provided Duration of the project Varies between 4% (new users of family planning) and 51% (patient referred to hospital) Quality scores Third quarter of 2015 Between 4% and 96% Num. of patients missing from community tracing* Third quarter of 2013 (first community survey) Between 12% and 47% Patients’ satisfaction* - - * Data collected, but not systematically analyzed
  10. 10. Findings Implementation issues in verification processes Planned Observed Provide data, analyzed for feedback and coaching to facilities Little time for data analysis and coaching activities. No analysis at all for community verif. data Avoid fraud and provide a threat of sanctions to providers Sanctions for frauds and discrepancies in quantity reported are rarely applied. Rewards for higher patients’ satisfaction, or lower discrepancies in data reported are also not implemented Transparent, regular and timely payment Delays in the payment of bonus Improve governance and stewardship DHMTs rarely involved (they don’t have time, nor resources and RBF verif. is not «motivating» enough) Channel the «voice» of communities and patients - Elite appropriation of CBOs - No analysis (rewards/sanctions) based on patients’ satisfaction survey Financially viable Relatively costly, esp. community verification
  11. 11. Findings Little time for data analysis, feedback and coaching Proportion of time spent on different activities for imp. agency staff in the field, based on timesheets of implem. agency staff Quantitative verification 46% Community verification 26% Quality evaluation 18% Data recording 10% Verification 67% Others 16% Data analysis and Meeting at central level 9% Training and Coaching 8%
  12. 12. Findings verification implementation issues Planned Observed Provide data, analyzed for feedback and coaching to facilities Little time for data analysis and coaching activities. No analysis at all for community verif. data Avoid fraud and provide a threat of sanctions to providers Sanctions for frauds and discrepancies in quantity reported are rarely applied. Rewards for higher patients’ satisfaction, or lower discrepancies in data reported are also not implemented Transparent, regular and timely payment Delays in the payment of bonus Improve governance and stewardship DHMTs rarely involved (they don’t have time, nor resources and RBF verif. is not «motivating» enough) Channel the «voice» of communities and patients - Elite appropriation of CBOs - No analysis (rewards/sanctions) based on patients’ satisfaction survey Financially viable Relatively costly, esp. community verification
  13. 13. Findings Verification implementation issues Planned Observed Provide data, analyzed for feedback and coaching to facilities Little time for data analysis and coaching activities. No analysis at all for community verif. data Avoid fraud and provide a threat of sanctions to providers Sanctions for frauds and discrepancies in quantity reported are rarely applied. Rewards for higher patients’ satisfaction, or lower discrepancies in data reported are also not implemented Transparent, regular and timely payment Delays in the payment of bonus Improve governance and stewardship DHMTs rarely involved (they don’t have time, nor resources and RBF verif. is not «motivating» enough) Channel the «voice» of communities and patients - Elite appropriation of CBOs - No analysis (rewards/sanctions) based on patients’ satisfaction survey Financially viable Relatively costly, esp. community verification
  14. 14. Findings Payment delays Quarter Quarter End Invoice Transmission Delay due to verif. Transfer Delay due to transfer Total Delay (month) Q2 2012 Jun-12 Sep-12 3,5 Nov-12 1,5 5 Q3 2012 Sep-12 Dec-12 3,5 Apr-13 3 6,5 Q4 2012 Dec-12 Apr-13 4,0 Jul-13 3,5 7,5 Q1 2013 Mar-13 Jul-13 4,5 Nov-13 3,5 8 Q2 2013 Jun-13 Sep-13 3,5 Dec-13 3 6,5 Q3 2013 Sep-13 Dec-13 3,5 Feb-14 2 5,5 Q4 2013 Dec-13 Apr-14 4,0 May-14 0,5 4,5 Q1 2014 Mar-14 Jun-14 3,5 Jul-14 0 3,5 Q2 2014 Jun-14 Sep-14 3,5 Jan-15 3,5 7,0 Q3 2014 Sep-14 Dec-14 3,0 Feb-15 2 5
  15. 15. Findings Verification implementation issues Planned Observed Provide data, analyzed for feedback and coaching to facilities Little time for data analysis and coaching activities. No analysis at all for community verif. data Avoid fraud and provide a threat of sanctions to providers Sanctions for frauds and discrepancies in quantity reported are rarely applied. Rewards for higher patients’ satisfaction, or lower discrepancies in data reported are also not implemented Transparent, regular and timely payment Delays in the payment of bonus Improve governance and stewardship DHMTs rarely involved (they don’t have time, nor resources and RBF verif. is not «motivating» enough) Channel the «voice» of communities and patients - Elite appropriation of CBOs - No analysis (rewards/sanctions) based on patients’ satisfaction survey Financially viable Relatively costly, esp. community verification
  16. 16. Findings Verification implementation issues Planned Observed Provide data, analyzed for feedback and coaching to facilities Little time for data analysis and coaching activities. No analysis at all for community verif. data Avoid fraud and provide a threat of sanctions to providers Sanctions for frauds and discrepancies in quantity reported are rarely applied. Rewards for higher patients’ satisfaction, or lower discrepancies in data reported are also not implemented Transparent, regular and timely payment Delays in the payment of bonus Improve governance and stewardship DHMTs rarely involved (they don’t have time, nor resources and RBF verif. is not «motivating» enough) Channel the “voice” of communities and patients - Elite appropriation of CBOs - No analysis of (no rewards/sanctions based on) patients’ satisfaction survey Financially viable Relatively costly, esp. community verification
  17. 17. Findings Verification implementation issues Planned Observed Provide data, analyzed for feedback and coaching to facilities Little time for data analysis and coaching activities. No analysis at all for community verif. data Avoid fraud and provide a threat of sanctions to providers Sanctions for frauds and discrepancies in quantity reported are rarely applied. Rewards for higher patients’ satisfaction, or lower discrepancies in data reported are also not implemented Transparent, regular and timely payment Delays in the payment of bonus Improve governance and stewardship DHMTs rarely involved (they don’t have time, nor resources and RBF verif. is not «motivating» enough) Channel the «voice» of communities and patients - Elite appropriation of CBOs - No analysis (rewards/sanctions) based on patients’ satisfaction survey Financially viable Relatively costly, esp. community verification
  18. 18. Findings Verification processes are costly Funds to implem agency for verification activities (only) Funds to CBOs 0.33 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Funds for RBF bonus to facilities Total funds for verification activities 1,595,644 USD 718,295 USD 0.12
  19. 19. Discussion & conclusions • Methodologically What are Impact Evaluation really testing, if the theory of change is modified by implementation challenges?  it is essential to include an analysis of the implementation processes in the RBF schemes’ evaluations • Design The design of the key elements of RBF schemes should be adapted to the context and iteratively modified and improved during implementation • Verification Our analysis leads us to question whether the rational for three-pronged (incl. community verif.) and thorough (i.e. non-random) verification is valid under all conditions.
  20. 20. Acknowledgment • Thanks to all the Technical Assistants of the Implementation Agency who provided time, insight and expertise that greatly assisted the research • To Maria Paola Bertone and Olivier Barthes for assistance with methodology, and all the AEDES team for comments that greatly improved this presentation • Many thanks also to Dr Akpamoli and his team and to Maud Juquois and Ibrahim Magazi from the World Bank for their constant support during the implementation of the project and with this research.

×