1. Scaling Up Performance Based
Financing in
Rwanda
2004-2008
Rwanda PBF team:
Paulin Basinga, Ghyuri Fritsche,
Bruno Meessen, Laurent
Musango, Louis Rusa, Claude
Sekabaraga, Agnes Soucat et al 1

2. Outline
2
Reconstruction and Innovations
1.
Scaling Up Performance Based
2.
Financing
Results
3.
Reforms
4.
Impact Evaluation
5.
2

3. Outline
3
Reconstruction and Innovations

Scaling Up Performance Based

Financing
Results

Reforms

Impact Evaluation

3

4. The post-colonial times ..
Modern health introduced in Rwanda free of charge to users and

funded through direct public subsidy : infrastructure,
equipments, personnel etc
1980s : shortages and rationing, dilapidation of health services

1992: community participation for financing and management of

health care (Bamako Initiative).
1994 : Genocide

1995-97: Reconstruction after the genocide : emergency

situations, NGOs, services free of charge.
1998-Willingness to come back to development and government

leadership: drug revolving funds re-established and cost sharing
reintroduced

5. Innovation II: Scaling Up of Community
Health Insurance
Source: Cellule d'appui aux mutuelles de sante. Ministry of Health / Rwanda 5

6. Innovation III: Scaling Up of
Performance Based Financing
Phase -0 (white shaded): the three PBF pilot projects
Phase-1 (pink shaded): districts in which PBF was started in Jan 2006
Phase-2 (red shaded): the seven ‘control districts’ in which PBF was implemented in April 2008.

7. Outline
7
Reconstruction and Innovations

Scaling Up Performance Based

Financing
Results

Reforms

Impact Evaluation

7

8. Chronology of Performance-Based
Financing in Rwanda

9. The PBF pilot experiments (2002-2005)
Three pilot schemes:
 Butare (since 2002)
 Cyangugu (since 2003)
 BTC (since 2005)
Led at provincial level by International NGOs.

Priority health interventions: child immunisation, ANC,

assisted deliveries, family planning, curative care.
A fee-for service at health center level

9

10. Scaling up: 2005-2008
2004: Evaluation of Butare and Cyangugu pilots

2005: Institutionalization:

2006: Scaling Up to 23 districts with 7 controls

2008: All districts


11. Evaluation: Cyagungu and Butare PBF
2004
Use of Assisted Deliveries over time
Performance Based Contracts Control
30
25
% of births
20
2001
15 2002
2004
10
5
0
Butare Cyangugu Gikongoro Kibungo

12. Institutionalization: integration in
country budget
Since 2005, government pays outputs through recurrent

budget (PBF budget line):
2005 US$ 800,000 for 4 districts

2006 US$ 5,000,000 for the country

Funds flow quarterly from Treasury directly to health

facilities’ Bank Account on the basis of results of
previous quarter
Since 2007, budget line item for PBF scheme for the

District Steering Committee activities based at District
level

13. Institutionalization: the HIVAIDS
money
One national approach, one institutional set-up, same unit

costs and same admin system facilitates alignment: Global
Fund pays for HIV indicators into their supported sites .
Payment through same Bank Account: e.g MSH and ICAP

– USG contractor-, FHI and BTC
Careful assessment of incentives through HIV monies in

PBF: protecting PHC services by linking payments of HIV
and PHC monies to levels of quality of general services.
Unit Fee * Quantity * % Quality = Payment;

14. National PBF Model : 2005-2008

15. National PBF model : 2005-2008
The national model for health centers is based on contracts

between different levels:
 Steering Committee (comité de pilotage) with
representation of health authorities
 Three layers of contracts:
Contract between CAAC and comité de pilotage

Contract between comité de pilotage and health facility

Contract between Health facility and individual health workers

In-depth verification activities :

Done by one focal point per administrative district for quantitative

evaluation
Done by hospital for qualitative evaluation

Separation of functions

15

16. PBF Payment at Facility Level
Payments for performance are based on the quantity of outputs

achieved (through case-based remuneration) conditional on the
quality of services rendered.
The outputs (quantity) are measured monthly

The quality is measured quarterly through the use of an elaborate

supervisory checklist. (13 services)
PBF Payment HC = Quantity * % Quality
The formula :

‘PBF Payment HC’ is the consolidated quarterly health center invoice (for either general

or HIV),
‘Quantity’ stands for the quarterly provisory health center invoice (the sum of all

indicators multiplied with their unit fees),
‘% Quality’ stands for the consolidated score—expressed as a percentage—obtained

from the quarterly quality supervisory checklist
16

17. General health indicators and PBF prices
Amount paid per
Nu Amount paid per case (US$)
m INDICATEUR case (Rwf)
1 Number of New cases 100 $ 0.18
2 Number of New cases received at the prenatal care (first visits) 50 $ 0.09
3 Number of Women who received 4 prenatal consultations 200 $ 0.37
Number of women completed the 2 or 3 or 4 or 5 Tetanus
4 vaccines 250 $ 0.46
Number of Women who received the 2nd dose of Intermittent
5 Preventive Treatment of malaria 250 $ 0.46
Number of at risk pregnancies Referred before 9 months of
6 pregnancy 1000 $ 1.83
Number of child aged 12-59 months seen at the curative care
7 service for growth monitoring 100 $ 0.18
Family planning P new users (DIU, Pills, injections, implants)
100 $ 0.18
8
Family planning : number of users: DIU, Pills, injections,
9 implants 1000 $ 1.83
10 Fully Vaccinated Child 500 $ 0.92
11 Institutional Deliveries at the health center 2500 $ 4.59
12 Emergency referrals to the Hospital for obstetric care 2500 $ 4.59
13 Malnourished children referred 1000 $ 1.83
17
14 Others Emergency referrals 1000 $ 1.83

18. HIV/AIDS indicators and PBF prices
(1$ = 545 RWF)
Amount Amount paid
Nu paid per per case (US$)
m INDICATORS case (Rwf)
1Number of clients tested for HIV at the VCT center 500 $ 0.92
Number of couples/partners tested during the reporting
2 month 2500 $ 4.59
Number of HIV+ pregnant women on ART treatment
3 during labor 2500 $ 4.59
4Number of infants born topatients who received CD4
Number of HIV positives HIV+ mothers tested 5000 $ 9.17
5 test 2500 $ 4.59
Number of HIV + patients traited with cotrimoxazole
6 each month 250 $ 0.46
7Number of new adults HIV+ on ART treatment 2500 $ 4.59
8Number of new infants HIV+ on ART treatment 3750 $ 6.88
9Number of HIV+ women on contraception 1500 $ 2.75
10Total number of HIV+ patients tested for tuberculosis 1500 $ 2.75
18

19. Process evaluation
Meeting of the Steering committee : quantity and

quality assessment
Counter verification of the patients in the

community : looking for the phantom patients
Counter verification of the quality score by hospital

team: randomly selected site
Comparison of PBF data with HMIS data

Peers evaluation (Hospital PBF)

19

20. Administrative & management
coordination
PBF admin system with internet based data entry and

retrieval facilitate decentralized management and
future decentralized payments (by districts);
Semi-automated payment module, linked to central

database, witch allow for ease of payments by MOF
(Ministry of finances) and others (MSH; BTC; FHI
and GF);
Central database allows for following trends and

forecast accurately financial risk;

21. •ICT management tools: www.pbfrwanda.org.rw
•INSERT GRAPHIC TO ADD MAP
•MAP IS 6.17” TALL

22. How many persons to do that?
MOH central PBF Unit (CAAC): 1 coordinator and two

full-time staffs;
A key role for partners (members of the CAAC and on

the field)
An Extended team approach has been put in place to

cover 23 districts, and includes PBF focal points from the
MOH, eight NGOs and a bilateral agency as a
coordination structure

23. Outline
23
Reconstruction and Innovations

Scaling Up Performance Based

Financing
Results

Reforms

Impact Evaluation

23

24. Results
Increases in the Volume of Services
1.
Increase of the Quality of Services
2.
Increase of staff productivity
3.
Provider Enthusiasm and Motivation
4.

25. Outline
25
Reconstruction and Innovations

Scaling Up Performance Based

Financing
Results

Reforms

Impact Evaluation

25

26. Increase in Volume of Services (after 27
months)
PBF Indicator January 2006 average/ March 2008 Percentage increase
month/ average/month/ (linear/log R2)
health center health center
( 258 health centers on (286 health centers on
average) average)
Institutional 21 37.5 78%
Deliveries (log 0.75)
New Curative 985 1,489 51%
Consultations (log 0.19)
ANC: second dose 21 52.5 150%
of Tetanus Toxid (log 0.63)
Family Planning 15.5 47.9 209%
new users (linear 0.88)
Family Planning 175.2 711.6 306%
users at the end of (linear 0.98)
the month

27. Results for Family Planning Users at the
end of the Month Family Planning, Modern Methods, Users at the End of the Month
Average Per Health Center per Month
700
640
600
Average number per month
500
R2 = 0.9784
400
300
175
200
100
0
1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11 12
2006 2007

28. FP Injections and oral methods at Health
Centers % Increase in Prevalence over 24
months; through December 2007
•January 2006

29. Other improvements
Over 16 months of PBF, the Quality increased on

average by 7% across these 13 services.
A sharp increase in staff productivity.

Whilst all providers appreciate the additional bonuses

that they earn through PBF, most also see clear
advantages in the better services they provide, and
take clear pride and ownership of these activities
which originate ‘from within’ as opposed to being
dictated from above.

30. Improvements confirmed by survey
based data

31. Assisted delivery – Modern contraceptive
use
Source : Enquêtes démographiques et de santé 1992, 2000 , 2005 et intérimaire 2007 31
Meilleures pratiques en SM au Rwanda

32. Decrease of fertility
Fertility Rate 2005-2008
6.2
6.1
6
5.9
5.8
5.7
5.6
5.5
5.4
5.3
5.2
2005 2007
Source : Rwanda DHS 2005-2007

33. Dramatic Increase of Coverage of
Insecticide Treated Nets
ITNs coverage 2005-2007
Proportion of Children less than 5 sleeping under a
bed net
80%
70%
60%
50%
40%
30%
20%
10%
0%
2005 2007
Source : Rwanda DHS 2005-2007

34. Decrease of malaria incidence
Malaria out patient Non Malaria out patient
 
Sheet 1 Sheet 1
Age group Age group
5 years and above 5 years and above
220K
Under 5 years Under 5 years
25K
200K
180K
20K
Sum of Conf# outpatient malaria
160K
Sum of Non malaria OPD
140K
15K
120K
100K
10K
80K
60K
5K
40K
20K
0K
0K
2001 2002 2003 2004 2005 2006 2007
2001 2002 2003 2004 2005 2006 2007
The trend of sum of Conf# outpatient malaria for Year G#C. Color shows details
The trend of sum of Non malaria OPD for Year G#C. Color shows details about
about Age group . The data is filtered on Country , which keeps Rwanda. The
Age group . The data is filtered on Country , which keeps Rwanda. The view is
view is filtered on Age group , which keeps 5 years and above and Under 5
filtered on Age group , which keeps 5 years and above and Under 5 years.
years.

35. Malaria deaths decreased
Malaria death

Non-Malaria Death

Sheet 1 Sheet 1
1000
Age group Age group
300
5 years and above 5 years and above
Under 5 years Under 5 years
900
250
800
alaria death (clinical + conf)
700
200
eath
600
on alaria D
Sum of N M
500
150
Sum of M
400
100
300
200
50
100
0
0
2001 2002 2003 2004 2005 2006 2007
2001 2002 2003 2004 2005 2006 2007
The trend of sum of Malaria death (clinical + conf) for Year G#C. Color shows
The trend of sum of Non Malaria Death for Year G#C. Color shows details about
details about Age group . The data is filtered on Country , which keeps Rwanda.
Age group . The data is filtered on Country , which keeps Rwanda. The view is
The view is filtered on Age group , which keeps 5 years and above and Under 5
filtered on Age group , which keeps 5 years and above and Under 5 years.
years.

36. Infant and Under 5 mortality rate
36

37. Outline
37
Reconstruction and Innovations

Scaling Up Performance Based

Financing
Results

Reforms

Impact Evaluation

37

38. Reforms 2000-2008
Autonomization

Performance Based Budgeting

Decentralization,


39. Reform I: Autonomization
Based on Bamako Initiative

Health centers and hospitals fully autonomous : 60%

healh centers public autonomous, 40% faith- based
private not for profit
Facilities are financially autonomous: Commercial bank

account, revenue from user payments and payments from
community insurance
Subsidized by the government: Needs based block grant

for wages, Performance Based Grant for recurrent costs,
and specific financing from public health programs
(vaccines, contraceptives, TB drugs and ARV etc)

40. Reform II: Performance Based
Transfers
“IMIHIGO”: contract between the President of

the Republic and the district mayors
Key health indicators integrated in the contract (in

2007: ITNs, Mutuelles, FP, safe deliveries,
hygiene..)
Strong political commitment to results

Quartely review with Prime Minister, President

attending twice a year

41. Reform III: Decentralization
Administrative and fiscal

decentralization gives Fiscal and Financial Decentralization
flexibility to local 80,000,000,000
governments by providing
them with needs and
performance based block 60,000,000,000
grants
Amount in RWF
Decentralization of wages
 Transfers to Districts
40,000,000,000 CDF
sent as a block grant to Transfers to Provinces
facilities
20,000,000,000
Facilities have the

authority to hire and fire
Facilities receive
 0
Disbursed 2002
Disbursed 2003
Disbursed 2004
Disbursed 2005
Projected 2007
Budget 2006
blockgrant from
government
“People follow the
 Ye ar
money”

42. Outline
42
Reconstruction and Innovations

Scaling Up Performance Based

Financing
Results

Reforms

Impact Evaluation

42

43. Study Rationale
43
No examples of rigorously evaluated bonus

payment schemes to public sector health care
providers in developing countries
No distinction between the incentive effect and

the effect of an increase in resources for the health
facilities
No unbundling of extrinsic and intrinsic-altruistic-

motivation
Link between worker motivation programs and

quality of care
43

44. Hypotheses
44
For both general health services and HIV/AIDS
services, we test whether PBC:
 Increases the quantity of contracted health
services delivered
 Improves the quality of contracted health
services provided
 Does not decrease the quantity or quality of non-
contracted services provided,
 Decreases average household out-of-pocket
expenditures per service delivered
 Improves the health status of the population 44

45. Evaluation Design
Make use of expansion of PBC schemes over time
45 
The rollout took place at the District level; random assignment at the

district level
Treatment and control facilities were allocated as follows:

Identified districts without PBC in health centers in 2005

Group the districts in “similar sets” based on characteristics:

rainfall

population density

livelihoods

Flip a coin to assign districts within each “similar” to treatment

and control groups.
45

46. More money vs. More incentives
Incentive based payments increase the total

amount of money available for health center,
which can also affect services
Phase II area receive equivalent amounts of

transfers
average of what Phase I receives

Not linked to production of services

Money to be allocated by the health center

Preliminary finding: most of it goes to salaries

46

47. Baseline Survey: Sampling Strategy
47
47

48. Lessons Learned: PBF
Start with easy things and then go progressively to complexity.

Health centers before hospitals

Simple quality indicators

Need for strong leadership and political will from authorities

Need for strong implementation oriented coordination structures

and large pool of trainers

49. Lessons Learned: PBF
Institutionalization is the key phase:

Importance of institutional contracts
Critical role of validation institution

PBF was used as a lever for reform:

Allowed to raise reuneration n assoication with

performance
Progressively shifted management of humn resources

Assoicated decentralization with rapid results


50. Lessons learned
Fiscal decentralization can help increase resources for health

facilities if well designed.. To serve purpose of service delivery..
autonomy of provision is essential…
Results Based Financing is a powerful mechanism to achieve the

twin objective of increased performance and increased retention
of qualified service providers
Combining public subsidy and private funding leads to increased

remuneration and better adequacy with needs
Delinking healh workers from the central wage bill and civil

service is possible..and health workers like it ..
Rwanda is back on track to reach the MDGs including MDG5


51. IMPACT EVALUATION OF
PERFORMANCE BASED
FINANCING
for
GENERAL HEALTH AND HIV/AIDS SERVICES
in
RWANDA
A collaboration between the Rwanda Ministry of Health, CNLS,
and SPH, the INSP in Mexico, UC Berkeley and the World Bank
51

52. Roll-out plan
52
Phase 0 districts (white) are those districts in which PBF

was piloted
Cyangugu = Nyamasheke + Rusizi districts

Butare = Huye + Gisagara districts

BTC = Rulindo + Muhanga + Ruhango + Bugesera + Kigali ville

Phase 1 districts (yellow) are districts in which PBF is being

implemented in 2006, following the ‘roll-out plan’
Phase 2 districts (green) are districts in which PBF is not

yet phased in; these are the so-called ‘Phase 2’ or ‘control
districts’ following the roll-out plan. According to plan, PBF
will be introduced in these districts by 2008.
52

53. Rollout plan for PBC in General Health
Sets Phase I Phase II
1 Kibungo all Kirehe all
2 Nyanza rem aining Kam onyi all
3 Gakenke rem aining
Rulindo rem aining
Byum ba rem aining
4 Rwam agana rem aining Kayonza west, all
Gatsibo west, all
5 Gatsibo east, all Nyagatare east,all
Kayonza east, all
6 Rutsiro all, south west Kibuye west, all
Nyam asheke rem aining
7 Ngororero all Kibuye east, all
Gasiza
8 Rutsiro all, except south west Kabaya all
9 Nyaruguru rem aining Gikongoro all
10 Burera all Ruhengeri all
Nyagatare west, all
53

54. Rollout plan for PBC in HIV/AIDS services
Imm ediately Phase I Phase II
These places already HIV / A IDS PBC to be introduced HIV / A IDS PBC to be introduced
have PBC in health A FTE R or SIMULA TE OUSLY A FTE R or SIMULA TE OUSLY
centers for non- with PBC for general services (ie. with PBC for general services (ie.
HIV / A IDS services N OT before) N OT before)
Rulindo Kibungo Kirehe
Kam onyi Nyanza Kam onyi
Gisozi Gakenke
Gisagara Rulindo
Butare Byum ba
Bugesera Rwam agana Kayonza
Gasabo Gatsibo Nyagatare
Nyarugenge Nyam asheke Kibuye
Cyangugu Ngororero Gasiza
Rutsiro Kabaya
Nyaruguru Gikongoro
Burera Ruhengeri
54

55. Program and Evaluation Roll-Out Plan
55
Timeline
Jan-06 Mar-06 Jun-06 2007 Feb-08 Apr-08 May-08 Jul-08
Treatment Start of intervention
Program
Implementation Control Start of intervention
FACILITY GH Household HIV Household
Impact Evaluation
SURVEYS Baseline FOLLOW-UP
55

56. Sampling Issues
Law of large numbers does not apply here…

Proposed solution:

Propensity scores matching of communities in treatment and

comparison based on observable characteristics
Over-sample “similar” communities in Phase I & Phase II

It turned out

Couldn’t find enough characteristics to predict assignment to

Phase I
Took a leap of faith and did simple stratified sampling

56

57. Analysis Plan
57
All analyses will be clustered at the district level
 Compare the average outcomes of facilities and
individuals in the treatment group to those in the
control group 24 months after the intervention began.
 Use of multivariate regression (or non-parametric
matching) : control confounding factors
 Test for differential individual impacts by:
Gender, poverty level

Parental background (If infant : maternal education,

HH wealth)
57

58. Difference in differences models
To test the robustness of the analysis

58
Control sample (both observed and unobserved) heterogeneity

A two-way fixed effect linear regression:

∑
yit = α PBC it + Bk X itk + γ i + δ t + ε it
k
Where:
yit is an outcome variable for facilitiy i in period t
or for individual i who lives in the catchment area of facility i,
PBC it is an indicator of whether facility i is being paid by PBC in period t,
X itk are time varying control variables,
i is a facility fixed effect,
δ t is a year fixed effect,
ε it is an error term.
58

59. SECTION 4:
Impact evaluation
Implementation
SECTION 4.1 :
Baseline surveys
59

60. Baseline Survey: Sampling Strategy
60
60

61. The baseline has 4 surveys
December 2005-March 2006:

General Health facility survey (166 centers)

Phase 1 : 80 facilities

Phase 2 : 86 facilities

General Health household survey (2,016 HH)

August – November 2006:

HIV/AIDS facility survey (64 centers)

HIV/AIDS household survey (1994 HH)

HIV/AIDS study for another presentation

61

62. General Health Centers Survey:
Content
62
General characteristics

Human resources module: Skills, experience and

motivations of the staff
Services and pricing

Equipment and resources

Vignettes: Pre-natal care, child care, adult care

VCT, PMTCT, AIDS detection services
Exit interviews: Pre-natal care, child care, adult

care, VCT, PMTCT
62

63. Baseline Health Facility: Utilization
NUM BER OF CONSULTATIONS IN AUGUST 2005
Phas e I Phas e II
Num be r Num be r
Se rvice s Provide d of Obs . cs td of Obs . cs td T-Stat
Curative child care 79 307.165 314.925 81 222.247 162.384 -2.151
Curative adult care 79 581.608 799.890 81 461.543 632.451 -1.055
Child grow th monitoring 74 129.770 227.201 70 152.100 357.436 0.450
Prenatal care 77 77.065 45.741 77 76.013 62.327 -0.119
Institutional delivery 74 17.041 17.940 74 13.230 12.221 -1.510
Home delivery 49 1.408 5.330 49 1.776 7.811 0.272
TB treatment 61 2.393 11.960 62 0.403 0.877 -1.307
Malaria treatment 79 305.557 238.645 80 234.838 211.342 -1.979
VCT 62 127.048 153.189 57 96.860 185.501 -0.971
63

64. General Health Household Level:
Sampling Method
64
Identified sectors and cells served by each of the 164

health facilities in the sample,
Randomly selected four cells from the catchment

area,
For each cell, obtained number of zones (10-15 hh)

Randomly selected three zones in each cell

Obtained household lists for each of the zones

Randomly selected one household for each zone

Produced random sample of 12 households per health

facility, with a final sample size of 2,016 households.64

65. Baseline Field Sampling: GH HH
HEALTH FACILITY
CELL 1 CELL 2 CELL 3 CELL 4
HH 1 HH 2 HH 3 HH 4 HH 5 HH 6 HH 7 HH 8 HH 9 HH 10 HH 11 HH 12
65

66. Baseline General Household Sample
2159 HH, 10,880 individuals

Average HH size is 5.71 individuals,

75% of the sample is under the age of 30 years old.

(Sampling strategy)
Not a nationally representative sample:

 Sample of rural households with children < 6 years
old
66

67. Baseline General Household Content
Socio-economic information

Anemia finger prick test: children 12-71 months old

Malaria dip stick test: children under age 6

Anthropometrics: <6 years old

Mental health: mothers, pregnant women, adults over

age 20
Sexual history and preventative behavior knowledge

Pre-natal care utilization and results

Parents or caretakers were asked for information

regarding child (<5 years) health status

68. Baseline Household Data: Education
68
Sample: ages 6 and up P HASE I (Intervention) P HASE II (Control)
Variable Nr. Obs Mean Std. Error Nr. Obs Mean Std. Error T -stat
Ever attended 3104 79.68% 0.011 4042 80.88% 0.010 -0.825
No schooling 2458 10.26% 0.007 3252 10.88% 0.007 -0.606
At least some P rimary 2458 82.93% 0.010 3252 79.97% 0.010 2.195 **
At least some Secondary 2458 6.46% 0.008 3252 8.42% 0.007 -1.841 *
Attended school in last12 months 2391 41.13% 0.015 3181 41.34% 0.012 -0.112
Able to read Kinyarwanda 2532 63.59% 0.015 3359 66.37% 0.014 -1.375
68

69. Baseline Household Data: Assets
P HASE I P HASE II
Variable Nr. Obs Mean Std. Error Nr. Obs Mean Std. Error T -stat
Complete sofa set 921 2.18% 0.005 1147 3.08% 0.007 -1.073
Radio 921 49.63% 0.023 1147 50.27% 0.017 -0.221
Radio-cassette or music system 921 4.29% 0.009 1147 5.89% 0.008 -1.274
Telephone Mobile 921 1.07% 0.005 1147 3.50% 0.007 -2.778 ***
Mosquito nets 921 22.53% 0.023 1147 28.00% 0.027 -1.542
Sewing machine 921 0.76% 0.003 1147 1.18% 0.004 -0.883
A bed 921 62.82% 0.023 1147 58.71% 0.022 1.290
W ardrobe 921 4.53% 0.009 1147 6.07% 0.010 -1.155
Metalic library 921 0.57% 0.002 1147 1.28% 0.004 -1.645
T able 921 63.47% 0.022 1147 63.40% 0.021 0.021
Chair 921 85.48% 0.018 1147 84.82% 0.019 0.255
A bicycle 921 16.37% 0.020 1147 17.26% 0.027 -0.262
69

70. Baseline Household Data:
Activities of Daily Living (21+ years)
Adults, mothers and pregnant women Phase I Phase II
Variable Obs Mean std Obs Mean std tstat
Total Sample
Accepts to perform ADL 1705 79.71% 0.011 2081 81.28% 0.012 -0.988
Nr of seconds for 1 sit-to-stand 1376 8.815 0.796 1680 8.807 0.680 0.008
Nr of seconds for 5 sit-to-stand 1373 16.991 0.406 1676 16.161 0.437 1.392
Nr of sit-to-stand in previous 1329 4.932 0.019 1619 4.904 0.015 1.114
Squat for 30 seconds, seconds 1373 29.194 0.228 1676 28.879 0.231 0.970
Balance on right foot, seconds 1374 29.074 0.223 1676 28.278 0.240 2.429 **
Balance on left foot, seconds 1373 28.893 0.238 1674 28.190 0.248 2.046 **
70

71. Baseline Household Data: Prenatal Care
71
P hase I P hase II
Variable Nr.Obs. Mean St d.Error Nr.Obs Mean St d.Error T -st at
Of all births since Jan. 2005
In facility birth 1238 32.05% 0.026 1462 34.87% 0.025 -0.777
Of most recent pregnancy
Times received PNC 723 2.781 0.046 900 2.687 0.052 1.368
Injection to prevent tetanus 728 74.52% 0.023 900 72.56% 0.019 0.654
71

72. Baseline Household Data:
Child Immunization
72
P hase I P hase II
Variable Nr.Obs. Mean St d.Error Nr.Obs Mean St d.Error T -st at
12-23 months old
fully_immunized 262 75.36% 0.03 295 77.04% 0.04 -0.33
12-71 months old
fully_immunized 1154 63.77% 0.03 1387 65.28% 0.02 -0.43
72

73. Baseline Household Data:
Child Health Care Utilization (<6 years)
P hase I P hase II
Variable Nr.Obs. M ean St d.Error Nr.Obs M ean St d.Error T -st at
days_sick 1438 1.96 0.12 1717 2.59 0.13 -3.54 ***
receive_care 597 25.21% 0.02 803 25.58% 0.02 -0.13
times_receive_care 143 1.58 0.10 218 1.44 0.08 1.12
cost_fees 135 246.20 43.80 213 285.71 42.82 -0.65
cost_supplies 132 93.05 52.04 208 101.43 28.25 -0.14
cost_medicine 133 287.77 64.66 207 475.54 76.57 -1.87 *
cost_medicine_nopres162 73.48 19.46 283 109.43 30.67 -0.99
cost_lab 134 77.13 21.38 208 78.92 21.43 -0.06
cost_other 134 21.35 8.28 204 45.29 19.51 -1.13
73

74. Baseline Household Data:
Child Biomarkers (<6 years)
27.82%
30.00%
25.00% 22.16%
19.53%
18.67%
20.00%
Phase I
15.00%
Phase II
10.00% 7.51% 7.33%
5.00%
0.00%
Anemic Malaria Fever and Malaria
74

75. Validity of Sample
Require two different validations:

Validate the sampling for the evaluation design

Diff in means tests between Phase I and Phase II to determine

if intervention and comparison groups balanced at baseline
Validate the quality of data

Compare descriptive stats to other sources of national data

(i.e.: 2005 & 2007 DHS, MOH data)
75

76. Validity of Sample and Data
76
Evaluation design

Of 110 key characteristics and output variables of HF, the

sample is balanced on 104 of the indicators.
Of 80 key HH output variables, the sample is balanced on 73

of the variables.
Majority of the indicators which differ between Phase I and Phase II are

results from patient exit interview, which is not a random sample.
Quality of data

HH Results comparable to the 2005 DHS, MOH data

76

77. SECTION 4.2 :
Program implementation and
monitoring
77

78. Timeline of Activities
2005 2006 2007 2008
12 1 2 3 4 5 6 7 8 9 10 11 12 123456789
General Health Facility
BASELINE DATA General Health Household
COLLECTION HIV/ AIDS Facility
HIV/ AIDS Household
PBF TRAINING PHASE I DISTRICTS
PHASE I DISTRICTS: OUTPUT
PBF PAYMENTS
PHASE II DISTRICTS: INPUT
MONITORING
DATA COLLECTION
Health Facility
FOLLOW UP DATA
General Health Household
COLLECTION
HIV/ AIDS Household
PHASE I DISTRICTS
PBF TRAINING
PHASE II DISTRICTS
78

79. Monitoring Program Roll-out
Regular participation in the PBF technical committee

meetings by the impact evaluation team members
Monitor threat to internal validity of sample

from:
Political pressure to expand PBF into Phase II districts

before 2008
Many facility directors and providers in Phase II districts

heard of PBF through colleagues or media so attempt to
imitate treatment
Ensure exposure to PBC for enough time

Avoid contamination: Training of phase 2 started after

data collection for HF in May 2008
79

80. Monitoring Program Roll-out
Additional data collection effort focused on

monitoring PBF roll-out at facility level
Date received training

Relationship with Comite de Pilotage: Number of

audits conducted
Amounts received at facility due to PBF

Allocation of PBF to salaries and other

Monitoring helped to ensure the evaluation team

understood the actual roll-out
80

81. Monitoring Program Roll-out
Key points for GH analysis

Baseline: Collected prior to training or first payments.

Found little evidence of imitation of treatment in Phase

II
Follow-up: The MOH initiated a revised training

course for ALL districts in 2008.
Phase I districts received March-April 2008, and Phase II

districts received in May 2008
May look at indicators up to March 2008 for all health

facilities as health facility data collection didn’t end until July
2008
81

82. SECTION 4.3 :
Follow up surveys
82

83. Follow-up surveys
February-September 2008

3 surveys:

Combined health facilities survey for General Health -

HIV/AIDS
Household survey for General Health (panel data)

Household survey for HIV/AIDS (panel data)

83

84. Follow-up Field Sampling: GH HF
Return to 166 facilities

Some GH facilities began offering HIV/AIDS services

(VCT, PMTCT and/or ARV) between 2006-2008
Identified in the field and used the HIV/AIDS HF

questionnaire; all GH HF questions still asked but in
different format
94 (56.63%) GH 2006 & 2008

60 (36.14%) GH 2006; HIV/AIDS 2008

12 (7.23%) incomplete information

84

85. Follow-up Field Sampling: GH HH
Objective:

Return to the same households to create panel data set

(2006-2008)
Print baseline roster (names, codes) and keep consistent

across waves
Account for household members who left and new

arrivals from 2006-2008
85

86. Follow-up Field Sampling: GH HH
Return to same households:

In total 2159 were suppose to be surveyed in the

catchment area of 167 facilities.
1888 (87%) were interviewed in 2006 and 2008

267 (12%) were replaced

4 (0.2%) were not found and not replaced

% of replacement by region:

South (24%), North (14%), East (4%) and West (13 %)

86

87. Follow-up Field Sampling: GH HH
External reasons:

Migration result of 1) avoiding Gacaca, 2) employment in Kigali,

3) famine in South during the last 2 years
Decentralization in 2006 renamed some areas in study sample;

impossible to locate based on baseline location
Internal reasons:

For some health facilities, the baseline HH team didn’t follow

sampling procedure
Given a cell to survey by the SPH team but difficult to reach

Used the same cell information but surveyed households in another

area
Health facilities with 10-13 hh replaced

87

88. Follow-up Field Sampling: GH HH
WHAT DOES THIS MEAN FOR ANALYSIS?

Restrict to only matched households

1,888 households

88

89. Assisted delivery – Modern contraceptive
use
Source : Enquêtes démographiques et de santé 1992, 2000 , 2005 et intérimaire 2007 89
Meilleures pratiques en SM au Rwanda

90. P roportionnal Malaria morbidity in Health
C entres vs Health Utilization R ate
80
75
73,5
71,1
70,3
70
67,4
60
50,4
50
44,4
40
37,8 37,9
30 29,9 28,4
27,4
25
20
15
10
0
2001 2002 2003 2004 2005 2006 2007
Malaria morbidity Health utilis ation rate 90

91. Infant and Under 5 mortality rate
91

92. Trend of Maternal Mortality ratio
92
Source : Demographic and Heath survey 1992, 2000 et 2005.

93. Section 5:
Current and next steps
93

94. Current and Next Steps
Reformat, clean data bases to create panel data

Initial GH HF results: January 2009

Initial GH HH results: March 2009

Plan dissemination workshop in Kigali to discuss

initial results with key stakeholders and TWG

95. QUESTIONS?
SUGGESTIONS?
95

96. Performance Based Financing
Rwanda : Increase in utilization of services
2006-2008
(average of 206 health centers)
350
300
250
200
%
1 50
1 00
50
0
Planning
Institutional
Planning New
New Curative
Consultations
Tetanus
Deliveries
Family
Toxid
Total
Family
Health Interventions