Shared at a data dissemination and data use workshop on the results of the impact evaluation of the Strengthening Tuberculosis Control in Ukraine project. Access another presentation at https://www.slideshare.net/measureevaluation/evaluation-of-the-impact-of-a-social-support-strategy-on-treatment-outcomes/.

1. Evaluation of the Impact
Zulfiya Charyeva
Nicole Judice
MEASURE Evaluation, Palladium
TB-HIV Integration
Strategy on Treatment
Outcomes

2. Strengthening TB Control in Ukraine
Project (STbCU)
 Goal – Reduce the burden of TB through
specific quality assurance and system
strengthening measures for routine TB
services, MDR-TB, and HIV co-infection
• Provide social support to promote patient
adherence to TB treatment (social support
study)
• Improve access to and use of timely
diagnostic and treatment for HIV
co-infected patients to reduce
mortality (TB-HIV integration study)

3. TB-HIV Integration Program Objectives
 Identify gaps in TB-HIV co-infection services
and build capacity to
address them
 Ensure HIV testing for TB patients and
effective referral of those found to be HIV
positive
 Provide TB screening of HIV patients
and referral to TB services for suspected TB
cases

4. Activities Implemented by the STbCU
Project
 Work with the government to institutionalize
best practices for TB-HIV management
 Develop databases and protocols to
support reporting and sharing of data
across TB and HIV services
 Provide numerous trainings to TB, HIV, and
infectious disease (ID) specialists in caring
for TB-HIV co-infected patients

5. Evaluation Design
A mixed-methods approach with
a quasi-experimental quantitative
evaluation design complemented
by qualitative descriptive work to
inform the findings.

6. Impact Evaluation Questions:
TB-HIV Integration Study
 A. Completion of TB-HIV service cascade:
What proportion of TB and HIV/AIDS
patients complete each step in the cascade
of services from screening
to treatment per national protocol?
 B. Factors affecting the use of TB-HIV
services: What facilitates or impedes timely
access to and use of testing and treatment
for TB and HIV/AIDS patients?

7.  C. Impact of service integration on time
to services: Do service integration, training
and support between TB and HIV/AIDS services
decrease the time lag between each step of
service (screening, testing,
and treatment) for TB and HIV/AIDS patients?
 D. Impact of service integration on all-cause
mortality: Do service integration, training and
support between TB and HIV/AIDS services
decrease all-cause mortality among the TB-HIV
coinfected patients?
Impact Evaluation Questions:
TB-HIV Integration Study (2)

8. Summary of Methods, Table 1
Question Data
collection
Data sources Sample Sample size Analysis
A, C, D Chart
abstraction
Patient medical
records;
electronic TB
manager
Systematic
random
sampling
Baseline: 1,427 charts from
facilities and 1,064 charts
from AIDS centers. End line:
1,448 charts from TB facilities
and 1,529 charts from AIDS
centers.
Survival analysis,
proportional
models with a
difference-in-
differences
approach
B In-depth
interviews
(IDIs)
Patients,
providers,
STbCU staff
Purposive Baseline: 18 IDIs with
providers in six oblasts.
End line: 30 IDIs with
17 IDIs and 6 focus group
discussions with providers in
3 intervention oblasts, 6 IDIs
with STbCU staff.
Qualitative
data analysis
Context Facility survey Facility lead
doctors and
administrators
All facilities
in the
regions
Baseline: 18 TB and 9 HIV
facilities. End line: 17 TB and 8
HIV facilities.
Descriptive
statistics

9. TB-HIV Integration Study Oblasts
Intervention oblasts
 Kharkiv, Odessa, and Zaporizhzhya
 Selected based on TB and HIV case
counts and co-infection rates
Comparison oblasts
 Kiev, Mykolaiv, and Zhytomyr
 Loosely matched to intervention oblasts
on TB and HIV disease rates, population
density, and level of socio-economic
development

10. Study Windows – Questions A, C, D
 Baseline: January – December 2012
 End line: April 2014 – June 2015

11. Sampling for Questions A, C, D
TB facilities patient sampling:
 First random sample (S1) of patients was
selected without replacement from all new
TB patients in the baseline/end line study
window, proportionate to size of
the oblast
 A second sample (S2) was then selected
from the remaining identified co-infected
patients

12. AIDS centers patient sampling:
 First random sample (S1) of patients was
selected without replacement from the oblast
AIDS centers registration journals in the
baseline/end line study window, proportionate
to size of the oblast
 A second sample (S2) – the ID specialists
in each oblast provided a list of all coinfected
patients in the oblast
• Systematic random sampling in Odessa
• Use all remaining charts in other oblasts
Sampling for Questions A, C, D

13. Difference-in-Differences Definition
Source: Wikipedia, https://en.wikipedia.org/wiki/Difference_in_differences

14. Findings

15. RQA: Completion of TB-HIV
Service Cascade – Findings
from AIDS Centers

16. TB Screening and Testing Cascade for
HIV Patients (Sample 1) – Figure 4.1

17. TB and HIV Treatment Cascade for HIV
Patients (Co-Infected Patients) – Figure 4.2

18. RQA: Completion of TB-HIV
Service Cascade – Findings
from TB Facilities

19. RQA: Completion of TB-HIV Service
Cascade – Findings from TB Facilities
HIV testing:
 In intervention oblasts, 91% of new TB
patients with no prior HIV diagnosis
received an HIV diagnostic test at baseline,
compared with 99%
at end line

20. 85% 82% 78%
7%
87% 86% 86%
14%
15%
5%
13%
5%
1%
93%
88% 88%
11%
92% 91% 91%
14%
7%
1%
8%
2%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
New TB
patients
HIV VCT HIV diagnostic
test
HIV case
confirmed
New TB
patients
HIV VCT HIV diagnostic
test
HIV case
confirmed
Baseline Endline
Intervention No prior HIV Intervention Prior HIV Comparison No prior HIV Comparison Prior HIV
HIV Testing Cascade for Newly Diagnosed TB
Patients (Sample 1) – Figure 4.3

21. RQA: Completion of TB-HIV Service
Cascade – Findings from TB Facilities
ART initiation:
 The percentage of HIV-positive TB
patients with no prior HIV diagnosis that
started ART increased over time in the
intervention group from 21% at baseline
to 51% at end line
 ART initiation decreased from 48%
to 47% in comparison oblasts

22. HIV Treatment Cascade for Co-Infected TB
Patients – Figure 4.4
88%
35%
18% 18%
84%
56%
42% 42%
12%
3%
2% 2%
16%
8%
4% 4%
98%
78%
48% 48%
98%
71%
46% 46%
2%
1%
2%
1%
0%
20%
40%
60%
80%
100%
HIV case
confirmed
HIV
registration
Started
ART
TB outcome
recorded
HIV case
confirmed
HIV
registration
Started
ART
TB outcome
recorded
Baseline Endline
Intervention No prior HIV Intervention Prior HIV Comparison No prior HIV Comparison Prior HIV

23. RQB: Factors Affecting the Use
of TB – HIV Services

24. Factors That Facilitate Access to
and Use of Services
 Improvements in timely TB diagnostic testing
 Enhanced services for TB patients in
AIDS centers
 Tracking of TB clients who were successfully
treated
 Good communication between TB and ID
specialists
 Awareness of medical staff concerning HIV
 Availability of free ART

25. Diagnostics became faster. New methods of
sputum testing have appeared. Rapid tests for
patients with co-infection. And Bactec and gin
expert in case of TB … Informational support,
laboratory diagnostics, methods of treatment
– everything got systematized and improved. There
has been integration of two services
and by now we have pretty good services.
[Focus group discussion participant]
Factors That Facilitate Access to
and Use of Services

26. … if the the patient has a fever, or if there are any other
symptoms like cough, sweating, weight loss and etc., I
immediately connect TB doctor.
Thank God we have one in our facility. And, in general,
it is very good, because, when there was no TB doctor,
it was very difficult for us in this respect. And now, right
here we can make a common decision whether to do
a CT, or X-ray.
[Provider]
Factors That Facilitate Access to
and Use of Services

27. Barriers to Timely Access to and Use of
Services – Providers’ Perspectives
 Clients’ inability to accept
their HIV diagnosis and follow
treatment instructions
 Short-staffed facilities
 Infrastructure issues

28. I came to work here in 2005 and the staff
has not increased since that time, despite
the fact that we have more and more
patients. There should be 12 patients [per
doctor], but in fact we have 36–40
[patients].
[Focus group discussion participant]
Barriers to Timely Access to and Use
of Services – Providers’ Perspectives

29.  Dealing with HIV-related stigma
 Long lines at facilities
 High out-of-pocket costs associated with
travel, inpatient stay, laboratory work, and
medications
 Confusion about where to go to receive
treatment
 Confusion about medication regimens and
their debilitating side effects
Barriers to Timely Access to and Use
of Services – Clients’ Perspectives

30. No, I don’t get the treatment by the place of my
residence, but in the facility of XXX district. My
treatment costs me a penny. I spend around 100
UAH only to get here and around three hours at
my best, and I have to make as much as three
transport changes. I have to travel to receive my
treatment every day, which is very inconvenient.
[Patient]
Barriers to Timely Access to and Use
of Services – Clients’ Perspectives

31. Barriers to Timely Access to and Use
of Services
Client databases are not consistently shared
across all TB and HIV services
 Makes coordination challenging
 Further increases travel costs for patients,
as they have to travel between TB and HIV
clinics

32. RQB – Conclusion
 The study suggests that while
improvements in diagnostic testing and
coordination across TB and HIV facilities is
well underway, factors such as stigma,
emotional burden, adequate education to
deal with the side effects of the medication,
and high patient out-of-pocket costs still
need to be addressed.

33. RQC: Impact of Service
Integration on Time
to Services

34. RQC: Impact of Service Integration on Time
to Services – Findings from AIDS Centers
HIV Testing:
 Patients in the intervention group were twice as likely
at baseline (p<0.001) and 16% less likely
at end line (p=0.115) to be tested for TB
 Over the course of the TB-HIV integration program,
TB testing improved significantly
for both groups
 In the intervention group relative to the comparison
group, the net impact of the program on TB testing
was negative (HR=0.40, p<0.001)

35. Figure 6.1: Time to TB Testing for Patients
at the AIDS Centers (Sample 1)
0.000.250.500.751.00
0 200 400 600 800 1000
Time (days)
Baseline
0.000.250.500.751.00
0 200 400 600 800 1000
Time (days)
Endline
TB testing among HIV patients
Comparison Oblasts Intervention Oblasts

36. RQC: Impact of Service Integration on Time
to Services – Findings from AIDS Centers
ART initiation:
 At baseline, patients in the intervention group
were 37% less likely to begin ART compared
to those in the comparison group (p<0.05)
• This difference reduced at end line to 22%
 The difference-in-differences model: the TB-HIV
integration program resulted in a significantly
positive impact on increase in ART testing in the
intervention oblasts (HR=1.49, p<0.05)

37. Figure 6.2: Time to ART Initiation among
Co-Infected Patients by Intervention Status
Wald chi-square test: p= 0.292
0.000.250.500.751.00
0 200 400 600 800 1000
Time (days)
Baseline
0.000.250.500.751.00
0 200 400 600 800 1000
Time (days)
Endline
ART initiation among co-infected patients by intervention status
Comparison Oblasts Intervention Oblasts

38. RQC: Impact of Service Integration on Time
to Services – Findings from TB Facilities
HIV Testing:
 TB patients in intervention oblasts were
42% less likely at baseline (p<0.001) and
23% less likely at end line (p<0.01) to be tested
for HIV compared to TB patients
in comparison oblasts
 Difference-in-differences results model:
a positive impact on the likelihood of receiving
an HIV diagnostic test (HR=1.28, p<0.05)

39. Figure 6.3: Time to HIV Testing for Patients
at TB Dispensaries (Sample 1)0.000.250.500.751.00
0 200 40 0 600 8 00
Time (days)
Baseline
0.000.250.500.751.00
0 200 400 600 800
Time (days)
Endline
HIV testing among TB patients
Comparison Oblasts Intervention Oblasts

40. RQC: Impact of Service Integration on Time
to Services – Findings from TB Facilities
ART initiation:
 Patients in intervention oblasts were 53% less
likely to initiate ART than patients in
comparison oblasts at baseline (p<0.001), but
were 35% more likely to initiate ART than the
comparison group at end line (p<0.01)
 Difference-in-differences model: a very strong
and positive estimate of program impact on
the likelihood of ART initiation (HR=2.91,
p<0.001).

41. Figure 6.4: Time to ART Initiation among Co-Infected
Patients at TB Dispensaries by Intervention Status0.000.250.500.751.00
0 200 400 600 800 1 000
Time (days)
Baseline
0.000.250.500.751.00
0 200 400 60 0 800 1000
Time (days)
Endline
ART initiation among co-infected patients by intervention status
Comparison Oblasts Intervention Oblasts

42. RQD: Impact of Service
Integration on All-Cause
Mortality

43. RQD: Impact of Service Integration on All-Cause
Mortality – Findings from AIDS Centers
 At baseline, there were no difference in
survival between intervention and comparison
groups
 At end line, patients in the intervention group
are about 14% less likely to die compared to
the comparison group
• The difference is not statistically significant
 Difference-in-differences model: we do not
detect a significant impact of the integration
program on all-cause mortality

44. Figure 7.1. Time to death among coinfected patients
at AIDS centers by intervention status

45. RQD: Impact of Service Integration on All-Cause
Mortality – Findings from TB Facilities
 No difference in the likelihood of
all-cause mortality between
intervention and comparison groups
at baseline or at end line
 Do not detect a significant program impact
on the likelihood of death

46. Figure 7.2. Time to Death among Co-Infected
Patients at TB Facilities by Intervention Status0.000.250.500.751.00
Proportionalive
0 100 200 300 400 500 600
Time (days)
Baseline
0.000.250.500.751.00
0 100 200 300 400 500 600
Time (days)
Endline
Survival by intervention status
Comparison Oblasts Intervention Oblasts

47. Conclusions

48. Conclusions
Qualitative findings
 The TB-HIV integration program affected
several positive changes in the integration of
services, especially around availability of
diagnostic tests across facilities, and training
of providers

49. Findings from HIV center records
 The TB-HIV integration program is
associated with a significant increase
in timely initiation of ART
Findings from TB facilities records
 Significantly positive impact of the program on
the likelihood of patients receiving a diagnostic
HIV test and starting ARTs
Conclusions (cont.)

50.  We do not detect an impact on survival based
on data from either the TB or
HIV facilities
Conclusions (cont.)

51. Factors to Explain No Detection
of Program Impact on Survival
 At the time the patients entered AIDS
centers, those in the intervention facilities
might have been sicker
 We were not able to account for disease
severity variables such as CD4 cell count or
TB disease stage in our impact models, due
to the large amount of missing disease
characteristic data at baseline, especially at
AIDS centers

52. Questions and Discussion

53. MEASURE Evaluation is funded by the U.S. Agency for
International Development (USAID) under terms of
Cooperative Agreement AID-OAA-L-14-00004 and
implemented by the Carolina Population Center, University
of North Carolina at Chapel Hill in partnership with ICF
International, John Snow, Inc., Management Sciences for
Health, Palladium Group, and Tulane University. The views
expressed in this presentation do not necessarily reflect the
views of USAID or the United States government.
www.measureevaluation.org