GHME 2013 Conference
Session: Verbal autopsy
Date: June 18 2013
Presenter: Peter Serina
Institution:
Institute for Health Metrics and Evaluation (IHME),
University of Washington
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
The development of a shortened verbal autopsy instrument for routine surveillance
1. The development of a shortened
verbal autopsy instrument for
routine surveillance
June 18, 2013
Peter Serina
Post-Bachelor Fellow
2. Outline
• The context
• Construction of a shortened instrument
• Measuring performance
• Next steps
2
3. Cause of death data
3
• Cause of death is important
• Vital registration systems are limited
• An information gap exists
4. One solution: verbal autopsy
• Trained interviewer
• Structured questionnaire
o Signs
o Symptoms
o Demographic characteristics
• Lay respondent familiar with the
deceased
• Population Health Metrics
Research Consortium
(PHMRC) instrument:
o 153 questions
o 45 minutes
4
PHMRC GC13 Project
5. Verbal autopsy: an evolution
5
• Survey structure
• Analytic techniques
• Data capture
Wikimedia commons
6. Outline
• The Context
• Construction of a shortened instrument
• Measuring performance
• Next steps
• Conclusion
6
7. Construction of shortened instrument
Two approaches:
1. Expert-driven
2. Data-driven
7
Cost
(Time and
money)
Quality
(Survey
performance)
Coverage
(Population, caus
es, and areas)
8. Data-driven approach
The data:
• Population Health Metrics
Research Consortium (PHMRC)
• Gold standard verbal autopsy
validity study
The method:
• Tariff
• Systematic reduction
8
Country Site
GS
deaths
Mexico Mexico City 2,031
Tanzania
Pemba Island 822
Dar es Salaam 3,239
India
Uttar Pradesh 2,170
Andhra Pradesh 2,382
Philippines Bohol 1,898
Total 12,542
9. Tariff method
9
AIDS Malaria Fall
Suffered a fall 0 0 32
Decedent had
TB 5 0 0
Fever 1 1.5 -1
Decedent had
dementia 0 -1 0
Tariff
standard
deviation
18.5
2.9
1.3
0.6
Symptoms
Causes
10. Performance metrics
• Mean chance-corrected concordance for cause j
o Performance at the individual level
• Cause-specific mortality fraction (CSMF) accuracy
o Performance at the population level
10
11. Individual performance vs. question quantity for adults
11
Chance-
corrected
concordance
Number of questions
12. Outline
12
• The context
• Construction of a shortened instrument
• Measuring performance
• Next steps
• Conclusion
16. Outline
16
• The context
• Construction of a shortened instrument
• Measuring performance
• Next steps
• Conclusion
17. Next steps
• Link survey size to time to
implement
• Pilot surveys in:
o Philippines
o Bangladesh
o Sri Lanka
17
2.2: Did Juan Garcia have a fever?
18. Outline
18
• The Context
• Construction of a shortened instrument
• Measuring performance
• Next steps
• Conclusion
19. Conclusion
Our findings:
• > 40% fewer questions chosen in systematic, data-driven
process
• Negligible effects on performance with same population size
Impact:
• Greater number of verbal autopsies can be collected
• Strengthen routine surveillance
• Address the information gap
19
Purpose:Reasoning behind and scientific basis for shortened verbal autopsy survey instrument.
COD importantTrack changes in burden of disease across space and timeSet priorities for interventionsMonitor public health programsInform allocation and distribution of resourcesVital RegistrationVR being viewed as the best, though most expensive to set up and maintain. Vital registration also limitedFor adults, concordance of 67% when corrected for chanceWork at IHME: 38% of deaths in 2010 were covered by vital registration systems.How do we fill this critical information gap
VA used to determine COD, using survey-interview methods to ask about symptoms a deceased experienced prior to death from caregivers, family, and friends.
FamousRembrant Painting of an autopsyNot a new idea17th century London – death searchers (communicable disease)informal, taken on paper1950s through 1970s Survey structure change to be more systemmaticQuestionnaires were certified by physiciansPast 10 yearsNew advancements in analyzing techniques using statistical models and computer algorithmsData capture paper-based electronicHow can we use these advancements to create a shortened instrument both in terms of time and data entry that is able help fill that information gapEven with all of these advancements, surveys take 45 min, logging entry
Instrument too longDon’t mention ByassWhat you hope to achieve with a reduced instrumentCostCoverage QualityExpert-based: drop items from a long instrument based on expert opinion Been done by other research teamsCould take less time and money to createEmpirical: use a validation dataset and test the effect of dropping items or sets of items on chance-corrected concordance and CSMF accuracy.
Explain table
Don’t call it a toy, It’s a simplified examplehigh tariff= more related0 tariff= no relationNegative tariff= counter-indicativePresent as a story/anecdote imagine if family member of recently deceased…
NUMBER of QUESTIONSLooking at performance as we systematically reduce the number of questions based on the tariff standard deviationSet 90 questions as the starting point that we don’t want to go below (we can add questions back in if needed, for example health record information)Question=SymptomExplain axes
IHME study I cited earlier had 78% CSMF Accuracy for AdultsExplain axes, colors
CCC- concordance, % correct, corrected for chanceExplain axes, colors
A reduced VA tool has important implications for the costs, upkeep, and strengthening of routine surveillance systems, especially for accurately capturing COD dataAs we’ve learned, question reduction can really benefit from data-driven testing and approaches as the quantity and quality of questions – and their pairings – make a difference in the performance of the toolEnd on what you hope to achieve with the pilot study and how it will feed into the bigger COD/information systems/data collection issues and challenges Give it a go J