This document outlines the SAFTINet project, which aims to build an infrastructure for comparative effectiveness research using distributed healthcare data. The project will create four patient cohorts to study how healthcare delivery system factors relate to outcomes for conditions like asthma, hypertension, and hypercholesterolemia. Challenges include linking variables measured at different levels like organization, practice, provider and patient. Methods like hierarchical models are proposed to account for this multilevel structure.

1. +
SAFTINet: ScalableArchitecture for
FederatedTranslational Inquiries Network
October 2011

2. +
Outline
 Overview
 CER Objectives

3. +
SAFTINet Overview and Aims
 Funding Mechanism: AHRQ ARRA OS: Recovery Act 2009:
Scalable Distributed Research Networks forComparative
Effectiveness Research (R01)
 To build the infrastructure for a distributed data network that
supports comparative effectiveness research
 Intended components of the network
 EHR and Medicaid claims data harmonized to a common
data model (modifications to OMOP)
 Made available for distributed query usingTRIAD grid
technology
 Supplemented with PROs and practice-level survey data

4. +
SAFTINet Research Objectives
Develop four cohorts of patients with
 asthma (childhood and adult)
 hypertension
 hypercholesterolemia
Conduct comparative effectiveness
research on healthcare delivery
system factors as they relate to
health outcomes in these cohorts

5. +
CER Hypothesis
 Health care delivery system factors, such as the patient-
centered medical home…
DELIVERY SYSTEM
FACTORS
+ COVARIATES →
OUTCOMES
(chronic disease
control)

6. +
CER Hypothesis
 Health care delivery system factors, such as the patient-
centered medical home…
are important determinants of the control of asthma, high
blood pressure and hypercholesterolemia.
DELIVERY SYSTEM
FACTORS
+ COVARIATES →
OUTCOMES
(chronic disease
control)

7. +
Building infrastructure for CER
 Considerations
 Are we collecting the right data to support CER?
 Are the data of sufficient quality to conduct high
quality CER?
 Do we have sufficient power to detect significant
effects?
 Have we identified the right covariates to control
for bias and confounding?

8. +
Methods and analysis
 Example hypothesis:
 Asthma outcomes among adults are better at health centers that
implement PCMH functions
 Unit of analysis?
 Outcomes are measured at the patient level
 Predictors are measured at the practice level
 Covariates (mediators, confounders, etc) exist at multiple levels
(patient, provider, practice, organization)
 Analytic strategy: Hierarchical linear models (aka mixed effects
or multilevel models)
 Can handle multiple levels of analysis in a single regression
equation
 Can handle missing data (requires at least two data points per unit
of analysis)

9. +
Challenges
 Limitations in use of “real world” clinical data for research
purposes
 Variability in documentation across providers and systems
 E.g., ICD-9 code may not mean a patient HAS that diagnosis – may
represent a “rule out” or “considered” code
 Differences across systems and practices in the collection of
patient-reported outcomes data (point of care vs non point of care)
 Sample size and power
 Highest-level unit of analysis?
 Organizations (n = 5)
 Practices (n = 50)
 Providers (n = 190)
 Patients (n = 440,000)

10. +
Challenges: Level of Analysis
 At what level(s) do we measure each variable?
 Is PCMH a factor that varies at the clinic level or the
organizational level?
 How do we link patients to a provider or practice?
 How do we structure our analytic plan?
 What are the potential confounders of the relationship
between PCMH and disease outcomes?
 How do we minimize the number of covariates given
the limited degrees of freedom?
 What biases in the data do we expect (and at what
level of analysis)?

11. +
Challenges: Level of Analysis
LEVEL
DELIVERY SYSTEM
FACTORS
COVARIATES
OUTCOMES
(chronic disease
control)
Organization
Practice
Provider
Patient

12. +
Challenges: Level of Analysis
LEVEL
DELIVERY SYSTEM
FACTORS
COVARIATES
OUTCOMES
(chronic disease
control)
Organization
Practice
Provider
Patient
Partner organizations report
PCMH is implemented at the
organization level

13. +
Challenges: Level of Analysis
LEVEL
DELIVERY SYSTEM
FACTORS
COVARIATES
OUTCOMES
(chronic disease
control)
Organization
Practice
Provider
Patient
Partner organizations report
PCMH is implemented at the
organization level
PCMH survey asks about how
providers use the elements of
PCMH in clinical care

14. +
Challenges: Level of Analysis
LEVEL
DELIVERY SYSTEM
FACTORS
COVARIATES
OUTCOMES
(chronic disease
control)
Organization
Practice
Provider
Patient
Partner organizations report
PCMH is implemented at the
organization level
PCMH survey asks about how
providers use the elements of
PCMH in clinical care
The “PC” is about patients—
should we ask for their input or
assess their behavior?

15. +
Challenges: Confounding
 In real-world setting, different practices measure
variables differently
 establish minimum requirements, e.g., for implementing
and reporting data from a PRO
 How do we address common-cause variables?
 Use of Directed Acyclic Graphs (DAGs) to identify a
minimal set of covariates to remove confounding

16. +
Challenges: Confounding
LEVEL
DELIVERY SYSTEM
FACTORS
COVARIATES
OUTCOMES
(chronic disease
control)
Organization
Practice
Provider
Patient
A practice’s degree of
PCMH-ness is likely
associated with how it
implements a PRO and with
the quantity and quality of
data it reports for a PRO

17. +
Questions/Comments/Input
“Knowing is not enough; we must apply.Willing is not enough; we must do.” —
Goethe