A talk on methods for representing potential drug-drug interactions (PDDIs) so that they can be more useful for clinical decision support. The talk has three sections: 1) a novel "evidential" approach to evaluating the validity of PDDIs, 2) a discussion of what information is necessary to make a judgement on the clinical relevance of a PDDI exposure, and 3) a proposal to use predictive risk models to actively monitor of the risk of specific adverse events during a PDDI exposure. This talk was given at the AHRQ-sponsored "Generating, Evaluating, and Implementing Evidence for Drug-Drug Interactions in Health Information Technology to Improve Patient Safety: A Multi-Stakeholder Conference"

1. Effective communication of Drug-drug
interaction (DDI) knowledge
Richard Boyce, PhD
Postdoctoral Fellow in Biomedical Informatics
University of Pittsburgh
Page 1

2. Objectives
1. Present a new approach to representing and making
predictions with drug mechanism knowledge
2. Identify core knowledge elements for DDI decision
support and suggest the possibility of a common
model for representing and sharing DDI knowledge
3. Suggest how further research on clinical trigger systems
could lead to reduced DDI alert fatigue while improving
patient safety
Page 2

3. Objective 1:
Present a new approach to representing and making
predictions with drug mechanism knowledge
Page 3

4. A large number of potential interactions can
be inferred based on mechanisms
pre-clinical and pre-market
post-market in vitro and in vivo
For example, metabolic interactions:
Page 4

5. Three informatics challenges to reasoning
with drug mechanisms1
Drug mechanism knowledge is dynamic, sometimes
missing, and often uncertain
Hypothesis:
A computable model of drug-mechanism evidence
will enable powerful methods for addressing these
issues.
[1] Richard Boyce, Carol Collins, John Horn, and Ira Kalet. Modeling Drug Mechanism
Knowledge Using Evidence and Truth Maintenance. IEEE Transactions on Information
Technology in Biomedicine, 11(4):386-397, 2007. Page 5

6. Case study: The Drug Interaction Knowledge
Base (DIKB)
The DIKB:
infers drug-mechanism knowledge from a computable
representation of evidence
makes interaction and non-interaction predictions with
measurable performance characteristics
Current focus:
DDIs that occur by metabolic inhibition
Page 6

7. The DIKB architecture
Page 7

8. The reasoning system's current DDI
prediction model1
Uses facts in the knowledge-base that meet explicit
belief criteria
Provides a "ball-park" magnitude estimate
based on the importance of the affected metabolic
pathway to the object drug
Predicts enzyme-specific non-interactions
enables the suggestion of alternative drugs
[1] Richard Boyce. An Evidential Knowledge Representation for Drug-mechanisms and
its Application to Drug Safety. PhD thesis, University of Washington, August 2008.
Page 8

9. The DIKB is an evidential system
All assertions are linked to their evidence for or
against:
(FLUCONAZOLE inhibits CYP2C9)
Evidence for Evidence against
helps assess credibility, draw attention to errors,
and establish confidence
Page 9

10. All assumptions are linked to evidence
Enables the system to identify when assumptions
are no longer valid
Helps identify poor uses of evidence
Page 10

11. All evidence is classified using a study
taxonomy
Oriented toward user confidence assignment
36 types over several sub-hierarchies
A DDI clinical trial
...randomized
...non-randomized
...
A non-traceable statement
...in drug product labelling
...
A metabolic enzyme inhibition experiment
...focusing on CYP450 enzymes
...done in human liver microsomes
...
Page 11

12. Basic quality standards are defined for every
evidence type
Example: Pharmacokinetic clinical trials
adequate duration and magnitude of dosing
patient genotype/phenotype is noted if the target
enzyme is polymorphic
Example: in vitro enzyme metabolism identification
human hepatocyte or recombinant CYP450
'selective' inhibitors and 'probe' substrates
Page 12

13. This DIKB's architecture helps identify
evidence use strategies
What evidence is sufficient to justify that a drug
possesses certain mechanistic properties in vivo?
What evidence-use strategy enables the system to
make the best predictions?
How do these predictions compare with the most
rigorous treatment of evidence possible?
Page 13

14. An experiment focusing on statins1
Study overview:
Collect evidence on statins and a sub-set of co-
prescribed drugs (16 drugs, 19 metabolites)‫‏‬
Make predictions using a “belief criteria strategy”
designed by two drug experts
Explore computer-generated strategies to find the
ones that were “best-performing”
Examine the feasibility of novel DDI predictions
[1] R. Boyce, C. Collins, J. Horn, and I. Kalet. Computing with evidence part II: An
evidential approach to predicting metabolic drug-drug interactions. J Biomed Inform,
2009. (2009) DOI:10.1016/j.jbi.2009.05.010. Page 14

15. An experiment focusing on statins1
Study overview:
(16 drugs, 19 metabolites)‫‏‬
36,000 strategies
[1] R. Boyce, C. Collins, J. Horn, and I. Kalet. Computing with evidence part II: An
evidential approach to predicting metabolic drug-drug interactions. J Biomed Inform,
2009. (2009) DOI:10.1016/j.jbi.2009.05.010. Page 15

16. Metrics
sensitivity, specificity, positive predictive value (PPV)‫‏‬
accuracy = (true pos + true neg) / (all predictions)‫‏‬
kappa Page 16

17. The validation set
drug pairs 586
interactions 41 (7%)
non-interactions 7 (1.2%)
unknown 538 (91.8%)
'unknown': a pair for which no interaction or non-
interaction can be confirmed
Page 17

18. Test 1 – the DIKB's most rigorous evidence-
use strategy possible
Sample of the two most critical assertions:
D inhibits ENZ :
FOR: clinical trials with D and another drug known to
selectively inhibit ENZ in vivo
AGAINST: metabolic inhibition studies with probe
substrates
D substrate-of ENZ
FOR/AGAINST: drug metabolism studies involving
participants with known CYP450 phenotypes or
genotypes
Page 18

19. Test 1 – results using the most rigorous
criteria
Statistic Value
number of predictions 17
True positive 14
True negative 0
novel 3
Positive predictive value 1.00
Sensitivity 1.00
Specificity undefined
Coverage of validation set 14/49 = 29%
Page 19

20. Test 2 – computer-derived 'best' strategy
LOE inhibits substrate-of
High in vitro study AND PK authoritative
Med
Low
Page 20

21. Test 2 – computer-derived 'best' strategy
LOE inhibits substrate-of
High authoritative
Med authoritative or in vitro
Low
Page 21

22. Test 2 – computer-derived 'best' strategy
LOE inhibits substrate-of
High authoritative
Med
Low in vitro study
Page 22

23. Test 2 – computer-derived 'best' strategy
LOE inhibits substrate-of
High in vitro study AND PK
Med in vitro study
Low
Page 23

24. Test 2 – computer-derived 'best' strategy
LOE inhibits substrate-of
High
Med authoritative or in vitro in vitro study
Low
Page 24

25. Test 2 – computer-derived 'best' strategy
LOE inhibits substrate-of
High
Med in vitro study
Low in vitro study
Page 25

26. Test 2 – better-performing strategies
8,351 (23%) of 36,000 tested strategies
Equal or better sensitivity, positive predictive value,
and agreement than the expert-designed strategy
1,152 (3%) performed at the top level
Statistic Value Statistic Value
true positives 34 false positives 0
true negatives 0 false negatives 0
sensitivity 1.00 specificity n/a
PPV 1.00 kappa 0.52
Page 26

27. Test 2 – relaxing belief criteria often
improved performance
more stringent less stringent
Page 27

28. Test 2 – relaxing belief criteria often
improved performance
n=~1k
n=~17k
n=8
n=~18k
more stringent less stringent
Page 28

29. Test 2 Novel Predictions
31 novel interaction predictions
13 matched to 15 published case reports
18 other non-refuted interaction predictions
Two interaction predictions supported by case
reports that were not present in product labelling:
Interaction prediction Level Evidence
diltiazem – atorvastatin > 2-fold 2 case reports – possible
fluconazole – simvastatin > 2-fold 1 case report – possible
Page 29

30. Conclusions from the pilot experiment
The new approach to representing drug mechanism
knowledge
can suggest the best use of available evidence for
metabolic DDI prediction
is less subjective than ranking evidence and selecting
the most rigorous
This is interesting because
drug mechanism evidence base varies dramatically
across marketed drugs
Page 30

31. Objective 2:
Identify core knowledge elements for DDI decision
support and suggest the possibility of a common
model for representing and sharing DDI knowledge
Page 31

32. Example - A possible observed DDI
Interaction prediction Level Evidence
diltiazem – atorvastatin > 2-fold 2 case reports – possible
Two case reports reporting on five individuals who
developed symptoms of myopathy or
rhabdomyolysis1,2
All cases provide some evidence that an adverse
event (AE) was caused by this DDI
[1] P. Gladding, H. Pilmore, and C. Edwards. Potentially fatal interaction between
diltiazem and statins. Ann Intern Med, 140(8):W31, 2004.
[2] J. J. Lewin 3rd, J. M. Nappi, and M. H. Taylor. Rhabdomyolysis with concurrent
atorvastatin and diltiazem. Ann Pharmacother, 36(10):1546-1549, 2002. Page 32

33. Assessing the example DDI
Interaction prediction Level Evidence
diltiazem – atorvastatin > 2-fold 2 case reports – possible
The DDI:
is reasonable
could lead to a serious or fatal adverse event
...but, we don't know:
patient-specific risk factors
prevalence of co-prescribing and various outcomes
Page 33

34. Structured DDI assessment
A structured assessment scores evidence and
potential severity1
[1] E. N. van Roon, S. Flikweert, M. le Comte, P. N. Langendijk, W. J. Kwee-
Zuiderwijk, P. Smits, and J. R. Brouwers. Clinical relevance of drug-drug interactions :
a structured assessment procedure. Drug Saf, 28(12):1131-1139, 2005. Page 34

35. Evidence for/against a DDI
pre- and post-market studies
in vitro experiments
known or theoretical mechanisms
case reports and case series
pharmacovigilance Page 35

36. Risk factors
patient characteristics X potential adverse event
patient characteristics X DDI mechanism
drug characteristics
route of administration, dose, timing, sequence
Page 36

37. Risk factors depend on evidence
Page 37

38. Incidence
prevalence of co-prescription
prevalence of AE
incidence of AE in exposed and non-exposed
Page 38

39. Incidence and evidence strengthen each-
other
Page 39

40. Seriousness of the AE
Classified by specific clinical outcome
...but, can any seriousness ranking be generally
accepted?
no effect death
?
Page 40

41. Re-assessing the example DDI
drug pair diltiazem – atorvastatin
potential AE rhabdomyolysis
other HMG-CoA, inhibitors,
risk factors
phenotype, ...
five possible cases, established
evidence
mechanism
prevalance of pair common
general incidence of AE rare
incidence in exposed ...
incidence in non-exposed ...
Page 41

42. Structured assessments vary
focus and content
methods for ranking severity
across compendia, vendors1, and implementations2
[1] F.D. Min, B. Smyth, N. Berry, H. Lee, and B.C. Knollmann. Critical evaluation of
hand-held electronic prescribing guides for physicians. In American Society for
Clinical Pharmacology and Therapeutics, volume 75. 2004.
[2] Thomas Hazlet, Todd A. Lee, Phillip Hansten, and John R. Horn. Performance of
community pharmacy drug interaction software. J Am Pharm Assoc, 41(2):200-204,
2001.
Page 42

43. Agreement on common elements might be
possible
drug pair
potential AE
risk factors
evidence
prevalance of pair
general incidence of AE
incidence in exposed
incidence in non-exposed
...
...and could form the basis for a sharing DDI
knowledge across resources Page 43

44. Objective 3:
Suggest how further research on clinical trigger
systems might lead to reduced DDI alert fatigue while
improving patient safety
Page 44

45. Results from a recent review on medication
alerting1
“Adverse events were observed in 2.3%, 2.5%, and
6% of the overridden alerts, respectively, in studies
with override rates of 57%, 90%, and 80%.”
“The most important reason for overriding was alert
fatigue caused by poor signal-to-noise ratio”
Only one study looked at error reductions for DDI
alerts – the results were statistically non-significant
[1] H. van der Sijs, J. Aarts, A. Vulto, and M. Berg. Overriding of drug safety alerts
in computerized physician order entry. J Am Med Inform Assoc, 13(2):138-147,
2006. Page 45

46. What does the literature suggest?
find a balance between push vs. pull alerts
tier DDI alerts by severity
give users the ability to set preferences for some
types of alerts
provide value e.g. changing meds or correcting the
medical record from the alert
make alert systems more intelligent
Page 46

47. A potential complementary approach
Clinical event monitor -
a system that identifies
and flags clinical data
indicative of a potentially
risky patient state
Page 47

48. Example – UPMC MARS-AiDE
Page 48

49. Example triggers from UPMC MARS-AiDE1
signal alerts ADRs PPV
Na+ polystyrene given; taking a drug
2 1 0.5
that may cause hyperkalemia
Vitamin K given; taking warfarin 4 4 1
Clostridium difficile toxin positive;
taking a drug that may cause 2 2 1
pseudomembranous colitis
Elevated alanine aminotransferase
concentration; taking a drug that may 5 5 1
cause/worsen hepatocellular toxicity
[1] S. M. Handler, J. T. Hanlon, S. Perera, M. I. Saul, D. B. Fridsma, S. Visweswaran, S. A.
Studenski, Y. F. Roumani, N. G. Castle, D. A. Nace, and M. J. Becich. Assessing the
performance characteristics of signals used by a clinical event monitor to detect adverse
drug reactions in the nursing home. AMIA Annu Symp Proc, pages 278-282, 2008. Page 49

50. DDI-aware clinical event monitoring
Page 50

51. Potential benefits and risks of DDI-aware
clinical event monitoring
Benefits
automatic consideration of patient-specific risk factors
a possible safety net for some 'potential DDI' alerts
may provide implicit management options (e.g.
stop/change interacting drug)‫‏‬
Risks
potential for additional alert burden
is it ethical to not alert prescribers?
...
Page 51

52. Conclusions
We've looked briefly at three areas of research that
aim to make more effective use of DDI knowledge in
clinical care:
representing and using evidence for predicting DDIs
DDI knowledge sharing
integrating DDI knowledge with clinical event
monitoring
Page 52

53. Acknowledgements
Thanks to AHRQ for sponsoring this conference
Advisors/mentors: Steve Handler, Ira Kalet, Carol
Collins, John Horn, Tom Hazlet, Joe Hanlon, Roger
Day
Funding:
University of Pittsburgh Department of Biomedical
Informatics
NIH grant T15 LM07442
Elmer M. Plein Endowment Research Fund from the
UW School of Pharmacy
University of Pittsburgh Institute on Aging Page 53