1. It’s Back: Predicting Adverse
Drug Reactions Using PubChem
Screening Data
TITLE
Yannick Pouliot
(with significant contributions from Annie Chiang)
8/31/2010

2. Motivation
Short-term: Determine feasibility of
predicting specific classes of adverse
drug reactions (ADRs) using machine
learning and compound screening data
Long-term: Use collection of simple
screens to assess likelihood of tissuespecific ADRs

3. Understanding “BioAssay” Notion
• Usually, BioAssay = collection of activity
measurements for compounds screened against
a specific target in a cell type at one or more
concentrations
• However, scope of BioAssay DB goes beyond
compound screening:
▫ Cell-free assays
▫ In vivo assays

4. What’s a SOC?
• SOC = System of Organ Classes
• A SOC groups “… adverse reaction Preferred
Terms pertaining to the same system-organ”.
• Example: SOC C0236104 - “Resistance
Mechanism Disorders”

5. Knowns
• Drugs frequently exhibit a higher frequency of
tissue-specific ADRs beyond generic liver and
kidney damage.
• Pubchem Bioassays DB offers a large number of
assays involving a significant number of protein
targets

6. Hypothesis
H1: Drugs with increased frequency of SOCspecific ADRs can be identified from patterns of
reactivity in PubChem BioAssay screens.
Ho: Reactivity patterns in PubChem BioAssay do
not distinguish drugs with increased frequency
of tissue-specific ADRs .

7. Data Features
• For a given SOC, matrix of
▫ PRR
▫ drug CUI
▫ BioAssay ID (“AID”)
• Sparse matrix: most compounds have been
screened in a few assays only
▫  limited overlap between CVAR and BioAssay
• Very large data sets (more later)

8. Data
Integration

9. Analytical
Process
Binarized PRR (PRR>=2  1
else 0)

10. Selected Statistic:
Proportional Risk Ratio (PRR)
Event of
interest
Other events
Drug of
interest
A
Other drugs
C
D
B
• PRR = OBS/EXP = [A / (A+C)] / [B / (B+D)]
• Serious ADR Threshold PRR≥2, w/at least
3 cases reported

11. Results: Max PRR by SOC for Statins
Active Ingredient
Atorvastatin
Cerivastatin
Fluvastatin
Lovastatin
Pravastatin
Rosuvastatin
Simvastatin
PRR (no. cases)
6.2 (958)
10.47 (284)
5.12 (7)
4.62 (11)
5.13 (104)
7.34 (803)
5.79 (186)
SOC
Musculo-skeletal
Musculo-skeletal
Musculo-skeletal
Musculo-skeletal
Musculo-skeletal
Musculo-skeletal
Musculo-skeletal

12. Addressing Zero ADR
• Many drugs do not have a SOC-specific PRR
▫ Unclear if this means they are unusually safe
(could be due to e.g. low prescription volume)
▫ Approach: Assign SOC-specific PRR = 0 if at least
10 ADR reports exist overall

13. Results Since Last Meeting

14. Properties of CVAR drug ingredients
Ingredients with drug reports in CVAR
Ingredients with drug reports in CVAR WITH `health_product_role` = 'suspect' and `reaction_type` = 'Adverse
Reaction'
Number
2,901
2,746
Ingredients with drug reports in CVAR with `health_product_role` = 'suspect' and `reaction_type` = 'Adverse
Reaction' AND whoart_soc_cui is not null
2,731
Ingredients with drug reports in CVAR with `health_product_role` = 'suspect' and `reaction_type` = 'Adverse
Reaction' and whoart_soc_cui is not null AND total_number_reports >= 10
1,550
Ingredients with drug reports in CVAR with `health_product_role` = 'suspect' and `reaction_type` = 'Adverse
Reaction' and whoart_soc_cui is not null and total_number_reports >= 10 AND present in PUBCHEM_BIOASSAY
485

15. BioAssay Subset Properties
Assays and Drugs in PubChem BioAssay with SOC-identified CVAR drug
ingredients and ADR reports >=10
AssayType
confirmatory
in vivo_screening
other
screening
NumberOfAssays
545
81
93
466
6
summary
Total:
NumberCVARCmpds
664
341
790
629
202
1,191
2,626

16. Mapping Results
All SIDs
CVAR drug ingredients mapped to SIDs
Number
913,742
7,913
CVAR drug ingredients with SOC-identified ADRs mapped to SIDs
4,382
CVAR drug ingredients with SOC-identified ADRs and >= 10 reports mapped to SIDs
3,136

17. SOC ID
Predictive
Modeling
Results - 1
SOCName
resistance
C0236104 mechanism
disorders
Avg Model AUC
InitCmpds CmpdsRetained
0.92 (0.000593)
468
70
C0221016
red blood cell
disorders
0.79 (0.000318)
468
185
C0236099
reproductive
disorders - male
0.77 (0.000167)
468
271
0.76 (0.000802)
468
115
0.76 (0.000465)
468
177
0.74 (0.000272)
468
376
0.72 (0.000721)
468
126
C0042790 vision disorders
0.72 (0.000174)
468
286
skin and
C0037272 appendages
disorders
0.7 (0.000196)
468
250
C0027651 neoplasms
C0035204
respiratory system
disorders
centr & periph
C0027765 nervous system
disorders
C0014130
endocrine
disorders

18. AssayType
Avg Model AUC
AID1
resistance
mechanism
disorders
0.92 (0.000593)
AID119
confirmatory
Small molecule inhibitors of tumor cell
growth in implanted CCRF-CEM
leukemia cells in mice
2.95E-004
(1.05E-005)
1.15E+000
(5.08E-003)
red blood cell
disorders
0.79 (0.000318)
AID330
in vivo
screening
Small molecule inhibitors of tumor cell
growth in implanted P388 leukemia
CD2F1 (CDF1) tumors in mice
1.15E-004
(1.30E-006)
2.34E-001 (6.33E004)
reproductive
0.77 (0.000167)
disorders - male
AID1461
confirmatory
7.49E-008
(1.45E-009)
5.55E-001 (3.87E004)
neoplasms
0.76 (0.000802)
AID543
confirmatory
Small molecules cytoxic to H-4-II-E rat
hepatoma cell line
2.16E-005
(8.19E-007)
9.39E-001 (2.21E003)
respiratory
0.76 (0.000465)
system disorders
AID774
other
Small molecule inhibitors of Inhibition of
dnzymes frequently used to reach a
NAD/NADH Endpoint
2.14E-003
(3.85E-005)
-9.21E+000
(2.31E-002)
Target
Small molecule inhibitors of
G protein-coupled receptor for asthma
neuropeptide S receptor (NPSR)
susceptibility isoform A (NPRS A) [Homo sapiens]
signaling
Avg Coeff AID1
Small molecule inhibitors of inwardrectifying potassium ion channel Kir2.1
screening
in HEK293 cells (human embryonic
kidney)
potassium inwardly-rectifying channel J2 [Mus
musculus]
1.70E-007
(5.27E-009)
3.08E-001 (1.70E004)
Small molecule inhibitors of cytochrome
P450 3A4 (cell-free)
cytochrome P450_ subfamily IIIA-polypeptide 4
[Homo sapiens]
1.22E-003
(6.53E-005)
8.75E-001 (2.65E003)
Small molecule inhibitors of transient
short transient receptor potential channel 6 [Mus
receptor potential cation channel C6
musculus]
(TRPC6) in HEK293 cells
5.04E-004
(1.09E-005)
1.97E-001 (2.23E004)
tyrosine 3-monooxygenase/tryptophan 5monooxygenase activation protein-zeta
polypeptide [Bos taurus]
6.23E-005
(1.37E-006)
4.95E-001 (4.66E004)
centr & periph
nervous system 0.74 (0.000272)
disorders
AID1672
endocrine
disorders
0.72 (0.000721)
AID885
confirmatory
vision disorders 0.72 (0.000174)
AID2553
screening
skin and
appendages
disorders
AID781
screening
0.7 (0.000196)
Objective
Avg p-value
AID1
SOCName
Small molecule inhibitors of 14-3-3/Bad
interactions (cell-free)

19. ROC AUC For C0236104 “resistance mechanism disorders”

20. LOOCV Validation For C0236104 “resistance mechanism disorders”

21. Universe of Data For SOC 0236104

22. Disorders Associated With SOC C0236104
(“Resistance Mechanism Disorders”)
Allergic conditions
Autoimmune disorders
Immune disorders NEC
Immunodeficiency syndromes
Ancillary infectious topics
Bacterial infectious disorders
Chlamydial infectious disorders
Ectoparasitic disorders
Fungal infectious disorders
Helminthic disorders
Infections - pathogen unspecified
Mycobacterial infectious disorders
Mycoplasmal infectious disorders
Protozoal infectious disorders
Rickettsial infectious disorders
Viral infectious disorders

23. Indications For Drugs Correlated with Model For SOC
C0236104 (“Resistance Mechanism Disorders”)
Antineoplastic Agents
Anti-Bacterial Agents
Anti-inflammatory Agents
Anticholesteremic Agents
Anti-Inflammatory
Agents, Non-Steroidal
Anti-Allergic Agents
Analgesics
Anti-Dyskinesia Agents

24. Lessons Learned
• Limitation of relational databases sans partitioning
▫ Queries won’t return if >50M rows
• Sneaky MySQL loader
▫ Can fail to load records w/o reporting error
▫ Problem when on can’t easily verify expected number of records
from XML files
▫ Solution: Write your own loader (can include data validation)
• BMIR cluster has serious NFS problems
▫ Couldn’t run more than a few parsing jobs at same time
• … and my favorite: The dreaded NCBI surprise!

25. The Case Of The Missing Atorvastatin
… and no
• Problem: Why were some statins missing from my
synonyms!
dataset?
▫ E.g.: Atorvastatin
• Answer: It is present, but there is no way to identify it as such
• Example from AID 881  Atorvastatin SID = 29215408

26. And Now For Some Test Marketing

27. Acknowledgements
• Alex and Chirag, for contributing secret R
knowledge
• Atul, for being helpfully skeptical and patient
• Alex S for quickly addressing DB issues
• NCBI, for providing DBs and messing up my life

28. Need To Standardize And Normalize Assay Activity Metrics
Types of activity metrics (substr 1-12)
% Cell Viabi
% cellular A
% CPE Inhibi
% Inhibition
%Activity at
%displacemen
%Efficacy at
%Inhibition
%Response of
Activity at
AF_20uM
AreaNm
AreaoftheNuc
Ave %Efficac
Ave %Inhibit
AverageInteg
AverageInten
AverageSpots
Baseline-Act
Cell-Activit
CellCount
CellsNucInte
Donor-Activi
Fed-Activity
FP-Activity
F_Ratio
GFP-Activity
Mean High
Mean Low
Mean_NC
Mean_PC
MPIPiCm
MPIPiNm
MS % Inhibit
NucleiNucAre
NumberofCell
Parental-Act
PercentagePo
PiNmbyPiCm
Primary % In
Rate-Activit
Ratio-Activi
RatioofSpoti
RFP-Activity
STD Deviatio
Std.Err(Repe
StdDev_NC
StdDev_PC
TIINiNM
TotalCytopla
TotalIntegra
TotalSpotInt
Total_fluore
TSHR-Activit
W460-Activit
W530-Activit
ZScore
ZScore at 10
ZScore at 20