Extreme Spatio-Temporal Data Analysis

Extreme Spatio Temporal Data Analysis in
Biomedical Informatics

Joel Saltz MD, PhD
Director Center for Comprehensive
Informatics

Contributions

• Computer Science: Methods and middleware
Center for Comprehensive Informatics

for analysis, classification of very large
datasets from low dimensional spatio-
temporal sensors; methods to carry out
comparisons and change detection between
sensor datasets
• Biomedical: Mine whole slide image datasets
to better predict outcome and response to
treatments, generate basic insights into
pathophysiology and identify new treatment
targets

Outline of Talk

• Pathology: Analysis of Digitized Tissue for Research

and Practice
• Feature Clustering: Morphologic Tumor Subtypes in
GBM Brain Tumors and Relationship to “omic”
classifications
• Whole Slide Image Analysis in Clinical Practice:
Neuroblastoma
• Tissue Flow: Multiplex Quantum Dot
• HPC/BIGDATA Feature Pipeline
• Pathology data analytic tools and techniques


Whole Slide Imaging: Scale

Pathology Computer Assisted Diagnosis

Shimada, Gurcan, Kong, Saltz

Computerized Classification System
for Grading Neuroblastoma
Initialization Yes
Image Tile Background? Label
I=L
• Background Identification
No

Create Image I(L)
• Image Decomposition (Multi-
Training Tiles resolution levels)
Segmentation I = I -1 • Image Segmentation
Down-sampling
(EMLDA)
Segmentation
Feature Construction
• Feature Construction (2nd
Yes

No order statistics, Tonal
Feature Extraction I > 1?
Feature Construction
Features)
Feature Extraction
Classification
• Feature Extraction (LDA) +
Classification (Bayesian)
Classifier Training
No
• Multi-resolution Layer
Within Confidence
Region ? Controller (Confidence
Yes
TRAINING Region)
TESTING

Direct Study of Relationship Between
vs

In Silico Brain Tumor Center

Anaplastic Astrocytoma
(WHO grade III)

Glioblastoma
(WHO grade IV)

Morphological Tissue Classification
Whole Slide Imaging Cellular Features

Nuclei Segmentation

Lee Cooper,
Jun Kong

Nuclear Features Used to Classify GBMs

50
3 2 1
20
1
45
40
Silhouette Area

40 60

Cluster
80
2
35
100

120
30
140
3

25 160
2 3 4 5 6 7 20 40 60 80 100 120 140 160
# Clusters 0 0.5 1
Silhouette Value

Consensus clustering of morphological
signatures

Study includes 200 million nuclei taken from 480
slides corresponding to 167 distinct patients

Each possibility evaluated using 2000 iterations of K-
means to quantify co-clustering

Clustering identifies three morphological groups

• Analyzed 200 million nuclei from 162 TCGA GBMs (462 slides)
• Named for functions of associated genes:
Cell Cycle (CC), Chromatin Modification (CM),
Protein Biosynthesis (PB)
• Prognostically-significant (logrank p=4.5e-4)

CC CM PB
1
CC
10 0.8 CM
PB
20
Feature Indices

0.6

Survival
30 0.4

40 0.2

50
0
0 500 1000 1500 2000 2500 3000
Days

Gene Expression Class Associations

• Cox proportional hazards
– Gene expression class not significant p=0.58
– Morphology clustering p=5.0e-3

100
Classical
Mesenchymal
80
Subtype Percentage (%)

Neural
Proneural
60

40

20

0
CC CM PB
Cluster

Clustering Validation

• Separate set of 84 GBMs from Henry Ford Hospital
• ClusterRepro: CC p=7.2e-3, CM p=1.3e-2

CC Mixed CM
1

10 CC
0.8 Mixed
Feature Indices

20 CM
0.6
30 Survival

0.4
40

0.2
50

0
0 20 40 60 80 100
Months


Associations

Novel Pathology Modalities
Genomics Imaging
Excellent Molecular Resolution Excellent Spatial Resolution
Limited Spatial Resolution Limited Molecular Resolution

1000’s of genes

Quantum Dots

Professor Robin Bostick

Imaging Pipeline – Feature Extraction

Example Application: Cancer Stem Cell
Niche
• Cancer stem cells
– Rare(?), proliferative cells, regenerative
– Do they prefer to live near blood vessels, or necrosis?

Extreme Spatio-Temporal Sensor Data Analytics

• Leverage exascale data and
computer resources to
squeeze the most out of
image, sensor or simulation
data
• Run lots of different
algorithms to derive same
features
• Run lots of algorithms to
derive complementary
features
• Data models and data
management infrastructure
to manage data products,
feature sets and results from
classification and machine
learning algorithms

Application Targets

• Multi-dimensional spatial-temporal datasets
– Microscopy image analyses
– Biomass monitoring using satellite imagery
– Weather prediction using satellite and ground sensor
data
– Large scale simulations
• Can we analyze 100,000+ microscopy images per
hour?
• Correlative and cooperative analysis of data from
multiple sensor modalities and sources
• What-if scenarios and multiple design choices or
initial conditions

Biomass Monitoring (joint with ORNL)
• Investigate changes in vegetation and land use

• Hierarchical, multi-resolution coarse/fine-grained analytics into a
unified framework
• Changes identified using high temporal/low spatial resolution MODIS
data
• Segmentation and classification methods used to characterize
changes using higher resolution data (e.g. multitemporal AWiFS
data)
• Segmentation and classification to identify man-made structures.

Core Transformations
• Data Cleaning and Low Level Transformations
• Data Subsetting, Filtering, Subsampling
• Spatio-temporal Mapping and Registration
• Object Segmentation
• Feature Extraction, Object Classification
• Spatio-temporal Aggregation
• Change Detection, Comparison, and Quantification

Extreme DataCutter
DataCutter
Pipeline of filters connected though logical streams
In transit processing
Flow control between filters and streams
Developed 1990s-2000s; led to IBM System S
Extreme DataCutter
Two level hierarchical pipeline framework
In transit processing
Coarse grained components coordinated by Manager that
coordinates work on pipeline stages between nodes
Fine grained pipeline operations managed at the node level
Both levels employ filter/stream paradigm

Extreme DataCutter – Two Level Model

Node Level Work Scheduling

• Features of Node Level Architectures
– Nodes contain CPUs, GPUs
– Each CPU contains multiple cores
– GPU has complex internal architecture
– Data locality within node
– Data paths between CPUs and GPUs

Keeneland Node


• Attempt to minimize data movement
• Identify and assign operations that perform
well on GPU
• Balance load between CPUs and GPUs
• Prefetch data
• Identify and use high bandwidth CPU/GPU
data paths
• Schedule exclusive GPU access for
components (e.g. morphological
reconstruction) requiring fine grained
parallelism

Brain Tumor Pipeline Scaling on Keeneland
(100 Nodes)

Control Structures for Handling Fine
Grained/Runtime Dependent Parallelism in GPUs

Morphological Reconstruction:
8-15 Fold speedup vis one CPU core (Intel i7 2.66 GHz) on NVIDIA C2070
and GTX580 GPUs

Large Scale Data Management

 Implemented with IBM DB2 for large scale
pathology image metadata (~million markups per
slide)
 Represented by a complex data model capturing
multi-faceted information including markups,
annotations, algorithm provenance, specimen, etc.
 Support for complex relationships and spatial
query: multi-level granularities, relationships
between markups and annotations, spatial and
nested relationships
 Highly optimized spatial query and analyses

Spatial Centric – Pathology Imaging “GIS”
Point query: human marked point Window query: return markups
inside a nucleus contained in a rectangle

.

Containment query: nuclear feature Spatial join query: algorithm
aggregation in tumor regions validation/comparison

PAIS
PAIS (Pathology Analytical Imaging Standards)
Supported by caBIG, R01 and ACTSI
class Domain Mo...
 PAIS Logical Model
WholeSlideImageReference TMAImageReference
Patient

Region
 62 UML classes
MicroscopyImageReference
Subject

DICOMImageReference
0..1

1
0..1  markups, annotations,
1 Specimen

0..*
ImageReference 1 0..*
0..1
imageReferences,
1 0..1 AnatomicEntity
User
0..1
0..* 1
1
provenance
1 1 Equipment
Group

 PAIS Data Representation
0..1
1
0..1 1 PAIS
1 0..*
Project AnnotationReference
1

 XML (compressed) or HDF5
0..1

0..* 1 1 1

Collection 0..* 0..*

0..*
1

Markup
0..*
0..*

0..1
Annotation 0..*
 PAIS Databases
0..*
GeometricShape 1

 loading, managing and
1 1
0..* 0..* 0..*
1

Observation Calculation Inference
Surface Field
1 1
querying and sharing data
0..1 0..1 0..1

Provenance
0..*
 Native XML DBMS or
RDBMS + SDBMS

PAIS: Example Queries

Example Query for Integrative Studies
• Find mean nuclear feature vector and covariance on

tumor regions for each patient grouped by tumor
subtype
SELECT c.pais_uid, pc.subtype,
AVG(area), AVG(perimeter), AVG(eccentricity),
COVARIANCE(area, perimeter), COVARIANCE(area, eccentricity)
FROM pais.calculation_flat c,TCGA.PATIENT_CHARACTERISTIC pc,
pais.patient p
WHERE p.patientid = pc.patient_id AND
p.pais_uid = c.pais_uid
GROUP BY c.pais_uid, pc.subtype;
2 1 3 4
1
1 Cluster 1
20 10 0.9 Cluster 2
20 Cluster 3
40 0.8
Cluster 4
30
0.7
60 40
2
Feature Indices

0.6
Cluster

50

Survival
80
0.5
60
100
0.4
70
3
120 80 0.3

140 90 0.2

100 0.1
160 4
110
50 100 150 0
0 500 1000 1500 2000 2500 3000
0 0.2 0.4 0.6 0.8 1 Days
Silhouette Value

Algorithm Validation: Intersection
between Two Result Sets (Spatial Join)
PAIS: Example Queries

. .

VLDB 2012

Change Detection, Comparison, and Quantification

Summary and Perspective

• Large scale integrative data analytic methods and

tools to integrate clinical, molecular, Pathology,
Radiology data
• Characterize new cancer subtypes and biomarkers,
predict outcome, treatment response
• Algorithms to quantify Pathology classification
• HPC/BIGDATA analysis pipelines

Importance:

• Computer Science: general approaches to analysis

and classification of very large datasets from low
dimensional spatio-temporal sensors
• Biomedical: generate basic insights into
pathophysiology, clues to new treatments, better
ways of evaluating existing treatments and core
infrastructure needed for comparative effectiveness
research studies

Thanks to:
• In silico center team: Dan Brat (Science PI), Tahsin Kurc, Ashish Sharma, Tony Pan, David
Gutman, Jun Kong, Sharath Cholleti, Carlos Moreno, Chad Holder, Erwin Van Meir, Daniel
Rubin, Tom Mikkelsen, Adam Flanders, Joel Saltz (Director)
• caGrid Knowledge Center: Joel Saltz, Mike Caliguiri, Steve Langella co-Directors; Tahsin
Kurc, Himanshu Rathod Emory leads
• caBIG In vivo imaging team: Eliot Siegel, Paul Mulhern, Adam Flanders, David Channon,
Daniel Rubin, Fred Prior, Larry Tarbox and many others
• In vivo imaging Emory team: Tony Pan, Ashish Sharma, Joel Saltz
• Emory ATC Supplement team: Tim Fox, Ashish Sharma, Tony Pan, Edi Schreibmann, Paul
Pantalone
• Digital Pathology R01: Foran and Saltz; Jun Kong, Sharath Cholleti, Fusheng Wang, Tony
Pan, Tahsin Kurc, Ashish Sharma, David Gutman (Emory), Wenjin Chen, Vicky Chu, Jun
Hu, Lin Yang, David J. Foran (Rutgers)
• NIH/in silico TCGA Imaging Group: Scott Hwang, Bob Clifford, Erich Huang, Dima
Hammoud, Manal Jilwan, Prashant Raghavan, Max Wintermark, David Gutman, Carlos
Moreno, Lee Cooper, John Freymann, Justin Kirby, Arun Krishnan, Seena Dehkharghani,
Carl Jaffe
• ACTSI Biomedical Informatics Program: Marc Overcash, Tim Morris, Tahsin Kurc,
Alexander Quarshie, Circe Tsui, Adam Davis, Sharon Mason, Andrew Post, Alfredo Tirado-
Ramos
• NSF Scientific Workflow Collaboration: Vijay Kumar, Yolanda Gil, Mary Hall, Ewa Deelman,
Tahsin Kurc, P. Sadayappan, Gaurang Mehta, Karan Vahi

Extreme Spatio-Temporal Data Analysis

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (18)

Similaire à Extreme Spatio-Temporal Data Analysis

Similaire à Extreme Spatio-Temporal Data Analysis (20)

Plus de Joel Saltz

Plus de Joel Saltz (20)

Dernier

Dernier (20)

Extreme Spatio-Temporal Data Analysis

Notes de l'éditeur