Automated image analysis has had a long history but continues to grow with massive improvements in algorithms, speed, performance, and with emerging opportunities for high throughput tissue biomarker analysis and automated decision support for primary diagnostics. Of particular importance is the development of computer vision and image analysis for H&E stained samples. This talk will outline recent advances in automated tissue analysis for biomarker discovery and diagnostics and how adoption of digital pathology will drive the demand for quantitative imaging and decision support. As an example, PathXL have developed TissueMark for the automated identification and analysis of tumour in lung, colon, breast and prostate cancer digital H&E slides. The conventional pathological estimation of % tumour nuclei in H&E samples shows gross variation between pathologists, undermining the quality of next generation sequencing, molecular testing and patient therapy and potential of false negative diagnoses. TissueMark uses a combination of pattern recognition, glandular analysis and nuclear segmentation to identify premaligant and invasive cancer patterns in H&E stained tissues and use this to assess tumour cell numbers and annotate samples for nucleic acid extraction and molecular profiling. Benchmark data was generated to validate TissueMark technology and showed concordance of automated data with manual counts, accelerating tumour markup and improving sample quality assessment. This represents an example of how automated imaging of tissue samples can be of immense value in quantitative tumour analysis for molecular diagnostics, thereby improving reliability in discovery and diagnostics. This together with other examples in pathology research and practice will demonstrate that next generation tissue imaging technology in digital pathology could radically change how pathology is practiced.

1. Peter W Hamilton
Professor of Pathology Bioimaging and Informatics
Centre for Cancer Research & Cell Biology
Queen’s University of Belfast
Vice President, Research and Development PathXL
Next generation imaging and
Computer vision in Pathology:
Pipedream to reality

2. Digital Pathology Growth
Digital Pathology Market worth $437 Million by 2018

3. Digital Pathology is not new!
Histopathology 1987;9:901-911
Classification of normal colorectal mucosa and
adenocarcinoma by morphometry.
HAMILTON PW*, ALLEN DC*, WATT PCH PATTERSON CC,
BIGGART JD.

4. The Regrowth of Digital Pathology
1970 1980 1990 2000 2010
Academicactivity
Whole Slide Imaging
Pathology &
Personalised medicine

5. Whole Slide Imaging

6. Precision (Personalised) Medicine

7. Target Discovery Lead Optimization Preclinical/animal
Studies
Clinical Development
I II III
Approval Clinic
Drug Development
Biomarker Discovery Biomarker Validation Assay Development Clinical Utility Testing Approval Clinic
Biomarker Development
The Challenge of Precision Medicine
Therapeutic/diagnostic
co-development
7

8. Target Discovery Lead Optimization Preclinical/animal
Studies
Clinical Development
I II III
Approval Clinic
Drug Development
Biomarker Discovery Biomarker Validation Assay Development Clinical Utility Testing Approval Clinic
Biomarker Development
Biobanking
Biobanks supply high
quality tissue samples
and images for target
and biomarker
identification
Tissue Microarrays
(TMAs) and remote
biomarker analysis
Digital TMA management,
review and biomarker scoring
for discovery and validation
Image Analysis
Companion Algorithms
Multicentre Clinical Trials
Remote review of tissue
biomarkers for trial and
therapeutic arm selection across
institutions, networks and
countries
Toxicological Pathology
Remote review of slides to
ensure integrity of
pathological interpretation
and interobserver variation
Digital Pathology
in the Drug/Biomarker Development Pipeline
Tumor
Identification
Automated tumor
annotation and % tumor
measurements for
Molecular Diagnostics
Quantitative assays to support
patient stratification and
therapeutic selection
Quantitative automated
assessment of tissue
biomarkers (IHC, ISH)
8

9. NI Molecular Pathology Lab
Centre for Cancer Research
Queen’s University Belfast

10. NEXT GENERATION SEQUENCINGHT GE ARRAYS / METHYL / CNV
IMAGING SCANNING LT TESTING (SEQ, Q-PCR)AUT NA EXTR
AUTOMATED IHCAUTOM. FISHAUTOMATED H+E
MICROSCOPYSAMPLE PREPARATION

11. Integrated Digital Pathology
Biomarker Research Primary Molecular Diagnostics Accredited Laboratory

12. Cloud Storage and Serving
Integrated Digital Pathology
Central Archive and Image Server
Whole Slide Scanners
Archiving
Biobaking
Training
Tumor Board Meetings
Internal Quality Control
Remote Slide Review
Biomarker Discovery and Validation
Mutisite Collaboration
Multicentre Clinical Trials
The pathologist no longer needs to be in the same room as the glass slide

15. Errors in Pathology
Subjectivity of visual scoring

16. Kappa
Pre-invasive lesions of the bronchus 0.55
(Nicholson – Histopathology 2001;38:202-208)
Cervical cytology 0.46
(Stoler – JAMA 2001;285:1500-1505)
Cervical Histology 0.15 – 0.62
(McCluggage – Br J Obs Gynae 1998;105:206-210)
Prostate Cancer 0.58
(Egevad – Urology 2001;57:291-295)
Oral Dysplasia 0.27 – 0.45
(Warnakulasuriya – J Pathol 2001;194:294-297)
Variation in interpretation of renal transplant biopsiesFurness et al.
Aberrant diagnoses by surgical pathologists Wakely et al
Dysplasia classification: pathology in disgrace Bosman.
“Individuality” in the specialty of surgical pathology Ackerman
Errors in pathological diagnosis

17. Automated
Computer Vision and Analysis of Tissues
Nuclear Staining Cytoplasmic Staining Membrane Staining

19. Biomarker Marker Discovery Studies
458 samples across 4 TMAs
BAX IHC
Scored by x2 experienced pathologists
BAX & BAK as predictors of patient outcome
Automated imaging of BAX IHC

20. MANUAL SCORE
QPATH AUTOMATED
Num.scored > 100
Num.scored > 100

21. Computerised imaging allows you to do difficult
things…

22. Augmented Visualisation in Pathology
(AVP)
Allows you to measure the seeable
Allows you to detect the unseeable

24. Computerised imaging allows you to do difficult things…
Tumour
Stroma
Q Nuclear H-score
Q Cyto H-score
Q Nuclear H-score
Q Cyto H-score
FLIP Pro-caspase 8
Adenocarcinoma Squamous carcinoma
Phenotypic signature
FLIP CASP8
High High
Low Low
High Low
Low High
HET
FLIP
Adenocarcinoma: Q H-score>170 = High
Squamous cell: Q H-score>245 = High
CASP8
Adenocarcinoma: Q H-score>160 = High
Squamous cell: Q H-score>195 = High

25. p= <0.0001
HR 14.37
95% CI 3.41-60.49
p= 0.05
HR 2.57
95% CI 0.67-6.77
Adenocarcinoma specific H-score
p= 0.03
HR 3.15
95% CI 1.12-8.84
Squamous cell carcinoma specific H-score
Cytoplasmic expression (not nuclear) was prognostic in NSCLC – Ad and Sq

26. Image analysis of Tissue Heterogeneity
Potts et al. Lab Invest 2012;92:1342-57

27. Immuno-oncology and immuno-therapy

29. ER
PR
HER2
Mib1 (KI67)
p53
CK5/6
CK14
CK-17
Baseline IHC BiomarkersOropharynx TMA 1
Mesothelioma TMA 1
Ovarian TMA 1
Ovarian TMA 2
Ovarian TMA 2A (Stroma)
Ovarian TMA 3B
Gastric Cancer TMA Sing
Oesophageal TMA ICR
NSCLC TMA1
COIN TRIAL (TMA 1-40)
Breast TMA 1-4
CK20
E-cadherin
Retrospective tissue series & TMAS
S100
HBME1
p16
CA125
CA19.9
High Throughput Image Analysis of Baseline Biomarkers
Breast Cancer
Colorectal Cancer
Ovarian Cancer
Prostate Cancer
Head & Neck Cancer
Lung Cancer
Prospective Biobank Collections
Bladder TMA 1-3
Moving from small local cohorts to large mutinational patient populations

30. High Performance Image Analysis
HP Blade System Cluster 900 processor cores
MS Message Passage Interface (MPI)
Centralised Dynamic Load Balancing

31. HPC provides significant analytical speedup for
automated TMA analysis
• Evaluation and fine tuning of biomarker algorithms on large datasets
• Multiplex Biomarker experiments across large tissue cohorts, multiple TMAs and multiple markers
• FAST-PATH FP7 Marie Curie Programme
Wang Y & Hamilton , et al. Ultrafast processing of gigapixel TMA images using HPC. PLoS ONE 2010; 6(2): e15818
X50 – X100 fold speed up in processing time
300 tissue core arrays - IHC

32. Accelerator Award
A national digital pathology and image analysis programme for solid tumour analysis
Clinical Fellowship programme in Molecular Pathology
Belfast
Southampton
ICR/Royal Marsden
Manchester
Newcastle
Leicester

33. Automated Imaging in tissue research is going to drive discovery
of next generation of tissue biomarkers
for precision medicine

34. But won’t tissue pathology be redundant
in next few years?

35. Transforming how we practice pathology

36. Gene Panels and Clinical Sequencing

37. Molecular testing, FFPE and H&E Review
EGFR
KRAS
BRAF
NRAS
CMET
MMR
Oncotype Dx
Mammaprint
Foundation One
Clinical Sequencing
Sample
FFPE
Tumour Markup
Tumour
Sufficiency
Macrodissection
DNA Extraction
DNA
Quantification
Platform
Molecular Assay Output
Sanger
QPCR
NGS
Pre-Analytical
Analytical
OperatorVariability

39. To automatically identify tumour and calculate tumour
percentage in digital H&E tissue sections using image analysis
Pathologist mark-up TissueMark mark-up

40. I. Tumour Identification
TissueMark
Molecular Diagnos cs
Image Viewing Image Management Image Conversion Image Serving Workflow crea on
Digital Image Handling
Tile Management Pa ern recogni on
Object management
& analysis
Image Processing Visualisa on
Image Processing and Analysis
Biomarker AnalysisImmunocell analysisCancer detec on Tumor boundary analysis
Tissue Recogni on and Cancer detec on
Gland recogni on Epithelial analysis Nuclear analysis
Tissue Architecture and Cellular Quan ta on
Histo iden fica on Tumor Cell Counts
Histological ScreeningBiomarker Clinical Trials Immuno-oncology
PathXL’s Tissue Recogition Engine

41. II. Computation of a macrodissection boundary

42. Original
Original
Original
Lung
Breast
Colon
Across different tissue types

43. % Tumour cells ?
III. % Tumour cells

44. KRAS: COBAS 5%, Sanger 15%
EGFR: COBAS 5%, Sanger 30%
BRAF: COBAS 5%, Sanger 30%
Next Generation Sequencing: 5% - 70%
Foundation One: 20%
TCGA: 80%
Limits of sensitivity & Percentage Tumour DNA

45. • 20 High resolution images NSCLC
▪ Circulated to 4 pathologists
▪ % tumour estimates
Variation in lung % tumour cell estimates
amongst pathologists

46. Lung Cancer % Tumour Estimates

47. Patented algorithms for the counting of cells and calculating
% tumor in H&E tissue samples

48. r = 0.972
P<0.0001
TissueMark Validation
Lung Tumours

49. Workflow for easy integration

51. Automated Imaging and Decision
Support for Primary Diagnostics

53. Significantly improves objectivity and reliability of diagnosis
FDA have given 510k approval for use of algorithms for Her2 measurement routine
ASCO/CAP Recommendations (Wolff et al 2007)
Health insurers in USA reimburse for Her2 image analysis tests
0 2+ 3+
Subjective: 20% misclassification
1+
Her2 IHC - biomarker in breast cancer

54. Routine Adoption of Quantitative Imaging
is
Reliant on Adoption of Digital Pathology for
Primary Review and Diagnosis

55. https://digitalpathologyassociation.org/healthcare-faqs
FDA and digital pathology

56. These applications make your life easier
These applications make the quality of
your work better
https://digitalpathologyassociation.org/healthcare-faqs

57. Is Digital Pathology Safe?
Is Digital Pathology Cost Effective?

58. Is digital pathology for primary review safe?

59. Is digital pathology for primary cost-effective?
5-years:
Total cost savings based on
anticipated improvements in
pathology productivity and
histology lab consolidation
were estimated at
$12.4 million
for an institution with 219,000
annual accessions.
Potentially reduce costs of
incorrect treatment by $5.4
million

60. The Digital Pathology Cockpit

61. 95%
5%
IHC
H&E
Reducing Error
Rates in Pathology
Computerised Imaging and H&E analysis?

62. Image Analysis for H&E evaluation

63. Image Analysis for H&E evaluation

65. Mapping Tissue Phenotype and Morphological Heterogeneity

66. Integrating phenotype and genotype to capture tumour heterogenity

67. Next generation imaging:
From pipedream to reality

68. Professor Manuel Salto-Tellez
PhD students
Mr Ryan Hutchinson
Mr Nick McCarthy
Post-doctoral Researchers
Dr Peter Bankhead, PhD
Dr Darragh McArt, PhD
Dr Yinhai Wang, PhD
Dr Ching-Wei Wang, PhD
Dr Stephen Keenan, PhD
Dr Andrena McCavinagh, PhD
Pathologists
Dr Jackie James, MD
Dr Maurice Loughrey, MD
Dr Damian McManus, MD
Professor R Montironi, MD
Professor R Williams, MD
PathXL
Dr Jim Diamond (PathXL)
Mr David McCleary (PathXL)
Mr Jonathon Tunstall (PathXL)
Dr Giussepe Lippolis (Fast-Path)
Dr Nick McCarthy (Fast-Path)
Acknowledgements