Federated Learning (FL) is a learning paradigm that enables collaborative learning without centralizing datasets. In this webinar, NVIDIA present the concept of FL and discuss how it can help overcome some of the barriers seen in the development of AI-based solutions for pharma, genomics and healthcare. Following the presentation, the panel debate on other elements that could drive the adoption of digital approaches more widely and help answer currently intractable science and business questions.

1. Pistoia Alliance Webinar
How Can Federated AI/ML Learning Support Genomics
and Patient Data Analysis to Enable Precision Medicine
at Scale?
4th May 2020
15.00 to 16.00 BST

2. ©PistoiaAlliance
Agenda
2
Time Title Presenter
15:00 Introduction, housekeeping Zahid Tharia, Pistoia Alliance
15:05 Pistoia Alliance Digital Strategy Paul Denny-Gouldson, Pistoia
Alliance
15:10 How can federated learning support genomics and enable
precision medicine at scale?
Craig Rhodes, EMEA Industry Lead
for Healthcare and Life Science,
NVIDIA
Nicola Rieke, Senior Deep Learning
Solution Architect – Healthcare,
NVIDIA
15:35 Panel: How do we collect and leverage patient data to
develop tools that support digital health, what data is
important and what barriers exist?
To include:
Jennifer Goldsack – CEO Digital
Medicine Society (DiMe)
Tim McCarthy, Head of Digital
Medicine & Translational Imaging,
Pfizer
Marissa Dockendorf, Director, Global
Digital Analytics & Technologies,
Merck
15:55 Wrap up Paul Denny-Gouldson, Pistoia
Alliance

3. Introduction
Paul Denny-Gouldson, Consultant
Pistoia Alliance

4. Poll:
With COVID-19 will the potential accelerated move to
Digital be sustainable?
Yes/No
4

5. ©PistoiaAlliance
The Digital Seed project: Quality generation and ethical
use of digital health data in clinical studies
❑ Collaboration with DiMe
❑ Project Funded by Pistoia - $40k
❑ Project Manager being recruited, Steering committee being formed
❑ Project Goals
Make recommendations on developing EULAs and TOUs for digital technologies used in health and biomedical research. Accelerate
digitally-powered investigator-initiated and sponsored research for the betterment of public health.
❑ Proposed Deliverables
• Identify and synthesize existing best practices from bioethics, health, technology, data science, and cyber-security disciplines with regards to
protecting confidentiality, privacy, and control over data
• Articulate a standard lexicon of digital study types & define quality metrics for categorizing evidence used to designate digital tools as fit-for-
purpose in a clinical application - limit to clinical trials to constrain scope
• Develop consensus recommendations and resources for the development and use of digital tools for data capture
• Disseminate recommendations and resources to the manufacturers of digital tools, patient communities conducting citizen science, regulators
and policymakers, IRBs, research sponsors, and investigators
❑ We need more steering committee members – please volunteer if you are interested

6. ©PistoiaAlliance
Other IP3 Digital Projects
6
2107 Building a strong evidentiary base for the adoption of digital medicine
tools to support clinical applications
2105 Assembling domain-centric digital data sets for use as a testing
environment for new digital health measures
2104 Driving multi-stakeholder acceptance of patient generated health data
(PGHD) for use in clinical trials
2102 Advancing the Ethical Oversight of Biomedical Research to Keep Pace
with Rapid Advancements in Digital Technology
2101 Powering studies using a sensor to generate data to inform the
endpoints
2100 Quality generation and ethical use of digital health data in clinical
studies
If anyone is interested in any of these areas then please get in touch with us

7. How Can Federated Learning Support
Genomics and Enable Precision Medicine
at Scale?
Craig Rhodes, EMEA Industry Lead for Healthcare and Life Science, NVIDIA
Nicola Rieke, Senior Deep Learning Solution Architect – Healthcare, NVIDIA

8. Nicola Rieke | Sr. Deep Learning Solution Architect - Healthcare
Craig Rhodes| EMEA IBD for AI/Deep Learning – Health and Life Science
HOW CAN FEDERATED LEARNING
SUPPORT GENOMICS AND ENABLE
PRECISION MEDICINE AT SCALE?

9. 9
AI IN MEDICINE
RADIOLOGY
CT, MRI, US, X-RAY
PATHOLOGY
TISSUE & CELL
DERMATOLOGY
OPHTHALMOLOGY
ELECTRONIC HEALTH
RECORDS
...
27K Medical AI papers
~30 FDA Approved products
~7 Billion USD investment by 2021
GENOMICS

10. 10
FROM RESEARCH...
Improving state of the art performance in controlled settings

11. 11
...TO APPLICATIONS
Achieving human-level performance on large data and clinical settings
Hegde et al., Similar Image Search for Histopathology: SMILY,
Nature digital medicine 2019
• 127000 image patches
• 128,000,000 8 × 8 μm regions
• Histopathology image search
Luo et al., Real-time artificial intelligence for detection of upper
gastrointestinal cancer by endoscopy: a multicentre, case-control,
diagnostic study, Lancet Oncology 2019
• 6 hospitals in China
• 84424 individuals
• 1036496 endoscopy images
• Gastrointestinal cancer detection
• Perf. similar to the expert endoscopist
Esteva et al, Dermatologist-level classification of skin cancer with
deep neural networks, Nature 2017
• 129450 clinical images
• 2032 diseases
• Skin cancer detection
• comparable to dermatologists
De Fauw et al., Clinically applicable deep learning for diagnosis and
referral in retinal disease, Nature Medicine 2018
• “Only” 14884 OCT 3D scans
• Resolution ~ 5 µm
• Volumetric multi-region segmentation
• Performance comparable to humans

12. 12
DATA-DRIVEN MEDICINE REQUIRES FEDERATED EFFORTS
- Training of robust and accurate DL models requires large
and diverse datasets
- Research is driven by data lakes
- Health data is highly sensitive, subject to regulations
and cannot easily be shared
- Regulatory and legal challenges related to ethics,
privacy and data protection, but also technical ones
Data Governance and Privacy
Demographic Bias / Healthcare in remote areas /
Hindered Research (e.g. Rare Diseases) ?
Data
GPU
Model
Rieke, N., Hancox, J., Li, W., Milletari, F., Roth, H., Albarqouni, S., Bakas, S., Galtier, M. N., Landman, B., Maier-Hein, K., Ourselin, S., Sheller, M.,
Summers, R. M., Trask, A., Xu, D., Baust, M. & Cardoso, M. J. (2020). The Future of Digital Health with Federated Learning. preprint arXiv:2003.08119.

13. 13
Data
GPU
Model Data
GPU
Model
Collaboration ?
Possible Solution:
Training algorithms
collaboratively without
sharing the raw data?
DATA-DRIVEN MEDICINE REQUIRES FEDERATED EFFORTS
Data Governance and Privacy
Federated Learning – allow algorithms to learn from non co-located data

14. 14
DATA-DRIVEN MEDICINE REQUIRES FEDERATED EFFORTS
- Address privacy and governance challenges
- FL could create new opportunities
- Large-scale validation directly in the institutions
- Enable novel research (e.g. rare diseases)
- Medical data is not duplicated
- Privacy protection with differential privacy
The Promise of Federated Efforts
Federated Learning
learning paradigm in which
multiple parties train
collaboratively without
centralizing datasets
Rieke, N., Hancox, J., Li, W., Milletari, F., Roth, H., Albarqouni, S., Bakas, S., Galtier, M. N., Landman, B., Maier-Hein, K., Ourselin, S., Sheller, M.,
Summers, R. M., Trask, A., Xu, D., Baust, M. & Cardoso, M. J. (2020). The Future of Digital Health with Federated Learning. preprint arXiv:2003.08119.

15. 15
IMPACT OF FEDERATED LEARNING
Increasing the value of AI for all healthcare stakeholders
Clinicians
Accurate assistance tools,
Standardization
Patients
Accurate and unbiased AI,
Data donor
Researchers
Access to large datasets,
Clinical relevant problems
Hospital and Practices
Full control of patient data,
Infrastructure
Manufacturers
Continuous improvement
of ML-based systems
Rieke, N., Hancox, J., Li, W., Milletari, F., Roth, H., Albarqouni, S., Bakas, S., Galtier, M. N., Landman, B., Maier-Hein, K., Ourselin, S., Sheller, M.,
Summers, R. M., Trask, A., Xu, D., Baust, M. & Cardoso, M. J. (2020). The Future of Digital Health with Federated Learning. preprint arXiv:2003.08119.

16. 16
FEDERATED LEARNING
• Broadly speaking, FL can be formalised as:
Let denote a global loss function via a weighted combination of K local losses computed on private data
• Rooted in older forms of collaborative learning. Main characteristics:
• Datasets are distributed
• Shared model
Definition
Rieke, N., Hancox, J., Li, W., Milletari, F., Roth, H., Albarqouni, S., Bakas, S., Galtier, M. N., Landman, B., Maier-Hein, K., Ourselin, S., Sheller, M.,
Summers, R. M., Trask, A., Xu, D., Baust, M. & Cardoso, M. J. (2020). The Future of Digital Health with Federated Learning. preprint arXiv:2003.08119.

17. 17
FEDERATED LEARNING
Communication Architectures
Rieke, N., Hancox, J., Li, W., Milletari, F., Roth, H., Albarqouni, S., Bakas, S., Galtier, M. N., Landman, B., Maier-Hein, K., Ourselin, S., Sheller, M.,
Summers, R. M., Trask, A., Xu, D., Baust, M. & Cardoso, M. J. (2020). The Future of Digital Health with Federated Learning. preprint arXiv:2003.08119.

18. 18
SERVER-CLIENT FEDERATED LEARNING
Changing the way AI algorithms are trained
GPU
GPU
GPU

19. 19
SERVER-CLIENT FEDERATED LEARNING
Changing the way AI algorithms are trained
GPU
GPU
GPU

20. 20
SERVER-CLIENT FEDERATED LEARNING
Changing the way AI algorithms are trained
GPU
GPU
GPU

21. 21
SERVER-CLIENT FEDERATED LEARNING
Changing the way AI algorithms are trained
GPU
GPU
GPU

22. 22
SERVER-CLIENT FEDERATED LEARNING
Changing the way AI algorithms are trained
GPU
GPU
GPU
Li, W., Milletarì, F., Xu, D., Rieke, N., Hancox, J., Zhu, W., ... & Feng, A. (2019, October). Privacy-preserving Federated Brain Tumour
Segmentation. In International Workshop on Machine Learning in Medical Imaging (pp. 133-141). Springer, Cham.

23. 23
PERFORMANCE & ACCURACY EXPERIMENTS
• Multi-modal multi-class brain tumour seg.
• 242 subjects
• Data-centralised training
• Federated averaging
• 13 clients
Privacy-preserving Federated Brain Tumour
Segmentation. arxiv.org/abs/1910.00962
Federated Learning building Robust AI

24. 24
TECHNICAL CONSIDERATIONS
FL does not solve all potential privacy issues.
Privacy – preserving techniques for FL offer levels
of protection that exceed general ML models.
Depending on level of trust in FL consortium,
different counter-measures may be implemented.
PRIVACY & SECURITY
Medical data is particularly diverse (type,
dimensionality, …). This poses a challenge if data
is not independent and identically distributed
across participants. Global solution may not be
the optimal local solution.
DATA HETEROGENEITY
In particular in non-trusted federations, traceability
and accountability processes require execution
integrity. It may also be helpful to measure the
amount of contribution from each participant to
determine relevant compensation and establish a
revenue model among the participants.
TRACEABILITY & ACCOUNTABILITY SYSTEM ARCHITECTURE
Rieke, N., Hancox, J., Li, W., Milletari, F., Roth, H., Albarqouni, S., Bakas, S., Galtier, M. N., Landman, B., Maier-Hein, K., Ourselin, S., Sheller, M.,
Summers, R. M., Trask, A., Xu, D., Baust, M. & Cardoso, M. J. (2020). The Future of Digital Health with Federated Learning. preprint arXiv:2003.08119.
Local training in each institution requires
computational infrastructure available on-site.
Data and annotation needs to be standardized.
For enabling the collaborative training, a training
protocol is needed.

25. 25
HEALTHCARE INDUSTRY ADOPTING FEDERATED LEARNING
MEDICAL IMAGING
Adopting NVIDIA Clara Federated Learning for Imaging
PHARMA
Machine Learning Ledger Orchestration for Drug Discovery
PHARMA PARTNERS
PUBLIC PARTNERS
This project has received funding from the Innovative Medicines
Initiative 2 Joint Undertaking under grant agreement N°831472.
This Joint Undertaking receives support from the European Union’s
Horizon 2020 research and innovation programme and EFPIA

26. 26
FEDERATED LEARNING
EXPERIMENT

27. 27
FEDERATED LEARNING EXPERIMENT
- Breast density is a significant risk factor for breast
cancer.
- Women with a high mammographic breast density (>75%)
have a four- to six-fold increased breast cancer risk
compared with women having a very low breast density.
- High mammographic breast density impairs the
sensitivity and specificity of breast cancer screening,
possibly because present (small) malignant lesions are
not detectable.
Classification of Breast Imaging Reporting and Data System (BI-RADS) Breast Density in 2D
Mammography
Ooms, E., et al. (2007). "Mammography: interobserver variability in breast density
assessment." The Breast 16(6): 568-576.

28. 28
FEDERATED LEARNING EXPERIMENT
Classification of Breast Imaging Reporting and Data System (BI-RADS) Breast Density in 2D
Mammography
Lehman, C. D., et al. (2019). "Mammographic breast density assessment using deep learning: clinical implementation." Radiology 290(1): 52-58.

29. 29
FEDERATED LEARNING EXPERIMENT
Classification of Breast Imaging Reporting and Data System (BI-RADS) Breast Density in 2D
Mammography

30. 30
FEDERATED LEARNING EXPERIMENT
Classification of Breast Imaging Reporting and Data System (BI-RADS) Breast Density in 2D
Mammography

31. 31
FEDERATED LEARNING EXPERIMENT
Average model performance

32. 32
Distributed Collaborative Learning
Build a common, robust AI model without sharing data
Using NVIDIA Clara to:
Authenticate and deliver Clara FL to participating hospitals
Locally train on private data
Securely Share partial-model weights
Apply Federated Averaging creating a new global model
BYOC to Federated Learning - New
Addressing Data Diversity & Privacy Global Model
w
CLARA FEDERATED LEARNING
Collaborative Distributed Learning

33. 33
BRINGING STATE-OF-THE-ART
AI TO HEALTHCARE

34. 34
NVIDIA IN HEALTHCARE
BREAKTHROUGHS AI STARTUPS
RESEARCHIMAGING DRUG DISCOVERY
DEVICES

35. 35
FEDERATED LEARNING FUTURE OF AI

36. 36
FEDERATED LEARNING
Clara Federated Learning: https://developer.nvidia.com/clara-medical-imaging
Papers:
• The Future of Digital Health with Federated Learning: https://arxiv.org/abs/2003.08119
• Privacy-preserving Federated Brain Tumour Segmentation: https://arxiv.org/abs/1910.00962
• Federated Deep Learning Among Multiple Institutions for Automated Classification of Breast Density:
https://cdn.ymaws.com/siim.org/resource/resmgr/siim20/abstracts-research/chang-kalpathycramer_federat.pdf
Blog:
• https://blogs.nvidia.com/blog/2019/10/13/what-is-federated-learning/
• https://devblogs.nvidia.com/federated-learning-clara/
Resources

38. Poll:
What Data is most important to support digital pharma
and health?
1. Lab Test results
2. Omics / Biobank data
3. Lifestyle + food
4. Trials data + Medical history
5. Depends on area of focus

39. Panel:
Jennifer Goldsack – CEO Digital Medicine Society (DiMe)
Tim McCarthy, Head of Digital Medicine & Translational Imaging, Pfizer
Marissa Dockendorf, Director, Global Digital Analytics & Technologies, Merck

40. ©PistoiaAlliance
Question one
40
• How do we collect and leverage patient data to
develop tools that support digital health, what
data is important and what barriers exist?

41. ©PistoiaAlliance
Question two
41
• Where do you see the next steps for “testing” in
trials and what might a remote testing model in
the near term look like?

42. ©PistoiaAlliance
Question Three
42
• How can we work collaboratively once data is
collected?

43. Panel Wrap up

44. Poll
What do you see as the biggest barrier to collecting
and leveraging patient data to support digital health
using federated AI/ML learning?
1. Costs
2. Lack of skills & technology
3. Lack of industry-wide data standards
4. Industry regulation
5. Internal resistance

45. Get Involved!
Digital, FAIR, AI/ML – IP3 project list
Steering Committee
Paul Denny-Gouldson
paul.denny-Gouldson@pistoiaalliance.org
Membership:
Beeta Balali-Mood
beeta.balalimood@pistoiaalliance.orgbeeta.balalimood@pistoiaalliance.o
r
General Enquiries:
Zahid Tharia – zahid.tharia@pistoiaalliance.org
www.pistoiaalliance.orgwww.pistoiaalliance.org

46. Next Webinar
UXLS "Happy Hour": Heartificial Intelligence - A Human
Centered Approach
Wed, May 13, 2020, 16:00 – 17:00 BST

47. info@pistoiaalliance.org @pistoiaalliance www.pistoiaalliance.org