microsoft ai on healthcare

1. Microsoft AI and cases sharing
蔡孟儒 Raymond
Sr. Program Manager
Customer Advisory Team, Azure C+E, GCR

3. Services &
Tools
Processing
Frameworks
AI Applications
Cognitive Services
Infrastructure
AML Studio & Web Services BOT Framework
Model & Experimentation
Management
Data Wrangling & Spark AI Batch
Training
Storage (Azure Data Services) & Hardware (CPU, GPU, FPGS & ASIC)
Inferencing
Spark, SQL,
Other Engines
DSVM
Machine Learning and Deep Learning Toolkits
CNTK Tensorflow ML Server Scikit-Learn Other Libs.
ACS
Docker
Tooling
CPUs
Edge
Dev
DS

4. Machine Learning & AI Portfolio
When to use what?
What engine(s) do you want
to use?
Deployment target?
Which experience do you
want?
Build your own or consume pre-
trained models?
Microsoft
ML & AI
products
Build your
own
Azure Machine Learning
Code first
(On-prem)
ML Server
(On-prem)
Hadoop
SQL
Server
(cloud)
AML Web Services
SQL
Server
Spark Hadoop Azure
Batch
DSVM Azure Container
Service
(K8, Docker,
DC/OS)
Visual tooling
(cloud)
AML Studio
Consume
Cognitive services, bots

5. Tap into rich
knowledge
amassed from
the web,
academia, or
your own data
Access billions of
web pages,
images, videos,
and news with
the power of
Bing APIs
Process text and
learn how to
recognize what
users want
Hear and speak
to your users by
filtering noise,
identifying
speakers, and
understanding
intent
Emerging
Cognitive
Services
technologies for
early adopters
From faces to
feelings, allow
your apps to
understand
images
and video

8. AI-powered
Data Wrangling
+
E2E ML Dev
Productivity
+
Deploy
Anywhere
=
E2E Tooling for AI
Development
Program Synthesis
Docker, Spark, IOT Edge,
On prem, AWS/GCP…
SPARK, GPU, Open Source
Lifecycle Management

9. Data Prep with Program Synthesis
Rapidly sample, understand, and prep
data
Leverage PROSE and more for intelligent
data prep by example
Extend/customize transforms and
featurization through Python
Generate Python and Pyspark for
execution at scale

10. Built-in AI-powered Data Wrangling
Collaboration with notebooks & Git
Version control & reproducibility
Metrics, lineage, run history, asset
management, and more

11. Azure Container Service
(scale out with Kubernetes clusters)
Azure IoT Edge
Spark on HDInsight
On-prem, AWS, GCP….

13. Azure Data Science Virtual Machine
• Popular tools Pre-installed & Pre-configured
 Includes RStudio & JuliaPro
• Deep Learning Extension for Azure GPU VM
• Developer Editions of SQL & R Server
• Now available on Azure Batch
• Supports Popular Workflows:
 SQL Server R Services: -
Dev>Train>Test>Deploy>Score
 Using the Local Spark instance on the DSVM for Dev
& Test
 Training and Deploying Deep Learning Models Using
the ‘Deep Learning Toolkit for the DSVM’ on GPU
based Azure VMs

14. Accelerating adoption of AI by developers
(consuming models)
Rise of hybrid training and scoring scenarios
Push scoring/inference to the event (edge,
cloud, on-prem)
Some developers moving into deep learning as
non-traditional path to DS / AI dev
Growth of diverse hardware arms race across all
form factors (CPU / GPU / FPGA / ASIC /
device)
Data prep
Model deployment &
management
Model lineage & auditing
Explain-ability
A D O P T I N G A I :
T R E N D S A N D C H A L L E N G E S
C H A L L E N G E SK E Y T R E N D S

15. Services &
Tools
Processing
Frameworks
AI Applications
Cognitive Services
Infrastructure
AML Studio & Web Services BOT Framework
Model & Experimentation
Management
Data Wrangling & Spark AI Batch
Training
Storage (Azure Data Services) & Hardware (CPU, GPU, FPGS & ASIC)
Inferencing
Spark, SQL,
Other Engines
DSVM
Machine Learning and Deep Learning Toolkits
CNTK Tensorflow ML Server Scikit-Learn Other Libs.
ACS
Docker
Tooling
CPUs
Edge
Dev
DS

17. Multiple Instance Learning
MIL
Training
Cardinal
Fish
Classifier
Weakly labeled training data
Positive
Instances
Negative
Instances
(Maron 1997, Viola 2005)

18. Multiple Instance Learning
MIL
Training
Cancer Image
Classifier
Weakly labeled training data
Positive
Instances
Negative
Instances

19. Parallel Multiple Instance Learning
• A standard histopathology
slice Resolution: 200,000 x
200,000
• Most existing medical
imaging tools infeasible
• Compute on multiple
machines

20. Context-Constrained Multiple Instance Learning
for Histopathology Image Segmentation

21. Using eye movement patterns for
early detection of dyslexia in children
Optolexia
“The flexibility and ease of use of the Azure Machine Learning analytics platform
makes it a perfect foundation for expanding our existing solution into new areas.“
Fredrik Wetterhall
Chief Executive Officer at Optolexia
The challenge
Optolexia wanted the ability to iterate and
scale their dyslexia detection model in order
to accommodate expansion into schools,
new environments, and enable additional
condition screenings.
Machine Learning in action
• Visualized models, scoring, and results without
writing new code to refine the testing tool
• Screened over 1k students and identified signs of
dyslexia earlier than ever before, leading to improved
student care, education, and self-esteem
• Created a scalable model enabling experimentation and
testing with new languages and conditions
Watch video

22. 數據分析改善閱讀障礙 (短片輔助說明)
透過機器學習來Learn普通讀者與閱讀障礙患者的閱讀軌跡
X

23. patient medical
history care
[1]

24. • Entity extraction is a subtask of information extraction (also known as
Named-entity recognition (NER), entity chunking and entity identification).
• Biomedical named entity recognition is a critical step for complex
biomedical NLP tasks such as:
• Extraction of diseases, symptoms from electronic medical or health
records.
• Understanding the interactions between different entity types such as
drug-drug interaction, drug-disease relationship and gene-protein
relationship, e.g.,
• Drug A cures Disease B.
• Drug A causes Disease B.
• Generic solution: Similar for other domains (e.g., legal, finance)

26. Machine Learning & AI Portfolio
When to use what?
What engine(s) do you want
to use?
Deployment target
Which experience do you
want?
Build your own or consume pre-
trained models?
Microsoft
ML & AI
products
Build your
own
Azure Machine Learning
Code first
(On-prem)
ML Server
(On-prem )
Hadoop
SQL
Server
(cloud)
AML Web Services
SQL
Server
Spark Hadoop Azure
Batch
DSVM Azure Container
Service
(K8, Docker,
DC/OS)
Visual tooling
(cloud)
AML Studio
Consume
Cognitive services, bots

27. https://azure.microsoft.com/en-
us/overview/ai-platform/?v=17.42w
https://azure.microsoft.com/en-us/services/cognitive-
services/custom-vision-service/

28. Appendix

29. Experimental Condition
Datasets
30 non-cancer (NC) images and 53 colon cancer histopathology images
Obtained from the Department of Pathology of Zhejiang University using Hamamatsu Nano
Zoomer 2.0HT digital slice scanner
Each image is independently labeled by two pathologists, the third pathologist moderates their
discussion
MTA—Moderately or well differentiated tubular adenocarcinoma
PTA—Poorly differentiated tubular adenocarcinoma
MA—Mucinous adenocarcinoma
SRC—Signet-ring carcinoma

30. Results
• Results: Pixel-level segmentation