these slides cover basic deep learning models and one application case in skin cancer

1. Group meeting
--Journal Club
Liangqun Lu
2017-09-20

2. Outline
● Background on Deep learning
○ Definition
○ Common Models (AE, DBN, CNN and RNN)
○ Transfer Learning
● Paper 1: Dermatologist-level classification of skin
cancer with deep neural networks
● Paper 2: Deep learning for healthcare: review,
opportunities and challenges
2

3. Deep learning Definition
Deep learning is a class of machine learning algorithms that:
● use a cascade of many layers of nonlinear processing units for feature
extraction and transformation.
● Each successive layer uses the output from the previous layer as input.
Higher level features are derived from lower level features to form a
hierarchical representation.
3

4. H = WT
. X + b -- Logistic regression --- Multiple perceptron
Output nodes: label; input; ….
Can be applied on supervised / unsupervised / semi-supervised problems
samples
Features
4

5. Becoming popular (2006 -- )
● Big Data
● Strong
computing
capability
● Bottleneck
solution in DL
https://beamandrew.github.io/deeplearning/2017/02/23/deep_learning_101_part1.html 5

6. Breakthrough in areas
● Computer vision /image recognition
● Speech recognition
● Text analysis
● Natural Language Processing
● Self-driving cars
● Drug development and genomics
● …..
6

7. https://www.nervanasys.com/deep-learning-and-the-need-for-unified-tools/ 7

8. http://recognize-speech.com/acoustic-model/knn/benchmarks-comparison-of-different-architectures
WER: word
error rate
8

9. https://medium.com/@edersantana/towards-a-scientific-method-for-self-driving-cars-research-4db960833
677
9

10. http://www.ausy.tu-darmstadt.de/Research/Research 10

11. http://www.asimovinstitute.org/neural-network-zoo/
Unsupervised Supervised
Image related Sequence data
DL models
11

12. DL models: Autoencoder
12

13. 13

14. DL models: CNN
14

15. 15

16. DL models: RNN
16

17. 17

18. Transfer Learning
18

19. 19

20. 20

21. 21

22. 22

23. Skin cancer
● 5.4 m new cases in US every year
● Melanomas(< 5%) causing 75% skin-cancer-related deaths
● 5-year survival 99% for early stage diagnosis and 14% for late
stage diagnosis
● Classification from dermoscopy and histological image lacking
generalization capability
23

24. Analysis flow
Image data: 129,450 with 2032 labels
(dermoscopy and histological)
CNN
Google inception v3
Performance evaluation
ACC; AUC;
specificity and sensitivity
*Pre-trained DL
architecture,
including 1.28 m
images with 1000
labels
24

25. Deep CNN layout
25

26. Results
26

27. t-SNE visualization of the last hidden layer representations in the CNN for
four disease classes.
27

28. Conclusions
● Similar performance of deep learning model with 21
dermatologists
● The method can be deployable on mobile device to track skin
cancer
28

29. 29

30. 30

31. 31

32. DL some points
● RBM, GANs, RL,
● Weight Initialization
● Activation function
● Hyperparameters optimization
● Regularization (Dropout etc.)
● Evaluation (Ensembles etc.)
● Transfer learning / fine-tuning
32

33. Take-home messages
● What can we conclude from several successful cases?
● Is DL appropriate in molecule-level biological data?
● What is the potential hierarchical architecture in gene
representation?
● How much data is enough for gene expression-based DL training?
● How to consider the one-shot information from transcriptome?
● How to combine omic data from different resources to DL models?
33

34. Thank you!
34