Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

ICCV 2019 REVIEW [CDM]

1 568 vues

Publié le

ICCV 2019 review

Publié dans : Technologie

ICCV 2019 REVIEW [CDM]

  1. 1. Yonsei University Severance Hospital CCIDS ICCV 2019 Review Dongmin Choi
  2. 2. ICCV - ICCV : International Conference on Computer Vision - 2019년 국내에서 첫 개최 : 10.27 ~ 11.2, COEX - 2019 ICCV : 약 7,500명 참가 총 1,077편의 논문 Yonsei University Severance Hospital CCIDS
  3. 3. ICCV ICCV 2019 Summary 1. Visual Recognition for Medical Images 2. Image and Video Synthesis: How, Why and What if? 3. Interpretating and Explaining Visual AI Models 4. SinGAN: Learning a Generative Model from a Single Natural Images (Best Paper) Yonsei University Severance Hospital CCIDS
  4. 4. Yonsei University Severance Hospital CCIDS Visual Recognition for Medical Images https://sites.google.com/view/iccv19-vrmi
  5. 5. Visual Recognition for Medical Images Yonsei University Severance Hospital CCIDS Deep Learning in Neuroimaging and Radiotherapy Dinggang Shen University of North Carolina at Chapel Hill
  6. 6. Visual Recognition for Medical Images Yonsei University Severance Hospital CCIDS Image Registration Template T Subject S Deep-learning based Registration J. Fan, X. Cao, Z. Xue, PT. Yap, D. Shen, “Adversarial Similarity Network for Evaluating Image Alignment in Deep Learning based Registration”
  7. 7. Visual Recognition for Medical Images Yonsei University Severance Hospital CCIDS Train the registration network by Similarity Metric Template T Subject S Deep Registration Network (U-Net) T S 68 * 68 * 68 68 * 68 * 68 Deformation Field 3*28*28*28 Spatial Transformation Warped subject 28*28*28 Dissimilarity Loss LD(T, S') S'φ J. Fan, X. Cao, Z. Xue, PT. Yap, D. Shen, “Adversarial Similarity Network for Evaluating Image Alignment in Deep Learning based Registration”
  8. 8. Visual Recognition for Medical Images Yonsei University Severance Hospital CCIDS Train the registration network by Similarity Metric (inspired by GAN) Template T Subject S Deep Registration Network (U-Net) T S 68 * 68 * 68 68 * 68 * 68 Deformation Field 3*28*28*28 Spatial Transformation Warped subject 28*28*28 Discriminator S'φ J. Fan, X. Cao, Z. Xue, PT. Yap, D. Shen, “Adversarial Similarity Network for Evaluating Image Alignment in Deep Learning based Registration” Generator
  9. 9. Visual Recognition for Medical Images Yonsei University Severance Hospital CCIDS On Unseen Testing Dataset J. Fan, X. Cao, Z. Xue, PT. Yap, D. Shen, “Adversarial Similarity Network for Evaluating Image Alignment in Deep Learning based Registration”
  10. 10. Visual Recognition for Medical Images Yonsei University Severance Hospital CCIDS Registration Result J. Fan, X. Cao, Z. Xue, PT. Yap, D. Shen, “Adversarial Similarity Network for Evaluating Image Alignment in Deep Learning based Registration”
  11. 11. Visual Recognition for Medical Images Yonsei University Severance Hospital CCIDS Image Synthesis MRI CT Can we synthesize CT image from a single MRI Image? CT images are highly desired for - Specific diagnosis - Dose planning - PET attenuation correction D. Nie et al, “Medical Image Synthesis with Deep Convolutional Adversarial Networks”
  12. 12. Visual Recognition for Medical Images Yonsei University Severance Hospital CCIDS D. Nie et al, “Medical Image Synthesis with Deep Convolutional Adversarial Networks” Auto-context Use context features to iteratively refine the training results
  13. 13. Visual Recognition for Medical Images Yonsei University Severance Hospital CCIDS D. Nie et al, “Medical Image Synthesis with Deep Convolutional Adversarial Networks” Auto-context Refinements Difference maps
  14. 14. Visual Recognition for Medical Images Yonsei University Severance Hospital CCIDS Q&A Q. Whole Image가 아니라 Patch를 사용하는 이유? A. 데이터 수가 딥러닝을 학습하기에는 부족하기 때문이다. Patch를 사용하면 훨씬 많은 학습 데이터를 확보할 수 있기 때문에 patch를 사용한다. 물론 데이터 수가 충분히 많다면 patch가 아니라 whole image를 사용하는 것이 더욱 좋다.
  15. 15. Yonsei University Severance Hospital CCIDS Image and Video Synthesis: How, Why and What if https://sites.google.com/berkeley.edu/iccv-2019-image-and-video-syn
  16. 16. Image and Video Synthesis: How, Why and What if? Yonsei University Severance Hospital CCIDS Seeing What a GAN Cannot Generate David Bau MIT http://ganseeing.csail.mit.edu//
  17. 17. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS https://software.intel.com/en-us/blogs/2017/08/21/mode-collapse-in-gans Model Collapse in GAN is serious Problem ※ Model Collapse : A problem when all the generator outputs are identical (all of them or most of the samples are equal)
  18. 18. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS Fake Images Real Images http://ganseeing.csail.mit.edu//
  19. 19. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS Fake Images Real Images Inception Inception Fake Inception Feature Space Real Inception Feature Space http://ganseeing.csail.mit.edu// FID (Frechet Inception Distance) Measuring GAN Quality
  20. 20. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS Fake Images Real Images http://ganseeing.csail.mit.edu// 1. What is actually missing in the distribution? 2. What is actually missing in each image?
  21. 21. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS http://ganseeing.csail.mit.edu// 1. Understanding Omissions in the Distribution Generated Image Semantic segmentation Real Image Semantic segmentation
  22. 22. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS http://ganseeing.csail.mit.edu// 1. Understanding Omissions in the Distribution Generated Image Semantic segmentation Real Image Semantic segmentation
  23. 23. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS http://ganseeing.csail.mit.edu// 2. Understanding Omissions in Individual Images Real Image x Synthesized Image G(z)
  24. 24. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS http://ganseeing.csail.mit.edu// 2. Understanding Omissions in Individual Images Real Image x Synthesized Image G(z) Pairs (x, G(z*)) reveals omissions Objective : z* = argminz Loss(x, G(z))
  25. 25. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS http://ganseeing.csail.mit.edu// Two steps to invert a large generator
  26. 26. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS http://ganseeing.csail.mit.edu// x = G(z) x = G(z*) x = real x = G(z*) Generated Reconstruction Real Photo Reconstruction When G generates x, reconstruction is precise When reconstruction is imperfect we know G cannot generate x
  27. 27. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS http://ganseeing.csail.mit.edu// GANs don’t like people
  28. 28. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS http://ganseeing.csail.mit.edu// The Cheese Hypothesis Original Image
  29. 29. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS http://ganseeing.csail.mit.edu// The Cheese Hypothesis Original Image Optimized z
  30. 30. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS http://ganseeing.csail.mit.edu// The Cheese Hypothesis Original Image Adapted Cheese
  31. 31. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS Real Image x
  32. 32. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS Optimized vector z* G Real Image x Reconstructed Image G(z*) z* = argminz Loss(x, G(z))
  33. 33. Seeing What a GAN Cannot Generate Yonsei University Severance Hospital CCIDS Optimized vector z* G Real Image x Reconstructed Image G(z*, θ*) θ* z*, θ* = argminz,θ Loss(x, G(z)) + R(θ) Regularizer Inspired by Deep Image Prior [Ulyanove et al, 2018]
  34. 34. Yonsei University Severance Hospital CCIDS Interpretating and Explaining Visual AI Models http://xai.unist.ac.kr/workshop/2019/
  35. 35. Interpretating and Explaining Visual AI Models Yonsei University Severance Hospital CCIDS Recent progress towards XAI at UC Berkeley Trevor Darrel UC Berkeley
  36. 36. Recent progress towards XAI at UC Berkeley Yonsei University Severance Hospital CCIDS
  37. 37. Recent progress towards XAI at UC Berkeley Yonsei University Severance Hospital CCIDS Explainable NN
  38. 38. Recent progress towards XAI at UC Berkeley Yonsei University Severance Hospital CCIDS Salience for Introspection Image LIME GradCAM RISE V Petsiuk et al. RISE: Randomized Input Sampling for Explanation of Black-box Models
  39. 39. Recent progress towards XAI at UC Berkeley Yonsei University Severance Hospital CCIDS Overview of RISE V Petsiuk et al. RISE: Randomized Input Sampling for Explanation of Black-box Models
  40. 40. Recent progress towards XAI at UC Berkeley Yonsei University Severance Hospital CCIDS Pros of RISE V Petsiuk et al. RISE: Randomized Input Sampling for Explanation of Black-box Models Cons of RISE A more general framework as the importance map is obtained with access to only the input and output of the base model Time and resource complexity
  41. 41. Recent progress towards XAI at UC Berkeley Yonsei University Severance Hospital CCIDS LRP : Layer-wise Relevance Propagation Explaining Decisions of Neural Networks by LRP. Alexander Binder @ Deep Learning: Theory, Algorithms, and Applications. Berlin, June 2017
  42. 42. Recent progress towards XAI at UC Berkeley Yonsei University Severance Hospital CCIDS LRP : Layer-wise Relevance Propagation Explaining Decisions of Neural Networks by LRP. Alexander Binder @ Deep Learning: Theory, Algorithms, and Applications. Berlin, June 2017
  43. 43. Recent progress towards XAI at UC Berkeley Yonsei University Severance Hospital CCIDS LRP : Layer-wise Relevance Propagation Explaining Decisions of Neural Networks by LRP. Alexander Binder @ Deep Learning: Theory, Algorithms, and Applications. Berlin, June 2017
  44. 44. Recent progress towards XAI at UC Berkeley Yonsei University Severance Hospital CCIDS Unmasking Clever Hans Predictors S Lapuschkin et al. Unmasking Clever Hans Predictors and Assessing What Machines Really Learn Images with a copyright watermark (a, c) are classified to “horse” class, but images without an added copyright watermark (b, d) aren’t.
  45. 45. Recent progress towards XAI at UC Berkeley Yonsei University Severance Hospital CCIDS Understanding Learning Behaviour S Lapuschkin et al. Unmasking Clever Hans Predictors and Assessing What Machines Really Learn Model learns 1. Track the ball 2. Focus on paddle 3. Focus in the tunnel
  46. 46. Yonsei University Severance Hospital CCIDS Learning a Generative Model from a Single Natural Image https://sites.google.com/berkeley.edu/iccv-2019-image-and-video-syn SinGAN Tamar Rott Shaham Google AI ICCV 2019 Best Paper
  47. 47. SinGAN Yonsei University Severance Hospital CCIDS
  48. 48. SinGAN Yonsei University Severance Hospital CCIDS
  49. 49. SinGAN Yonsei University Severance Hospital CCIDS
  50. 50. SinGAN Yonsei University Severance Hospital CCIDS ▶ Same model ▶ No extra training SinGAN for single image animation https://www.youtube.com/watch?v=xk8bWLZk4DU&feature=youtu.be
  51. 51. SinGAN Yonsei University Severance Hospital CCIDS Model
  52. 52. SinGAN Yonsei University Severance Hospital CCIDS Summary - SinGAN : Single Image GAN - Controllable Generation - Image Manipulation Applications
  53. 53. SinGAN Yonsei University Severance Hospital CCIDS MRI Toy Example Training Image
  54. 54. SinGAN Yonsei University Severance Hospital CCIDS MRI Toy Example Training Image Generated High-Resolution Image
  55. 55. Thank You

×