MULTI-DOMAIN UNPAIRED ULTRASOUND IMAGE ARTIFACT REMOVAL USING A SINGLE CONVOLUTIONAL NEURAL NETWORK

1. Jaeyoung Huh1,
Shujaat Khan1, and Jong Chul Ye2
BISPL - BioImaging, Signal Processing, and Learning lab.
1 Dept. Bio & Brain Engineering
2 Kim Jaechul Graduate School of AI
KAIST, Korea
Multi-Domain Unpaired Ultrasound Image
Artifact Removal Using
a Single Convolutional Neural Network

2. Introduction
 Ultrasound image artifact
- Blurring artifact from limitation by the
bandwidth of the transducer.
"Unsupervised deconvolution neural network for high quality ultrasound imaging." International Ultrasonics Symposium (IUS). IEEE, 2020.
No interference Constructive
interference
Destructive
interference
Signal
1
Signal
2
Signal
summation
 Deconvolution  Despeckle
- Speckle pattern generated by
instructive and destructive signal
interference.

3. Motivation
 Conventional Method
 US artifact hinder accurate diagnosis.
 Many researchers have proposed
various model-based iterative
algorithms
 computationally expensive
"Increasing axial resolution of ultrasonic imaging with a joint sparse representation model." TUFFC, 2016.
"A non-local low-rank framework for ultrasound speckle reduction." CVPR, 2017.

4. Motivation
 Deep learning based single domain image to
image translation such as CycleGAN.
 Still have some technical hurdles to wide
acceptance.
 Different types of artifact needs distinct choice of
artifact suppression algorithms.
 Single domain translation model

5. Motivation
 Recently, multi-domain image translation
models are proposed such as StarGAN.
 StarGAN require bi-directional translation
which is unnecessary in US image.
 Multi-domain translation model

6. Background
 Representative single-domain unpaired image-to-image translation method.
 It minimizes the distance between 𝜇, 𝜇𝜃 and 𝜈, 𝜈𝜙.
 Geometry of CycleGAN  CycleGAN structure
 CycleGAN
"Unpaired image-to-image translation using cycle-consistent adversarial networks.“ CVPR, 2017.

7. Background
 StarGAN
"Stargan: Unified generative adversarial networks for multi-domain image-to-image translation.“ CVPR, 2018.
 StarGAN structure
 Representative multi-domain unpaired image-to-image translation method.
 It can translate into each domain by using single generator.
 Each domain is separated by mask vector composed of one-hot vector.

8. Strategy
 Multi-domain Ultrasound image artifact removal using a single CNN
 Delay-And-Sum (DAS) image is generated from
raw data obtained from the scanner.
=> not necessary to re-generate from other
domain image.
 By combining the concept of CycleGAN and
StarGAN, we proposed uni-directional multi-
domain translation method for US image artifact
removal.

9. Our contribution
 Proposed method

10. Our contribution
 Generator, Discriminator Structure
 The generator is U-Net with residual block.
 The discriminator is multi-head structure.
3x3 Convolution with stride=1 + InstNorm + Relu
3x3 Convolution with stride=2 + InstNorm + Relu
3x3 Convolution with stride=1 + InstNorm
3x3 Transposed convolution with stride=2 + InstNorm + Relu
3x3 Convolution with stride=1 + Tanh
4x4 Convolution with stride=2 + LeakyRelu
3x3 Convolution with stride=1
4x4 Convolution with stride=1
Domain
classification
Real/Fake
classification

11. Training Details
Implementation Details
Total Epoch 1000
Learning Rate
Linear decreasing from 1e-4 after the
half of total epoch
Optimizer Adam Optimizer
Batch size 4
Parameter (𝜆𝑐𝑦𝑐, 𝜆𝐺𝑃, 𝜆𝑐𝑙𝑠)= (20,30,1)
Optimization
Formulation
WGAN with Gradient-Penalty
Dataset Details
Training set
Input - 304 images (8 subjects)
Input -125 images (Phantom)
Target - 429 images
(generated using input image)
Test set
80 images (2 subjects)
16 images (tissue mimicking phantom)
Data
Augmentation
Flipping, Rotating, Random Scaling
Normalization Normalized all images to -1~1
Target
generation
- Deconvolution
"Increasing axial resolution of ultrasonic imaging
with a joint sparse representation model." TUFFC,
2016.
- Despeckle
"A non-local low-rank framework for ultrasound
speckle reduction." CVPR, 2017.

12. Results
 Qualitative

13. Results
 Quantitative

14. Conclusion
 Ultrasound (US) image suffer from many artifacts from various sources.
 We proposed multi-domain US image artifact removal method using single convolution
neural network.
 The method is based on representative image-to-image (I2I) translation algorithm,
CycleGAN and StarGAN.
 A single network can provide blurring removed or speckle suppressed image.

15. Thank You