https://www.youtube.com/watch?v=sR7hBJGpwQo&list=PLWKf9beHi3TgstcIn8K6dI_85_ppAxzB8&index=57&t=0s
================================================================================
Apply domain adaptation to medical image
================================================================================
* Heart image via MRI
* Use domain shift to convert MRI image to CT image
* Use network which is trained by MRI data
* See whether segmentations can be possible on CT image
* Lower domain adaptation results in bad segmentation (1st image from right image group)
* Good domain adaptation results in good segmentation
================================================================================
* Domain shift lowers performance
* Train network over MNIST dataset
* Good classification
* Apply model trained over MNIST data to SVHN image data
* Worsen performance
================================================================================
Domain shift
* Train network over domain A data
* Apply trained network over domain B data
================================================================================
* Why performance degradation happens in domain shift?
* Due to different data distributions
* You can see classifier trained over MNIST doesn't work for SVHN data
================================================================================
* Domain adaptation
================================================================================
Feature adaptation (paper: adversarial discriminative domain adaptation, 2017)
* Domain adaptation in feature level
* train network
* train network by using "domain label"
* use discriminator which create "good fake feature for SVHN and MNIST"
* Classifier which is trained over SVHN data at first pretrain step can classify MNIST data
================================================================================
Domain adaptation method 2:
Combine 2 adaptive strategies
* Train CycleGAN first for image level domain adaptation
* Use ADDA for feature level domain adaptation
================================================================================
* Images of heart
* Different moodalities
================================================================================
* Problem
* Domain experts create label for each region of heart of CT image
* Domain experts should create labels for each region of heart of MRI image
================================================================================
* Problem 2
* Performance degradation in domain adaptation
================================================================================
* Model arcitecture
================================================================================
* Result
(a): without adaptation
(b): image adaptation using CycleGAN (less performance)
(c): feature adaptation
(c): Image+feature adaptation
================================================================================
Workflow
* Perform image adaptation by using pixel-to-pixel image transformation (GAN model)
* Train generator $$$G_t$$$ which makes CT-looking image
* Train discriminator which distinguishes fake-created-CT-image
* Use GAN loss to train both networks
* GAN loss
$$$\mathcal{L}_{adv}^{t}(G_t,D_t)
= \mathbb{E}_{x^t \sim X^t} [\log{D_t(x^t)}] +
\mathbb{E}_{x^s \sim X^s} [\log{(1-D_t(G_t(x^s)))}]$$$
* Role: create CT-looking image from MRI image
================================================================================
* Pass CT-looking image into encoder to create feature images
* Pass feature images into Decoder U to create MRI-looking image
* If D_s can't distinguish CT-looking and MRI-looking images,
it means good training on encoder and decoder
================================================================================
Add segmentation network: Classifier C, $$$D_p$$$
================================================================================
================================================================================