Few-shot biomedical image segmentation using diffusion models: Beyond image generation.
Diffusion models
Generative AI
Orthopedics surgery
Pelvis radiographs
Semantic segmentation
Synthetic data
Journal
Computer methods and programs in biomedicine
ISSN: 1872-7565
Titre abrégé: Comput Methods Programs Biomed
Pays: Ireland
ID NLM: 8506513
Informations de publication
Date de publication:
Dec 2023
Dec 2023
Historique:
received:
03
05
2023
revised:
12
09
2023
accepted:
25
09
2023
medline:
14
11
2023
pubmed:
2
10
2023
entrez:
1
10
2023
Statut:
ppublish
Résumé
Medical image analysis pipelines often involve segmentation, which requires a large amount of annotated training data, which is time-consuming and costly. To address this issue, we proposed leveraging generative models to achieve few-shot image segmentation. We trained a denoising diffusion probabilistic model (DDPM) on 480,407 pelvis radiographs to generate 256 ✕ 256 px synthetic images. The DDPM was conditioned on demographic and radiologic characteristics and was rigorously validated by domain experts and objective image quality metrics (Frechet inception distance [FID] and inception score [IS]). For the next step, three landmarks (greater trochanter [GT], lesser trochanter [LT], and obturator foramen [OF]) were annotated on 45 real-patient radiographs; 25 for training and 20 for testing. To extract features, each image was passed through the pre-trained DDPM at three timesteps and for each pass, features from specific blocks were extracted. The features were concatenated with the real image to form an image with 4225 channels. The feature-set was broken into random patches, which were fed to a U-Net. Dice Similarity Coefficient (DSC) was used to compare the performance with a vanilla U-Net trained on radiographs. Expert accuracy was 57.5 % in determining real versus generated images, while the model reached an FID = 7.2 and IS = 210. The segmentation UNet trained on the 20 feature-sets achieved a DSC of 0.90, 0.84, and 0.61 for OF, GT, and LT segmentation, respectively, which was at least 0.30 points higher than the naively trained model. We demonstrated the applicability of DDPMs as feature extractors, facilitating medical image segmentation with few annotated samples.
Sections du résumé
BACKGROUND
BACKGROUND
Medical image analysis pipelines often involve segmentation, which requires a large amount of annotated training data, which is time-consuming and costly. To address this issue, we proposed leveraging generative models to achieve few-shot image segmentation.
METHODS
METHODS
We trained a denoising diffusion probabilistic model (DDPM) on 480,407 pelvis radiographs to generate 256 ✕ 256 px synthetic images. The DDPM was conditioned on demographic and radiologic characteristics and was rigorously validated by domain experts and objective image quality metrics (Frechet inception distance [FID] and inception score [IS]). For the next step, three landmarks (greater trochanter [GT], lesser trochanter [LT], and obturator foramen [OF]) were annotated on 45 real-patient radiographs; 25 for training and 20 for testing. To extract features, each image was passed through the pre-trained DDPM at three timesteps and for each pass, features from specific blocks were extracted. The features were concatenated with the real image to form an image with 4225 channels. The feature-set was broken into random patches, which were fed to a U-Net. Dice Similarity Coefficient (DSC) was used to compare the performance with a vanilla U-Net trained on radiographs.
RESULTS
RESULTS
Expert accuracy was 57.5 % in determining real versus generated images, while the model reached an FID = 7.2 and IS = 210. The segmentation UNet trained on the 20 feature-sets achieved a DSC of 0.90, 0.84, and 0.61 for OF, GT, and LT segmentation, respectively, which was at least 0.30 points higher than the naively trained model.
CONCLUSION
CONCLUSIONS
We demonstrated the applicability of DDPMs as feature extractors, facilitating medical image segmentation with few annotated samples.
Identifiants
pubmed: 37778140
pii: S0169-2607(23)00498-4
doi: 10.1016/j.cmpb.2023.107832
pii:
doi:
Substances chimiques
dihydroxydiphenyl-pyridyl methane
R09078E41Y
Bisacodyl
10X0709Y6I
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
107832Informations de copyright
Copyright © 2023 Elsevier B.V. All rights reserved.
Déclaration de conflit d'intérêts
Declaration of Competing Interest The authors have no actual or potential conflict of interest.