Towards a guideline for evaluation metrics in medical image segmentation.
Biomedical image segmentation; Semantic segmentation; Medical Image Analysis
Evaluation
Guideline
Performance assessment
Reproducibility
Journal
BMC research notes
ISSN: 1756-0500
Titre abrégé: BMC Res Notes
Pays: England
ID NLM: 101462768
Informations de publication
Date de publication:
20 Jun 2022
20 Jun 2022
Historique:
received:
12
02
2022
accepted:
07
06
2022
entrez:
20
6
2022
pubmed:
21
6
2022
medline:
23
6
2022
Statut:
epublish
Résumé
In the last decade, research on artificial intelligence has seen rapid growth with deep learning models, especially in the field of medical image segmentation. Various studies demonstrated that these models have powerful prediction capabilities and achieved similar results as clinicians. However, recent studies revealed that the evaluation in image segmentation studies lacks reliable model performance assessment and showed statistical bias by incorrect metric implementation or usage. Thus, this work provides an overview and interpretation guide on the following metrics for medical image segmentation evaluation in binary as well as multi-class problems: Dice similarity coefficient, Jaccard, Sensitivity, Specificity, Rand index, ROC curves, Cohen's Kappa, and Hausdorff distance. Furthermore, common issues like class imbalance and statistical as well as interpretation biases in evaluation are discussed. As a summary, we propose a guideline for standardized medical image segmentation evaluation to improve evaluation quality, reproducibility, and comparability in the research field.
Identifiants
pubmed: 35725483
doi: 10.1186/s13104-022-06096-y
pii: 10.1186/s13104-022-06096-y
pmc: PMC9208116
doi:
Types de publication
Letter
Review
Langues
eng
Sous-ensembles de citation
IM
Pagination
210Subventions
Organisme : Bundesministerium für Bildung und Forschung
ID : FKZ01ZZ1804E
Informations de copyright
© 2022. The Author(s).
Références
Sci Rep. 2020 Aug 13;10(1):13724
pubmed: 32792540
Nat Methods. 2019 Dec;16(12):1247-1253
pubmed: 31636459
Stud Health Technol Inform. 2021 May 27;281:518-519
pubmed: 34042629
IEEE Trans Med Imaging. 2020 Feb;39(2):499-513
pubmed: 31329113
BMC Med Imaging. 2015 Aug 12;15:29
pubmed: 26263899
Comput Med Imaging Graph. 2015 Jul;43:99-111
pubmed: 25863519
Nat Methods. 2021 Feb;18(2):203-211
pubmed: 33288961
BMC Med Imaging. 2021 Jan 18;21(1):12
pubmed: 33461500
Med Image Anal. 2017 Dec;42:60-88
pubmed: 28778026
JAMA. 2019 Dec 24;322(24):2377-2378
pubmed: 31755905
Comput Biol Med. 2021 Jul;134:104497
pubmed: 34022486
Stud Health Technol Inform. 2021 Sep 21;283:23-31
pubmed: 34545816
PLoS One. 2015 Oct 08;10(10):e0140381
pubmed: 26447861
Curr Probl Diagn Radiol. 2021 Mar-Apr;50(2):262-267
pubmed: 32591104
J Pathol Inform. 2018 Nov 02;9:37
pubmed: 30533276
Eur Radiol Exp. 2021 Jan 21;5(1):4
pubmed: 33474675
J Med Imaging (Bellingham). 2021 Jan;8(Suppl 1):010902-10902
pubmed: 34646912
Inform Med Unlocked. 2021;25:100681
pubmed: 34337140
J Gastroenterol Hepatol. 2021 Jan;36(1):31-38
pubmed: 33140875
Brain Commun. 2020 Jul 09;2(2):fcaa096
pubmed: 33134913
Data Brief. 2019 Nov 21;28:104863
pubmed: 31867417
Radiology. 1982 Apr;143(1):29-36
pubmed: 7063747
PLoS One. 2019 Sep 26;14(9):e0222916
pubmed: 31557204
Med Image Anal. 2017 Dec;42:44-59
pubmed: 28772163
Stud Health Technol Inform. 2022 May 25;294:33-37
pubmed: 35612011
J Am Acad Dermatol. 2018 Feb;78(2):270-277.e1
pubmed: 28969863
PLoS One. 2016 Jun 06;11(6):e0157112
pubmed: 27273091