Inter-rater agreement in glioma segmentations on longitudinal MRI.
Glioblastoma
Glioma
Inter-rater agreement
Low-grade glioma
MRI
Manual segmentation
Journal
NeuroImage. Clinical
ISSN: 2213-1582
Titre abrégé: Neuroimage Clin
Pays: Netherlands
ID NLM: 101597070
Informations de publication
Date de publication:
2019
2019
Historique:
received:
12
07
2018
revised:
06
02
2019
accepted:
19
02
2019
pubmed:
3
3
2019
medline:
27
12
2019
entrez:
3
3
2019
Statut:
ppublish
Résumé
Tumor segmentation of glioma on MRI is a technique to monitor, quantify and report disease progression. Manual MRI segmentation is the gold standard but very labor intensive. At present the quality of this gold standard is not known for different stages of the disease, and prior work has mainly focused on treatment-naive glioblastoma. In this paper we studied the inter-rater agreement of manual MRI segmentation of glioblastoma and WHO grade II-III glioma for novices and experts at three stages of disease. We also studied the impact of inter-observer variation on extent of resection and growth rate. In 20 patients with WHO grade IV glioblastoma and 20 patients with WHO grade II-III glioma (defined as non-glioblastoma) both the enhancing and non-enhancing tumor elements were segmented on MRI, using specialized software, by four novices and four experts before surgery, after surgery and at time of tumor progression. We used the generalized conformity index (GCI) and the intra-class correlation coefficient (ICC) of tumor volume as main outcome measures for inter-rater agreement. For glioblastoma, segmentations by experts and novices were comparable. The inter-rater agreement of enhancing tumor elements was excellent before surgery (GCI 0.79, ICC 0.99) poor after surgery (GCI 0.32, ICC 0.92), and good at progression (GCI 0.65, ICC 0.91). For non-glioblastoma, the inter-rater agreement was generally higher between experts than between novices. The inter-rater agreement was excellent between experts before surgery (GCI 0.77, ICC 0.92), was reasonable after surgery (GCI 0.48, ICC 0.84), and good at progression (GCI 0.60, ICC 0.80). The inter-rater agreement was good between novices before surgery (GCI 0.66, ICC 0.73), was poor after surgery (GCI 0.33, ICC 0.55), and poor at progression (GCI 0.36, ICC 0.73). Further analysis showed that the lower inter-rater agreement of segmentation on postoperative MRI could only partly be explained by the smaller volumes and fragmentation of residual tumor. The median interquartile range of extent of resection between raters was 8.3% and of growth rate was 0.22 mm/year. Manual tumor segmentations on MRI have reasonable agreement for use in spatial and volumetric analysis. Agreement in spatial overlap is of concern with segmentation after surgery for glioblastoma and with segmentation of non-glioblastoma by non-experts.
Sections du résumé
BACKGROUND
Tumor segmentation of glioma on MRI is a technique to monitor, quantify and report disease progression. Manual MRI segmentation is the gold standard but very labor intensive. At present the quality of this gold standard is not known for different stages of the disease, and prior work has mainly focused on treatment-naive glioblastoma. In this paper we studied the inter-rater agreement of manual MRI segmentation of glioblastoma and WHO grade II-III glioma for novices and experts at three stages of disease. We also studied the impact of inter-observer variation on extent of resection and growth rate.
METHODS
In 20 patients with WHO grade IV glioblastoma and 20 patients with WHO grade II-III glioma (defined as non-glioblastoma) both the enhancing and non-enhancing tumor elements were segmented on MRI, using specialized software, by four novices and four experts before surgery, after surgery and at time of tumor progression. We used the generalized conformity index (GCI) and the intra-class correlation coefficient (ICC) of tumor volume as main outcome measures for inter-rater agreement.
RESULTS
For glioblastoma, segmentations by experts and novices were comparable. The inter-rater agreement of enhancing tumor elements was excellent before surgery (GCI 0.79, ICC 0.99) poor after surgery (GCI 0.32, ICC 0.92), and good at progression (GCI 0.65, ICC 0.91). For non-glioblastoma, the inter-rater agreement was generally higher between experts than between novices. The inter-rater agreement was excellent between experts before surgery (GCI 0.77, ICC 0.92), was reasonable after surgery (GCI 0.48, ICC 0.84), and good at progression (GCI 0.60, ICC 0.80). The inter-rater agreement was good between novices before surgery (GCI 0.66, ICC 0.73), was poor after surgery (GCI 0.33, ICC 0.55), and poor at progression (GCI 0.36, ICC 0.73). Further analysis showed that the lower inter-rater agreement of segmentation on postoperative MRI could only partly be explained by the smaller volumes and fragmentation of residual tumor. The median interquartile range of extent of resection between raters was 8.3% and of growth rate was 0.22 mm/year.
CONCLUSION
Manual tumor segmentations on MRI have reasonable agreement for use in spatial and volumetric analysis. Agreement in spatial overlap is of concern with segmentation after surgery for glioblastoma and with segmentation of non-glioblastoma by non-experts.
Identifiants
pubmed: 30825711
pii: S2213-1582(19)30077-4
doi: 10.1016/j.nicl.2019.101727
pmc: PMC6396436
pii:
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
101727Informations de copyright
Copyright © 2019 The Authors. Published by Elsevier Inc. All rights reserved.
Références
Conf Proc IEEE Eng Med Biol Soc. 2016 Aug;2016:4403-4406
pubmed: 28269254
Neuro Oncol. 2012 Apr;14(4):496-505
pubmed: 22416109
J Neurooncol. 2018 Aug;139(1):167-175
pubmed: 29633111
Sci Data. 2017 Sep 05;4:170117
pubmed: 28872634
IEEE Trans Med Imaging. 2015 Oct;34(10):1993-2024
pubmed: 25494501
Radiother Oncol. 2001 Jul;60(1):49-59
pubmed: 11410304
Radiother Oncol. 2005 May;75(2):217-23
pubmed: 15885831
AJNR Am J Neuroradiol. 2013 Mar;34(3):533-40
pubmed: 22997168
Nat Rev Clin Oncol. 2018 Feb;15(2):112-125
pubmed: 29158591
Eur J Cancer. 2012 Jul;48(10):1532-42
pubmed: 22227039
Phys Med Biol. 2009 May 7;54(9):2863-73
pubmed: 19384002
PLoS One. 2014 May 07;9(5):e96873
pubmed: 24804720
AJR Am J Roentgenol. 2009 Dec;193(6):W515-22
pubmed: 19933626
J Digit Imaging. 2013 Dec;26(6):1045-57
pubmed: 23884657
Schizophr Bull. 1991;17(3):483-9
pubmed: 1947873
Neurosurg Rev. 2010 Jan;33(1):91-6
pubmed: 19847462
J Neurooncol. 2012 Jul;108(3):477-83
pubmed: 22407175
Neuroradiology. 2017 Oct;59(10):1013-1020
pubmed: 28842741
J Neurooncol. 2017 Jan;131(2):393-402
pubmed: 27837437
Healthc Technol Lett. 2018 Jan 26;5(1):13-17
pubmed: 29515811
Transl Oncol. 2014 Feb 01;7(1):40-7
pubmed: 24772206
PLoS One. 2013 Sep 06;8(9):e73353
pubmed: 24039922
Acta Neuropathol. 2007 Aug;114(2):97-109
pubmed: 17618441
Neuro Oncol. 2017 Nov 6;19(suppl_5):v1-v88
pubmed: 29117289
AJNR Am J Neuroradiol. 2017 Oct;38(10):1884-1891
pubmed: 28882867
PLoS One. 2016 Nov 2;11(11):e0165302
pubmed: 27806121
Radiology. 2013 May;267(2):560-9
pubmed: 23392431
J Neurosurg. 2001 Aug;95(2):190-8
pubmed: 11780887
Sci Rep. 2016 Apr 27;6:25007
pubmed: 27118379
Neuro Oncol. 2007 Jan;9(1):63-9
pubmed: 17018698
Neurosurgery. 2017 May 1;80(5):793-799
pubmed: 28387870
Psychol Bull. 1979 Mar;86(2):420-8
pubmed: 18839484
Clin Neuroradiol. 2017 Jun;27(2):153-161
pubmed: 26490369
Neurosurgery. 2010 Nov;67(5):1329-34
pubmed: 20871451
Neurosurgery. 2012 Sep;71(3):729-39; discussion 739-40
pubmed: 22668885
Neuro Oncol. 2015 Feb;17(2):282-8
pubmed: 25031032
Neurosurg Rev. 2008 Jul;31(3):263-9
pubmed: 18299912
Radiother Oncol. 2005 Nov;77(2):182-90
pubmed: 16256231
IEEE Trans Med Imaging. 1994;13(4):716-24
pubmed: 18218550
IEEE J Biomed Health Inform. 2019 Jan;23(1):38-46
pubmed: 29993901
IEEE Trans Med Imaging. 2012 Oct;31(10):1941-54
pubmed: 22907965
Neurochirurgie. 2013 Apr;59(2):89-92
pubmed: 23623033
J Clin Oncol. 2001 Jan 15;19(2):551-7
pubmed: 11208850
JAMA Oncol. 2016 Nov 1;2(11):1460-1469
pubmed: 27310651
Sci Rep. 2016 Mar 22;6:23376
pubmed: 27001047