VertXNet: an ensemble method for vertebral body segmentation and identification from cervical and lumbar spinal X-rays.
Journal
Scientific reports
ISSN: 2045-2322
Titre abrégé: Sci Rep
Pays: England
ID NLM: 101563288
Informations de publication
Date de publication:
09 Feb 2024
09 Feb 2024
Historique:
received:
06
03
2023
accepted:
13
12
2023
medline:
10
2
2024
pubmed:
10
2
2024
entrez:
9
2
2024
Statut:
epublish
Résumé
Accurate annotation of vertebral bodies is crucial for automating the analysis of spinal X-ray images. However, manual annotation of these structures is a laborious and costly process due to their complex nature, including small sizes and varying shapes. To address this challenge and expedite the annotation process, we propose an ensemble pipeline called VertXNet. This pipeline currently combines two segmentation mechanisms, semantic segmentation using U-Net, and instance segmentation using Mask R-CNN, to automatically segment and label vertebral bodies in lateral cervical and lumbar spinal X-ray images. VertXNet enhances its effectiveness by adopting a rule-based strategy (termed the ensemble rule) for effectively combining segmentation outcomes from U-Net and Mask R-CNN. It determines vertebral body labels by recognizing specific reference vertebral instances, such as cervical vertebra 2 ('C2') in cervical spine X-rays and sacral vertebra 1 ('S1') in lumbar spine X-rays. Those references are commonly relatively easy to identify at the edge of the spine. To assess the performance of our proposed pipeline, we conducted evaluations on three spinal X-ray datasets, including two in-house datasets and one publicly available dataset. The ground truth annotations were provided by radiologists for comparison. Our experimental results have shown that the proposed pipeline outperformed two state-of-the-art (SOTA) segmentation models on our test dataset with a mean Dice of 0.90, vs. a mean Dice of 0.73 for Mask R-CNN and 0.72 for U-Net. We also demonstrated that VertXNet is a modular pipeline that enables using other SOTA model, like nnU-Net to further improve its performance. Furthermore, to evaluate the generalization ability of VertXNet on spinal X-rays, we directly tested the pre-trained pipeline on two additional datasets. A consistently strong performance was observed, with mean Dice coefficients of 0.89 and 0.88, respectively. In summary, VertXNet demonstrated significantly improved performance in vertebral body segmentation and labeling for spinal X-ray imaging. Its robustness and generalization were presented through the evaluation of both in-house clinical trial data and publicly available datasets.
Identifiants
pubmed: 38336974
doi: 10.1038/s41598-023-49923-3
pii: 10.1038/s41598-023-49923-3
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
3341Informations de copyright
© 2024. The Author(s).
Références
Burns, J. E., Yao, J., Muñoz, H. & Summers, R. M. Automated detection, localization, and classification of traumatic vertebral body fractures in the thoracic and lumbar spine at ct. Radiology 278, 64 (2016).
doi: 10.1148/radiol.2015142346
pubmed: 26172532
Ronneberger, O., Fischer, P. & Brox, T. U-Net: Convolutional networks biomedical image segmentation. Med. Image Comput. Comput.-Assist. Interv. 234–241, 2015. https://doi.org/10.1007/978-3-319-24574-4_28 (2015).
doi: 10.1007/978-3-319-24574-4_28
He, K. et al. Mask R-CNN. In Proceedings of the IEEE International Conference on Computer Vision, 2961–2969 (2017).
Chen, Y. et al. Vertxnet: Automatic segmentation and identification of lumbar and cervical vertebrae from spinal x-ray images. arXiv preprint arXiv:2207.05476 (2022).
Masood, S., Sharif, M., Masood, A., Yasmin, M. & Raza, M. A survey on medical image segmentation. Curr. Med. Imaging 11, 3–14 (2015).
doi: 10.2174/157340561101150423103441
Kass, M., Witkin, A. & Terzopoulos, D. Snakes: Active contour models. Int. J. Comput. Vis. 1, 321–331. https://doi.org/10.1007/BF00133570 (1988).
doi: 10.1007/BF00133570
de Brebisson, A. & Montana, G. Deep neural networks for anatomical brain segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 20–28 (2015).
Ciresan, D., Giusti, A., Gambardella, L. M. & Schmidhuber, J. Deep neural networks segment neuronal membranes in electron microscopy images. In Advances in Neural Information Processing Systems, 2843–2851 (2012).
Rashid, R., Akram, M. U. & Hassan, T. Fully convolutional neural network for lungs segmentation from chest x-rays. In International Conference Image Analysis and Recognition, 71–80 (Springer, 2018).
Novikov, A. A. et al. Fully convolutional architectures for multiclass segmentation in chest radiographs. IEEE Trans. Med. Imaging 37, 1865–1876 (2018).
doi: 10.1109/TMI.2018.2806086
pubmed: 29994439
Al Arif, S., Knapp, K. & Slabaugh, G. Spnet: Shape prediction using a fully convolutional neural network. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 430–439 (Springer, 2018).
Tran, V. L., Lin, H.-Y. & Liu, H.-W. Mbnet: a multi-task deep neural network for semantic segmentation and lumbar vertebra inspection on x-ray images. In Proceedings of the Asian Conference on Computer Vision (2020).
Li, Y., Liang, W., Zhang, Y., An, H. & Tan, J. Automatic lumbar vertebrae detection based on feature fusion deep learning for partial occluded c-arm x-ray images. In 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 647–650 (IEEE, 2016).
Kurachka, K. & Tsalka, I. Vertebrae detection in x-ray images based on deep convolutional neural networks. In 2017 IEEE 14th International Scientific Conference on Informatics, 194–196 (IEEE, 2017).
Li, C. et al. SPA-RESUNET: Strip pooling attention resunet for multi-class segmentation of vertebrae and intervertebral discs. In 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI), 1–5 (IEEE, 2022).
Whitehead, W., Moran, S., Gaonkar, B., Macyszyn, L. & Iyer, S. A deep learning approach to spine segmentation using a feed-forward chain of pixel-wise convolutional networks. In 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), 868–871 (IEEE, 2018).
Khanal, B., Dahal, L., Adhikari, P. & Khanal, B. Automatic cobb angle detection using vertebra detector and vertebra corners regression, https://doi.org/10.48550/ARXIV.1910.14202 .
Sa, R. et al. Intervertebral disc detection in x-ray images using faster r-CNN. In 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 564–567 (IEEE, 2017).
Kim, D. H., Jeong, J. G., Kim, Y. J., Kim, K. G. & Jeon, J. Y. Automated vertebral segmentation and measurement of vertebral compression ratio based on deep learning in x-ray images. J. Digit. Imaging 34, 853–861 (2021).
doi: 10.1007/s10278-021-00471-0
pubmed: 34236562
pmcid: 8455797
Windsor, R., Jamaludin, A., Kadir, T. & Zisserman, A. A convolutional approach to vertebrae detection and labelling in whole spine MRI. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 712–722 (Springer, 2020).
Cho, B. H. et al. Automated measurement of lumbar lordosis on radiographs using machine learning and computer vision. Glob. Spine J. 10, 611–618 (2020).
doi: 10.1177/2192568219868190
Zhang, T. et al. A novel tool to provide predictable alignment data irrespective of source and image quality acquired on mobile phones: What engineers can offer clinicians. Eur. Spine J. 29, 387–395 (2020).
doi: 10.1007/s00586-019-06264-y
pubmed: 31897731
Shin, Y., Han, K. & Lee, Y. H. Temporal trends in cervical spine curvature of South Korean adults assessed by deep learning system segmentation, 2006–2018. JAMA Netw. Open 3, e2020961–e2020961 (2020).
doi: 10.1001/jamanetworkopen.2020.20961
pubmed: 33057644
pmcid: 7563081
Qadri, S. F. et al. OP-convNet: A patch classification-based framework for CT vertebrae segmentation. IEEE Access 9, 158227–158240 (2021).
doi: 10.1109/ACCESS.2021.3131216
Altini, N. et al. Segmentation and identification of vertebrae in CT scans using CNN, k-means clustering and k-NN. In Informatics, vol. 8, 40 (MDPI, 2021).
Cheng, P., Yang, Y., Yu, H. & He, Y. Automatic vertebrae localization and segmentation in CT with a two-stage Dense-U-Net. Sci. Rep. 11, 1–13 (2021).
doi: 10.1038/s41598-021-01296-1
Baeten, D. et al. Secukinumab, an interleukin-17a inhibitor, in ankylosing spondylitis. N. Engl. J. Med. 373, 2534–2548 (2015).
doi: 10.1056/NEJMoa1505066
pubmed: 26699169
Isensee, F., Jaeger, P. F., Kohl, S. A., Petersen, J. & Maier-Hein, K. H. nnU-Net: A self-configuring method for deep learning-based biomedical image segmentation. Nat. Methods 18, 203–211 (2021).
doi: 10.1038/s41592-020-01008-z
pubmed: 33288961
Ren, S., He, K., Girshick, R. B. & Sun, J. Faster R-CNN: towards real-time object detection with region proposal networks. CoRR arXiv:abs/1506.01497 (2015).
Paszke, A. et al. Pytorch: An imperative style, high-performance deep learning library. CoRR arXiv:abs/1912.01703 (2019).
Meng, D., Mohammed, E., Boyer, E. & Pujades, S. Vertebrae localization, segmentation and identification using a graph optimization and an anatomic consistency cycle (2022).
Koo, B. S. et al. A pilot study on deep learning-based grading of corners of vertebral bodies for assessment of radiographic progression in patients with ankylosing spondylitis. Ther. Adv. Musculoskelet. Dis. 14, 1759720X221114097. https://doi.org/10.1177/1759720X221114097 (2022).
doi: 10.1177/1759720X221114097
pubmed: 35898565
pmcid: 9310199
Deodhar, A. et al. Improvement of signs and symptoms of nonradiographic axial spondyloarthritis in patients treated with secukinumab: Primary results of a randomized, placebo-controlled phase III study. Arthritis Rheumatol. 73, 110–120 (2021).
doi: 10.1002/art.41477
pubmed: 32770640
Kirillov, A., He, K., Girshick, R., Rother, C. & Dollar, P. Panoptic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019).
Jamaludin, A., Kadir, T. & Zisserman, A. SpineNet: Automated classification and evidence visualization in spinal MRIs. Med. Image Anal. 41, 63–73 (2017).
doi: 10.1016/j.media.2017.07.002
pubmed: 28756059
Mallon, A.-M. et al. Advancing data science in drug development through an innovative computational framework for data sharing and statistical analysis. BMC Med. Res. Methodol. 21, 1–11 (2021).
doi: 10.1186/s12874-021-01409-4