A deep learning approach for projection and body-side classification in musculoskeletal radiographs.
Artificial intelligence
Bone and bones
Deep learning
Musculoskeletal diseases
Radiography
Journal
European radiology experimental
ISSN: 2509-9280
Titre abrégé: Eur Radiol Exp
Pays: England
ID NLM: 101721752
Informations de publication
Date de publication:
14 Feb 2024
14 Feb 2024
Historique:
received:
18
09
2023
accepted:
29
11
2023
medline:
14
2
2024
pubmed:
14
2
2024
entrez:
14
2
2024
Statut:
epublish
Résumé
The growing prevalence of musculoskeletal diseases increases radiologic workload, highlighting the need for optimized workflow management and automated metadata classification systems. We developed a large-scale, well-characterized dataset of musculoskeletal radiographs and trained deep learning neural networks to classify radiographic projection and body side. In this IRB-approved retrospective single-center study, a dataset of musculoskeletal radiographs from 2011 to 2019 was retrieved and manually labeled for one of 45 possible radiographic projections and the depicted body side. Two classification networks were trained for the respective tasks using the Xception architecture with a custom network top and pretrained weights. Performance was evaluated on a hold-out test sample, and gradient-weighted class activation mapping (Grad-CAM) heatmaps were computed to visualize the influential image regions for network predictions. A total of 13,098 studies comprising 23,663 radiographs were included with a patient-level dataset split, resulting in 19,183 training, 2,145 validation, and 2,335 test images. Focusing on paired body regions, training for side detection included 16,319 radiographs (13,284 training, 1,443 validation, and 1,592 test images). The models achieved an overall accuracy of 0.975 for projection and 0.976 for body-side classification on the respective hold-out test sample. Errors were primarily observed in projections with seamless anatomical transitions or non-orthograde adjustment techniques. The deep learning neural networks demonstrated excellent performance in classifying radiographic projection and body side across a wide range of musculoskeletal radiographs. These networks have the potential to serve as presorting algorithms, optimizing radiologic workflow and enhancing patient care. The developed networks excel at classifying musculoskeletal radiographs, providing valuable tools for research data extraction, standardized image sorting, and minimizing misclassifications in artificial intelligence systems, ultimately enhancing radiology workflow efficiency and patient care. • A large-scale, well-characterized dataset was developed, covering a broad spectrum of musculoskeletal radiographs. • Deep learning neural networks achieved high accuracy in classifying radiographic projection and body side. • Grad-CAM heatmaps provided insight into network decisions, contributing to their interpretability and trustworthiness. • The trained models can help optimize radiologic workflow and manage large amounts of data.
Sections du résumé
BACKGROUND
BACKGROUND
The growing prevalence of musculoskeletal diseases increases radiologic workload, highlighting the need for optimized workflow management and automated metadata classification systems. We developed a large-scale, well-characterized dataset of musculoskeletal radiographs and trained deep learning neural networks to classify radiographic projection and body side.
METHODS
METHODS
In this IRB-approved retrospective single-center study, a dataset of musculoskeletal radiographs from 2011 to 2019 was retrieved and manually labeled for one of 45 possible radiographic projections and the depicted body side. Two classification networks were trained for the respective tasks using the Xception architecture with a custom network top and pretrained weights. Performance was evaluated on a hold-out test sample, and gradient-weighted class activation mapping (Grad-CAM) heatmaps were computed to visualize the influential image regions for network predictions.
RESULTS
RESULTS
A total of 13,098 studies comprising 23,663 radiographs were included with a patient-level dataset split, resulting in 19,183 training, 2,145 validation, and 2,335 test images. Focusing on paired body regions, training for side detection included 16,319 radiographs (13,284 training, 1,443 validation, and 1,592 test images). The models achieved an overall accuracy of 0.975 for projection and 0.976 for body-side classification on the respective hold-out test sample. Errors were primarily observed in projections with seamless anatomical transitions or non-orthograde adjustment techniques.
CONCLUSIONS
CONCLUSIONS
The deep learning neural networks demonstrated excellent performance in classifying radiographic projection and body side across a wide range of musculoskeletal radiographs. These networks have the potential to serve as presorting algorithms, optimizing radiologic workflow and enhancing patient care.
RELEVANCE STATEMENT
CONCLUSIONS
The developed networks excel at classifying musculoskeletal radiographs, providing valuable tools for research data extraction, standardized image sorting, and minimizing misclassifications in artificial intelligence systems, ultimately enhancing radiology workflow efficiency and patient care.
KEY POINTS
CONCLUSIONS
• A large-scale, well-characterized dataset was developed, covering a broad spectrum of musculoskeletal radiographs. • Deep learning neural networks achieved high accuracy in classifying radiographic projection and body side. • Grad-CAM heatmaps provided insight into network decisions, contributing to their interpretability and trustworthiness. • The trained models can help optimize radiologic workflow and manage large amounts of data.
Identifiants
pubmed: 38353812
doi: 10.1186/s41747-023-00417-x
pii: 10.1186/s41747-023-00417-x
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
23Informations de copyright
© 2024. The Author(s).
Références
Vos T, Lim SS, Abbafati C et al (2020) Global burden of 369 diseases and injuries in 204 countries and territories, 1990–2019: a systematic analysis for the Global Burden of Disease Study 2019. Lancet 396:1204–1222. https://doi.org/10.1016/S0140-6736(20)30925-9
doi: 10.1016/S0140-6736(20)30925-9
Nekolla EA, Schegerer AA, Griebel J, Brix G (2017) Häufigkeit und Dosis diagnostischer und interventioneller Röntgenanwendungen. Radiologe 57:555–562. https://doi.org/10.1007/s00117-017-0242-y
doi: 10.1007/s00117-017-0242-y
pubmed: 28361179
Bhargavan M, Kaye AH, Forman HP, Sunshine JH (2009) Workload of radiologists in United States in 2006–2007 and trends since 1991–1992. Radiology 252:458–467. https://doi.org/10.1148/radiol.2522081895
doi: 10.1148/radiol.2522081895
pubmed: 19508987
Jiménez-Sánchez A, Kazi A, Albarqouni S et al (2020) Precise proximal femur fracture classification for interactive training and surgical planning. Int J Comput Assist Radiol Surg 15:847–857. https://doi.org/10.1007/s11548-020-02150-x
doi: 10.1007/s11548-020-02150-x
pubmed: 32335786
Thian YL, Li Y, Jagmohan P et al (2019) Convolutional neural networks for automated fracture detection and localization on wrist radiographs. Radiol Artif Intell 1:e180001. https://doi.org/10.1148/ryai.2019180001
doi: 10.1148/ryai.2019180001
pubmed: 33937780
pmcid: 8017412
Norman B, Pedoia V, Noworolski A et al (2019) Applying densely connected convolutional neural networks for staging osteoarthritis severity from plain radiographs. J Digit Imaging 32:471–477. https://doi.org/10.1007/s10278-018-0098-3
doi: 10.1007/s10278-018-0098-3
pubmed: 30306418
Larson DB, Chen MC, Lungren MP et al (2017) Performance of a deep-learning neural network model in assessing skeletal maturity on pediatric hand radiographs. Radiology. https://doi.org/10.1148/radiol.2017170236
doi: 10.1148/radiol.2017170236
pubmed: 29232185
Gao Y, Zhu T, Xu X (2020) Bone age assessment based on deep convolution neural network incorporated with segmentation. Int J Comput Assist Radiol Surg 15:1951–1962. https://doi.org/10.1007/s11548-020-02266-0
doi: 10.1007/s11548-020-02266-0
pubmed: 32986142
Juba B, Le HS (2019) Precision-recall versus accuracy and the role of large data sets. Proc AAAI Conf Artif Intell 33:4039–4048. https://doi.org/10.1609/aaai.v33i01.33014039
doi: 10.1609/aaai.v33i01.33014039
Rajkomar A, Lingam S, Taylor AG et al (2017) High-throughput classification of radiographs using deep convolutional neural networks. J Digit Imaging 30:95–101. https://doi.org/10.1007/s10278-016-9914-9
doi: 10.1007/s10278-016-9914-9
pubmed: 27730417
Yi PH, Kim TK, Wei J et al (2019) Automated semantic labeling of pediatric musculoskeletal radiographs using deep learning. Pediatr Radiol 49:1066–1070. https://doi.org/10.1007/s00247-019-04408-2
doi: 10.1007/s00247-019-04408-2
pubmed: 31041454
Hinterwimmer F, Consalvo S, Wilhelm N et al (2023) SAM-X: sorting algorithm for musculoskeletal x-ray radiography. Eur Radiol 33:1537–1544. https://doi.org/10.1007/s00330-022-09184-6
doi: 10.1007/s00330-022-09184-6
pubmed: 36307553
Kim TK, Yi PH, Wei J et al (2019) Deep learning method for automated classification of anteroposterior and posteroanterior chest radiographs. J Digit Imaging 32:925–930. https://doi.org/10.1007/s10278-019-00208-0
doi: 10.1007/s10278-019-00208-0
pubmed: 30972585
pmcid: 6841900
Cao F, Huang HK, Pietka E, et al (2003) Image database for digital hand atlas. Proc. SPIE 5033, Medical Imaging 2003: PACS and Integrated Medical Information Systems: Design and Evaluation. 5033:461–470. https://doi.org/10.1117/12.480681
Eckstein F, Wirth W, Nevitt MC (2012) Recent advances in osteoarthritis imaging—the Osteoarthritis Initiative. Nat Rev Rheumatol 8:622–630. https://doi.org/10.1038/nrrheum.2012.113
doi: 10.1038/nrrheum.2012.113
pubmed: 22782003
pmcid: 6459017
Rajpurkar P, Irvin J, Bagul A et al (2018) MURA: Large dataset for abnormality detection in musculoskeletal radiographs. 1st Conference on Medical Imaging with Deep Learning (MIDL 2018), Amsterdam, The Netherlands
Varma M, Lu M, Gardner R et al (2019) Automated abnormality detection in lower extremity radiographs using deep learning. Nat Mach Intell 1:578–583. https://doi.org/10.1038/s42256-019-0126-0
doi: 10.1038/s42256-019-0126-0
Nora - The medical imaging platform. https://www.nora-imaging.com/ . Accessed 1 Sep 2023
Abadi M, Agarwal A, Barham P, et al (2016) TensorFlow: Large-scale machine learning on heterogeneous distributed systems. arXiv:1603.04467 https://doi.org/10.48550/arXiv.1603.04467
Keras: Deep learning for humans. https://keras.io/ . Accessed 1 Sep 2023
Chollet F (2017) Xception: Deep learning with depthwise separable convolutions. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Honolulu, HI, pp 1800–1807
Deng J, Dong W, Socher R, et al (2009) ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, 2009, pp 248-255. https://doi.org/10.1109/CVPR.2009.5206848 .
Rahman T, Khandakar A, Qiblawey Y et al (2021) Exploring the effect of image enhancement techniques on COVID-19 detection using chest X-ray images. Comput Biol Med 132:104319. https://doi.org/10.1016/j.compbiomed.2021.104319
doi: 10.1016/j.compbiomed.2021.104319
pubmed: 33799220
pmcid: 7946571
Pedregosa F, Varoquaux G, Gramfort A, et al (2018) Scikit-learn: machine learning in python. arXiv:1201.0490 https://doi.org/10.48550/arXiv.1201.0490
Selvaraju RR, Cogswell M, Das A et al (2020) Grad-CAM: Visual explanations from deep networks via gradient-based localization. Int J Comput Vis 128:336–359. https://doi.org/10.1007/s11263-019-01228-7
doi: 10.1007/s11263-019-01228-7
Guan B, Zhang G, Yao J et al (2020) Arm fracture detection in X-rays based on improved deep convolutional neural network. Comput Electr Eng 81:106530. https://doi.org/10.1016/j.compeleceng.2019.106530
doi: 10.1016/j.compeleceng.2019.106530
Liang S, Gu Y (2020) Towards robust and accurate detection of abnormalities in musculoskeletal radiographs with a multi-network model. Sensors (Basel) 20:3153. https://doi.org/10.3390/s20113153
doi: 10.3390/s20113153
pubmed: 32498374
Urinbayev K, Orazbek Y, Nurambek Y et al (2020) End-to-end deep diagnosis of x-ray images. 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). IEEE, Montreal, QC, Canada, pp 2182–2185
doi: 10.1109/EMBC44109.2020.9175208
Dratsch T, Korenkov M, Zopfs D et al (2021) Practical applications of deep learning: classifying the most common categories of plain radiographs in a PACS using a neural network. Eur Radiol 31:1812–1818. https://doi.org/10.1007/s00330-020-07241-6
doi: 10.1007/s00330-020-07241-6
pubmed: 32986160