The impact of large language models on radiology: a guide for radiologists on the latest innovations in AI.
Artificial intelligence
Deep learning
Diagnostic radiology
Large language model
Radiological workflow
Journal
Japanese journal of radiology
ISSN: 1867-108X
Titre abrégé: Jpn J Radiol
Pays: Japan
ID NLM: 101490689
Informations de publication
Date de publication:
29 Mar 2024
29 Mar 2024
Historique:
received:
15
11
2023
accepted:
21
02
2024
medline:
29
3
2024
pubmed:
29
3
2024
entrez:
29
3
2024
Statut:
aheadofprint
Résumé
The advent of Deep Learning (DL) has significantly propelled the field of diagnostic radiology forward by enhancing image analysis and interpretation. The introduction of the Transformer architecture, followed by the development of Large Language Models (LLMs), has further revolutionized this domain. LLMs now possess the potential to automate and refine the radiology workflow, extending from report generation to assistance in diagnostics and patient care. The integration of multimodal technology with LLMs could potentially leapfrog these applications to unprecedented levels.However, LLMs come with unresolved challenges such as information hallucinations and biases, which can affect clinical reliability. Despite these issues, the legislative and guideline frameworks have yet to catch up with technological advancements. Radiologists must acquire a thorough understanding of these technologies to leverage LLMs' potential to the fullest while maintaining medical safety and ethics. This review aims to aid in that endeavor.
Identifiants
pubmed: 38551772
doi: 10.1007/s11604-024-01552-0
pii: 10.1007/s11604-024-01552-0
doi:
Types de publication
Journal Article
Review
Langues
eng
Sous-ensembles de citation
IM
Informations de copyright
© 2024. The Author(s).
Références
Nakaura T, Higaki T, Awai K, Ikeda O, Yamashita Y. A primer for understanding radiology articles about machine learning and deep learning. Diagn Interv Imaging. 2020;101:765–70.
pubmed: 33121910
doi: 10.1016/j.diii.2020.10.001
Williams RJ, Zipser D. A learning algorithm for continually running fully recurrent neural networks. Neural Comput. 1989;1:270–80.
doi: 10.1162/neco.1989.1.2.270
Lu L, Wang X, Carneiro G, Yang L. Deep learning and convolutional neural networks for medical imaging and clinical informatics. Cham: Springer Nature; 2019.
doi: 10.1007/978-3-030-13969-8
Higaki T, Nakamura Y, Tatsugami F, Nakaura T, Awai K. Improvement of image quality at CT and MRI using deep learning. Jpn J Radiol. 2019;37:73–80.
pubmed: 30498876
doi: 10.1007/s11604-018-0796-2
Ozaki J, Fujioka T, Yamaga E, Hayashi A, Kujiraoka Y, Imokawa T, et al. Deep learning method with a convolutional neural network for image classification of normal and metastatic axillary lymph nodes on breast ultrasonography. Jpn J Radiol. 2022;40:814–22.
pubmed: 35284996
doi: 10.1007/s11604-022-01261-6
Ishihara M, Shiiba M, Maruno H, Kato M, Ohmoto-Sekine Y, Antoine C, et al. Detection of intracranial aneurysms using deep learning-based CAD system: usefulness of the scores of CNN’s final layer for distinguishing between aneurysm and infundibular dilatation. Jpn J Radiol. 2023;41:131–41.
pubmed: 36173510
doi: 10.1007/s11604-022-01341-7
Koretsune Y, Sone M, Sugawara S, Wakatsuki Y, Ishihara T, Hattori C, et al. Validation of a convolutional neural network for the automated creation of curved planar reconstruction images along the main pancreatic duct. Jpn J Radiol. 2023;41:228–34.
pubmed: 36121623
doi: 10.1007/s11604-022-01339-1
Kitahara H, Nagatani Y, Otani H, Nakayama R, Kida Y, Sonoda A, et al. A novel strategy to develop deep learning for image super-resolution using original ultra-high-resolution computed tomography images of lung as training dataset. Jpn J Radiol. 2022;40:38–47.
pubmed: 34318444
doi: 10.1007/s11604-021-01184-8
Nai Y-H, Loi HY, O’Doherty S, Tan TH, Reilhac A. Comparison of the performances of machine learning and deep learning in improving the quality of low dose lung cancer PET images. Jpn J Radiol. 2022;40:1290–9.
pubmed: 35809210
doi: 10.1007/s11604-022-01311-z
Yasaka K, Akai H, Sugawara H, Tajima T, Akahane M, Yoshioka N, et al. Impact of deep learning reconstruction on intracranial 1.5 T magnetic resonance angiography. Jpn J Radiol. 2022;40:476–83.
pubmed: 34851499
doi: 10.1007/s11604-021-01225-2
Kaga T, Noda Y, Mori T, Kawai N, Miyoshi T, Hyodo F, et al. Unenhanced abdominal low-dose CT reconstructed with deep learning-based image reconstruction: image quality and anatomical structure depiction. Jpn J Radiol. 2022;40:703–11.
pubmed: 35286578
pmcid: 9252942
doi: 10.1007/s11604-022-01259-0
Hosoi R, Yasaka K, Mizuki M, Yamaguchi H, Miyo R, Hamada A, et al. Deep learning reconstruction with single-energy metal artifact reduction in pelvic computed tomography for patients with metal hip prostheses. Jpn J Radiol. 2023;41:863–71.
pubmed: 36862290
pmcid: 10366278
doi: 10.1007/s11604-023-01402-5
Hamabuchi N, Ohno Y, Kimata H, Ito Y, Fujii K, Akino N, et al. Effectiveness of deep learning reconstruction on standard to ultra-low-dose high-definition chest CT images [Internet]. Jpn J Radiol. 2023. https://doi.org/10.1007/s11604-023-01470-7 .
doi: 10.1007/s11604-023-01470-7
pubmed: 37498483
pmcid: 10687108
Uematsu T, Nakashima K, Harada TL, Nasu H, Igarashi T. Comparisons between artificial intelligence computer-aided detection synthesized mammograms and digital mammograms when used alone and in combination with tomosynthesis images in a virtual screening setting. Jpn J Radiol. 2022;41:63–70.
pubmed: 36068450
pmcid: 9813079
doi: 10.1007/s11604-022-01327-5
Oshima S, Fushimi Y, Miyake KK, Nakajima S, Sakata A, Okuchi S, et al. Denoising approach with deep learning-based reconstruction for neuromelanin-sensitive MRI: image quality and diagnostic performance. Jpn J Radiol. 2023;41:1216–25.
pubmed: 37256470
pmcid: 10613599
doi: 10.1007/s11604-023-01452-9
Nakao T, Hanaoka S, Nomura Y, Hayashi N, Abe O. Anomaly detection in chest 18F-FDG PET/CT by Bayesian deep learning. Jpn J Radiol. 2022;40:730–9.
pubmed: 35094221
pmcid: 9252947
doi: 10.1007/s11604-022-01249-2
Toda N, Hashimoto M, Iwabuchi Y, Nagasaka M, Takeshita R, Yamada M, et al. Validation of deep learning-based computer-aided detection software use for interpretation of pulmonary abnormalities on chest radiographs and examination of factors that influence readers’ performance and final diagnosis. Jpn J Radiol. 2023;41:38–44.
pubmed: 36121622
doi: 10.1007/s11604-022-01330-w
Azuma M, Nakada H, Takei M, Nakamura K, Katsuragawa S, Shinkawa N, et al. Detection of acute rib fractures on CT images with convolutional neural networks: effect of location and type of fracture and reader’s experience. Emerg Radiol [Internet]. 2022. Accessed 3 Nov 2023;29. Available from: https://pubmed.ncbi.nlm.nih.gov/34855002/
Goto M, Sakai K, Toyama Y, Nakai Y, Yamada K. Use of a deep learning algorithm for non-mass enhancement on breast MRI: comparison with radiologists’ interpretations at various levels. Jpn J Radiol. 2023;41:1094–103.
pubmed: 37071250
pmcid: 10543141
doi: 10.1007/s11604-023-01435-w
Chen J, Li K, Peng X, Li L, Yang H, Huang L, et al. A transfer learning approach for staging diagnosis of anterior cruciate ligament injury on a new modified MR dual precision positioning of thin-slice oblique sagittal FS-PDWI sequence. Jpn J Radiol. 2023;41:637–47.
pubmed: 36607553
doi: 10.1007/s11604-022-01385-9
Liu Z, Liu Y, Zhang W, Hong Y, Meng J, Wang J, et al. Deep learning for prediction of hepatocellular carcinoma recurrence after resection or liver transplantation: a discovery and validation study. Hepatol Int. 2022;16:577.
pubmed: 35352293
doi: 10.1007/s12072-022-10321-y
Zeng GL. A deep-network piecewise linear approximation formula. IEEE Access. 2021;9:120665–74.
pubmed: 34532202
pmcid: 8442618
doi: 10.1109/ACCESS.2021.3109173
Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput. 1997;9:1735–80.
pubmed: 9377276
doi: 10.1162/neco.1997.9.8.1735
Peng B, Alcaide E, Anthony Q, Albalak A, Arcadinho S, Cao H, et al. RWKV: reinventing RNNs for the transformer era [Internet]. 2023. Accessed 31 Oct 2023] Available from: http://arxiv.org/abs/2305.13048 .
Brown TB, Mann B, Ryder N, Subbiah M, Kaplan J, Dhariwal P, et al. Language Models are Few-Shot Learners [Internet]. 2020 [Accessed 31 Oct 2023]. Available from: http://arxiv.org/abs/2005.14165 .
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, et al. Attention is all you need [Internet]. 2017 [Accessed 31 Oct 2023]. Available from: http://arxiv.org/abs/1706.03762 .
Jain SM. Introduction to transformers for NLP: with the hugging face library and models to solve problems. Apress; 2022
Ross Gruetzemacher Wichita State University, W. Frank Barton School of Business, David Paradice Auburn University, Harbert College of Business. Deep transfer learning & beyond: transformer language models in information systems research. ACM Comput Surv (CSUR). 2022. https://doi.org/10.1145/3505245 .
doi: 10.1145/3505245
Improving language understanding with unsupervised learning [Internet]. Accessed 31 Oct 2023. Available from: https://openai.com/research/language-unsupervised
Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I. Language Models are unsupervised multitask learners. 2019. Accessed 31 Oct 2023. Available from: https://www.semanticscholar.org/paper/Language-Models-are-Unsupervised-Multitask-Learners-Radford-Wu/9405cc0d6169988371b2755e573cc28650d14dfe
Devlin J, Chang M-W, Lee K, Toutanova K. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding [Internet]. 2018 [Accessed 31 Oct 2023]. Available from: http://arxiv.org/abs/1810.04805
Agrawal A, Suzgun M, Mackey L, Kalai AT. Do language models know when they’re hallucinating references? [Internet]. 2023. Accessed 31 Oct 2023. Available from: http://arxiv.org/abs/2305.18248
Athaluri SA, Manthena SV, Kesapragada VSRKM, Yarlagadda V, Dave T, Duddumpudi RTS. Exploring the boundaries of reality: investigating the phenomenon of artificial intelligence hallucination in scientific writing through ChatGPT references. Cureus. 2023;15:e37432.
pubmed: 37182055
pmcid: 10173677
McKenna N, Li T, Cheng L, Hosseini MJ, Johnson M, Steedman M. Sources of Hallucination by Large Language Models on Inference Tasks [Internet]. 2023 [Accessed 31 Oct 2023]. Available from: http://arxiv.org/abs/2305.14552
Azamfirei R, Kudchadkar SR, Fackler J. Large language models and the perils of their hallucinations. Crit Care. 2023;27:120.
pubmed: 36945051
pmcid: 10032023
doi: 10.1186/s13054-023-04393-x
Thirunavukarasu AJ, Ting DSJ, Elangovan K, Gutierrez L, Tan TF, Ting DSW. Large language models in medicine. Nat Med. 2023;29:1930–40.
pubmed: 37460753
doi: 10.1038/s41591-023-02448-8
Battaglia PW, Hamrick JB, Bapst V, Sanchez-Gonzalez A, Zambaldi V, Malinowski M, et al. Relational inductive biases, deep learning, and graph networks [Internet]. 2018. Accessed Oct 31 2023. Available from: http://arxiv.org/abs/1806.01261
Ueda D, Kakinuma T, Fujita S, Kamagata K, Fushimi Y, Ito R, et al. Fairness of artificial intelligence in healthcare: review and recommendations. Jpn J Radiol. 2023;42:1–13.
Stiennon N, Ouyang L, Wu J, Ziegler DM, Lowe R, Voss C, et al. Learning to summarize from human feedback [Internet]. 2020. Accessed 31 Oct 2023. Available from: http://arxiv.org/abs/2009.01325
Kung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, Elepaño C, et al. Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models. PLOS Digit Health. 2023;2: e0000198.
pubmed: 36812645
pmcid: 9931230
doi: 10.1371/journal.pdig.0000198
Ayers JW, Poliak A, Dredze M, Leas EC, Zhu Z, Kelley JB, et al. Comparing physician and artificial intelligence chatbot responses to patient questions posted to a public social media forum. JAMA Intern Med. 2023;183:589–96.
pubmed: 37115527
doi: 10.1001/jamainternmed.2023.1838
Parikh JR, Wolfman D, Bender CE, Arleo E. Radiologist burnout according to surveyed radiology practice leaders. J Am Coll Radiol. 2020;17:78–81.
pubmed: 31398308
doi: 10.1016/j.jacr.2019.07.008
Bhayana R, Krishna S, Bleakney RR. Performance of ChatGPT on a radiology board-style examination: insights into current strengths and limitations. Radiology. 2023;307: e230582.
pubmed: 37191485
doi: 10.1148/radiol.230582
Toyama Y, Harigai A, Abe M, Nagano M, Kawabata M, Seki Y, et al. Performance evaluation of ChatGPT, GPT-4, and Bard on the official board examination of the Japan Radiology Society. Jpn J Radiol. 2023. https://doi.org/10.1007/s11604-023-01491-2 .
doi: 10.1007/s11604-023-01491-2
pubmed: 37995008
pmcid: 10811006
Kufel J, Paszkiewicz I, Bielówka M, Bartnikowska W, Janik M, Stencel M, et al. Will ChatGPT pass the Polish specialty exam in radiology and diagnostic imaging? Insights into strengths and limitations. Pol J Radiol. 2023;88:e430–4.
pubmed: 37808173
pmcid: 10551734
doi: 10.5114/pjr.2023.131215
Seghier ML. ChatGPT: not all languages are equal. Nature. 2023;615:216.
pubmed: 36882613
doi: 10.1038/d41586-023-00680-3
Akinci D’Antonoli T, Stanzione A, Bluethgen C, Vernuccio F, Ugga L, Klontzas ME, et al. Large language models in radiology: fundamentals, applications, ethical considerations, risks, and future directions. Diagn Interv Radiol. 2023. https://doi.org/10.4274/dir.2023.232417 .
doi: 10.4274/dir.2023.232417
pubmed: 37789676
López-Úbeda P, Martín-Noguerol T, Juluru K, Luna A. Natural language processing in radiology: update on clinical applications. J Am Coll Radiol. 2022;19:1271–85.
pubmed: 36029890
doi: 10.1016/j.jacr.2022.06.016
Tinn R, Cheng H, Gu Y, Usuyama N, Liu X, Naumann T, et al. Fine-tuning large neural language models for biomedical natural language processing. Patterns (N Y). 2023;4: 100729.
pubmed: 37123444
doi: 10.1016/j.patter.2023.100729
Mahbub M, Srinivasan S, Danciu I, Peluso A, Begoli E, Tamang S, et al. Unstructured clinical notes within the 24 hours since admission predict short, mid & long-term mortality in adult ICU patients. PLoS ONE. 2022;17: e0262182.
pubmed: 34990485
pmcid: 8735614
doi: 10.1371/journal.pone.0262182
Gertz RJ, Bunck AC, Lennartz S, Dratsch T, Iuga A-I, Maintz D, et al. GPT-4 for automated determination of radiological study and protocol based on radiology request forms: a feasibility study. Radiology. 2023;307: e230877.
pubmed: 37310247
doi: 10.1148/radiol.230877
Doi K, Takegawa H, Yui M, Anetai Y, Koike Y, Nakamura S, et al. Deep learning-based detection of patients with bone metastasis from Japanese radiology reports. Jpn J Radiol. 2023;41:900–8.
pubmed: 36988827
doi: 10.1007/s11604-023-01413-2
Adams LC, Truhn D, Busch F, Kader A, Niehues SM, Makowski MR, et al. Leveraging GPT-4 for post hoc transformation of free-text radiology reports into structured reporting: a multilingual feasibility study. Radiology. 2023;307: e230725.
pubmed: 37014240
doi: 10.1148/radiol.230725
Lyu Q, Tan J, Zapadka ME, Ponnatapura J, Niu C, Myers KJ, et al. Translating radiology reports into plain language using ChatGPT and GPT-4 with prompt learning: results, limitations, and potential. Vis Comput Ind Biomed Art. 2023;6:9.
pubmed: 37198498
pmcid: 10192466
doi: 10.1186/s42492-023-00136-5
Wang X, Peng Y, Lu L, Lu Z, Summers RM. TieNet: Text-image embedding network for common thorax disease classification and reporting in chest X-rays. 2018 IEEE/CVF conference on computer vision and pattern recognition [Internet]. IEEE; 2018 [Accessed 26 Oct 2023]. Available from: https://ieeexplore.ieee.org/document/8579041/ .
Alfarghaly O, Khaled R, Elkorany A, Helal M, Fahmy A. Automated radiology report generation using conditioned transformers. Inform Med Unlocked. 2021;24: 100557.
doi: 10.1016/j.imu.2021.100557
Sirshar M, Paracha MFK, Akram MU, Alghamdi NS, Zaidi SZY, Fatima T. Attention based automated radiology report generation using CNN and LSTM. PLoS ONE. 2022;17: e0262209.
pubmed: 34990477
pmcid: 8736265
doi: 10.1371/journal.pone.0262209
Nakaura T, Yoshida N, Kobayashi N, Shiraishi K, Nagayama Y, Uetani H, et al. Preliminary assessment of automated radiology report generation with generative pre-trained transformers: comparing results to radiologist-generated reports. Jpn J Radiol. 2023. https://doi.org/10.1007/s11604-023-01487-y .
doi: 10.1007/s11604-023-01487-y
pubmed: 37831317
pmcid: 10764412
Hartung MP, Bickle IC, Gaillard F, Kanne JP. How to create a great radiology report. Radiographics. 2020;40:1658–70.
pubmed: 33001790
doi: 10.1148/rg.2020200020
Else H. Abstracts written by ChatGPT fool scientists. Nature. 2023;613:423–423.
pubmed: 36635510
doi: 10.1038/d41586-023-00056-7
Hwang SI, Lim JS, Lee RW, Matsui Y, Iguchi T, Hiraki T, et al. Is ChatGPT a “Fire of prometheus” for non-native english-speaking researchers in academic writing? Korean J Radiol. 2023;24:952–9.
pubmed: 37793668
pmcid: 10550740
doi: 10.3348/kjr.2023.0773
Liang W, Zhang Y, Cao H, Wang B, Ding D, Yang X, et al. Can large language models provide useful feedback on research papers? A large-scale empirical analysis [Internet]. arXiv.org. 2023. Accessed 27 Oct 2023. Available from: https://arxiv.org/pdf/2310.01783.pdf
Stokel-Walker C. ChatGPT listed as author on research papers: many scientists disapprove. Nature. 2023. https://doi.org/10.1038/d41586-023-00107-z .
doi: 10.1038/d41586-023-00107-z
pubmed: 37857883
Thorp HH. ChatGPT is fun, but not an author. Science. 2023;379:313.
pubmed: 36701446
doi: 10.1126/science.adg7879
Moy L. Guidelines for use of large language models by authors, reviewers, and editors: considerations for imaging journals. Radiology. 2023;309: e239024.
pubmed: 37815449
doi: 10.1148/radiol.239024
Tu T, Azizi S, Driess D, Schaekermann M, Amin M, Chang P-C, et al. Towards generalist Biomedical AI [Internet]. 2023. Accessed 30 Oct 2023. Available from: http://arxiv.org/abs/2307.14334
Khader F, Müller-Franzes G, Wang T, Han T, Tayebi Arasteh S, Haarburger C, et al. Multimodal deep learning for integrating chest radiographs and clinical parameters: a case for transformers. Radiology. 2023;309: e230806.
pubmed: 37787671
doi: 10.1148/radiol.230806
Lake BM, Baroni M. Human-like systematic generalization through a meta-learning neural network. Nature. 2023. https://doi.org/10.1038/s41586-023-06668-3 .
doi: 10.1038/s41586-023-06668-3
pubmed: 37880371
pmcid: 10620072