Artificial intelligence and clinical guidance in male reproductive health: ChatGPT4.0's AUA/ASRM guideline compliance evaluation.
AUA/ASRM guideline
ChatGPT4.0
artificial intelligence
male infertility
Journal
Andrology
ISSN: 2047-2927
Titre abrégé: Andrology
Pays: England
ID NLM: 101585129
Informations de publication
Date de publication:
17 Jul 2024
17 Jul 2024
Historique:
revised:
25
06
2024
received:
25
05
2024
accepted:
27
06
2024
medline:
17
7
2024
pubmed:
17
7
2024
entrez:
17
7
2024
Statut:
aheadofprint
Résumé
Male infertility is defined as the inability of a male to achieve a pregnancy in a fertile female by the American Urological Association (AUA) and the American Society for Reproductive Medicine (ASRM). Artificial intelligence, particularly in language processing models like ChatGPT4.0, offers new possibilities for supporting clinical decision-making. This study aims to assess the effectiveness of ChatGPT4.0 in responding to clinical queries regarding male infertility, which is aligned with AUA/ASRM guidelines. This observational study employed a design to evaluate the performance of ChatGPT4.0 across 1073 structured clinical queries categorized into true/false, multiple-choice, and open-ended. Two independent reviewers specializing in reproductive medicine assessed the responses using a six-point Likert scale to evaluate accuracy, relevance, and guideline adherence. In the true/false category, the initial accuracy was 92%, which increased to 94% by the end of the study period. For multiple-choice questions, accuracy improved from 85% to 89%. The most significant gains were seen in open-ended questions, where accuracy rose from 78% to 86%. Initially, some responses did not fully align with the AUA/ASRM guidelines. However, by the end of the 60 days, these responses had become more comprehensive and clinically relevant, indicating an improvement in the model's ability to generate guideline-conformant answers (p < 0.05). The depth and accuracy of responses for higher difficulty questions also showed enhancement (p < 0.01). ChatGPT4.0 can serve as a valuable support tool in managing male infertility, providing reliable, guideline-based information that enhances the accuracy of clinical decision-making tools and supports patient education.
Sections du résumé
BACKGROUND
BACKGROUND
Male infertility is defined as the inability of a male to achieve a pregnancy in a fertile female by the American Urological Association (AUA) and the American Society for Reproductive Medicine (ASRM). Artificial intelligence, particularly in language processing models like ChatGPT4.0, offers new possibilities for supporting clinical decision-making. This study aims to assess the effectiveness of ChatGPT4.0 in responding to clinical queries regarding male infertility, which is aligned with AUA/ASRM guidelines.
METHODS
METHODS
This observational study employed a design to evaluate the performance of ChatGPT4.0 across 1073 structured clinical queries categorized into true/false, multiple-choice, and open-ended. Two independent reviewers specializing in reproductive medicine assessed the responses using a six-point Likert scale to evaluate accuracy, relevance, and guideline adherence.
RESULTS
RESULTS
In the true/false category, the initial accuracy was 92%, which increased to 94% by the end of the study period. For multiple-choice questions, accuracy improved from 85% to 89%. The most significant gains were seen in open-ended questions, where accuracy rose from 78% to 86%. Initially, some responses did not fully align with the AUA/ASRM guidelines. However, by the end of the 60 days, these responses had become more comprehensive and clinically relevant, indicating an improvement in the model's ability to generate guideline-conformant answers (p < 0.05). The depth and accuracy of responses for higher difficulty questions also showed enhancement (p < 0.01).
CONCLUSION
CONCLUSIONS
ChatGPT4.0 can serve as a valuable support tool in managing male infertility, providing reliable, guideline-based information that enhances the accuracy of clinical decision-making tools and supports patient education.
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Informations de copyright
© 2024 American Society of Andrology and European Academy of Andrology.
Références
Amor H, Hammadeh ME. A systematic review of the impact of mitochondrial variations on male infertility. Genes (Basel). 2022;13(7):1182.
Mazzilli R, Vaiarelli A, Dovere L, et al. Severe male factor in in vitro fertilization: definition, prevalence, and treatment. An update. Asian J Androl. 2022;24(2):125‐134.
Turner KA, Rambhatla A, Schon S, et al. Male infertility is a women's health issue‐research and clinical evaluation of male infertility is needed. Cells. 2020;9(4):990.
Mazzilli R, Rucci C, Vaiarelli A, et al. Male factor infertility and assisted reproductive technologies: indications, minimum access criteria and outcomes. J Endocrinol Invest. 2023;46(6):1079‐1085.
Honig SC, Lipshultz LI, Jarow J. Significant medical pathology uncovered by a comprehensive male infertility evaluation. Fertil Steril. 1994;62(5):1028‐1034.
Kolettis PN, Sabanegh ES. Significant medical pathology discovered during a male infertility evaluation. J Urol. 2001;166(1):178‐180.
Schlegel PN, Sigman M, Collura B, et al. Diagnosis and treatment of infertility in men: AUA/ASRM guideline part II. Fertil Steril. 2021;115(1):62‐69.
Schlegel PN, Sigman M, Collura B, et al. Diagnosis and treatment of infertility in men: AUA/ASRM guideline part I. J Urol. 2021;205(1):36‐43.
Blumenfeld Z. What is the best regimen for ovarian stimulation of poor responders in ART/IVF. Front Endocrinol (Lausanne). 2020;11:192.
Yurchuk T, Petrushko M, Fuller B. State of the art in assisted reproductive technologies for patients with advanced maternal age. Zygote. 2023;31(2):149‐156.
Chiware TM, Vermeulen N, Blondeel K, et al. IVF and other ART in low‐ and middle‐income countries: a systematic landscape analysis. Hum Reprod Update. 2021;27(2):213‐228.
Medenica S, Zivanovic D, Batkoska L, et al. The future is coming: artificial intelligence in the treatment of infertility could improve assisted reproduction outcomes—the value of regulatory frameworks. Diagnostics (Basel). 2022;12(12):2979.
Ghayda RA, Cannarella R, Calogero AE, et al. Artificial intelligence in andrology: from semen analysis to image diagnostics. World J Mens Health. 2024;42(1):39‐61.
Coskun B, Ocakoglu G, Yetemen M, Kaygisiz O. Can ChatGPT, an artificial intelligence language model, provide accurate and high‐quality patient information on prostate cancer. Urology. 2023;180:35‐58.
Cil G, Dogan K. The efficacy of artificial intelligence in urology: a detailed analysis of kidney stone‐related queries. World J Urol. 2024;42(1):158.
Cakir H, Caglar U, Yildiz O, Meric A, Ayranci A, Ozgor F. Evaluating the performance of ChatGPT in answering questions related to urolithiasis. Int Urol Nephrol. 2024;56(1):17‐21.
Aksakalli T, Aksakalli IK, Cinislioglu AE, et al. Prediction of spontaneous distal ureteral stone passage using artificial intelligence. Int Urol Nephrol. 2024;56(7):2179‐2186.
Anastasiadis A, Koudonas A, Langas G, et al. Transforming urinary stone disease management by artificial intelligence‐based methods: a comprehensive review. Asian J Urol. 2023;10(3):258‐274.
Bach PV, Patel N, Najari BB, et al. Changes in practice patterns in male infertility cases in the United States: the trend toward subspecialization. Fertil Steril. 2018;110(1):76‐82.
Allahqoli L, Ghiasvand MM, Mazidimoradi A, Salehiniya H, Alkatout I. Diagnostic and management performance of ChatGPT in obstetrics and gynecology. Gynecol Obstet Invest. 2023;88(5):310‐313.
Suhag A, Kidd J, McGath M, et al. ChatGPT: a pioneering approach to complex prenatal differential diagnosis. Am J Obstet Gynecol Maternal‐Fetal Med. 2023;5(8):101029.
Brochu F, Robins S, Miner SA. Searching the internet for infertility information: a survey of patient needs and preferences. J Med Internet Res. 2019;21(12):e15132.
Lensen S, Chen S, Goodman L. IVF add‐ons in Australia and New Zealand: a systematic assessment of IVF clinic websites. Aust N Z J Obstet Gynaecol. 2021;61:430.
Jia X, Pang Y, Liu LS. Online health information seeking behavior: a systematic review. Healthcare (Basel). 2021;9:1740.
Biswas SS. Role of ChatGPT in public health. Ann Biomed Eng. 2023;51:868.
Beilby K, Dudink I, Kablar D. The quality of information about elective oocyte cryopreservation (EOC) on Australian fertility clinic websites. Aust N Z J Obstet Gynaecol. 2020;60:605.
Copp T, Nickel B, Lensen S. Anti‐Mullerian hormone (AMH) test information on Australian and New Zealand fertility clinic websites: a content analysis. BMJ Open. 2021;11:e046927.
Ayoub NF, Lee YJ, Grimm D. Comparison between ChatGPT and Google search as sources of postoperative patient instructions. JAMA Otolaryngol Head Neck Surg. 2023;149:556.
Deniz MS, Guler BY. Assessment of ChatGPT's adherence to ETA‐thyroid nodule management guideline over two different time intervals 14 days apart: in binary and multiple‐choice queries. Endocrine. 2024;84:312‐327.
Caterson J, Ambler O, Cereceda‐Monteoliva N, Horner M, Jones A, Poacher AT. Application of generative language models to orthopaedic practice. BMJ Open. 2024;14(3):e076484.
Ferres JML, Weeks WB, Chu LC, Rowe SP, Fishman EK. Beyond chatting: the opportunities and challenges of ChatGPT in medicine and radiology. Diagn Interv Imaging. 2023;104(6):263‐264.
Stevenson E, Walsh C, Hibberd L. Can artificial intelligence replace biochemists? A study comparing interpretation of thyroid function test results by ChatGPT and Google Bard to practising biochemists. Ann Clin Biochem. 2023;61(2):143‐149.