Could generative artificial intelligence replace fieldwork in pain research?
acute pain
artificial intelligence
climbing
generative pretrained transformer
Journal
Scandinavian journal of pain
ISSN: 1877-8879
Titre abrégé: Scand J Pain
Pays: Germany
ID NLM: 101520867
Informations de publication
Date de publication:
01 Jan 2024
01 Jan 2024
Historique:
received:
22
11
2023
accepted:
11
12
2023
medline:
7
3
2024
pubmed:
7
3
2024
entrez:
7
3
2024
Statut:
epublish
Résumé
Generative artificial intelligence (AI) models offer potential assistance in pain research data acquisition, yet concerns persist regarding data accuracy and reliability. In a comparative study, we evaluated open generative AI models' capacity to acquire data on acute pain in rock climbers comparable to field research. Fifty-two rock climbers (33 m/19 f; age 29.0 [24.0-35.75] years) were asked to report pain location and intensity during a single climbing session. Five generative pretrained transformer models were tasked with responses to the same questions. Climbers identified the back of the forearm (19.2%) and toes (17.3%) as primary pain sites, with reported median pain intensity at 4 [3-5] and median maximum pain intensity at 7 [5-8]. Conversely, AI models yielded divergent findings, indicating fingers, hands, shoulders, legs, and feet as primary pain localizations with average and maximum pain intensity ranging from 3 to 4.4 and 5 to 10, respectively. Only two AI models provided references that were untraceable in PubMed and Google searches. Our findings reveal that, currently, open generative AI models cannot match the quality of field-collected data on acute pain in rock climbers. Moreover, the models generated nonexistent references, raising concerns about their reliability.
Sections du résumé
BACKGROUND
BACKGROUND
Generative artificial intelligence (AI) models offer potential assistance in pain research data acquisition, yet concerns persist regarding data accuracy and reliability. In a comparative study, we evaluated open generative AI models' capacity to acquire data on acute pain in rock climbers comparable to field research.
METHODS
METHODS
Fifty-two rock climbers (33 m/19 f; age 29.0 [24.0-35.75] years) were asked to report pain location and intensity during a single climbing session. Five generative pretrained transformer models were tasked with responses to the same questions.
RESULTS
RESULTS
Climbers identified the back of the forearm (19.2%) and toes (17.3%) as primary pain sites, with reported median pain intensity at 4 [3-5] and median maximum pain intensity at 7 [5-8]. Conversely, AI models yielded divergent findings, indicating fingers, hands, shoulders, legs, and feet as primary pain localizations with average and maximum pain intensity ranging from 3 to 4.4 and 5 to 10, respectively. Only two AI models provided references that were untraceable in PubMed and Google searches.
CONCLUSION
CONCLUSIONS
Our findings reveal that, currently, open generative AI models cannot match the quality of field-collected data on acute pain in rock climbers. Moreover, the models generated nonexistent references, raising concerns about their reliability.
Identifiants
pubmed: 38452184
pii: sjpain-2023-0136
doi: 10.1515/sjpain-2023-0136
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Informations de copyright
© 2024 the author(s), published by De Gruyter.
Références
Stokel-Walker C, Van Noorden R. ChatGPT and generative AI’s impact on science. Nature. 2023;614(7947):214–6. 10.1038/d41586-023-00340-6 .
doi: 10.1038/d41586-023-00340-6
pubmed: 36747115
Schedlbauer J, Raptis G, Ludwig B. Web crawling, web scraping, and text mining for medical informatics labor market analysis. Int J Med Inform. 2021;150:104453. 10.1016/j.ijmedinf.2021.104453 .
doi: 10.1016/j.ijmedinf.2021.104453
pubmed: 33862508
Borji A. A categorical archive of ChatGPT failures. ArXiv; 2023. 10.48550/arXiv.2302.03494 .
doi: 10.48550/arXiv.2302.03494
Spitale G, Biller-Andorno N, Germani F. GPT-3 AI model (dis)informs better than humans. Sci Adv. 2023;9(26):eadh1850. 10.1126/sciadv.adh1850 .
doi: 10.1126/sciadv.adh1850
pubmed: 37379395
pmcid: PMC10306283
Moskatel LS, Zhang N. An observational, qualitative study on the utility of ChatGPT in assessing literature on migraine prevention. Front Neurol. 2023;14:1225223. 10.3389/fneur.2023.1225223 .
doi: 10.3389/fneur.2023.1225223
pubmed: 37662036
pmcid: PMC10469750