Artificial Intelligence-Supported Development of Health Guideline Questions.

Journal

Annals of internal medicine
ISSN: 1539-3704
Titre abrégé: Ann Intern Med
Pays: United States
ID NLM: 0372351

Informations de publication

Date de publication:
24 Sep 2024
Historique:
medline: 23 9 2024
pubmed: 23 9 2024
entrez: 23 9 2024
Statut: aheadofprint

Résumé

Guideline questions are typically proposed by experts. To assess how large language models (LLMs) can support the development of guideline questions, providing insights on approaches and lessons learned. Two approaches for guideline question generation were assessed: 1) identification of questions conveyed by online search queries and 2) direct generation of guideline questions by LLMs. For the former, the researchers retrieved popular queries on allergic rhinitis using Google Trends (GT) and identified those conveying questions using both manual and LLM-based methods. They then manually structured as guideline questions the queries that conveyed relevant questions. For the second approach, they tasked an LLM with proposing guideline questions, assuming the role of either a patient or a clinician. Allergic Rhinitis and its Impact on Asthma (ARIA) 2024 guidelines. None. Frequency of relevant questions generated. The authors retrieved 3975 unique queries using GT. From these, they identified 37 questions, of which 22 had not been previously posed by guideline panel members and 2 were eventually prioritized by the panel. Direct interactions with LLMs resulted in the generation of 22 unique relevant questions (11 not previously suggested by panel members), and 4 were eventually prioritized by the panel. In total, 6 of 39 final questions prioritized for the 2024 ARIA guidelines were not initially thought of by the panel. The researchers provide a set of practical insights on the implementation of their approaches based on the lessons learned. Single case study (ARIA guidelines). Approaches using LLMs can support the development of guideline questions, complementing traditional methods and potentially augmenting questions prioritized by guideline panels. Fraunhofer Cluster of Excellence for Immune-Mediated Diseases.

Sections du résumé

BACKGROUND UNASSIGNED
Guideline questions are typically proposed by experts.
OBJECTIVE UNASSIGNED
To assess how large language models (LLMs) can support the development of guideline questions, providing insights on approaches and lessons learned.
DESIGN UNASSIGNED
Two approaches for guideline question generation were assessed: 1) identification of questions conveyed by online search queries and 2) direct generation of guideline questions by LLMs. For the former, the researchers retrieved popular queries on allergic rhinitis using Google Trends (GT) and identified those conveying questions using both manual and LLM-based methods. They then manually structured as guideline questions the queries that conveyed relevant questions. For the second approach, they tasked an LLM with proposing guideline questions, assuming the role of either a patient or a clinician.
SETTING UNASSIGNED
Allergic Rhinitis and its Impact on Asthma (ARIA) 2024 guidelines.
PARTICIPANTS UNASSIGNED
None.
MEASUREMENTS UNASSIGNED
Frequency of relevant questions generated.
RESULTS UNASSIGNED
The authors retrieved 3975 unique queries using GT. From these, they identified 37 questions, of which 22 had not been previously posed by guideline panel members and 2 were eventually prioritized by the panel. Direct interactions with LLMs resulted in the generation of 22 unique relevant questions (11 not previously suggested by panel members), and 4 were eventually prioritized by the panel. In total, 6 of 39 final questions prioritized for the 2024 ARIA guidelines were not initially thought of by the panel. The researchers provide a set of practical insights on the implementation of their approaches based on the lessons learned.
LIMITATION UNASSIGNED
Single case study (ARIA guidelines).
CONCLUSION UNASSIGNED
Approaches using LLMs can support the development of guideline questions, complementing traditional methods and potentially augmenting questions prioritized by guideline panels.
PRIMARY FUNDING SOURCE UNASSIGNED
Fraunhofer Cluster of Excellence for Immune-Mediated Diseases.

Identifiants

DOI: 10.7326/ANNALS-24-00363 PMID: 39312778
pubmed: 39312778
doi: 10.7326/ANNALS-24-00363
doi:

Types de publication

Journal Article

Langues

eng

Sous-ensembles de citation

IM

Auteurs

Bernardo Sousa-Pinto (B)

Department of Community Medicine, Information and Health Decision Sciences, Faculty of Medicine, University of Porto, Porto, Portugal (B.S.-P., R.J.V., M.M.-C., S.G.-M., L.P., M.M., A.M.P.).

Rafael José Vieira (RJ)

Department of Community Medicine, Information and Health Decision Sciences, Faculty of Medicine, University of Porto, Porto, Portugal (B.S.-P., R.J.V., M.M.-C., S.G.-M., L.P., M.M., A.M.P.).
ORCID: 0000-0003-1834-3055

Manuel Marques-Cruz (M)

Department of Community Medicine, Information and Health Decision Sciences, Faculty of Medicine, University of Porto, Porto, Portugal (B.S.-P., R.J.V., M.M.-C., S.G.-M., L.P., M.M., A.M.P.).
ORCID: 0000-0002-9827-2551

Antonio Bognanni (A)

Department of Health Research Methods, Evidence, and Impact and Evidence in Allergy Group, Department of Medicine, McMaster University, Hamilton, Ontario, Canada (A.B.).
ORCID: 0000-0003-0128-903X

Sara Gil-Mata (S)

Department of Community Medicine, Information and Health Decision Sciences, Faculty of Medicine, University of Porto, Porto, Portugal (B.S.-P., R.J.V., M.M.-C., S.G.-M., L.P., M.M., A.M.P.).

Slava Jankin (S)

School of Government and School of Computer Science, University of Birmingham, Birmingham, United Kingdom (S.J.).
ORCID: 0000-0001-6915-177X

Joana Amaro (J)

Epidemiology Research Unit, Institute of Public Health of the University of Porto; Laboratory for Integrative and Translational Research in Population Health; and Department of Medicine, Faculty of Medicine, University of Porto, Porto, Portugal (J.A.).
ORCID: 0000-0003-4499-5524

Liliane Pinheiro (L)

Department of Community Medicine, Information and Health Decision Sciences, Faculty of Medicine, University of Porto, Porto, Portugal (B.S.-P., R.J.V., M.M.-C., S.G.-M., L.P., M.M., A.M.P.).

Marta Mota (M)

Department of Community Medicine, Information and Health Decision Sciences, Faculty of Medicine, University of Porto, Porto, Portugal (B.S.-P., R.J.V., M.M.-C., S.G.-M., L.P., M.M., A.M.P.).

Mattia Giovannini (M)

Allergy Unit, Meyer Children's Hospital IRCCS, and Department of Health Sciences, University of Florence, Florence, Italy (M.G.).
ORCID: 0000-0001-9568-6882

Leticia de Las Vecillas (L)

Department of Allergy, La Paz University Hospital, Madrid, Spain (L.V.).
ORCID: 0000-0003-4969-5678

Ana Margarida Pereira (AM)

Department of Community Medicine, Information and Health Decision Sciences, Faculty of Medicine, University of Porto, Porto, Portugal (B.S.-P., R.J.V., M.M.-C., S.G.-M., L.P., M.M., A.M.P.).
ORCID: 0000-0002-5468-0932

Justyna Lityńska (J)

Evidence Prime, Hamilton, Ontario, Canada (J.L.).

Boleslaw Samolinski (B)

Department of the Prevention of Environmental Hazards, Allergology and Immunology, Medical University of Warsaw, Warsaw, Poland (B.S.).
ORCID: 0000-0002-4043-7747

Jonathan Bernstein (J)

Division of Allergy and Immunology, Cincinnati Children's Hospital Medical Center, and Department of Internal Medicine, Division of Rheumatology, Allergy and Immunology, University of Cincinnati College of Medicine, Cincinnati, Ohio (J.B.).
ORCID: 0000-0002-3476-1196

Mark Dykewicz (M)

Section of Allergy and Immunology, Department of Internal Medicine, Saint Louis University School of Medicine, St. Louis, Missouri (M.D.).
ORCID: 0000-0001-9145-7636

Martin Hofmann-Apitius (M)

Department of Bioinformatics, Fraunhofer Institute for Algorithms and Scientific Computing, Bonn, Germany (M.H.-A., M.J.).
ORCID: 0000-0001-9012-6720

Marc Jacobs (M)

Department of Bioinformatics, Fraunhofer Institute for Algorithms and Scientific Computing, Bonn, Germany (M.H.-A., M.J.).
ORCID: 0000-0003-4153-3930

Nikolaos Papadopoulos (N)

Allergy Department, 2nd Pediatric Clinic, University of Athens, Athens, Greece (N.P.).
ORCID: 0000-0002-4448-3468

Sian Williams (S)

International Primary Care Respiratory Group, London, United Kingdom (S.W.).

Torsten Zuberbier (T)

Institute of Allergology, Charité-Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, and Allergology and Immunology, Fraunhofer Institute for Translational Medicine and Pharmacology, Berlin, Germany (T.Z.).

João A Fonseca (JA)

Department of Community Medicine, Information and Health Decision Sciences, University of Porto, Porto, Portugal (J.A.F.).

Ricardo Cruz-Correia (R)

Department of Community Medicine, Information and Health Decision, Faculty of Medicine, University of Porto, Porto, Portugal (R.C.-C.).
ORCID: 0000-0002-3764-5158

Jean Bousquet (J)

Institute of Allergology, Charité-Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany; Allergology and Immunology, Fraunhofer Institute for Translational Medicine and Pharmacology, Berlin, Germany; and Allergic Rhinitis and its Impact on Asthma, Montpellier, France (J.B.).
ORCID: 0000-0002-4061-4766

Holger J Schünemann (HJ)

Clinical Epidemiology and Research Center, Humanitas University and Research Hospital, Milan, Italy; and Fraunhofer Institute for Translational Medicine and Pharmacology, Allergology and Immunology, Charité, Berlin, Germany (H.J.S.).

Classifications MeSH