VaxBot-HPV: A GPT-based Chatbot for Answering HPV Vaccine-related Questions.
Cervical Cancer
Chatbot
GPT
HPV vaccine
Large Language model
Medical education
QA system
Vaccine
Journal
Research square
ISSN: 2693-5015
Titre abrégé: Res Sq
Pays: United States
ID NLM: 101768035
Informations de publication
Date de publication:
11 Sep 2024
11 Sep 2024
Historique:
pubmed:
24
9
2024
medline:
24
9
2024
entrez:
24
9
2024
Statut:
epublish
Résumé
HPV vaccine is an effective measure to prevent and control the diseases caused by Human Papillomavirus (HPV). This study addresses the development of VaxBot-HPV, a chatbot aimed at improving health literacy and promoting vaccination uptake by providing information and answering questions about the HPV vaccine. We constructed the knowledge base (KB) for VaxBot-HPV, which consists of 451 documents from biomedical literature and web sources on the HPV vaccine. We extracted 202 question-answer pairs from the KB and 39 questions generated by GPT-4 for training and testing purposes. To comprehensively understand the capabilities and potential of GPT-based chatbots, three models were involved in this study : GPT-3.5, VaxBot-HPV, and GPT-4. The evaluation criteria included answer relevancy and faithfulness. VaxBot-HPV demonstrated superior performance in answer relevancy and faithfulness compared to baselines (Answer relevancy: 0.85; Faithfulness: 0.97) for the test questions in KB, (Answer relevancy: 0.85; Faithfulness: 0.96) for GPT generated questions. This study underscores the importance of leveraging advanced language models and fine-tuning techniques in the development of chatbots for healthcare applications, with implications for improving medical education and public health communication.
Sections du résumé
Background
UNASSIGNED
HPV vaccine is an effective measure to prevent and control the diseases caused by Human Papillomavirus (HPV). This study addresses the development of VaxBot-HPV, a chatbot aimed at improving health literacy and promoting vaccination uptake by providing information and answering questions about the HPV vaccine.
Methods
UNASSIGNED
We constructed the knowledge base (KB) for VaxBot-HPV, which consists of 451 documents from biomedical literature and web sources on the HPV vaccine. We extracted 202 question-answer pairs from the KB and 39 questions generated by GPT-4 for training and testing purposes. To comprehensively understand the capabilities and potential of GPT-based chatbots, three models were involved in this study : GPT-3.5, VaxBot-HPV, and GPT-4. The evaluation criteria included answer relevancy and faithfulness.
Results
UNASSIGNED
VaxBot-HPV demonstrated superior performance in answer relevancy and faithfulness compared to baselines (Answer relevancy: 0.85; Faithfulness: 0.97) for the test questions in KB, (Answer relevancy: 0.85; Faithfulness: 0.96) for GPT generated questions.
Conclusions
UNASSIGNED
This study underscores the importance of leveraging advanced language models and fine-tuning techniques in the development of chatbots for healthcare applications, with implications for improving medical education and public health communication.
Identifiants
pubmed: 39315262
doi: 10.21203/rs.3.rs-4876692/v1
pmc: PMC11419187
pii:
doi:
Types de publication
Journal Article
Preprint
Langues
eng
Subventions
Organisme : NIAID NIH HHS
ID : R01 AI130460
Pays : United States
Organisme : NIAID NIH HHS
ID : U24 AI171008
Pays : United States
Déclaration de conflit d'intérêts
Declarations Competing Interests: The authors declare no conflicts of interest.
Références
Vaccine. 2020 Aug 27;38(38):6027-6037
pubmed: 32758380
Sci Rep. 2024 Feb 19;14(1):4053
pubmed: 38374354
J Infect. 2013 Mar;66(3):207-17
pubmed: 23103285
Cancer Control. 2024 Jan-Dec;31:10732748241238629
pubmed: 38462683
Adv Exp Med Biol. 2021;1313:1-14
pubmed: 34661888
Cancer Prev Res (Phila). 2012 Jan;5(1):18-23
pubmed: 22219162
Oncol Lett. 2020 Sep;20(3):2058-2074
pubmed: 32782524
Expert Rev Vaccines. 2024 Jan-Dec;23(1):53-59
pubmed: 38063069
J Healthc Inform Res. 2024 Feb 29;8(2):206-224
pubmed: 38681754
Cancers (Basel). 2024 Feb 05;16(3):
pubmed: 38339423
Vaccine. 2021 Apr 22;39(17):2416-2423
pubmed: 33775438
J Am Med Inform Assoc. 2024 Aug 29;:
pubmed: 39208311
New Microbiol. 2017 Apr;40(2):80-85
pubmed: 28368072
BMC Public Health. 2023 Apr 1;23(1):628
pubmed: 37005583
Gynecol Oncol. 2024 Feb;181:102-109
pubmed: 38150834
An Bras Dermatol. 2011 Mar-Apr;86(2):306-17
pubmed: 21603814
Vaccines (Basel). 2021 Jun 03;9(6):
pubmed: 34204971
J Obstet Gynaecol. 2020 Jul;40(5):602-608
pubmed: 31500479
Ann Oncol. 2011 Dec;22(12):2675-2686
pubmed: 21471563
J Biomed Semantics. 2024 Aug 10;15(1):14
pubmed: 39123237
Ann Biomed Eng. 2024 Aug;52(8):1928-1931
pubmed: 38310159
AMIA Annu Symp Proc. 2011;2011:171-80
pubmed: 22195068
Lancet Glob Health. 2020 Feb;8(2):e191-e203
pubmed: 31812369
PLoS One. 2024 Mar 21;19(3):e0300919
pubmed: 38512919
J Biomed Inform. 2024 Apr;152:104621
pubmed: 38447600