A conversational agent system for dietary supplements use.
Conversational agent
Deep learning
Dietary supplements
Named entity recognition
Natural language processing
Question answering
Journal
BMC medical informatics and decision making
ISSN: 1472-6947
Titre abrégé: BMC Med Inform Decis Mak
Pays: England
ID NLM: 101088682
Informations de publication
Date de publication:
07 07 2022
07 07 2022
Historique:
received:
27
04
2022
accepted:
23
05
2022
entrez:
7
7
2022
pubmed:
8
7
2022
medline:
12
7
2022
Statut:
epublish
Résumé
Dietary supplements (DS) have been widely used by consumers, but the information around the efficacy and safety of DS is disparate or incomplete, thus creating barriers for consumers to find information effectively. Conversational agent (CA) systems have been applied to healthcare domain, but there is no such system to answer consumers regarding DS use, although widespread use of DS. In this study, we develop the first CA system for DS use. Our CA system for DS use developed on the MindMeld framework, consists of three components: question understanding, DS knowledge base, and answer generation. We collected and annotated 1509 questions to develop a natural language understanding module (e.g., question type classifier, named entity recognizer) which was then integrated into MindMeld framework. CA then queries the DS knowledge base (i.e., iDISK) and generates answers using rule-based slot filling techniques. We evaluated the algorithms of each component and the CA system as a whole. CNN is the best question classifier with an F1 score of 0.81, and CRF is the best named entity recognizer with an F1 score of 0.87. The system achieves an overall accuracy of 81% and an average score of 1.82 with succ@3 + score of 76.2% and succ@2 + of 66% approximately. This study develops the first CA system for DS use using the MindMeld framework and iDISK domain knowledge base.
Sections du résumé
BACKGROUND
Dietary supplements (DS) have been widely used by consumers, but the information around the efficacy and safety of DS is disparate or incomplete, thus creating barriers for consumers to find information effectively. Conversational agent (CA) systems have been applied to healthcare domain, but there is no such system to answer consumers regarding DS use, although widespread use of DS. In this study, we develop the first CA system for DS use.
METHODS
Our CA system for DS use developed on the MindMeld framework, consists of three components: question understanding, DS knowledge base, and answer generation. We collected and annotated 1509 questions to develop a natural language understanding module (e.g., question type classifier, named entity recognizer) which was then integrated into MindMeld framework. CA then queries the DS knowledge base (i.e., iDISK) and generates answers using rule-based slot filling techniques. We evaluated the algorithms of each component and the CA system as a whole.
RESULTS
CNN is the best question classifier with an F1 score of 0.81, and CRF is the best named entity recognizer with an F1 score of 0.87. The system achieves an overall accuracy of 81% and an average score of 1.82 with succ@3 + score of 76.2% and succ@2 + of 66% approximately.
CONCLUSION
This study develops the first CA system for DS use using the MindMeld framework and iDISK domain knowledge base.
Identifiants
pubmed: 35799177
doi: 10.1186/s12911-022-01888-5
pii: 10.1186/s12911-022-01888-5
pmc: PMC9264487
doi:
Types de publication
Journal Article
Research Support, N.I.H., Extramural
Langues
eng
Sous-ensembles de citation
IM
Pagination
153Subventions
Organisme : NCCIH NIH HHS
ID : R01 AT009457
Pays : United States
Informations de copyright
© 2022. The Author(s).
Références
J Am Med Inform Assoc. 2016 Jul;23(4):802-11
pubmed: 27147494
Biochem Med (Zagreb). 2012;22(3):276-82
pubmed: 23092060
J Biomed Inform. 2011 Dec;44(6):1032-8
pubmed: 21856442
J Am Med Inform Assoc. 2020 Apr 1;27(4):539-548
pubmed: 32068839
Lancet Psychiatry. 2018 Jul;5(7):547
pubmed: 29941139
J Med Internet Res. 2019 Nov 7;21(11):e15360
pubmed: 31697237
J Am Med Inform Assoc. 2020 Feb 1;27(2):194-201
pubmed: 31592532
J Am Med Inform Assoc. 2017 May 01;24(3):596-606
pubmed: 28040687