Limitations of Deep Learning Attention Mechanisms in Clinical Research: Empirical Case Study Based on the Korean Diabetic Disease Setting.
Bayesian deep learning
artificial intelligence
attention
deep learning
explainable artificial intelligence
health data
uncertainty awareness
Journal
Journal of medical Internet research
ISSN: 1438-8871
Titre abrégé: J Med Internet Res
Pays: Canada
ID NLM: 100959882
Informations de publication
Date de publication:
16 12 2020
16 12 2020
Historique:
received:
25
02
2020
accepted:
11
11
2020
revised:
13
09
2020
entrez:
16
12
2020
pubmed:
17
12
2020
medline:
16
3
2021
Statut:
epublish
Résumé
Despite excellent prediction performance, noninterpretability has undermined the value of applying deep-learning algorithms in clinical practice. To overcome this limitation, attention mechanism has been introduced to clinical research as an explanatory modeling method. However, potential limitations of using this attractive method have not been clarified to clinical researchers. Furthermore, there has been a lack of introductory information explaining attention mechanisms to clinical researchers. The aim of this study was to introduce the basic concepts and design approaches of attention mechanisms. In addition, we aimed to empirically assess the potential limitations of current attention mechanisms in terms of prediction and interpretability performance. First, the basic concepts and several key considerations regarding attention mechanisms were identified. Second, four approaches to attention mechanisms were suggested according to a two-dimensional framework based on the degrees of freedom and uncertainty awareness. Third, the prediction performance, probability reliability, concentration of variable importance, consistency of attention results, and generalizability of attention results to conventional statistics were assessed in the diabetic classification modeling setting. Fourth, the potential limitations of attention mechanisms were considered. Prediction performance was very high for all models. Probability reliability was high in models with uncertainty awareness. Variable importance was concentrated in several variables when uncertainty awareness was not considered. The consistency of attention results was high when uncertainty awareness was considered. The generalizability of attention results to conventional statistics was poor regardless of the modeling approach. The attention mechanism is an attractive technique with potential to be very promising in the future. However, it may not yet be desirable to rely on this method to assess variable importance in clinical settings. Therefore, along with theoretical studies enhancing attention mechanisms, more empirical studies investigating potential limitations should be encouraged.
Sections du résumé
BACKGROUND
Despite excellent prediction performance, noninterpretability has undermined the value of applying deep-learning algorithms in clinical practice. To overcome this limitation, attention mechanism has been introduced to clinical research as an explanatory modeling method. However, potential limitations of using this attractive method have not been clarified to clinical researchers. Furthermore, there has been a lack of introductory information explaining attention mechanisms to clinical researchers.
OBJECTIVE
The aim of this study was to introduce the basic concepts and design approaches of attention mechanisms. In addition, we aimed to empirically assess the potential limitations of current attention mechanisms in terms of prediction and interpretability performance.
METHODS
First, the basic concepts and several key considerations regarding attention mechanisms were identified. Second, four approaches to attention mechanisms were suggested according to a two-dimensional framework based on the degrees of freedom and uncertainty awareness. Third, the prediction performance, probability reliability, concentration of variable importance, consistency of attention results, and generalizability of attention results to conventional statistics were assessed in the diabetic classification modeling setting. Fourth, the potential limitations of attention mechanisms were considered.
RESULTS
Prediction performance was very high for all models. Probability reliability was high in models with uncertainty awareness. Variable importance was concentrated in several variables when uncertainty awareness was not considered. The consistency of attention results was high when uncertainty awareness was considered. The generalizability of attention results to conventional statistics was poor regardless of the modeling approach.
CONCLUSIONS
The attention mechanism is an attractive technique with potential to be very promising in the future. However, it may not yet be desirable to rely on this method to assess variable importance in clinical settings. Therefore, along with theoretical studies enhancing attention mechanisms, more empirical studies investigating potential limitations should be encouraged.
Identifiants
pubmed: 33325832
pii: v22i12e18418
doi: 10.2196/18418
pmc: PMC7773508
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
e18418Informations de copyright
©Junetae Kim, Sangwon Lee, Eugene Hwang, Kwang Sun Ryu, Hanseok Jeong, Jae Wook Lee, Yul Hwangbo, Kui Son Choi, Hyo Soung Cha. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 16.12.2020.
Références
BMJ Open. 2014 Aug 19;4(8):e005710
pubmed: 25138810
JAMA Netw Open. 2019 Jul 3;2(7):e197700
pubmed: 31348505
JAMA. 2017 Dec 12;318(22):2199-2210
pubmed: 29234806
Cell. 2014 Aug 28;158(5):1199-1209
pubmed: 25171417
JAMA Netw Open. 2019 Mar 1;2(3):e191095
pubmed: 30901052
JAMA Netw Open. 2019 Nov 1;2(11):e1914645
pubmed: 31693124
Invest Ophthalmol Vis Sci. 2013 Oct 17;54(10):6827-33
pubmed: 24065813
Med Image Anal. 2019 Apr;53:197-207
pubmed: 30802813
Arch Intern Med. 1981 May;141(6):747-9
pubmed: 7235783
Cell. 2017 Feb 9;168(4):584-599
pubmed: 28187282
JAMA Dermatol. 2019 Sep 4;:
pubmed: 31483437
Diabetes Metab J. 2011 Dec;35(6):587-94
pubmed: 22247901
PLoS One. 2019 Feb 13;14(2):e0211057
pubmed: 30759094
Annu Rev Med. 1980;31:29-34
pubmed: 6994614
Int J Epidemiol. 2014 Feb;43(1):69-77
pubmed: 24585853
Diabetes Care. 2012 Dec;35(12):2650-64
pubmed: 23100048
JAMA Netw Open. 2019 Oct 2;2(10):e1914051
pubmed: 31651969
BMC Med Inform Decis Mak. 2019 Dec 5;19(Suppl 5):235
pubmed: 31801540
Ann Transl Med. 2018 Jun;6(11):216
pubmed: 30023379
Bioinformatics. 2003 Jan;19(1):90-7
pubmed: 12499298
Endocr J. 2014;61(1):61-70
pubmed: 24088600
Health Serv Res. 2007 Aug;42(4):1718-38
pubmed: 17610445
Sci Rep. 2020 Apr 28;10(1):7155
pubmed: 32346050
JAMA Netw Open. 2018 Oct 5;1(6):e183451
pubmed: 30646240
Genetics. 2013 Oct;195(2):573-87
pubmed: 23934883
Diabetes Metab J. 2014 Apr;38(2):109-19
pubmed: 24851205
Radiology. 1982 Apr;143(1):29-36
pubmed: 7063747
JAMA Netw Open. 2019 Mar 1;2(3):e190606
pubmed: 30874779
Bioinformatics. 2020 Sep 25;:
pubmed: 32976559