Interpretation of cluster structures in pain-related phenotype data using explainable artificial intelligence (XAI).
Journal
European journal of pain (London, England)
ISSN: 1532-2149
Titre abrégé: Eur J Pain
Pays: England
ID NLM: 9801774
Informations de publication
Date de publication:
02 2021
02 2021
Historique:
received:
07
04
2020
revised:
08
10
2020
accepted:
14
10
2020
pubmed:
17
10
2020
medline:
28
4
2021
entrez:
16
10
2020
Statut:
ppublish
Résumé
In pain research and clinics, it is common practice to subgroup subjects according to shared pain characteristics. This is often achieved by computer-aided clustering. In response to a recent EU recommendation that computer-aided decision making should be transparent, we propose an approach that uses machine learning to provide (1) an understandable interpretation of a cluster structure to (2) enable a transparent decision process about why a person concerned is placed in a particular cluster. Comprehensibility was achieved by transforming the interpretation problem into a classification problem: A sub-symbolic algorithm was used to estimate the importance of each pain measure for cluster assignment, followed by an item categorization technique to select the relevant variables. Subsequently, a symbolic algorithm as explainable artificial intelligence (XAI) provided understandable rules of cluster assignment. The approach was tested using 100-fold cross-validation. The importance of the variables of the data set (6 pain-related characteristics of 82 healthy subjects) changed with the clustering scenarios. The highest median accuracy was achieved by sub-symbolic classifiers. A generalized post-hoc interpretation of clustering strategies of the model led to a loss of median accuracy. XAI models were able to interpret the cluster structure almost as correctly, but with a slight loss of accuracy. Assessing the variables importance in clustering is important for understanding any cluster structure. XAI models are able to provide a human-understandable interpretation of the cluster structure. Model selection must be adapted individually to the clustering problem. The advantage of comprehensibility comes at an expense of accuracy.
Sections du résumé
BACKGROUND
In pain research and clinics, it is common practice to subgroup subjects according to shared pain characteristics. This is often achieved by computer-aided clustering. In response to a recent EU recommendation that computer-aided decision making should be transparent, we propose an approach that uses machine learning to provide (1) an understandable interpretation of a cluster structure to (2) enable a transparent decision process about why a person concerned is placed in a particular cluster.
METHODS
Comprehensibility was achieved by transforming the interpretation problem into a classification problem: A sub-symbolic algorithm was used to estimate the importance of each pain measure for cluster assignment, followed by an item categorization technique to select the relevant variables. Subsequently, a symbolic algorithm as explainable artificial intelligence (XAI) provided understandable rules of cluster assignment. The approach was tested using 100-fold cross-validation.
RESULTS
The importance of the variables of the data set (6 pain-related characteristics of 82 healthy subjects) changed with the clustering scenarios. The highest median accuracy was achieved by sub-symbolic classifiers. A generalized post-hoc interpretation of clustering strategies of the model led to a loss of median accuracy. XAI models were able to interpret the cluster structure almost as correctly, but with a slight loss of accuracy.
CONCLUSIONS
Assessing the variables importance in clustering is important for understanding any cluster structure. XAI models are able to provide a human-understandable interpretation of the cluster structure. Model selection must be adapted individually to the clustering problem. The advantage of comprehensibility comes at an expense of accuracy.
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
442-465Informations de copyright
© 2020 The Authors. European Journal of Pain published by John Wiley & Sons Ltd on behalf of European Pain Federation - EFIC®.
Références
Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19, 716-723. https://doi.org/10.1109/TAC.1974.1100705
Altman, D. G., & Bland, J. M. (1994a). Diagnostic tests 2: Predictive values. BMJ, 309, 102.
Altman, D. G., & Bland, J. M. (1994b). Diagnostic tests. 1: Sensitivity and specificity. BMJ, 308, 1552.
Anderson, E. (1935). The irises of the Gaspé peninsula. Bulletin of the American Iris Society, 59, 2-5.
Arnold, J. B. (2019). ggthemes: Extra Themes, Scales and Geoms for 'ggplot2'.
Arrieta, A. B., Díaz-Rodríguez, N., Ser, J. D., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R. et al (2019). Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI.
Badih, G., Pierre, M., & Laurent, B. (2019). Assessing variable importance in clustering: A new method based on unsupervised binary decision trees. Computational Statistics, 34, 301-321. https://doi.org/10.1007/s00180-018-0857-0
Bayes, M., & Price, M. (1763). An Essay towards Solving a Problem in the Doctrine of Chances. By the Late Rev. Mr. Bayes, F. R. S. Communicated by Mr. Price, in a Letter to John Canton, A. M. F. R. S. Philosophical Transactions 53, 370-418.
Bezdek, J. C., Keller, J. M., Krishnapuram, R., Kuncheva, L. I., & Pal, N. R. (1999). Will the real iris data please stand up? IEEE Transactions on Fuzzy Systems, 7, 368-369. https://doi.org/10.1109/91.771092
Bonferroni, C. E. (1936). Teoria statistica delle classi e calcolo delle probabilita. Pubblicazioni Del R Istituto Superiore Di Scienze Economiche E Commerciali Di Firenze, 8, 3-62.
Breiman, L. (2001). Random forests. Machine Learning, 45, 5-32.
Breimann, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1993). Classification and regression trees. Chapman and Hall.
Brodersen, K. H., Ong, C. S., Stephan, K. E., & Buhmann, J. M. (2010). The Balanced Accuracy and Its Posterior Distribution. In Pattern Recognition (ICPR), 2010 20th International Conference on, pp. 3121-3124.
Callsen, M. G., Moller, A. T., Sorensen, K., Jensen, T. S., & Finnerup, N. B. (2008). Cold hyposensitivity after topical application of capsaicin in humans. Experimental Brain Research, 191, 447-452. https://doi.org/10.1007/s00221-008-1535-1
Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155-159. https://doi.org/10.1037/0033-2909.112.1.155
Cohen, W. W. (1995). Fast effective rule induction. In ICML.
Dhar, V. (2013). Data science and prediction. Communications of the ACM, 56, 64-73. https://doi.org/10.1145/2500499
Diatchenko, L., Slade, G. D., Nackley, A. G., Bhalang, K., Sigurdsson, A., Belfer, I., Goldman, D., Xu, K. E., Shabalina, S. A., Shagin, D., Max, M. B., Makarov, S. S., & Maixner, W. (2005). Genetic basis for individual variations in pain perception and the development of a chronic pain condition. Human Molecular Genetics, 14, 135-143. https://doi.org/10.1093/hmg/ddi013
Dimova, V., Oertel, B. G., & Lötsch, J. (2016). Using a standardized clinical quantitative sensory testing battery to judge the clinical relevance of sensory differences between adjacent body areas. Clinical Journal of Pain. 33(1), 37-43.
Doehring, A., Küsener, N., Flühr, K., Neddermeyer, T. J., Schneider, G., & Lötsch, J. (2011). Effect sizes in experimental pain produced by gender, genetic variants and sensitization procedures. PLoS One, 6, e17724. https://doi.org/10.1371/journal.pone.0017724
Fillingim, R. B., Bruehl, S., Dworkin, R. H., Dworkin, S. F., Loeser, J. D., Turk, D. C., Widerstrom-Noga, E., Arnold, L., Bennett, R., Edwards, R. R., Freeman, R., Gewandter, J., Hertz, S., Hochberg, M., Krane, E., Mantyh, P. W., Markman, J., Neogi, T., Ohrbach, R., … Wesselmann, U. (2014). The ACTTION-American Pain Society Pain Taxonomy (AAPT): An evidence-based and multidimensional approach to classifying chronic pain conditions. The Journal of Pain, 15, 241-249. https://doi.org/10.1016/j.jpain.2014.01.004
Fisher, R. A. (1936). The use of multiple measurements in taxonomic problems. Annals of Eugenics, 7, 179-188. https://doi.org/10.1111/j.1469-1809.1936.tb02137.x
Fonti, V., & Belitser, E. (2017). Feature selection using lasso. VU Amsterdam Research Paper in Business Analytics, 30, 1-25.
Frank, E., & Witten, I. H. (1998). Generating Accurate Rule Sets Without Global Optimization. In ICML.
Gigerenzer, G., Todd, P. M., & The ABC Research Group, Evolution and cognition (1999). Fast and frugal heuristics: The adaptive toolbox. In G Gigerenzer & P.M. Todd (Eds.), Simple heuristics that make us smart (pp. 3-34). Oxford University Press).
Gustorff, B., Anzenhofer, S., Sycha, T., Lehr, S., & Kress, H. G. (2004). The sunburn pain model: The stability of primary and secondary hyperalgesia over 10 hours in a crossover setting. Anesthesia & Analgesia, 98, 173-177. table of contents. https://doi.org/10.1213/01.ane.0000093224.77281.a5.
Guttman, L. (1954). Some necessary conditions for common factor analysis. Psychometrika, 19, 149-161. https://doi.org/10.1007/BF02289162
Hamon, R., Junklewitz, H., & Sanchez, I. (2020). Robustness and Explainability of Artificial Intelligence - From technical to policy solutions. (Luxembourg, Publications Office of the European Union, Luxembourg).
Harrison, G. I., Young, A. R., & McMahon, S. B. (2004). Ultraviolet radiation-induced inflammation as a model for cutaneous hyperalgesia. The Journal of Investigative Dermatology, 122, 183-189. https://doi.org/10.1046/j.0022-202X.2003.22119.x
Ho, T. K. (1995). Random decision forests. In Proceedings of the Third International Conference on Document Analysis and Recognition (Volume 1) - Volume 1 (IEEE Computer Society), p. 278.
Hoffmann, R. T., & Schmelz, M. (1999). Time course of UVA- and UVB-induced inflammation and hyperalgesia in human skin. European Journal of Pain, 3, 131-139. https://doi.org/10.1053/eujp.1998.0106
Holte, R. C. (1993). Very simple classification rules perform well on most commonly used datasets. Machine Learning, 11, 63-90.
Hornik, K., Buchta, C., & Zeileis, A. (2009). Open-source machine learning: R meets Weka. Comput Stat, 24, 225-232. https://doi.org/10.1007/s00180-008-0119-7
Hotelling, H. (1933). Analysis of a complex of statistical variables into principal components. Journal of Educational Psychology, 24, 498-520. https://doi.org/10.1037/h0070888
Hothorn, T., Hornik, K., & Zeileis, A. (2006). Unbiased recursive partitioning: A conditional inference framework. Journal of Computational and Graphical Statistics, 15, 651-674. https://doi.org/10.1198/106186006X133933
Juran, J. M. (1975). The non-pareto principle; Mea culpa. Quality Progress, 8, 8-9.
Kaiser, H. F. (1958). The varimax criterion for analytic rotation in factor analysis. Psychometrika, 23, 187-200. https://doi.org/10.1007/BF02289233
Kaiser, U., Kopkow, C., Deckert, S., Neustadt, K., Jacobi, L., Cameron, P., De Angelis, V., Apfelbacher, C., Arnold, B., Birch, J., Bjarnegård, A., Christiansen, S., C de C Williams, A., Gossrau, G., Heinks, A., Hüppe, M., Kiers, H., Kleinert, U., Martelletti, P., … Schmitt, J. (2018). Developing a core outcome domain set to assessing effectiveness of interdisciplinary multimodal pain therapy: The VAPAIN consensus statement on core outcome domains. Pain, 159, 673-683. https://doi.org/10.1097/j.pain.0000000000001129
Kass, G. V. (1980). An exploratory technique for investigating large quantities of categorical data. Applied Statistics, 29, 119-127. https://doi.org/10.2307/2986296
Kim, H., & Loh, W.-Y. (2001). Classification trees with unbiased multiway splits. Journal of the American Statistical Association, 96, 589-604. https://doi.org/10.1198/016214501753168271
Kuhn, M. (2018). caret: Classification and Regression Training.
Kuhn, M., & Quinlan, R. (2018). C50: C5.0 decision trees and rule-based models.
Kursa, M. B., & Rudnicki, W. R. (2010). Feature selection with the Boruta package. Journal of Statistical Software, 36, 13.
Le, S., Josse, J., & Husson, F. C.(2008). FactoMineR: A package for multivariate analysis. Journal of Statistical Software, 25, 1-18.
Leisch, F. (2006). A toolbox for K-centroids cluster analysis. Computational Statistics & Data Analysis, 51, 526-544.
Lemon, J. (2006). Plotrix: A package in the red light district of R. R-News, 6, 8-12.
Lerch, F., Ultsch, A., & Lotsch, J. (2020). Distribution Optimization: An evolutionary algorithm to separate Gaussian mixtures. Scientific Reports, 10, 648. https://doi.org/10.1038/s41598-020-57432-w
Liaw, A., & Wiener, M. (2002). Classification and Regression by randomForest. R News, 2, 18-22.
Loh, W.-Y. (2009). Improving the precision of classification trees. Ann Appl Stat, 3, 1710-1737. https://doi.org/10.1214/09-AOAS260
Loh, W.-Y. (2011). Classification and regression trees. Wires Data Mining and Knowledge Discovery, 1, 14-23. https://doi.org/10.1002/widm.8
Loh, W.-Y. (2014). Fifty years of classification and regression trees. International Statistical Review, 82, 329-348. https://doi.org/10.1111/insr.12016
Loh, W.-Y., & Shih, Y.-S. (1997). Split selection methods for classification trees. Statistica Sinica, 7, 815-840.
Loh, W.-Y., & Vanichsetakul, N. (1988). Tree-structured classification via generalized discriminant analysis. Journal of the American Statistical Association, 83, 715-725. https://doi.org/10.1080/01621459.1988.10478652
Lötsch, J., Geisslinger, G., Heinemann, S., Lerch, F., Oertel, B. G., & Ultsch, A. (2017). Quantitative sensory testing response patterns to capsaicin- and UV-B-induced local skin hypersensitization in healthy subjects: A machine-learned analysis. Pain, 159, 11-24.
Lötsch, J., Kringel, D., Geisslinger, G., Oertel, B. G., Resch, E., & Malkusch, S. (2020). Machine-learned association of next-generation sequencing-derived variants in thermosensitive ion channels genes with human thermal pain sensitivity phenotypes. International Journal of Molecular Sciences, 21, 4367. https://doi.org/10.3390/ijms21124367
Lötsch, J., & Ultsch, A. (2019). Current projection methods-induced biases at subgroup detection for machine-learning based data-analysis of biomedical data. International Journal of Molecular Sciences, 21, 79 https://doi.org/10.3390/ijms21010079
Lötsch, J., & Ultsch, A. (2020). Random forests followed by computed ABC analysis as a feature selection method for machine-learning in biomedical data. In T. Imaizumi, A. Okada, S. Miyamoto, F. Sakaori, Y. Yamamoto, & M. Vichi (Eds.), Advanced Studies in Classification and Data Science, Singapore: Springer. http://doi-org-443.webvpn.fjmu.edu.cn/10.1007/978-981-15-3311-2_5.
MacQueen, J. (1967). Some methods for classification and analysis of multivariate observations. In Proceedings of the fifth berkeley symposium on mathematical statistics and probability, Volume 1: Statistics (pp. 281-297), University of California Press.
Maechler, M., Rousseeuw, P., Struyf, A., Hubert, M., & Hornik, K. (2017). cluster: Cluster analysis basics and extensions.
Magerl, W., Krumova, E. K., Baron, R., Tölle, T., Treede, R. D., & Maier, C. (2010). Reference data for quantitative sensory testing (QST): Refined stratification for age and a novel method for statistical comparison of group data. Pain, 151, 598-605. https://doi.org/10.1016/j.pain.2010.07.026
Milborrow, S. (2018). rpart.plot: Plot 'rpart' Models: An Enhanced Version of 'plot.rpart'.
Mohr, C., Leyendecker, S., Mangels, I., Machner, B., Sander, T., & Helmchen, C. (2008). Central representation of cold-evoked pain relief in capsaicin induced pain: An event-related fMRI study. Pain, 139, 416-430. https://doi.org/10.1016/j.pain.2008.05.020
Murphy, K. P. (2012). Machine learning. A Probabilistic Perspective (The MIT Press).
Newell, A., & Simon, H. A. (1976). Computer science as empirical inquiry: Symbols and search. Communications of the ACM, 19, 113-126. https://doi.org/10.1145/360018.360022
Palczewska, A., Palczewski, J., Marchese Robinson, R., & Neagu, D. (2014). Interpreting random forest classification models using a feature contribution method. In T. Bouabana-Tebibel & S. H. Rubin (Eds.), Integration of reusable systems (pp. 193-218). Springer International Publishing.
Pareto, V. (1909). Manuale di economia politica, Milan: Società editrice libraria, revised and translated into French as Manuel d’économie politique.
Pearson, K. (1900). On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling. Philosophical Magazine, Series, 5(50), 157-175.
Pearson, K. (1901). LIII. On lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 2, 559-572.
Pedersen, T. L., & Benesty, M. (2019). lime: Local Interpretable Model-Agnostic Explanations.
Peterson, W., Birdsall, T., & Fox, W. (1954). The theory of signal detectability. Transactions of the IRE Professional Group on Information Theory, 4, 171-212. https://doi.org/10.1109/TIT.1954.1057460
Pfaffel, O. (2020). FeatureImpCluster: Feature Importance for Partitional Clustering.
President's Information Technology Advisory, C. (2005). Report to the president: computational science: Ensuring America's Competitiveness.
Quinlan, J. R. (1986). Induction of decision trees. Machine Learning, 1, 81-106.
Quinlan, J. R. (2014). C4.5 : Programs for machine learning.
R Development Core Team (2008). R: A Language and Environment for Statistical Computing.
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). “Why Should I Trust You?”: Explaining the Predictions of Any Classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (San Francisco, California, USA, Association for Computing Machinery), pp. 1135-1144.
Rizopoulos, D. (2018). Max Kuhn and Kjell Johnson. applied predictive modeling. New York, Springer. Biometrics, 74, 383.
Robin, X., Turck, N., Hainard, A., Tiberti, N., Lisacek, F., Sanchez, J.-C., & Müller, M. (2011). pROC: An open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformatics, 12, 77.
Rolke, R., Baron, R., Maier, C., Tolle, T. R., Treede, R. D., Beyer, A., Binder, A., Birbaumer, N., Birklein, F., Botefur, I. C. et al (2006). Quantitative sensory testing in the German Research Network on Neuropathic Pain (DFNS): Standardized protocol and reference values. Pain, 123, 231-243.
Rolke, R., Magerl, W., Campbell, K. A., Schalber, C., Caspari, S., Birklein, F., & Treede, R. D. (2006). Quantitative sensory testing: A comprehensive protocol for clinical trials. European Journal of Pain, 10, 77-88.
Rousseeuw, P. J. (1987). Silhouettes: A graphical aid to the interpretation and validation of cluster analysis. Journal of Computational and Applied Mathematics, 20, 53-65. https://doi.org/10.1016/0377-0427(87)90125-7
Salzberg, S. L. (1994). C4.5: Programs for machine learning by J. Ross Quinlan. Morgan Kaufmann Publishers Inc, 1993. Machine Learning, 16, 235-240. https://doi.org/10.1007/BF00993309
Smirnov, N. (1948). Table for estimating the goodness of fit of empirical distributions. The Annals of Mathematical Statistics, 19(2), 279-281.
Smolensky, P. (2010). On the proper treatment of connectionism. Behavioral and Brain Sciences, 11, 1-23. https://doi.org/10.1017/S0140525X00052432
Student, (1908). Probable Error of a Correlation Coefficient Student. Biometrika, 6. 302-310. https://doi.org/10.1093/biomet/6.2-3.302.
Swets, J. A. (1973). The relative operating characteristic in psychology: A technique for isolating effects of response bias finds wide use in the study of perception and cognition. Science, 182, 990-1000. https://doi.org/10.1126/science.182.4116.990
Therneau, T., & Atkinson, B. (2019). rpart: Recursive Partitioning and Regression Trees.
Tjoa, E., & Guan, C. (2019). A Survey on Explainable Artificial Intelligence (XAI). Towards Medical XAI.
Ultsch, A. (2003). Pareto density estimation: A density estimation for knowledge discovery. In D. Baier & K. D. Werrnecke, (Eds.), Innovations in Classification, Data Science, and Information Systems - Proceedings 27th Annual Conference of the German Classification Society (GfKL), (621-628). Springer.
Ultsch, A., & Lötsch, J. (2015). Computed ABC analysis for rational selection of most informative variables in multivariate data. PLoS One, 10, e0129767. https://doi.org/10.1371/journal.pone.0129767
Ultsch, A., & Lötsch, J. (2017). Machine-learned cluster identification in high-dimensional data. Journal of Biomedical Informatics, 66, 95-104. https://doi.org/10.1016/j.jbi.2016.12.011
Ultsch, A., & Lötsch, J. (2020). The fundamental clustering and projection Suite (FCPS): A dataset collection to test the performance of clustering and data projection algorithms. Data, 5, 13. https://doi.org/10.3390/data5010013
Ultsch, A., & Moerchen, F. (2005). ESOM-Maps: Tools for clustering, visualization, and classification with Emergent SOM. Technical Report Dept of Mathematics and Computer Science. Germany: University of Marburg.
Vartiainen, P., Heiskanen, T., Sintonen, H., Roine, R. P., & Kalso, E. (2016). Health-related quality of life and burden of disease in chronic pain measured with the 15D instrument. Pain, 157, 2269-2276. https://doi.org/10.1097/j.pain.0000000000000641
Vollert, J., Mainka, T., Baron, R., Enax-Krumova, E. K., Hüllemann, P., Maier, C., Pfau, D. B., Tölle, T., & Treede, R. D. (2015). Quality assurance for Quantitative Sensory Testing laboratories: Development and validation of an automated evaluation tool for the analysis of declared healthy samples. Pain, 156, 2423-2430. https://doi.org/10.1097/j.pain.0000000000000300
Weyer-Menkhoff, I., & Lotsch, J. (2019). TRPA1 sensitization produces hyperalgesia to heat but not to cold stimuli in human volunteers. Clinical Journal of Pain. https://doi.org/10.1097/AJP.0000000000000677
Wickham, H. (2009). ggplot2: Elegant Graphics for Data Analysis. Springer-Verlag.
Youden, W. J. (1950). Index for rating diagnostic tests. Cancer, 3, 32-35. https://doi.org/10.1002/1097-0142(1950)3:1<32:AID-CNCR2820030106>3.0.CO;2-3