Using Human Resources Data to Predict Turnover of Community Mental Health Employees: Prediction and Interpretation of Machine Learning Methods.
human resources
machine learning
mental health employees
turnover
Journal
International journal of mental health nursing
ISSN: 1447-0349
Titre abrégé: Int J Ment Health Nurs
Pays: Australia
ID NLM: 101140527
Informations de publication
Date de publication:
03 Jul 2024
03 Jul 2024
Historique:
revised:
09
05
2024
received:
19
12
2023
accepted:
20
06
2024
medline:
4
7
2024
pubmed:
4
7
2024
entrez:
4
7
2024
Statut:
aheadofprint
Résumé
This study used machine learning (ML) to predict mental health employees' turnover in the following 12 months using human resources data in a community mental health centre. The data contain 621 employees' information (e.g., demographics, job information and client information served by employees) hired between 2011 and 2021 (56.5% turned over during the study period). Six ML methods (i.e., logistic regression, elastic net, random forest [RF], gradient boosting machine [GBM], neural network and support vector machine) were used to predict turnover, along with graphical and statistical tools to interpret predictive relationship patterns and potential interactions. The result suggests that RF and GBM led to better prediction according to specificity, sensitivity and area under the curve (>0.8). The turnover predictors (e.g., past work years, work hours, wage, age, exempt status, educational degree, marital status and employee type) were identified, including those that may be unique to the mental health employee population (e.g., training hours and the proportion of clients with schizophrenia diagnosis). It also revealed nonlinear and nonmonotonic predictive relationships (e.g., wage and employee age), as well as interaction effects, such that past work years interact with other variables in turnover prediction. The study indicates that ML methods showed the predictability of mental health employee turnover using human resources data. The identified predictors and the nonlinear and interactive relationships shed light on developing new predictive models for turnover that warrant further investigations.
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Subventions
Organisme : NIMH NIH HHS
ID : NIMH R34MH119411
Pays : United States
Informations de copyright
© 2024 The Author(s). International Journal of Mental Health Nursing published by John Wiley & Sons Australia, Ltd.
Références
Aarons, G. A., and A. C. Sawitzky. 2006. “Organizational Climate Partially Mediates the Effect of Culture on Work Attitudes and Staff Turnover in Mental Health Services.” Administration and Policy in Mental Health and Mental Health Services Research 33, no. 3: 289–301. https://doi.org/10.1007/s10488‐006‐0039‐1.
Beidas, R. S., S. Marcus, C. B. Wolk, et al. 2016. “A Prospective Examination of Clinician and Supervisor Turnover Within the Context of Implementation of Evidence‐Based Practices in a Publicly‐Funded Mental Health System.” Administration and Policy in Mental Health and Mental Health Services Research 43, no. 5: 640–649. https://doi.org/10.1007/s10488‐015‐0673‐6.
Brabson, L. A., A. D. Herschell, D. J. Kolko, and S. J. Mrozowski. 2019. “Associations Among Job Role, Training Type, and Staff Turnover in a Large‐Scale Implementation Initiative.” The Journal of Behavioral Health Services & Research 46, no. 3: 399–414. https://doi.org/10.1007/s11414‐018‐09645‐1.
Breiman, L. 2001. “Random Forests.” Machine Learning 45, no. 1: 5–32. https://doi.org/10.1023/A:1010933404324.
Breiman, L., J. H. Friedman, R. A. Olsen, and C. J. Stone. 2017. “Classification and Regression Trees.” Routledge. https://doi.org/10.1201/9781315139470.
Brereton, R. G., and G. R. Lloyd. 2010. “Support Vector Machines for Classification and Regression.” The Analyst 135, no. 2: 230–267. https://doi.org/10.1039/B918972F.
Bukach, A. M., F. K. Ejaz, N. Dawson, and R. J. Gitter. 2017. “Turnover Among Community Mental Health Workers in Ohio.” Administration and Policy in Mental Health and Mental Health Services Research 44, no. 1: 115–122. https://doi.org/10.1007/s10488‐015‐0706‐1.
Cascarano, A., J. Mur‐Petit, J. Hernández‐González, et al. 2023. “Machine and Deep Learning for Longitudinal Biomedical Data: A Review of Methods and Applications.” Artificial Intelligence Review 56, no. S2: 1711–1771. https://doi.org/10.1007/s10462‐023‐10561‐w.
Cho, Y. J., and H. J. Song. 2017. “Determinants of Turnover Intention of Social Workers: Effects of Emotional Labor and Organizational Trust.” Public Personnel Management 46, no. 1: 41–65. https://doi.org/10.1177/0091026017696395.
Domladovac, M. 2021. “Comparison of Neural Network With Gradient Boosted Trees, Random Forest, Logistic Regression and SVM in Predicting Student Achievement.” In 2021 44th International Convention on Information, Communication and Electronic Technology (MIPRO), 211–216. Opatija, Croatia: IEEE. https://doi.org/10.23919/MIPRO52101.2021.9596684.
Dunn, O. J. 1961. “Multiple Comparisons Among Means.” Journal of the American Statistical Association 56, no. 293: 52–64.
Esmaieeli Sikaroudi, A. M., R. Ghousi, and A. Sikaroudi. 2015. “A Data Mining Approach to Employee Turnover Prediction (Case Study: Arak Automotive Parts Manufacturing).” Journal of Industrial and Systems Engineering 8, no. 4: 106–121.
Friedman, J. H. 2001. “Greedy Function Approximation: A Gradient Boosting Machine.” The Annals of Statistics 29, no. 5: 1189–1232.
Friedman, J. H. 2002. “Stochastic Gradient Boosting.” Computational Statistics & Data Analysis 38, no. 4: 367–378. https://doi.org/10.1016/S0167‐9473(01)00065‐2.
Friedman, J. H., and B. E. Popescu. 2008. “Predictive Learning via Rule Ensembles.” The Annals of Applied Statistics 2, no. 3: 916–954.
Fukui, S., A. L. Rollins, and M. P. Salyers. 2020. “Characteristics and Job Stressors Associated With Turnover and Turnover Intention Among Community Mental Health Providers.” Psychiatric Services 71, no. 3: 289–292.
Fukui, S., W. Wu, J. Greenfield, et al. 2023. “Machine Learning With Human Resources Data: Predicting Turnover Among Community Mental Health Center Employees.” The Journal of Mental Health Policy and Economics 26, no. 2: 63–76.
Fukui, S., W. Wu, and M. P. Salyers. 2019. “Mediational Paths From Supervisor Support to Turnover Intention and Actual Turnover Among Community Mental Health Providers.” Psychiatric Rehabilitation Journal 42, no. 4: 350–357.
Gokcen, I., and J. Peng. 2002. “Comparing Linear Discriminant Analysis and Support Vector Machines.” In Advances in Information Systems, edited by T. Yakhno, 104–113. Berlin, Heidelberg: Springer. https://doi.org/10.1007/3‐540‐36077‐8_10.
Hallett, E., E. Simeon, V. Amba, D. Howington, K. J. McConnell, and J. M. Zhu. 2024. “Factors Influencing Turnover and Attrition in the Public Behavioral Health System Workforce: Qualitative Study.” Psychiatric Services 75, no. 1: 55–63. https://doi.org/10.1176/appi.ps.20220516.
Hart, S. 1989. “Shapley Value.” In Game Theory, edited by J. Eatwell, M. Milgate, and P. Newman, 210–216. UK: Palgrave Macmillan. https://doi.org/10.1007/978‐1‐349‐20181‐5_25.
Hastie, T., J. Friedman, and R. Tibshirani. 2001. The Elements of Statistical Learning. New York, NY: Springer. https://doi.org/10.1007/978‐0‐387‐21606‐5.
Hawkins, M. 2021. Review of Physician and Advanced Practitioner Recruiting Incentives [White Paper]. https://www.merritthawkins.com/uploadedFiles/physician‐advanced‐practitioner‐incentive‐review‐2021.pdf.
Holtom, B. C., T. R. Mitchell, T. W. Lee, and M. B. Eberly. 2008. “5 Turnover and Retention Research: A Glance at the Past, a Closer Review of the Present, and a Venture Into the Future.” The Academy of Management Annals 2, no. 1: 231–274. https://doi.org/10.1080/19416520802211552.
Hom, P. W., T. W. Lee, J. D. Shaw, and J. P. Hausknecht. 2017. “One Hundred Years of Employee Turnover Theory and Research.” Journal of Applied Psychology 102, no. 3: 530–545.
Hosmer, D. W., S. Lemeshow, and R. X. Sturdivant. 2013. Applied Logistic Regression. New York, NY: John Wiley & Sons.
James, G., D. Witten, T. Hastie, and R. Tibshirani. 2021. An Introduction to Statistical Learning: With Applications in R. New York, NY: Springer US. https://doi.org/10.1007/978‐1‐0716‐1418‐1.
Jerez, J. M., I. Molina, P. J. García‐Laencina, et al. 2010. “Missing Data Imputation Using Statistical and Machine Learning Methods in a Real Breast Cancer Problem.” Artificial Intelligence in Medicine 50, no. 2: 105–115. https://doi.org/10.1016/j.artmed.2010.05.002.
Jun, M.‐J. 2021. “A Comparison of a Gradient Boosting Decision Tree, Random Forests, and Artificial Neural Networks to Model Urban Land Use Changes: The Case of the Seoul Metropolitan Area.” International Journal of Geographical Information Science 35, no. 11: 2149–2167. https://doi.org/10.1080/13658816.2021.1887490.
Kassambara, A. 2018. Machine Learning Essentials: Practical Guide in R. 1st ed. CreateSpace Independent Publishing Platform.
Kavzoglu, T., and P. M. Mather. 2003. “The Use of Backpropagating Artificial Neural Networks in Land Cover Classification.” International Journal of Remote Sensing 24, no. 23: 4907–4938. https://doi.org/10.1080/0143116031000114851.
Kirasich, K., T. Smith, and B. Sadler. 2018. “Random Forest vs Logistic Regression: Binary Classification for Heterogeneous Datasets.” SMU Data Science Review 1, no. 3: 25.
Kuhn, M. 2022. caret: Classification and Regression Training. R Package Version 6.0‐92. https://CRAN.R‐project.org/package=caret.
Kuhn, M., and K. Johnson. 2013. Applied Predictive Modeling. New York, NY: Springer. https://doi.org/10.1007/978‐1‐4614‐6849‐3.
Loh, W.‐Y., and P. Zhou. 2021. Variable Importance Scores. (arXiv:2102.07765). https://doi.org/10.48550/arXiv.2102.07765.
Lundberg, S. M., and S.‐I. Lee. 2017. “A Unified Approach to Interpreting Model Predictions.” Advances in Neural Information Processing Systems 30: 4768–4777. https://proceedings.neurips.cc/paper/2017/hash/8a20a8621978632d76c43dfd28b67767‐Abstract.html.
Molnar, C. 2018. “Iml: An R Package for Interpretable Machine Learning.” Journal of Open Source Software 3, no. 26: 786. https://doi.org/10.21105/joss.00786.
Molnar, C., G. Casalicchio, and B. Bischl. 2020. “Interpretable Machine Learning—A Brief History, State‐of‐the‐Art and Challenges.” In Joint European conference on machine learning and knowledge discovery in databases, vol. 1323, 417–431. Cham, Switzerland: Springer International Publishing. https://doi.org/10.1007/978‐3‐030‐65965‐3_28.
Murugan, A., S. A. H. Nair, and K. P. S. Kumar. 2019. “Detection of Skin Cancer Using SVM, Random Forest and kNN Classifiers.” Journal of Medical Systems 43, no. 8: 269. https://doi.org/10.1007/s10916‐019‐1400‐8.
Pereira, J. M., M. Basto, and A. F. da Silva. 2016. “The Logistic Lasso and Ridge Regression in Predicting Corporate Failure.” Procedia Economics and Finance 39: 634–641. https://doi.org/10.1016/S2212‐5671(16)30310‐0.
Quinn, A., J. R. Rycraft, and D. Schoech. 2002. “Building a Model to Predict Caseworker and Supervisor Turnover Using a Neural Network and Logistic Regression.” Journal of Technology in Human Services 19, no. 4: 65–85. https://doi.org/10.1300/J017v19v04_05.
R Core Team. 2021. R: A Language and Environment for Statistical Computing [Computer software]. https://www.R‐project.org/.
Rodríguez‐Pérez, R., and J. Bajorath. 2020. “Interpretation of Machine Learning Models Using Shapley Values: Application to Compound Potency and Multi‐Target Activity Predictions.” Journal of Computer‐Aided Molecular Design 34, no. 10: 1013–1026. https://doi.org/10.1007/s10822‐020‐00314‐0.
Rollins, A. L., M. P. Salyers, J. Tsai, and J. M. Lydick. 2010. “Staff Turnover in Statewide Implementation of ACT: Relationship With ACT Fidelity and Other Team Characteristics.” Administration and Policy in Mental Health and Mental Health Services Research 37, no. 5: 417–426. https://doi.org/10.1007/s10488‐009‐0257‐4.
SAMHSA. 2022. Addressing Burnout in the Behavioral Health Workforce through Organizational Strategies. https://store.samhsa.gov/sites/default/files/SAMHSA_Digital_Download/pep22‐06‐02‐005‐.PDF.
Scanlan, J. N., and M. Still. 2013. “Job Satisfaction, Burnout and Turnover Intention in Occupational Therapists Working in Mental Health.” Australian Occupational Therapy Journal 60, no. 5: 310–318. https://doi.org/10.1111/1440‐1630.12067.
Sexton, R. S., S. McMurtrey, J. O. Michalopoulos, and A. M. Smith. 2005. “Employee Turnover: A Neural Network Solution.” Computers & Operations Research 32, no. 10: 2635–2651. https://doi.org/10.1016/j.cor.2004.06.022.
Strobl, C., J. Malley, and G. Tutz. 2009. “An Introduction to Recursive Partitioning: Rationale, Application, and Characteristics of Classification and Regression Trees, Bagging, and Random Forests.” Psychological Methods 14, no. 4: 323–348.
Sullivan, W. P., D. C. Kondrat, and D. Floyd. 2015. “The Pleasures and Pain of Mental Health Case Management.” Social Work in Mental Health 13, no. 4: 349–364. https://doi.org/10.1080/15332985.2014.955942.
Troyanskaya, O., M. Cantor, G. Sherlock, et al. 2001. “Missing Value Estimation Methods for DNA Microarrays.” Bioinformatics 17, no. 6: 520–525. https://doi.org/10.1093/bioinformatics/17.6.520.
Tsai, S. P., E. J. Bernacki, and L. J. Lucas. 1989. “A Longitudinal Method of Evaluating Employee Turnover.” Journal of Business and Psychology 3: 465–473.
Tzeng, H.‐M., J.‐G. Hsieh, and Y.‐L. Lin. 2004. “Predicting Nurses' Intention to Quit With a Support Vector Machine: A New Approach to Set up an Early Warning Mechanism in Human Resource Management.” Computers, Informatics, Nursing 22, no. 4: 232–242.
Yanchus, N. J., D. Periard, S. C. Moore, A. C. Carle, and K. Osatuke. 2015. “Predictors of Job Satisfaction and Turnover Intention in VHA Mental Health Employees: A Comparison Between Psychiatrists, Psychologists, Social Workers, and Mental Health Nurses.” Human Service Organizations: Management, Leadership & Governance 39, no. 3: 219–244. https://doi.org/10.1080/23303131.2015.1014953.
Yanchus, N. J., D. Periard, and K. Osatuke. 2017. “Further Examination of Predictors of Turnover Intention Among Mental Health Professionals.” Journal of Psychiatric and Mental Health Nursing 24, no. 1: 41–56. https://doi.org/10.1111/jpm.12354.
Zhang, Y., and A. Haghani. 2015. “A Gradient Boosting Method to Improve Travel Time Prediction.” Transportation Research Part C: Emerging Technologies 58: 308–324. https://doi.org/10.1016/j.trc.2015.02.019.
Zhao, Y., M. K. Hryniewicki, F. Cheng, B. Fu, and X. Zhu. 2019. “Employee Turnover Prediction With Machine Learning: A Reliable Approach.” In Intelligent Systems and Applications, edited by K. Arai, S. Kapoor, and R. Bhatia, vol. 869, 737–758. Cham, Switzerland: Springer International Publishing. https://doi.org/10.1007/978‐3‐030‐01057‐7_56.
Zou, H., and T. Hastie. 2005. “Regularization and Variable Selection via the Elastic Net.” Journal of the Royal Statistical Society, Series B: Statistical Methodology 67, no. 2: 301–320.