Estimating misclassification error in a binary performance indicator: case study of low value care in Australian hospitals.

Indicators based on hospital administrative data have potential for misclassification error, especially if they rely on clinical detail that may not be well recorded in the data. We applied an approach using modified logistic regression models to assess the misclassification (false-positive and false-negative) rates of low-value care indicators. We applied indicators involving 19 procedures to an extract from the New South Wales Admitted Patient Data Collection (1 January 2012 to 30 June 2015) to label episodes as low value. We fit four models (no misclassification, false-positive only, false-negative only, both false-positive and false-negative) for each indicator to estimate misclassification rates and used the posterior probabilities of the models to assess which model fit best. False-positive rates were low for most indicators-if the indicator labels care as low value, the care is most likely truly low value according to the relevant recommendation. False-negative rates were much higher but were poorly estimated (wide credible intervals). For most indicators, the models allowing no misclassification or allowing false-negatives but no false-positives had the highest posterior probability. The overall low-value care rate from the indicators was 12%. After adjusting for the estimated misclassification rates from the highest probability models, this increased to 35%. Binary performance indicators have a potential for misclassification error, especially if they depend on clinical information extracted from administrative data. Indicators should be validated by chart review, but this is resource-intensive and costly. The modelling approach presented here can be used as an initial validation step to identify and revise indicators that may have issues before continuing to a full chart review validation.

Informations de copyright

© Author(s) (or their employer(s)) 2020. No commercial re-use. See rights and permissions. Published by BMJ.

Déclaration de conflit d'intérêts

Competing interests: AGE holds a HCF Research Foundation Professorial Research Fellowship, and receives income as a Ministerial appointee to the (Australian) Medicare Benefits Schedule (MBS) Review Taskforce, a member of the Choosing Wisely Australia advisory group, the Choosing Wisely International Planning Committee, the ACSQHC’s Atlas of Healthcare Variation Advisory Group, a Board Member of the NSW Bureau of Health Information (BHI), and as a consultant to Private Healthcare Australia and the Queensland and Victoria state health departments. TB-P has received scholarship income from the University of Sydney and the Capital Markets Cooperative Research Centre, and consulting fees from the Capital Markets Cooperative Research Centre, Queensland Health, the Victorian Department of Health and Human Services, and Private Healthcare Australia.