The effect of missing data and imputation on the detection of bias in cognitive testing using differential item functioning methods.

Bias Humans Neuropsychological Tests
Cognition Differential item functioning Item response theory Measurement

Journal

BMC medical research methodology
ISSN: 1471-2288
Titre abrégé: BMC Med Res Methodol
Pays: England
ID NLM: 100968545

Informations de publication

Date de publication:
27 03 2022
Historique:
received: 18 11 2021
accepted: 14 03 2022
entrez: 29 3 2022
pubmed: 30 3 2022
medline: 5 4 2022
Statut: epublish

Résumé

Item response theory (IRT) methods for addressing differential item functioning (DIF) can detect group differences in responses to individual items (e.g., bias). IRT and DIF-detection methods have been used increasingly often to identify bias in cognitive test performance by characteristics (DIF grouping variables) such as hearing impairment, race, and educational attainment. Previous analyses have not considered the effect of missing data on inferences, although levels of missing cognitive data can be substantial in epidemiologic studies. We used data from Visit 6 (2016-2017) of the Atherosclerosis Risk in Communities Neurocognitive Study (N = 3,580) to explicate the effect of artificially imposed missing data patterns and imputation on DIF detection. When missing data was imposed among individuals in a specific DIF group but was unrelated to cognitive test performance, there was no systematic error. However, when missing data was related to cognitive test performance and DIF group membership, there was systematic error in DIF detection. Given this missing data pattern, the median DIF detection error associated with 10%, 30%, and 50% missingness was -0.03, -0.08, and -0.14 standard deviation (SD) units without imputation, but this decreased to -0.02, -0.04, and -0.08 SD units with multiple imputation. Incorrect inferences in DIF testing have downstream consequences for the use of cognitive tests in research. It is therefore crucial to consider the effect and reasons behind missing data when evaluating bias in cognitive testing.

Sections du résumé

BACKGROUND
Item response theory (IRT) methods for addressing differential item functioning (DIF) can detect group differences in responses to individual items (e.g., bias). IRT and DIF-detection methods have been used increasingly often to identify bias in cognitive test performance by characteristics (DIF grouping variables) such as hearing impairment, race, and educational attainment. Previous analyses have not considered the effect of missing data on inferences, although levels of missing cognitive data can be substantial in epidemiologic studies.
METHODS
We used data from Visit 6 (2016-2017) of the Atherosclerosis Risk in Communities Neurocognitive Study (N = 3,580) to explicate the effect of artificially imposed missing data patterns and imputation on DIF detection.
RESULTS
When missing data was imposed among individuals in a specific DIF group but was unrelated to cognitive test performance, there was no systematic error. However, when missing data was related to cognitive test performance and DIF group membership, there was systematic error in DIF detection. Given this missing data pattern, the median DIF detection error associated with 10%, 30%, and 50% missingness was -0.03, -0.08, and -0.14 standard deviation (SD) units without imputation, but this decreased to -0.02, -0.04, and -0.08 SD units with multiple imputation.
CONCLUSIONS
Incorrect inferences in DIF testing have downstream consequences for the use of cognitive tests in research. It is therefore crucial to consider the effect and reasons behind missing data when evaluating bias in cognitive testing.

Identifiants

DOI: 10.1186/s12874-022-01572-2 PMID: 35346056 PMC: PMC8961895
pubmed: 35346056
doi: 10.1186/s12874-022-01572-2
pii: 10.1186/s12874-022-01572-2
pmc: PMC8961895
doi:

Types de publication

Journal Article Research Support, N.I.H., Extramural Research Support, Non-U.S. Gov't

Langues

eng

Sous-ensembles de citation

IM

Pagination

81

Subventions

Organisme : NIA NIH HHS
ID : P30 AG066507
Pays : United States
Organisme : NIA NIH HHS
ID : K01 AG050699
Pays : United States
Organisme : NHLBI NIH HHS
ID : HHSN268201700001I
Pays : United States
Organisme : NHLBI NIH HHS
ID : HHSN268201700004I
Pays : United States
Organisme : NIA NIH HHS
ID : K01 AG052640
Pays : United States
Organisme : NHLBI NIH HHS
ID : U01 HL096902
Pays : United States
Organisme : NIA NIH HHS
ID : R21 AG060243
Pays : United States
Organisme : NHLBI NIH HHS
ID : HHSN268201700005I
Pays : United States
Organisme : NHLBI NIH HHS
ID : U01 HL096899
Pays : United States
Organisme : NHLBI NIH HHS
ID : HHSN268201700003I
Pays : United States
Organisme : NHLBI NIH HHS
ID : U01 HL096814
Pays : United States
Organisme : NIA NIH HHS
ID : P30 AG066587
Pays : United States
Organisme : NIA NIH HHS
ID : K01 AG054693
Pays : United States
Organisme : NHLBI NIH HHS
ID : U01 HL096917
Pays : United States
Organisme : NHLBI NIH HHS
ID : HHSN268201700002I
Pays : United States
Organisme : NHLBI NIH HHS
ID : U01 HL096812
Pays : United States
Organisme : NHLBI NIH HHS
ID : R01 HL070825
Pays : United States

Informations de copyright

© 2022. The Author(s).

Références

Front Psychol. 2019 May 17;10:1137
pubmed: 31164855
Psychol Methods. 2017 Sep;22(3):507-526
pubmed: 27266798
Neuropsychol Rev. 2008 Sep;18(3):194-213
pubmed: 18819008
Neuroepidemiology. 2014;43(3-4):194-205
pubmed: 25402421
Psychol Aging. 2013 Sep;28(3):633-45
pubmed: 23437898
J Gerontol B Psychol Sci Soc Sci. 2002 Nov;57(6):P548-58
pubmed: 12426438
Med Care. 2006 Nov;44(11 Suppl 3):S124-33
pubmed: 17060819
J Assoc Res Otolaryngol. 2021 Jul;22(4):387-403
pubmed: 34008037
Int Psychogeriatr. 2019 Sep;31(9):1331-1341
pubmed: 30782222
J Int Neuropsychol Soc. 2020 May;26(5):515-526
pubmed: 31973776
Med Care. 2006 Nov;44(11 Suppl 3):S176-81
pubmed: 17060825
Am J Epidemiol. 1999 May 1;149(9):789-93
pubmed: 10221314
J Int Neuropsychol Soc. 2015 Jan;21(1):85-90
pubmed: 25389642
Neuropsychologia. 1989;27(8):1073-9
pubmed: 2797414
Am J Ophthalmol. 1982 Jul;94(1):91-6
pubmed: 7091289
Am J Epidemiol. 1989 Apr;129(4):687-702
pubmed: 2646917
BMC Med Inform Decis Mak. 2016 Jul 25;16 Suppl 3:74
pubmed: 27454392
Int J Methods Psychiatr Res. 2011 Mar;20(1):40-9
pubmed: 21499542
Clin Otolaryngol. 2016 Dec;41(6):718-729
pubmed: 26670203
Arch Gerontol Geriatr. 2017 Nov;73:43-49
pubmed: 28755569
J Am Geriatr Soc. 2009 Apr;57(4):722-9
pubmed: 19220562
Eur J Investig Health Psychol Educ. 2021 Dec 14;11(4):1653-1687
pubmed: 34940395
J Stat Softw. 2011 Mar 1;39(8):1-30
pubmed: 21572908
Neuropsychol Dev Cogn B Aging Neuropsychol Cogn. 2015;22(4):413-37
pubmed: 25325767
J Gerontol B Psychol Sci Soc Sci. 2005 Sep;60(5):P242-50
pubmed: 16131618
Neuroepidemiology. 2014;42(3):144-53
pubmed: 24481241
Ageing Res. 2011 Apr 28;2(1):19-25
pubmed: 22900138
J Aging Health. 2012 Sep;24(6):1044-76
pubmed: 22422759
J Consult Psychol. 1957 Apr;21(2):139-43
pubmed: 13416432
Psychol Aging. 1997 Dec;12(4):718-725
pubmed: 9416639
Neuropsychol Dev Cogn B Aging Neuropsychol Cogn. 2007 Jan;14(1):40-54
pubmed: 17164189
Aging Ment Health. 2014;18(6):671-3
pubmed: 24875093
Struct Equ Modeling. 2018;25(4):621-638
pubmed: 30083048
Epidemiology. 2015 Nov;26(6):878-87
pubmed: 26414855
J Gerontol A Biol Sci Med Sci. 2017 May 01;72(5):703-709
pubmed: 27071780
BMC Med Res Methodol. 2018 Aug 31;18(1):90
pubmed: 30170561

Auteurs

E Nichols (E)

Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, 615 N. Wolfe St, Baltimore, MD, USA. enicho30@jh.edu.

J A Deal (JA)

Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, 615 N. Wolfe St, Baltimore, MD, USA.
Cochlear Center for Hearing and Public Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA.

B K Swenor (BK)

Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, 615 N. Wolfe St, Baltimore, MD, USA.
Wilmer Eye Institute, Johns Hopkins Hospital, Baltimore, MD, USA.

A G Abraham (AG)

Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, 615 N. Wolfe St, Baltimore, MD, USA.
Department of Epidemiology, School of Public Health, University of Colorado Denver, Denver, CO, USA.

N M Armstrong (NM)

Department of Psychiatry and Human Behavior, Brown University Warren Alpert Medical School, Providence, RI, USA.

K Bandeen-Roche (K)

Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA.

M C Carlson (MC)

Department of Mental Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA.

M Griswold (M)

Memory Impairment and Neurodegenerative Dementia Center, University of Mississippi Medical Center, Jackson, MS, USA.

F R Lin (FR)

Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, 615 N. Wolfe St, Baltimore, MD, USA.
Cochlear Center for Hearing and Public Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA.
Department of Mental Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA.

T H Mosley (TH)

Memory Impairment and Neurodegenerative Dementia Center, University of Mississippi Medical Center, Jackson, MS, USA.

P Y Ramulu (PY)

Wilmer Eye Institute, Johns Hopkins Hospital, Baltimore, MD, USA.

N S Reed (NS)

Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, 615 N. Wolfe St, Baltimore, MD, USA.
Cochlear Center for Hearing and Public Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA.

A R Sharrett (AR)

Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, 615 N. Wolfe St, Baltimore, MD, USA.

A L Gross (AL)

Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, 615 N. Wolfe St, Baltimore, MD, USA.

Articles similaires

[Redispensing of expensive oral anticancer medicines: a practical application].

Lisanne N van Merendonk, Kübra Akgöl, Bastiaan Nuijen
1.00
Humans Antineoplastic Agents Administration, Oral Drug Costs Counterfeit Drugs

Smoking Cessation and Incident Cardiovascular Disease.

Jun Hwan Cho, Seung Yong Shin, Hoseob Kim et al.
1.00
Humans Male Smoking Cessation Cardiovascular Diseases Female

Evaluation of Low-Value Services Across Major Medicare Advantage Insurers and Traditional Medicare.

Ciara Duggan, Adam L Beckman, Ishani Ganguli et al.
1.00
Humans United States Aged Cross-Sectional Studies Medicare Part C

Effectiveness of Virtual Yoga for Chronic Low Back Pain: A Randomized Clinical Trial.

Hallie Tankha, Devyn Gaskins, Amanda Shallcross et al.
1.00
Humans Yoga Low Back Pain Female Male

Classifications MeSH

questionsmedicales.fr › Environnement et santé publique › Santé publique › Facteurs épidémiologiques › Biais (épidémiologie) › The effect of missing data and imputation on...
questionsmedicales.fr › Eucaryotes › Animaux › Chordés › Vertébrés › Mammifères › Eutheria › Primates › Haplorhini › Catarrhini › Hominidae › Humains › The effect of missing data and imputation on...
questionsmedicales.fr › Disciplines et activités comportementales › Tests psychologiques › Tests neuropsychologiques › The effect of missing data and imputation on...