If it's information, it's not "bias": a scoping review and proposed nomenclature for future response-shift research.
Glossary
Longitudinal trends
Methods
Nomenclature
Response shift
Scoping review
Journal
Quality of life research : an international journal of quality of life aspects of treatment, care and rehabilitation
ISSN: 1573-2649
Titre abrégé: Qual Life Res
Pays: Netherlands
ID NLM: 9210257
Informations de publication
Date de publication:
Aug 2022
Aug 2022
Historique:
accepted:
15
10
2021
pubmed:
28
10
2021
medline:
7
7
2022
entrez:
27
10
2021
Statut:
ppublish
Résumé
The growth in response-shift methods has enabled a stronger empirical foundation to investigate response-shift phenomena in quality-of-life (QOL) research; but many of these methods utilize certain language in framing the research question(s) and interpreting results that treats response-shift effects as "bias," "noise," "nuisance," or otherwise warranting removal from the results rather than as information that matters. The present project will describe the various ways in which researchers have framed the questions for investigating response-shift issues and interpreted the findings, and will develop a nomenclature for such that highlights the important information about resilience reflected by response-shift findings. A scoping review was done of the QOL and response-shift literature (n = 1100 articles) from 1963 to 2020. After culling only empirical response-shift articles, raters characterized how investigators framed and interpreted study research questions (n = 164 articles). Of 10 methods used, papers using four of them utilized terms like "bias" and aimed to remove response-shift effects to reveal "true change." Yet, the investigators' reflections on their own conclusions suggested that they do not truly believe that response shift is error to be removed. A structured nomenclature is proposed for discussing response-shift results in a range of research contexts and response-shift detection methods. It is time for a concerted and focused effort to change the nomenclature of those methods that demonstrated this misinterpretation. Only by framing and interpreting response shift as information, not bias, can we improve our understanding and methods to help to distill outcomes with and without response-shift effects.
Sections du résumé
BACKGROUND
BACKGROUND
The growth in response-shift methods has enabled a stronger empirical foundation to investigate response-shift phenomena in quality-of-life (QOL) research; but many of these methods utilize certain language in framing the research question(s) and interpreting results that treats response-shift effects as "bias," "noise," "nuisance," or otherwise warranting removal from the results rather than as information that matters. The present project will describe the various ways in which researchers have framed the questions for investigating response-shift issues and interpreted the findings, and will develop a nomenclature for such that highlights the important information about resilience reflected by response-shift findings.
METHODS
METHODS
A scoping review was done of the QOL and response-shift literature (n = 1100 articles) from 1963 to 2020. After culling only empirical response-shift articles, raters characterized how investigators framed and interpreted study research questions (n = 164 articles).
RESULTS
RESULTS
Of 10 methods used, papers using four of them utilized terms like "bias" and aimed to remove response-shift effects to reveal "true change." Yet, the investigators' reflections on their own conclusions suggested that they do not truly believe that response shift is error to be removed. A structured nomenclature is proposed for discussing response-shift results in a range of research contexts and response-shift detection methods.
CONCLUSIONS
CONCLUSIONS
It is time for a concerted and focused effort to change the nomenclature of those methods that demonstrated this misinterpretation. Only by framing and interpreting response shift as information, not bias, can we improve our understanding and methods to help to distill outcomes with and without response-shift effects.
Identifiants
pubmed: 34705159
doi: 10.1007/s11136-021-03023-9
pii: 10.1007/s11136-021-03023-9
doi:
Types de publication
Journal Article
Review
Langues
eng
Sous-ensembles de citation
IM
Pagination
2247-2257Informations de copyright
© 2021. The Author(s), under exclusive licence to Springer Nature Switzerland AG.
Références
Sprangers, M. A. G., & Schwartz, C. E. (1999). Integrating response shift into health-related quality of life research: A theoretical model. Social Science and Medicine, 48(11), 1507–1515.
doi: 10.1016/S0277-9536(99)00045-3
Rapkin, B. D., & Schwartz, C. E. (2004). Toward a theoretical model of quality-of-life appraisal: Implications of findings from studies of response shift. Health and Quality of Life Outcomes, 2(1), 14.
doi: 10.1186/1477-7525-2-14
Schwartz, C. E., & Sprangers, M. A. G. (1999). Methodological approaches for assessing response shift in longitudinal health-related quality-of-life research. Social Science and Medicine, 48, 1531–1548. https://doi.org/10.1016/s0277-9536(99)00047-7
doi: 10.1016/s0277-9536(99)00047-7
pubmed: 10400255
Schwartz, C. E., & Rapkin, B. D. (2004). Reconsidering the psychometrics of quality of life assessment in light of response shift and appraisal. Health and Quality of Life Outcomes, 2, 16.
doi: 10.1186/1477-7525-2-16
Bach, J. R., & Tilton, M. C. (1994). Life satisfaction and well-being measures in ventilator assisted individuals with traumatic tetraplegia. Archives of Physical Medicine and Rehabilitation, 75(6), 626–632.
doi: 10.1016/0003-9993(94)90183-X
Bar-On, D., Lazar, A., & Amir, M. (2000). Quantitative assessment of response shift in QOL research. Social Indicators Research, 49, 37–49. https://doi.org/10.1023/A:1006933612340
doi: 10.1023/A:1006933612340
Artavia-Mora, L., Wagner, N., Thiombiano, B. A., & Bedi, A. (2020). Adaptation and biomedical transition of people living with HIV to antiretroviral treatment in Burkina Faso. Global Public Health, 15(5), 638–653. https://doi.org/10.1080/17441692.2019.1704819
doi: 10.1080/17441692.2019.1704819
pubmed: 31916514
Dobkin, B. H., Nadeau, S. E., Behrman, A. L., Wu, S. S., Rose, D. K., Bowden, M., et al. (2014). Prediction of responders for outcome measures of locomotor experience applied post stroke trial. Journal of Rehabilitation Research and Development, 51(1), 39.
doi: 10.1682/JRRD.2013.04.0080
Prell, T., Gaur, N., Steinbach, R., Witte, O. W., & Grosskreutz, J. (2020). Modelling disease course in amyotrophic lateral Sclerosis: pseudo-longitudinal insights from cross-sectional health-related quality of life data. Health and Quality of Life Outcomes. https://doi.org/10.1186/s12955-020-01372-6
doi: 10.1186/s12955-020-01372-6
pubmed: 32357946
pmcid: 7195704
Bennett, J. A., & Winters-Stone, K. (2006). Conceptualizing and measuring physical functioning in cancer survivorship studies. Oncology Nursing Forum, 33(1), 41–49. https://doi.org/10.1188/06.ONF.41-49
doi: 10.1188/06.ONF.41-49
pubmed: 16470233
Balain, B., Ennis, O., Kanes, G., Singhal, R., Roberts, S. N., Rees, D., et al. (2009). Response shift in self-reported functional scores after knee microfracture for full thickness cartilage lesions. Osteoarthritis Cartilage, 17(8), 1009–1013. https://doi.org/10.1016/j.joca.2009.02.007
doi: 10.1016/j.joca.2009.02.007
pubmed: 19272855
Barclay-Goddard, R., Lix, L. M., Tate, R., Weinberg, L., & Mayo, N. E. (2011). Health-related quality of life after stroke: Does response shift occur in self-perceived physical function? Archives of Physical Medicine and Rehabilitation, 92(11), 1762–1769. https://doi.org/10.1016/j.apmr.2011.06.013
doi: 10.1016/j.apmr.2011.06.013
pubmed: 22032211
Schwartz, C. E., Stucky, B., Rivers, C. S., Noonan, V. K., & Finkelstein, J. A. (2018). Quality of life and adaptation in people with spinal cord injury: response shift effects from 1 to 5 years postinjury. Archives of Physical Medicine and Rehabilitation, 99, 1599-1608.e1591. https://doi.org/10.1016/j.apmr.2018.01.028
doi: 10.1016/j.apmr.2018.01.028
pubmed: 29481771
Oort, F. J. (2005). Using structural equation modeling to detect response shifts and true change. Quality of Life Research, 14, 587–598.
doi: 10.1007/s11136-004-0830-y
Oort, F. J., Visser, M. R. M., & Sprangers, M. A. G. (2005). An application of structural equation modeling to detect response shifts and true change in quality of life data from cancer patients undergoing invasive surgery. Quality of Life Research, 14, 599–609.
doi: 10.1007/s11136-004-0831-x
Lix, L. M., Sajobi, T. T., Sawatzky, R., Liu, J., Mayo, N. E., Huang, Y., et al. (2013). Relative importance measures for reprioritization response shift. Quality of Life Research, 22(4), 695–703. https://doi.org/10.1007/s11136-012-0198-3
doi: 10.1007/s11136-012-0198-3
pubmed: 22700163
Sajobi, T. T., Fiest, K. M., & Wiebe, S. (2014). Changes in quality of life after epilepsy surgery: the role of reprioritization response shift. Epilepsia, 55, 1331–1338.
doi: 10.1111/epi.12697
Sawatzky, R., Gadermann, A. M., Ratner, P. A., Zumbo, B., & Lix, L. (2012). Identifying individuals with inflammatory bowel disease who experienced response shift: A latent class analysis. Quality of Life Research, 21, 33.
Guilleux, A., Blanchin, M., Vanier, A., Guillemin, F., Falissard, B., Schwartz, C. E., et al. (2015). Response shift algorithm in item response theory (ROSALI) for response shift detection with missing data in patient-reported outcomes in longitudinal clinical trials. Quality of Life Research, 24(3), 553–564.
doi: 10.1007/s11136-014-0876-4
Schwartz, C. E., Bode, R., Repucci, N., Becker, J., Sprangers, M. A., & Fayers, P. M. (2006). The clinical significance of adaptation to changing health: A meta-analysis of response shift. Quality of Life Research, 15(9), 1533–1550. https://doi.org/10.1007/s11136-006-0025-9
doi: 10.1007/s11136-006-0025-9
pubmed: 17031503
Sajobi, T. T., Brahmbatt, R., Lix, L. M., Zumbo, B. D., & Sawatzky, R. (2018). Scoping review of response shift methods: Current reporting practices and recommendations. Quality of Life Research, 27(5), 1133–1146.
doi: 10.1007/s11136-017-1751-x
Schwartz, C. E., Powell, V. E., & Rapkin, B. D. (2017). When Global Rating of Change contradicts observed change: Examining appraisal processes underlying paradoxical responses over time. Quality of Life Research, 26, 847–857. https://doi.org/10.1007/s11136-016-1414-3
doi: 10.1007/s11136-016-1414-3
pubmed: 27722830
Schwartz, C. E., Zhang, J., Rapkin, B. D., & Finkelstein, J. A. (2019). Reconsidering the minimally important difference: Evidence of instability over time and across groups. Spine J, 19(4), 726–734. https://doi.org/10.1016/j.spinee.2018.09.010
doi: 10.1016/j.spinee.2018.09.010
pubmed: 30248391
Nunnally, J. C. (1994). Psychometric Theory (3rd ed.). Tata McGraw-Hill Education.
Edelen, M. O., Thissen, D., Teresi, J. A., Kleinman, M., & Ocepek-Welikson, K. (2006). Identification of differential item functioning using item response theory and the likelihood-based model comparison approach: application to the mini-mental state examination. Medical Care, 44, S134–S142.
doi: 10.1097/01.mlr.0000245251.83359.8c
Rapkin, B. D., & Schwartz, C. E. (2019). Advancing quality-of-life research by deepening our understanding of response shift: A unifying theory of appraisal. Quality of Life Research, 28(10), 2623–2630. https://doi.org/10.1007/s11136-019-02248-z
doi: 10.1007/s11136-019-02248-z
pubmed: 31321672
Oort, F. J. (2007) Bias is not a negatively valenced term In Personal Communication (Ed) Boston MA
Delgado-Rodriguez, M., & Llorca, J. (2004). Bias. Journal of Epidemiology & Community Health, 58(8), 635–641.
doi: 10.1136/jech.2003.008466
Sapir, E. (1929). The status of linguistics as a science. Language, 5(4), 207–214.
doi: 10.2307/409588
Whorf, B. L. (1940). Science and Linguistics: Bobbs-Merrill Indianapolis. IN.
Chandler, D. (1994). The Sapir-Whorf Hypothesis. Online documents at URL http://www.aber.ac.uk/media/Documents/short/whorf.html [30.06. 2004]
Martinez, J. E., Feldman, L. A., Feldman, M. J., & Cikara, M. (2021). Narratives shape cognitive representations of immigrants and immigration-policy preferences. Psychological Science, 32(2), 135–152.
doi: 10.1177/0956797620963610
Munn, Z., Peters, M. D., Stern, C., Tufanaru, C., McArthur, A., & Aromataris, E. (2018). Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Medical Research Methodology, 18(1), 1–7.
doi: 10.1186/s12874-018-0611-x
Shakespeare, W., Evans, G. B., Levin, H., & Shattuck, C. H. (1974). The Riverside Shakespeare. In (Vol. 1, pp. 1140): Houghton Mifflin Boston.
Howard, G. S., Ralph, K. M., Gulanick, N. A., Maxwell, S. E., Nance, D. W., & Gerber, S. K. (1979). Internal invalidity in pretest-posttest self-report evaluations and a re-evaluation of retrospective pretests. Applied Psychol Meas, 3(1), 1–23.
doi: 10.1177/014662167900300101
Wilson, I. B. (1999). Clinical understanding and clinical implications of response shift. Social Science and Medicine, 48(11), 1577–1588.
doi: 10.1016/S0277-9536(99)00050-7
Schwartz, C. E., & Sendor, M. (2000). Helping others helps oneself: Response shift effects in peer support. In C. E. Schwartz & M. A. G. Sprangers (Eds.), Adaptation to changing health: Response shift in quality-of-life research (pp. 175–188). American Psychological Association.
doi: 10.1037/10382-012
McPhail, S., & Haines, T. (2010). Response shift, recall bias and their effect on measuring change in health-related quality of life amongst older hospital patients. Health and Quality of Life Outcomes, 8, 65. https://doi.org/10.1186/1477-7525-8-65
doi: 10.1186/1477-7525-8-65
pubmed: 20618978
pmcid: 2912788
Li, Y., & Rapkin, B. (2009). Classification and regression tree uncovered hierarchy of psychosocial determinants underlying quality-of-life response shift in HIV/AIDS. Journal of Clinical Epidemiology, 62(11), 1138–1147. https://doi.org/10.1016/j.jclinepi.2009.03.021
doi: 10.1016/j.jclinepi.2009.03.021
pubmed: 19595576
pmcid: 2754600
Boucekine, M., Loundou, A., Baumstarck, K., Minaya-Flores, P., Pelletier, J., Ghattas, B., et al. (2013). Using the random forest method to detect a response shift in the quality of life of multiple sclerosis patients: a cohort study. BMC Medical Research Methodology, 13, 20.
doi: 10.1186/1471-2288-13-20
O’Boyle, C. A. (1994). The schedule for the evaluation of individual quality of life (SEIQoL). International Journal of Mental Health, 23(3), 3–23.
doi: 10.1080/00207411.1994.11449285
Ruta, D. A., Garratt, A. M., Leng, M., Russell, I. T., & MacDonald, L. M. (1994). A new approach to the measurement of quality of life. The Patient-Generated Index. Med Care, 32(11), 1109–1126.
pubmed: 7967852
Ahmed, S., Mayo, N., Scott, S., Kuspinar, A., & Schwartz, C. (2011). Using latent trajectory analysis of residuals to detect response shift in general health among patients with multiple sclerosis. Quality Life Research, 20(10), 1555–1560. https://doi.org/10.1007/s11136-011-0005-6
doi: 10.1007/s11136-011-0005-6
Mayo, N. E., Scott, S. C., Dendukuri, N., Ahmed, S., & Wood-Dauphinee, S. (2008). Identifying response shift statistically at the individual level. Quality of Life Research, 17(4), 627–639. https://doi.org/10.1007/s11136-008-9329-2
doi: 10.1007/s11136-008-9329-2
pubmed: 18386161
Sajobi, T. T., Lix, L. M., Singh, G., Lowerison, M., Engbers, J., & Mayo, N. E. (2015). Identifying reprioritization response shift in a stroke caregiver population: A comparison of missing data methods. Quality of Life Research, 24(3), 529–540. https://doi.org/10.1007/s11136-014-0824-3
doi: 10.1007/s11136-014-0824-3
pubmed: 25344817
Schwartz, C. E., Sajobi, T. T., Lix, L. M., Quaranto, B. R., & Finkelstein, J. A. (2013). Changing values, changing outcomes: The influence of reprioritization response shift on outcome assessment after spine surgery. Quality of Life Research, 22(9), 2255–2264.
doi: 10.1007/s11136-013-0377-x
Liu, Y., Wang, M., Tynjälä, J., Lv, Y., Villberg, J., Zhang, Z., et al. (2010). Test-retest reliability of selected items of health behaviour in school-aged children (HBSC) survey questionnaire in Beijing China. BMC Medical Research Methodology, 10, 73. https://doi.org/10.1186/1471-2288-10-73
doi: 10.1186/1471-2288-10-73
pubmed: 20696078
pmcid: 2927607
Lee, S. I., Huang, A., Mortazavi, B., Li, C., Hoffman, H. A., Garst, J., et al. (2016). Quantitative assessment of hand motor function in cervical spinal disorder patients using target tracking tests. Journal of Rehabilitation Research and Development, 53, 1007–1022. https://doi.org/10.1682/JRRD.2014.12.0319
doi: 10.1682/JRRD.2014.12.0319
pubmed: 28475202
Schwartz, C. E., Stucky, B. D., Michael, W., & Rapkin, B. D. (2020). Does response shift impact interpretation of change even among scales developed using item response theory? Journal of Patient-Reported Outcomes, 4(1), 8.
doi: 10.1186/s41687-019-0162-x
Westerman, M. J., Hak, T., The, A.-M., Echteld, M. A., Groen, H. J. M., & van der Wal, G. (2007). Change in what matters to palliative patients: Eliciting information about adaptation with SEIQoL-DW. Palliative Medicine, 21, 581–586. https://doi.org/10.1177/0269216307081938LK
doi: 10.1177/0269216307081938LK
pubmed: 17942496
Postulart, D., & Adang, E. M. (2000). Response shift and adaptation in chronically ill patients. Medical Decision Making, 20, 186–193. https://doi.org/10.1177/0272989X0002000204
doi: 10.1177/0272989X0002000204
pubmed: 10772356
Mistry, K., & Simpson, J. (2013). Exploring the transitional process from receiving a diagnosis to living with motor neurone disease. Psychology & Health, 28, 939–953. https://doi.org/10.1080/08870446.2013.770513
doi: 10.1080/08870446.2013.770513
Aburub, A. S., Gagnon, B., Ahmed, S., Rodríguez, A. M., & Mayo, N. E. (2018). Impact of reconceptualization response shift on rating of quality of life over time among people with advanced cancer. Supportive Care in Cancer, 26, 3063–3071.
doi: 10.1007/s00520-018-4156-7
Carlier, I. V. E., van Eeden, W. A., de Jong, K., Giltay, E. J., van Noorden, M. S., van der Feltz-Cornelis, C., et al. (2019). Testing for response shift in treatment evaluation of change in self-reported psychopathology amongst secondary psychiatric care outpatients. International Journal of Methods in Psychiatric Research, 28, e1785. https://doi.org/10.1002/mpr.1785
doi: 10.1002/mpr.1785
pubmed: 31206911
pmcid: 6852603
Dibb, B., & Yardley, L. (2006). How does social comparison within a self-help group influence adjustment to chronic illness? A longitudinal study. Social Science and Medicine, 63(6), 1602–1613. https://doi.org/10.1016/j.socscimed.2006.03.031
doi: 10.1016/j.socscimed.2006.03.031
pubmed: 16716478
Li, Y., & Schwartz, C. E. (2011). Data mining for response shift patterns in multiple sclerosis patients using recursive partitioning tree analysis. Quality of Life Research, 20, 1543–1553.
doi: 10.1007/s11136-011-0004-7
Boucekine, M., Boyer, L., Baumstarck, K., Millier, A., Ghattas, B., Auquier, P., et al. (2015). Exploring the response shift effect on the quality of life of patients with schizophrenia: An application of the random forest method. Medical Decision Making, 35(3), 388–397. https://doi.org/10.1177/0272989x14559273
doi: 10.1177/0272989x14559273
pubmed: 25398623
Ahmed, S., Mayo, N. E., Corbiere, M., Wood-Dauphinee, S., Hanley, J., & Cohen, R. (2005). Change in quality of life of people with stroke over time: True change or response shift? Quality of Life Research, 14, 611–627. https://doi.org/10.1007/s11136-004-3708-0
doi: 10.1007/s11136-004-3708-0
pubmed: 16022056
Schwartz, C. E., Sprangers, M. A. G., Carey, A., & Reed, G. (2004). Exploring response shift in longitudinal data. Psychology & Health, 19(1), 51–69.
doi: 10.1080/0887044031000118456
Schwartz, C. E., & Rapkin, B. D. (2012). Understanding appraisal processes underlying the thentest: A mixed methods investigation. Quality of Life Research, 21, 381–388. https://doi.org/10.1007/s11136-011-0023-4
doi: 10.1007/s11136-011-0023-4
pubmed: 21969088
Sprangers, M. A. G., Taminiau-Bloem, E. F., Van Zuuren, F. J., Koeneman, M. A., Visser, M. R. M., Tishelman, C., et al. (2015). Using a retrospective pretest instead of a conventional pretest is replacing biases: A qualitative study of cognitive processes underlying responses to thentest items. Quality of Life Research, 24, 10. https://doi.org/10.1007/s11136-015-1078-4LK
doi: 10.1007/s11136-015-1078-4LK
Oort, F. J. (2021). SEM method will only correctly find response shift if it applies to the majority of the respondents (or all respondents), in a minority of the indicator variables. Personal Communication, . Amsterdam, The Netherlands.
Kline, R. B. (2015). Principles and practice of structural equation modeling. Guilford Publications.
Sajobi, T., Anota, A., Awosoga, O., Lawal, O., Oomen, C., Brobbey, A., et al. (2020). A meta-analysis of response shift effects in patient-reported outcomes studies [Abstract]. Quality of Life Research, 29(Suppl 1), S2–S3.
Vanier, A., Leplège, A., Hardouin, J. B., Sébille, V., & Falissard, B. (2015). Semantic primes theory may be helpful in designing questionnaires such as to prevent response shift. Journal of Clinical Epidemiology, 68(6), 646–654. https://doi.org/10.1016/j.jclinepi.2015.01.023
doi: 10.1016/j.jclinepi.2015.01.023
pubmed: 25716903
Krägeloh, C. U., Bergomi, C., Siegert, R. J., & Medvedev, O. N. (2018). Response shift after a mindfulness-based intervention: Measurement invariance testing of the Comprehensive Inventory of Mindfulness Experiences. Mindfulness, 9(1), 212–220.
doi: 10.1007/s12671-017-0764-4
Osborne, R. H., Hawkins, M., & Sprangers, M. A. (2006). Change of perspective: A measurable and desired outcome of chronic disease self-management intervention programs that violates the premise of preintervention/postintervention assessment. Arthritis Care Re, 55(3), 458–465.
doi: 10.1002/art.21982
Schwartz, C. (2005). Decision making at the end of life: Shifting sands. Journal of the Royal Society of Medicine, 98(7), 297–298.
doi: 10.1177/014107680509800701
Schwartz, C. E., Stark, R. B., & Stucky, B. D. (2020). Response-shift effects in neuromyelitis optica spectrum disorder: a secondary analysis of clinical trial data. Quality of Life Research, 30, 1–16.
Schwartz, C. E., Stark, R. B., Stucky, B. D., Li, Y., & Rapkin, B. D. (2021). Response-shift effects in neuromyelitis optica spectrum disorder: estimating response-shift-adjusted scores using equating. Qual Life Res, 1–10.
Schwartz, C. E., Stark, R. B., Borowiec, K., Nolte, S., & Myren, K. J. (2021). Norm-based comparison of the quality-of-life impact of ravulizumab and eculizumab in paroxysmal nocturnal hemoglobinuria. Orphanet Journal of Rare Diseases. https://doi.org/10.1186/s13023-021-02016-8
doi: 10.1186/s13023-021-02016-8
pubmed: 34809687
pmcid: 8607700