The Crosswise Model for Surveys on Sensitive Topics: A General Framework for Item Selection and Statistical Analysis.
Bayesian statistics
item response theory
sensitive questions
surveys
truth-telling techniques
Journal
Psychometrika
ISSN: 1860-0980
Titre abrégé: Psychometrika
Pays: United States
ID NLM: 0376503
Informations de publication
Date de publication:
28 May 2024
28 May 2024
Historique:
received:
16
12
2022
medline:
29
5
2024
pubmed:
29
5
2024
entrez:
28
5
2024
Statut:
aheadofprint
Résumé
When surveys contain direct questions about sensitive topics, participants may not provide their true answers. Indirect question techniques incentivize truthful answers by concealing participants' responses in various ways. The Crosswise Model aims to do this by pairing a sensitive target item with a non-sensitive baseline item, and only asking participants to indicate whether their responses to the two items are the same or different. Selection of the baseline item is crucial to guarantee participants' perceived and actual privacy and to enable reliable estimates of the sensitive trait. This research makes the following contributions. First, it describes an integrated methodology to select the baseline item, based on conceptual and statistical considerations. The resulting methodology distinguishes four statistical models. Second, it proposes novel Bayesian estimation methods to implement these models. Third, it shows that the new models introduced here improve efficiency over common applications of the Crosswise Model and may relax the required statistical assumptions. These three contributions facilitate applying the methodology in a variety of settings. An empirical application on attitudes toward LGBT issues shows the potential of the Crosswise Model. An interactive app, Python and MATLAB codes support broader adoption of the model.
Identifiants
pubmed: 38806852
doi: 10.1007/s11336-024-09976-3
pii: 10.1007/s11336-024-09976-3
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Informations de copyright
© 2024. The Author(s).
Références
Atsusaka, Y., & Stevenson, R.T. (2021). A bias-corrected estimator for the crosswise model with inattentive respondents. Political Analysis, pp. 1–15.
Blair, G., Coppock, A., & Moor, M. (2020). When to worry about sensitivity bias: A social reference theory and evidence from 30 years of list experiments. American Political Science Review, 114(4), 1297–1315.
doi: 10.1017/S0003055420000374
Blair, G., & Imai, K. (2012). Statistical analysis of list experiments. Political Analysis, 20(1), 47–77.
doi: 10.1093/pan/mpr048
Blair, G., Imai, K., & Zhou, Y.-Y. (2015). Design and analysis of the randomized response technique. Journal of the American Statistical Association, 110(511), 1304–1319.
doi: 10.1080/01621459.2015.1050028
Chuang, E., Dupas, P., Huillery, E., & Seban, J. (2021). Sex, lies, and measurement: Consistency tests for indirect response survey methods. Journal of Development Economics, 148, 102582.
doi: 10.1016/j.jdeveco.2020.102582
Coffman, K. B., Coffman, L. C., & Ericson, K. M. M. (2016). The size of the “LGBT’’ population and the magnitude of antigay sentiment are substantially underestimated. Management Science, 63(10), 3168–3186.
doi: 10.1287/mnsc.2016.2503
De Jong, M. G., & Pieters, R. (2019). Assessing sensitive consumer behavior using the item count response technique. Journal of Marketing Research, 56(3), 345–360.
doi: 10.1177/0022243718821312
De Jong, M. G., Pieters, R., & Fox, J.-P. (2010). Reducing social desirability bias through item randomized response: An application to measure underreported desires. Journal of Marketing Research, 47(1), 14–27.
doi: 10.1509/jmkr.47.1.14
Fox, J.-P. (2010). Bayesian item response modeling: Theory and applications. Springer Science & Business Media.
Gelman, A., Meng, X.-L., & Stern, H. (1996). Posterior predictive assessment of model fitness via realized discrepancies. Statistica sinica, pp. 733–760.
Glynn, A. N. (2013). What can we learn with statistical truth serum? Design and analysis of the list experiment. Public Opinion Quarterly, 77(S1), 159–172.
doi: 10.1093/poq/nfs070
Goldberg, L. R. (1992). The development of markers for the big-five factor structure. Psychological Assessment, 4(1), 26–42.
doi: 10.1037/1040-3590.4.1.26
Hoffmann, A., De Puiseau, B. W., Schmidt, A. F., & Musch, J. (2017). On the comprehensibility and perceived privacy protection of indirect questioning techniques. Behavior Research Methods, 49(4), 1470–1483.
doi: 10.3758/s13428-016-0804-3
pubmed: 27631988
Höglinger, M., & Diekmann, A. (2017). Uncovering a blind spot in sensitive question research: False positives undermine the crosswise-model “RRT’’. Political Analysis, 25(1), 131–137.
doi: 10.1017/pan.2016.5
Höglinger, M., & Jann, B. (2018). More is not always better: An experimental individual-level validation of the randomized response technique and the crosswise model. PloS One, 13(8), e0201770.
doi: 10.1371/journal.pone.0201770
pubmed: 30106973
pmcid: 6091935
Imai, K., Park, B., & Greene, K. F. (2015). Using the predicted responses from list experiments as explanatory variables in regression models. Political Analysis, 23(2), 180–196.
doi: 10.1093/pan/mpu017
Jann, B., Jerke, J., & Krumpal, I. (2011). Asking sensitive questions using the crosswise model: An experimental survey measuring plagiarism. Public Opinion Quarterly, 76(1), 32–49.
doi: 10.1093/poq/nfr036
Jerke, J., Johann, D., Rauhut, H., Thomas, K., & Velicu, A. (2021). Handle with care: Implementation of the list experiment and crosswise model in a large-scale survey on academic misconduct. Field Methods, page Forthcoming.
Jerke, J., Johann, D., Rauhut, H., & Thomas, K. (2019). Too sophisticated even for highly educated survey respondents? A qualitative assessment of indirect question formats for sensitive questions. Survey Research Methods, 13(3), 319–351.
John, L. K., Loewenstein, G., Acquisti, A., & Vosgerau, J. (2018). When and why randomized response techniques (fail to) elicit the truth. Organizational Behavior and Human Decision Processes, 148, 101–123.
doi: 10.1016/j.obhdp.2018.07.004
Kuha, J., & Jackson, J. (2014). The item count method for sensitive survey questions: Modelling criminal behaviour. Journal of the Royal Statistical Society: Series C (Applied Statistics), 63(2), 321–341.
Kuklinski, J. H., Cobb, M. D., & Gilens, M. (1997). Racial attitudes and the “New South’’. The Journal of Politics, 59(2), 323–349.
doi: 10.1017/S0022381600053470
Kwan, S. S., So, M. K., & Tam, K. Y. (2010). Research note-applying the randomized response technique to elicit truthful responses to sensitive questions in is research: The case of software piracy behavior. Information Systems Research, 21(4), 941–959.
doi: 10.1287/isre.1090.0271
Landsheer, J. A., Van Der Heijden, P., & Van Gils, G. (1999). Trust and understanding, two psychological aspects of randomized response. Quality and Quantity, 33, 1–12.
doi: 10.1023/A:1004361819974
Lensvelt-Mulders, G. J., Hox, J. J., Van der Heijden, P. G., & Maas, C. J. (2005). Meta-analysis of randomized response research: Thirty-five years of validation. Sociological Methods & Research, 33(3), 319–348.
doi: 10.1177/0049124104268664
Mikkola, P., Martin, O. A., Chandramouli, S., Hartmann, M., Pla, O. A., Thomas, O., Pesonen, H., Corander, J., Vehtari, A., Kaski, S., et al. (2021). Prior knowledge elicitation: The past, present, and future. arXiv preprint arXiv:2112.01380 .
Mirzazadeh, A., Shokoohi, M., Navadeh, S., Danesh, A., Jain, J. P., Sedaghat, A., Farnia, M., & Haghdoost, A. (2018). Underreporting in HIV-related high-risk behaviors: Comparing the results of multiple data collection methods in a behavioral survey of prisoners in Iran. The Prison Journal, 98(2), 213–228.
doi: 10.1177/0032885517753163
pubmed: 30078913
pmcid: 6075723
Nepusz, T., Petróczi, A., Naughton, D. P., Epton, T., & Norman, P. (2014). Estimating the prevalence of socially sensitive behaviors: Attributing guilty and innocent noncompliance with the single sample count method. Psychological Methods, 19(3), 334–355.
doi: 10.1037/a0034961
pubmed: 24295152
Qiu, S.-F., Tang, M.-L., Tao, J.-R., & Wong, R. S. (2022). Sample size determination for interval estimation of the prevalence of a sensitive attribute under randomized response models. Psychometrika, pp. 1–29.
Reiber, F., Schnuerch, M., & Ulrich, R. (2020). Improving the efficiency of surveys with randomized response models: A sequential approach based on curtailed sampling. Psychological Methods, 27, 198.
doi: 10.1037/met0000353
pubmed: 32915000
Roberts, D. L., & John, F. A. S. (2014). Estimating the prevalence of researcher misconduct: A study of UK academics within biological sciences. PeerJ, 2, e562.
doi: 10.7717/peerj.562
pubmed: 25250215
pmcid: 4168756
Sagoe, D., Cruyff, M., Spendiff, O., Chegeni, R., De Hon, O., Saugy, M., van der Heijden, P. G., & Petróczi, A. (2021). Functionality of the crosswise model for assessing sensitive or transgressive behavior: A systematic review and meta-analysis. Frontiers in Psychology, 12, 655592.
doi: 10.3389/fpsyg.2021.655592
pubmed: 34248750
pmcid: 8260852
Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika monograph supplement.
Sayed, K. H., Cruyff, M. J., van der Heijden, P. G., & Petróczi, A. (2022). Refinement of the extended crosswise model with a number sequence randomizer: Evidence from three different studies in the uk. Plos One, 17(12), e0279741.
doi: 10.1371/journal.pone.0279741
pubmed: 36584205
pmcid: 9803288
Schnell, R., & Thomas, K. (2021). A meta-analysis of studies on the performance of the crosswise model. Sociological Methods & Research, 52, 1493–1518.
doi: 10.1177/0049124121995520
Spiegelhalter, D. J., Best, N. G., Carlin, B. P., & Van Der Linde, A. (2002). Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society Series B: Statistical Methodology, 64(4), 583–639.
doi: 10.1111/1467-9868.00353
Tourangeau, R., & Yan, T. (2007). Sensitive questions in surveys. Psychological Bulletin, 133(5), 859–883.
doi: 10.1037/0033-2909.133.5.859
pubmed: 17723033
Walzenbach, S., & Hinz, T. (2019). Pouring water into wine: Revisiting the advantages of the crosswise model for asking sensitive questions (pp. 1–16). Survey Methods: Insights from the Field.
Warner, S. L. (1965). Randomized response: A survey technique for eliminating evasive answer bias. Journal of the American Statistical Association, 60(309), 63–69.
doi: 10.1080/01621459.1965.10480775
pubmed: 12261830
Wolter, F., & Preisendörfer, P. (2013). Asking sensitive questions: An evaluation of the randomized response technique versus direct questioning using individual validation data. Sociological Methods & Research, 42(3), 321–353.
doi: 10.1177/0049124113500474
Yu, J.-W., Tian, G.-L., & Tang, M.-L. (2008). Two new models for survey sampling with sensitive characteristic: Design and analysis. Metrika, 67(3), 251–263.
doi: 10.1007/s00184-007-0131-x