Similarities and Differences in Tobacco Control Research Findings From Convenience and Probability Samples.
Convenience sampling
Health behavior
Mechanical Turk
Tobacco control
Validity
Journal
Annals of behavioral medicine : a publication of the Society of Behavioral Medicine
ISSN: 1532-4796
Titre abrégé: Ann Behav Med
Pays: England
ID NLM: 8510246
Informations de publication
Date de publication:
28 03 2019
28 03 2019
Historique:
pubmed:
28
7
2018
medline:
7
7
2020
entrez:
28
7
2018
Statut:
ppublish
Résumé
Online convenience samples are a quick and low-cost way to study health behavior, but the comparability to findings from probability samples is not yet well understood. We sought to compare convenience and probability samples' findings for experiments, correlates, and prevalence in the context of tobacco control research. Participants were a probability sample of 5,014 U.S. adults recruited by phone from September 2014 through May 2015 (cost ~U.S.$620,000) and an online convenience sample of 4,137 U.S. adults recruited through Amazon Mechanical Turk (MTurk) in December 2014 (cost ~U.S.$17,000). Participants completed a survey with experiments, measures of tobacco product use and demographic characteristics. MTurk convenience and probability samples showed the same pattern of statistical significance and direction in almost all experiments (21 of 24 analyses did not differ) and observational studies (19 of 25 associations did not differ). Demographic characteristics of the samples differed substantially (1 of 17 estimates did not differ), with the convenience sample being younger, having more years of education, and including more Whites and Asians. Tobacco product use also differed substantially (1 of 22 prevalence estimates did not differ), with the convenience sample reporting more cigarette and e-cigarette use (median error 19%). Using MTurk convenience samples can yield generalizable findings for experiments and observational studies. Prevalence estimates from MTurk convenience samples are likely to be over- or underestimates.
Sections du résumé
BACKGROUND
Online convenience samples are a quick and low-cost way to study health behavior, but the comparability to findings from probability samples is not yet well understood.
PURPOSE
We sought to compare convenience and probability samples' findings for experiments, correlates, and prevalence in the context of tobacco control research.
METHODS
Participants were a probability sample of 5,014 U.S. adults recruited by phone from September 2014 through May 2015 (cost ~U.S.$620,000) and an online convenience sample of 4,137 U.S. adults recruited through Amazon Mechanical Turk (MTurk) in December 2014 (cost ~U.S.$17,000). Participants completed a survey with experiments, measures of tobacco product use and demographic characteristics.
RESULTS
MTurk convenience and probability samples showed the same pattern of statistical significance and direction in almost all experiments (21 of 24 analyses did not differ) and observational studies (19 of 25 associations did not differ). Demographic characteristics of the samples differed substantially (1 of 17 estimates did not differ), with the convenience sample being younger, having more years of education, and including more Whites and Asians. Tobacco product use also differed substantially (1 of 22 prevalence estimates did not differ), with the convenience sample reporting more cigarette and e-cigarette use (median error 19%).
CONCLUSIONS
Using MTurk convenience samples can yield generalizable findings for experiments and observational studies. Prevalence estimates from MTurk convenience samples are likely to be over- or underestimates.
Identifiants
pubmed: 30052702
pii: 5056850
doi: 10.1093/abm/kay059
pmc: PMC6339836
mid: NIHMS984616
doi:
Types de publication
Journal Article
Research Support, N.I.H., Extramural
Research Support, U.S. Gov't, P.H.S.
Langues
eng
Sous-ensembles de citation
IM
Pagination
476-485Subventions
Organisme : NCI NIH HHS
ID : P50 CA180907
Pays : United States
Informations de copyright
© Society of Behavioral Medicine 2018. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.
Références
J Behav Med. 2017 Apr;40(2):352-359
pubmed: 27663553
Prev Med. 2017 Mar;96:144-148
pubmed: 28024860
PLoS One. 2018 Jan 11;13(1):e0189928
pubmed: 29324749
Tob Control. 2016 Sep;26(5):592-599
pubmed: 27924009
Annu Rev Clin Psychol. 2016;12:53-81
pubmed: 26772208
Perspect Psychol Sci. 2011 Jan;6(1):3-5
pubmed: 26162106
J Theor Biol. 2012 Apr 21;299:172-9
pubmed: 21402081
PLoS One. 2012;7(12):e51876
pubmed: 23272183
Soc Sci Med. 2017 Jun;182:136-141
pubmed: 28427731
Prev Med. 2017 Sep;102:93-99
pubmed: 28694063
Am J Public Health. 2017 Aug;107(8):1283-1289
pubmed: 28640681
BMC Public Health. 2016 Jun 23;16:516
pubmed: 27333921
Behav Res Methods. 2012 Mar;44(1):1-23
pubmed: 21717266