Crowdsourcing in health and medical research: a systematic review.
Challenge contest
Crowdsourcing
Health
Innovation
Medicine
Systematic review
Journal
Infectious diseases of poverty
ISSN: 2049-9957
Titre abrégé: Infect Dis Poverty
Pays: England
ID NLM: 101606645
Informations de publication
Date de publication:
20 Jan 2020
20 Jan 2020
Historique:
received:
17
05
2019
accepted:
07
01
2020
entrez:
22
1
2020
pubmed:
22
1
2020
medline:
21
4
2020
Statut:
epublish
Résumé
Crowdsourcing is used increasingly in health and medical research. Crowdsourcing is the process of aggregating crowd wisdom to solve a problem. The purpose of this systematic review is to summarize quantitative evidence on crowdsourcing to improve health. We followed Cochrane systematic review guidance and systematically searched seven databases up to September 4th 2019. Studies were included if they reported on crowdsourcing and related to health or medicine. Studies were excluded if recruitment was the only use of crowdsourcing. We determined the level of evidence associated with review findings using the GRADE approach. We screened 3508 citations, accessed 362 articles, and included 188 studies. Ninety-six studies examined effectiveness, 127 examined feasibility, and 37 examined cost. The most common purposes were to evaluate surgical skills (17 studies), to create sexual health messages (seven studies), and to provide layperson cardio-pulmonary resuscitation (CPR) out-of-hospital (six studies). Seventeen observational studies used crowdsourcing to evaluate surgical skills, finding that crowdsourcing evaluation was as effective as expert evaluation (low quality). Four studies used a challenge contest to solicit human immunodeficiency virus (HIV) testing promotion materials and increase HIV testing rates (moderate quality), and two of the four studies found this approach saved money. Three studies suggested that an interactive technology system increased rates of layperson initiated CPR out-of-hospital (moderate quality). However, studies analyzing crowdsourcing to evaluate surgical skills and layperson-initiated CPR were only from high-income countries. Five studies examined crowdsourcing to inform artificial intelligence projects, most often related to annotation of medical data. Crowdsourcing was evaluated using different outcomes, limiting the extent to which studies could be pooled. Crowdsourcing has been used to improve health in many settings. Although crowdsourcing is effective at improving behavioral outcomes, more research is needed to understand effects on clinical outcomes and costs. More research is needed on crowdsourcing as a tool to develop artificial intelligence systems in medicine. PROSPERO: CRD42017052835. December 27, 2016.
Sections du résumé
BACKGROUND
BACKGROUND
Crowdsourcing is used increasingly in health and medical research. Crowdsourcing is the process of aggregating crowd wisdom to solve a problem. The purpose of this systematic review is to summarize quantitative evidence on crowdsourcing to improve health.
METHODS
METHODS
We followed Cochrane systematic review guidance and systematically searched seven databases up to September 4th 2019. Studies were included if they reported on crowdsourcing and related to health or medicine. Studies were excluded if recruitment was the only use of crowdsourcing. We determined the level of evidence associated with review findings using the GRADE approach.
RESULTS
RESULTS
We screened 3508 citations, accessed 362 articles, and included 188 studies. Ninety-six studies examined effectiveness, 127 examined feasibility, and 37 examined cost. The most common purposes were to evaluate surgical skills (17 studies), to create sexual health messages (seven studies), and to provide layperson cardio-pulmonary resuscitation (CPR) out-of-hospital (six studies). Seventeen observational studies used crowdsourcing to evaluate surgical skills, finding that crowdsourcing evaluation was as effective as expert evaluation (low quality). Four studies used a challenge contest to solicit human immunodeficiency virus (HIV) testing promotion materials and increase HIV testing rates (moderate quality), and two of the four studies found this approach saved money. Three studies suggested that an interactive technology system increased rates of layperson initiated CPR out-of-hospital (moderate quality). However, studies analyzing crowdsourcing to evaluate surgical skills and layperson-initiated CPR were only from high-income countries. Five studies examined crowdsourcing to inform artificial intelligence projects, most often related to annotation of medical data. Crowdsourcing was evaluated using different outcomes, limiting the extent to which studies could be pooled.
CONCLUSIONS
CONCLUSIONS
Crowdsourcing has been used to improve health in many settings. Although crowdsourcing is effective at improving behavioral outcomes, more research is needed to understand effects on clinical outcomes and costs. More research is needed on crowdsourcing as a tool to develop artificial intelligence systems in medicine.
TRIAL REGISTRATION
BACKGROUND
PROSPERO: CRD42017052835. December 27, 2016.
Identifiants
pubmed: 31959234
doi: 10.1186/s40249-020-0622-9
pii: 10.1186/s40249-020-0622-9
pmc: PMC6971908
doi:
Types de publication
Journal Article
Systematic Review
Langues
eng
Sous-ensembles de citation
IM
Pagination
8Subventions
Organisme : NCATS NIH HHS
ID : UL1 TR001420
Pays : United States
Références
Am J Obstet Gynecol. 2016 Nov;215(5):644.e1-644.e7
pubmed: 27365004
Prehosp Disaster Med. 2014 Oct;29(5):484-8
pubmed: 25226347
Eur Urol. 2016 Apr;69(4):547-550
pubmed: 26755338
N Engl J Med. 2015 Jun 11;372(24):2316-25
pubmed: 26061836
J Med Internet Res. 2014 Oct 30;16(10):e233
pubmed: 25356929
J Med Internet Res. 2019 May 15;21(5):e12708
pubmed: 31094338
J Med Internet Res. 2015 Mar 30;17(3):e72
pubmed: 25835472
J Sch Health. 2004 Oct;74(8):325-8
pubmed: 15554118
Sex Transm Dis. 2017 Oct;44(10):630-636
pubmed: 28876322
J Gen Intern Med. 2014 Jan;29(1):187-203
pubmed: 23843021
J Glob Health. 2017 Jun;7(1):011003
pubmed: 28686749
Brain. 2017 Jun 1;140(6):1680-1691
pubmed: 28459961
J Urol. 2016 Jun;195(6):1859-65
pubmed: 26778711
J Med Imaging (Bellingham). 2018 Jul;5(3):034002
pubmed: 30840724
Artif Intell Med. 2019 Jul;98:77-86
pubmed: 31521254
J Endourol. 2016 Apr;30(4):447-52
pubmed: 26597352
Sex Transm Dis. 2015 Nov;42(11):625-8
pubmed: 26462186
Cancer Med. 2017 Nov;6(11):2595-2605
pubmed: 28960834
PeerJ. 2019 Apr 12;7:e6762
pubmed: 30997295
J Endourol. 2015 May;29(5):604-9
pubmed: 25356517
Int J Comput Assist Radiol Surg. 2015 Sep;10(9):1435-47
pubmed: 26133652
Sex Transm Infect. 2013 Dec;89(8):650-2
pubmed: 23702459
Resuscitation. 2011 Dec;82(12):1514-8
pubmed: 21854731
Clin Infect Dis. 2016 Jun 1;62(11):1436-1442
pubmed: 27129465
Lancet Glob Health. 2019 Sep;7(9):e1172-e1173
pubmed: 31401998
PLoS One. 2013 Aug 21;8(8):e71154
pubmed: 23990935
Am J Prev Med. 2014 Feb;46(2):179-87
pubmed: 24439353
J Sch Health. 1992 Oct;62(8):377-80
pubmed: 1479830
J Am Coll Radiol. 2017 Nov;14(11):1489-1497
pubmed: 29101973
Int J Health Geogr. 2018 Jun 5;17(1):17
pubmed: 29871687
BMC Med Inform Decis Mak. 2018 Jul 23;18(Suppl 2):55
pubmed: 30066655
Resuscitation. 2011 Oct;82(10):1273-8
pubmed: 21652136
Malar J. 2019 Jan 24;18(1):21
pubmed: 30678733
Netw Res Triangle Park N C. 1997 Spring;17(3):29-31
pubmed: 12292393
Med Image Comput Comput Assist Interv. 2014;17(Pt 2):438-45
pubmed: 25485409
J Virus Erad. 2017 Oct 1;3(4):223-228
pubmed: 29057087
BMJ Innov. 2017;3:227-237
pubmed: 29576873
Am J Surg. 2016 Feb;211(2):398-404
pubmed: 26709011
J Biomed Inform. 2017 May;69:86-92
pubmed: 28389234
PLoS Med. 2018 Aug 28;15(8):e1002645
pubmed: 30153265
J Surg Res. 2015 Jun 15;196(2):302-6
pubmed: 25888499
J Med Internet Res. 2012 Nov 29;14(6):e167
pubmed: 23196001
Ann Med. 2015;47(7):570-5
pubmed: 26469375
BMC Infect Dis. 2017 Nov 1;17(Suppl 1):701
pubmed: 29143673
J Endourol. 2017 Apr;31(S1):S95-S100
pubmed: 27633332
Circulation. 2016 Aug 23;134(8):e87-e108
pubmed: 27334603
Malar J. 2018 Jan 30;17(1):54
pubmed: 29378588
Brain. 2018 Sep 1;141(9):2619-2630
pubmed: 30101347
JAMA Oncol. 2019 May 1;5(5):654-661
pubmed: 30998808
J Surg Res. 2014 Mar;187(1):65-71
pubmed: 24555877
BMJ. 2011 Oct 18;343:d5928
pubmed: 22008217
Reprod Health Matters. 2013 May;21(41):57-68
pubmed: 23684188
Cult Health Sex. 2011 Sep;13(8):945-59
pubmed: 21787256
Resuscitation. 2007 Sep;74(3):487-94
pubmed: 17452068
Glob Public Health. 2013;8(8):890-903
pubmed: 24004339
J Endourol. 2015 Oct;29(10):1183-8
pubmed: 25867006
Tob Control. 2003 Sep;12(3):243-4
pubmed: 12958370
Sex Transm Dis. 2019 Mar;46(3):172-178
pubmed: 30741854
Int J Surg. 2010;8(5):336-41
pubmed: 20171303