Conducting two evidence syntheses in six weeks - experiences with and evaluation of a pilot project.
Evidence synthesis
Lean
Systematic review
Workflow
Journal
BMC medical research methodology
ISSN: 1471-2288
Titre abrégé: BMC Med Res Methodol
Pays: England
ID NLM: 100968545
Informations de publication
Date de publication:
16 Sep 2024
16 Sep 2024
Historique:
received:
13
06
2023
accepted:
06
09
2024
medline:
17
9
2024
pubmed:
17
9
2024
entrez:
16
9
2024
Statut:
epublish
Résumé
Evidence synthesis organisations are trying to meet commissioners' needs for rapid responses to their evidence synthesis commissions. In this project we piloted an intensive process, working to complete evidence syntheses within six-weeks, rather than the standard lead time of 4-6 months. Our objective was to explore how researchers experience working intensively, identify barriers and facilitators, and determine how a more intensive approach to evidence synthesis could be more systematically introduced in the future. In a pre-planning phase, an intensive work group was established, and two commissions were selected for this pilot project. The evidence synthesis process was divided into two phases: planning and intensive. The planning phase, involved scheduling the intensive phase, exploring new digital tools, and identifying peer reviewers. The intensive phase encompassed the entire evidence synthesis process. Two review teams were formed, each with a team lead supported by a process lead and leadership contact point. Throughout the project, teams engaged in reflective meetings to evaluate and adjust processes as needed. During the planning phase, teams identified significant uncertainties regarding scopes, research questions, and inclusion criteria. To address this, they engaged with commissioners earlier than originally planned, clarified these aspects, and prepared protocols. Despite some minor deviations from the original plan, both reviews were completed on schedule, with one team expanding their scope due to the absence of eligible studies. Teams operated flexibly, held regular meetings, and found the process seamless due to fewer interruptions. Machine learning tools facilitated rapid study selection. The process lead role, created to guide and evaluate the project, proved beneficial, providing structure and support, although clearer role delineation with the leadership contact point could have improved efficiency. Overall, the intensive process fostered focus and productivity, allowing teams to manage short-term deliverables effectively. The researchers preferred working intensively with one evidence synthesis over being involved with many projects at the same time. They felt that time use was more effective, and they were able to complete the tasks in a focused way. However, there are several implications that should be considered before implementing an intensive approach in future evidence syntheses.
Sections du résumé
BACKGROUND
BACKGROUND
Evidence synthesis organisations are trying to meet commissioners' needs for rapid responses to their evidence synthesis commissions. In this project we piloted an intensive process, working to complete evidence syntheses within six-weeks, rather than the standard lead time of 4-6 months. Our objective was to explore how researchers experience working intensively, identify barriers and facilitators, and determine how a more intensive approach to evidence synthesis could be more systematically introduced in the future.
METHODS
METHODS
In a pre-planning phase, an intensive work group was established, and two commissions were selected for this pilot project. The evidence synthesis process was divided into two phases: planning and intensive. The planning phase, involved scheduling the intensive phase, exploring new digital tools, and identifying peer reviewers. The intensive phase encompassed the entire evidence synthesis process. Two review teams were formed, each with a team lead supported by a process lead and leadership contact point. Throughout the project, teams engaged in reflective meetings to evaluate and adjust processes as needed.
RESULTS
RESULTS
During the planning phase, teams identified significant uncertainties regarding scopes, research questions, and inclusion criteria. To address this, they engaged with commissioners earlier than originally planned, clarified these aspects, and prepared protocols. Despite some minor deviations from the original plan, both reviews were completed on schedule, with one team expanding their scope due to the absence of eligible studies. Teams operated flexibly, held regular meetings, and found the process seamless due to fewer interruptions. Machine learning tools facilitated rapid study selection. The process lead role, created to guide and evaluate the project, proved beneficial, providing structure and support, although clearer role delineation with the leadership contact point could have improved efficiency.
CONCLUSIONS
CONCLUSIONS
Overall, the intensive process fostered focus and productivity, allowing teams to manage short-term deliverables effectively. The researchers preferred working intensively with one evidence synthesis over being involved with many projects at the same time. They felt that time use was more effective, and they were able to complete the tasks in a focused way. However, there are several implications that should be considered before implementing an intensive approach in future evidence syntheses.
Identifiants
pubmed: 39285338
doi: 10.1186/s12874-024-02334-y
pii: 10.1186/s12874-024-02334-y
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
208Informations de copyright
© 2024. The Author(s).
Références
Khangura S, Konnyu K, Cushman R, Grimshaw J, Moher D. Evidence summaries: the evolution of a rapid review approach. Syst Reviews. 2012;1(1):1–9.
doi: 10.1186/2046-4053-1-10
Clark J, Glasziou P, Del Mar C, Bannach-Brown A, Stehlik P, Scott AM. A full systematic review was completed in 2 weeks using automation tools: a case study. J Clin Epidemiol. 2020;121:81–90.
doi: 10.1016/j.jclinepi.2020.01.008
pubmed: 32004673
Scott AM, Glasziou P, Clark J. We extended the two-week systematic review (2weekSR) methodology to larger, more complex systematic reviews: a case series. J Clin Epidemiol. 2023.
Biesty L, Meskell P, Glenton C, Delaney H, Smalle M, Booth A, et al. A QuESt for speed: rapid qualitative evidence syntheses as a response to the COVID-19 pandemic. Syst Reviews. 2020;9(1):1–6.
doi: 10.1186/s13643-020-01512-5
Glenton C, Lewin S, Downe S, Paulsen E, Munabi-Babigumira S, Agarwal S, et al. Cochrane Effective Practice and Organisation of Care (EPOC) qualitative evidence syntheses, Differences from Reviews of Intervention Effectiveness and implications for Guidance. Int J Qualitative Methods. 2022;21:16094069211061950.
doi: 10.1177/16094069211061950
Womack JP, Jones DT. Beyond Toyota: how to root out waste and pursue perfection. Harvard Business Rev. 1996;74(5):140–51.
Jardim PSJ, Rose CJ, Ames HM, Echavez JFM, Van de Velde S, Muller AE. Automating risk of bias assessment in systematic reviews: a real-time mixed methods comparison of human researchers to a machine learning system. BMC Med Res Methodol. 2022;22(1):1–12.
doi: 10.1186/s12874-022-01649-y
Muller AE, Ames HMR, Himmels JPW, Jardim PSJ, Nguyen HL, Rose CJ et al. Implementation of machine learning in evidence syntheses in the Cluster for Reviews and Health Technology Assessments: Final report 2020–2021. 2021.
Muller AE, Ames HMR, Jardim PSJ, Rose CJ. Machine learning in systematic reviews: comparing automated text clustering with Lingo3G and human researcher categorization in a rapid review. Res Synthesis Methods. 2022;13(2):229–41.
doi: 10.1002/jrsm.1541
Muller AE, Berg RC, Meneses-Echavez JF, Ames HM, Borge TC, Jardim PSJ, et al. The effect of machine learning tools for evidence synthesis on resource use and time-to-completion: protocol for a retrospective pilot study. Syst Reviews. 2023;12(1):1–8.
doi: 10.1186/s13643-023-02171-y
Muller AE, Borge TC. Overdosevarslingssystemer: en forenklet metodevurdering-kartlegging. 2022.
Jacobsen Jardim PS, Gaustad J-V, Langøien LJ, Næss GE. Omfang av seksuell trakassering i Norge: en hurtigoversikt. 2022.
Hestevik CH, Evensen LH. Dødelighet ved forskrivning av benzodiazepiner til personer i legemiddelassistert rehabilitering (LAR): En systematisk oversikt. 2021.
Ames H, Hestevik CH. The acceptability, values and preferences of older people for chronic low back pain management; a qualitative evidence synthesis. 2022.
Cochrane handbook for systematic reviews of interventions. Version 6.2: Cochrane Collaboration. 2021. https://training.cochrane.org/handbook
Evensen LH, Lidal IB, Hafstad EV, Ames HMR. Nasjonale og regionale melde-og varselordninger for uønskede hendelser i pasientbehandlingen: systematisk oversikt. 2022.
Evensen LH, Lidal IB, Hafstad EV, Ames HMR. Utvalgt forskning om melde-og varselordninger i helse-og omsorgstje-nesten: forenklet metodevurdering–kartlegging. 2022.
Hafstad EV, Ames HMR. Internasjonale anbefalinger om nasjonale melde-og varselordninger for uønskede hendelser i helsetjenesten: et systematisk litteratursøk med sortering. 2022.
Hestevik CH, Jacobsen Jardim PS, Hval G. Helse-og omsorgstjenester til eldre innvandrere: en systematisk kartleggingsoversikt. 2022.
Halterman A, Mordecai. Full text Geoparsing and Event Geocoding. J Open Source Softw. 2017;2(9):91.
doi: 10.21105/joss.00091