Development and validation of paired MEDLINE and Embase search filters for cost-utility studies.
Cost-utility
Evidence selection
Paired analysis
Relative recall
Search filters
Journal
BMC medical research methodology
ISSN: 1471-2288
Titre abrégé: BMC Med Res Methodol
Pays: England
ID NLM: 100968545
Informations de publication
Date de publication:
03 12 2022
03 12 2022
Historique:
received:
31
03
2022
accepted:
21
11
2022
entrez:
3
12
2022
pubmed:
4
12
2022
medline:
7
12
2022
Statut:
epublish
Résumé
Search filters are standardised sets of search terms, with validated performance, that are designed to retrieve studies with specific characteristics. A cost-utility analysis (CUA) is the preferred type of economic evaluation to underpin decision-making at the National Institute for Health and Care Excellence (NICE). Until now, when searching for economic evidence for NICE guidelines, we have used a broad set of health economic-related search terms, even when the reviewer's interest is confined to CUAs alone. We developed search filters to retrieve CUAs from MEDLINE and Embase. Our aim was to achieve recall of 90% or better across both databases while reducing the overall yield compared with our existing broad economic filter. We used the relative recall method along with topic expert input to derive and validate 3 pairs of filters, assessed by their ability to identify a gold-standard set of CUAs that had been used in published NICE guidelines. We developed and validated MEDLINE and Embase filters in pairs (testing whether, when used together, they find target studies in at least 1 database), as this is how they are used in practice. We examined the proxy-precision of our new filters by comparing their overall yield with our previous approach using publications indexed in a randomly selected year (2010). All 3 filter-pairs exceeded our target recall and led to substantial improvements in search proxy-precision. Our paired 'sensitive' filters achieved 100% recall (95% CI 99.0 to 100%) in the validation set. Our paired 'precise' filters also had very good recall (97.6% [95%CI: 95.4 to 98.9%]). We estimate that, compared with our previous search strategy, using the paired 'sensitive' filters would reduce reviewer screening burden by a factor of 5 and the 'precise' versions would do so by a factor of more than 20. Each of the 3 paired cost-utility filters enable the identification of almost all CUAs from MEDLINE and Embase from the validation set, with substantial savings in screening workload compared to our previous search practice. We would encourage other researchers who regularly use multiple databases to consider validating search filters in combination as this will better reflect how they use databases in their everyday work.
Sections du résumé
BACKGROUND
Search filters are standardised sets of search terms, with validated performance, that are designed to retrieve studies with specific characteristics. A cost-utility analysis (CUA) is the preferred type of economic evaluation to underpin decision-making at the National Institute for Health and Care Excellence (NICE). Until now, when searching for economic evidence for NICE guidelines, we have used a broad set of health economic-related search terms, even when the reviewer's interest is confined to CUAs alone.
METHODS
We developed search filters to retrieve CUAs from MEDLINE and Embase. Our aim was to achieve recall of 90% or better across both databases while reducing the overall yield compared with our existing broad economic filter. We used the relative recall method along with topic expert input to derive and validate 3 pairs of filters, assessed by their ability to identify a gold-standard set of CUAs that had been used in published NICE guidelines. We developed and validated MEDLINE and Embase filters in pairs (testing whether, when used together, they find target studies in at least 1 database), as this is how they are used in practice. We examined the proxy-precision of our new filters by comparing their overall yield with our previous approach using publications indexed in a randomly selected year (2010).
RESULTS
All 3 filter-pairs exceeded our target recall and led to substantial improvements in search proxy-precision. Our paired 'sensitive' filters achieved 100% recall (95% CI 99.0 to 100%) in the validation set. Our paired 'precise' filters also had very good recall (97.6% [95%CI: 95.4 to 98.9%]). We estimate that, compared with our previous search strategy, using the paired 'sensitive' filters would reduce reviewer screening burden by a factor of 5 and the 'precise' versions would do so by a factor of more than 20.
CONCLUSIONS
Each of the 3 paired cost-utility filters enable the identification of almost all CUAs from MEDLINE and Embase from the validation set, with substantial savings in screening workload compared to our previous search practice. We would encourage other researchers who regularly use multiple databases to consider validating search filters in combination as this will better reflect how they use databases in their everyday work.
Identifiants
pubmed: 36463100
doi: 10.1186/s12874-022-01796-2
pii: 10.1186/s12874-022-01796-2
pmc: PMC9719242
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
310Informations de copyright
© 2022. The Author(s).
Références
J Med Libr Assoc. 2008 Oct;96(4):356-61
pubmed: 18974813
Res Synth Methods. 2018 Dec;9(4):602-614
pubmed: 29314757
Syst Rev. 2012 Feb 29;1:19
pubmed: 22587829
Health Info Libr J. 2009 Sep;26(3):187-202
pubmed: 19712211
BMC Med Res Methodol. 2006 Jul 18;6:33
pubmed: 16848895
Health Info Libr J. 2017 Sep;34(3):200-216
pubmed: 28703418
Int J Technol Assess Health Care. 2018 Jan;34(6):547-554
pubmed: 30442221