Quality Appraisal in Systematic Literature Reviews of Studies Eliciting Health State Utility Values: Conceptual Considerations.
Journal
PharmacoEconomics
ISSN: 1179-2027
Titre abrégé: Pharmacoeconomics
Pays: New Zealand
ID NLM: 9212404
Informations de publication
Date de publication:
29 Mar 2024
29 Mar 2024
Historique:
accepted:
22
02
2024
medline:
29
3
2024
pubmed:
29
3
2024
entrez:
29
3
2024
Statut:
aheadofprint
Résumé
The increasing number of studies that generate health state utility values (HSUVs) and the impact of HSUVs on cost-utility analyses make a robust tailored quality appraisal (QA) tool for systematic reviews of these studies necessary. This study aimed to address conceptual issues regarding QA in systematic reviews of studies eliciting HSUVs by establishing a consensus on the definitions, dimensions and scope of a QA tool specific to this context. A modified Delphi method was used in this study. An international multidisciplinary panel of seven experts was purposively assembled. The experts engaged in two anonymous online survey rounds. After each round, the experts received structured and controlled feedback on the previous phase. Controlled feedback allowed the experts to re-evaluate and adjust their positions based on collective insights. Following these surveys, a virtual face-to-face meeting was held to resolve outstanding issues. Consensus was defined a priori at all stages of the modified Delphi process. The response rates to the first-round and second-round questionnaires and the virtual consensus meeting were 100%, 86% and 71%, respectively. The entire process culminated in a consensus on the definitions of scientific quality, QA, the three QA dimensions-reporting, relevance and methodological quality-and the scope of a QA tool specific to studies that elicit HSUVs. Achieving this consensus marks a pivotal step towards developing a QA tool specific to systematic reviews of studies eliciting HSUVs. Future research will build on this foundation, identify QA items, signalling questions and response options, and develop a QA tool specific to studies eliciting HSUVs.
Sections du résumé
BACKGROUND
BACKGROUND
The increasing number of studies that generate health state utility values (HSUVs) and the impact of HSUVs on cost-utility analyses make a robust tailored quality appraisal (QA) tool for systematic reviews of these studies necessary.
OBJECTIVE
OBJECTIVE
This study aimed to address conceptual issues regarding QA in systematic reviews of studies eliciting HSUVs by establishing a consensus on the definitions, dimensions and scope of a QA tool specific to this context.
METHODS
METHODS
A modified Delphi method was used in this study. An international multidisciplinary panel of seven experts was purposively assembled. The experts engaged in two anonymous online survey rounds. After each round, the experts received structured and controlled feedback on the previous phase. Controlled feedback allowed the experts to re-evaluate and adjust their positions based on collective insights. Following these surveys, a virtual face-to-face meeting was held to resolve outstanding issues. Consensus was defined a priori at all stages of the modified Delphi process.
RESULTS
RESULTS
The response rates to the first-round and second-round questionnaires and the virtual consensus meeting were 100%, 86% and 71%, respectively. The entire process culminated in a consensus on the definitions of scientific quality, QA, the three QA dimensions-reporting, relevance and methodological quality-and the scope of a QA tool specific to studies that elicit HSUVs.
CONCLUSIONS
CONCLUSIONS
Achieving this consensus marks a pivotal step towards developing a QA tool specific to systematic reviews of studies eliciting HSUVs. Future research will build on this foundation, identify QA items, signalling questions and response options, and develop a QA tool specific to studies eliciting HSUVs.
Identifiants
pubmed: 38551803
doi: 10.1007/s40273-024-01365-z
pii: 10.1007/s40273-024-01365-z
doi:
Types de publication
Journal Article
Langues
eng
Informations de copyright
© 2024. The Author(s).
Références
Chang EM, Saigal CS, Raldow AC. Explaining health state utility assessment. JAMA. 2020;323(11):1085–6.
pubmed: 32091541
doi: 10.1001/jama.2020.0656
Wolowacz SE, Briggs A, Belozeroff V, Clarke P, Doward L, Goeree R, et al. Estimating health-state utility for economic models in clinical studies: an ISPOR Good Research Practices Task Force report. Value Health. 2016;19(6):704–19.
pubmed: 27712695
doi: 10.1016/j.jval.2016.06.001
Payakachat N, Murawski MM, Summers KH. Health utility and economic analysis: theoretical and practical issues. Expert Rev Pharmacoecon Outcomes Res. 2009;9(4):289–92.
pubmed: 19670986
doi: 10.1586/erp.09.36
Whitehead SJ, Ali S. Health outcomes in economic evaluation: the QALY and utilities. Br Med Bull. 2010;96:5–1.
pubmed: 21037243
doi: 10.1093/bmb/ldq033
Muchadeyi MT, Hernandez-Villafuerte K, Schlander M. Quality appraisal for systematic literature reviews of health state utility values: a descriptive analysis. BMC Med Res Methodol. 2022;22(1):303.
pubmed: 36434521
pmcid: 9700894
doi: 10.1186/s12874-022-01784-6
Zoratti MJ, Pickard AS, Stalmeier PFM, Ollendorf D, Lloyd A, Chan KKW, et al. Evaluating the conduct and application of health utility studies: a review of critical appraisal tools and reporting checklists. Eur J Health Econ. 2021;22(5):723–33.
pubmed: 33839968
doi: 10.1007/s10198-021-01286-0
Longworth L, Rowen D. NICE DSU Technical Support Document 10: The Use of Mapping Methods to Estimate Health State Utility Values [Internet]. London: National Institute for Health and Care Excellence (NICE); 2011. PMID: 28481491.
Petrou S, Kwon J, Madan J. A practical guide to conducting a systematic review and meta-analysis of health state utility values. Pharmacoeconomics. 2018;36(9):1043–61.
pubmed: 29750430
doi: 10.1007/s40273-018-0670-1
Xie F, Zoratti M, Chan K, Husereau D, Krahn M, Levine O, et al. Toward a centralized, systematic approach to the identification, appraisal, and use of health state utility values for reimbursement decision making: introducing the Health Utility Book (HUB). Med Decis Mak. 2019;39(4):370–8.
doi: 10.1177/0272989X19837969
Ara R, Brazier J, Peasgood T, Paisley S. The identification, review and synthesis of health state utility values from the literature. Pharmacoeconomics. 2017;35(Suppl. 1):43–55.
pubmed: 29052156
doi: 10.1007/s40273-017-0547-8
Ara R, Hill H, Lloyd A, Woods HB, Brazier J. Are current reporting standards used to describe health state utilities in cost-effectiveness models satisfactory? Value Health. 2020;23(3):397–405.
pubmed: 32197736
doi: 10.1016/j.jval.2019.12.004
Ara R, Peasgood T, Mukuria C, Chevrou-Severac H, Rowen D, Azzabi-Zouraq I, et al. Sourcing and using appropriate health state utility values in economic models in health care. Pharmacoeconomics. 2017;35(Suppl. 1):7–9.
pubmed: 29052161
doi: 10.1007/s40273-017-0543-z
Sanderson S, Tatt ID, Higgins JP. Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography. Int J Epidemiol. 2007;36(3):666–76.
pubmed: 17470488
doi: 10.1093/ije/dym018
Yepes-Nuñez JJ, Zhang Y, Xie F, Alonso-Coello P, Selva A, Schünemann H, et al. Forty-two systematic reviews generated 23 items for assessing the risk of bias in values and preferences’ studies. J Clin Epidemiol. 2017;85:21–31.
pubmed: 28478082
doi: 10.1016/j.jclinepi.2017.04.019
Büttner F, Winters M, Delahunt E, Elbers R, Lura CB, Khan KM, et al. Identifying the ’incredible’! Part 1: assessing the risk of bias in outcomes included in systematic reviews. Br J Sports Med. 2020;54(13):798–800.
pubmed: 31871015
doi: 10.1136/bjsports-2019-100806
Büttner F, Winters M, Delahunt E, Elbers R, Lura CB, Khan KM, et al. Identifying the ’incredible’! Part 2: spot the difference: a rigorous risk of bias assessment can alter the main findings of a systematic review. Br J Sports Med. 2020;54(13):801–8.
pubmed: 31871014
doi: 10.1136/bjsports-2019-101675
Katikireddi SV, Egan M, Petticrew M. How do systematic reviews incorporate risk of bias assessments into the synthesis of evidence? A methodological study. J Epidemiol Commun Health. 2015;69(2):189–95.
doi: 10.1136/jech-2014-204711
Marušić MF, Fidahić M, Cepeha CM, Farcaș LG, Tseke A, Puljak L. Methodological tools and sensitivity analysis for assessing quality or risk of bias used in systematic reviews published in the high-impact anesthesiology journals. BMC Med Res Methodol. 2020;20(1):121.
pubmed: 32423382
pmcid: 7236513
doi: 10.1186/s12874-020-00966-4
Walker DG, Wilson RF, Sharma R, Bridges J, Niessen L, Bass EB, et al. AHRQ methods for effective health care. Best practices for conducting economic evaluations in health care: a systematic review of quality assessment tools. Rockville: Agency for Healthcare Research and Quality (US); 2012.
Afshari S, Ameri H, Daroudi RA, Shiravani M, Karami H, Akbari SA. Health related quality of life in adults with asthma: a systematic review to identify the values of EQ-5D-5L instrument. J Asthma. 2022;59(6):1203–12.
pubmed: 33863264
doi: 10.1080/02770903.2021.1917607
Brennan VK, Mauskopf J, Colosia AD, Copley-Merriman C, Hass B, Palencia R. Utility estimates for patients with type 2 diabetes mellitus after experiencing a myocardial infarction or stroke: a systematic review. Expert Rev Pharmacoecon Outcomes Res. 2015;15(1):111–23.
pubmed: 25555462
doi: 10.1586/14737167.2015.965152
Buchanan-Hughes AM, Buti M, Hanman K, Langford B, Wright M, Eddowes LA. Health state utility values measured using the EuroQol 5-dimensions questionnaire in adults with chronic hepatitis C: a systematic literature review and meta-analysis. Qual Life Res. 2019;28(2):297–319.
pubmed: 30225787
doi: 10.1007/s11136-018-1992-3
Carrello J, Hayes A, Killedar A, Von Huben A, Baur LA, Petrou S, et al. Utility decrements associated with adult overweight and obesity in Australia: a systematic review and meta-analysis. Pharmacoeconomics. 2021;39(5):503–19.
pubmed: 33615427
doi: 10.1007/s40273-021-01004-x
Carter GC, King DT, Hess LM, Mitchell SA, Taipale KL, Kiiskinen U, et al. Health state utility values associated with advanced gastric, oesophageal, or gastro-oesophageal junction adenocarcinoma: a systematic review. J Med Econ. 2015;18(11):954–66.
pubmed: 26212479
doi: 10.3111/13696998.2015.1066380
Di Tanna GL, Urbich M, Wirtz HS, Potrata B, Heisen M, Bennison C, et al. Health state utilities of patients with heart failure: a systematic literature review. Pharmacoeconomics. 2021;39(2):211–29.
pubmed: 33251572
doi: 10.1007/s40273-020-00984-6
Gheorghe A, Moran G, Duffy H, Roberts T, Pinkney T, Calvert M. Health utility values associated with surgical site infection: a systematic review. Value Health. 2015;18(8):1126–37.
pubmed: 26686800
doi: 10.1016/j.jval.2015.08.004
Han R, François C, Toumi M. Systematic review of health state utility values used in European pharmacoeconomic evaluations for chronic hepatitis C: impact on cost-effectiveness results. Appl Health Econ Health Policy. 2021;19(1):29–44.
pubmed: 32661846
doi: 10.1007/s40258-020-00600-w
Kua WS, Davis S. PRS49: systematic review of health state utilities in children with asthma. Value Health. 2016;19(7):A557.
doi: 10.1016/j.jval.2016.09.1218
Li L, Severens JLH, Mandrik O. Disutility associated with cancer screening programs: a systematic review. PLoS ONE. 2019;14(7): e0220148.
pubmed: 31339958
pmcid: 6655768
doi: 10.1371/journal.pone.0220148
Paracha N, Abdulla A, MacGilchrist KS. Systematic review of health state utility values in metastatic non-small cell lung cancer with a focus on previously treated patients. Health Qual Life Outcomes. 2018;16(1):179.
pubmed: 30208899
pmcid: 6134713
doi: 10.1186/s12955-018-0994-8
Park HY, Cheon HB, Choi SH, Kwon JW. Health-related quality of life based on EQ-5D utility score in patients with tuberculosis: a systematic review. Front Pharmacol. 2021;12: 659675.
pubmed: 33935781
pmcid: 8080025
doi: 10.3389/fphar.2021.659675
Petrou S, Krabuanrat N, Khan K. Preference-based health-related quality of life outcomes associated with preterm birth: a systematic review and meta-analysis. Pharmacoeconomics. 2020;38(4):357–73.
pubmed: 31814079
doi: 10.1007/s40273-019-00865-7
Saeed YA, Phoon A, Bielecki JM, Mitsakakis N, Bremner KE, Abrahamyan L, et al. A systematic review and meta-analysis of health utilities in patients with chronic hepatitis C. Value Health. 2020;23(1):127–37.
pubmed: 31952667
doi: 10.1016/j.jval.2019.07.005
Szabo SM, Audhya IF, Malone DC, Feeny D, Gooch KL. Characterizing health state utilities associated with Duchenne muscular dystrophy: a systematic review. Qual Life Res. 2020;29(3):593–605.
pubmed: 31811595
doi: 10.1007/s11136-019-02355-x
Van Wilder L, Rammant E, Clays E, Devleesschauwer B, Pauwels N, De Smedt D. A comprehensive catalogue of EQ-5D scores in chronic disease: results of a systematic review. Qual Life Res. 2019;28(12):3153–61.
pubmed: 31531840
doi: 10.1007/s11136-019-02300-y
Ward Fuller G, Hernandez M, Pallot D, Lecky F, Stevenson M, Gabbe B. Health state preference weights for the Glasgow Outcome Scale following traumatic brain injury: a systematic review and mapping study. Value Health. 2017;20(1):141–51.
pubmed: 28212955
pmcid: 5339086
doi: 10.1016/j.jval.2016.09.2398
Yuan Y, Xiao Y, Chen X, Li J, Shen M. A systematic review and meta-analysis of health utility estimates in chronic spontaneous urticaria. Front Med (Lausanne). 2020;7: 543290.
pubmed: 33425930
pmcid: 7793814
doi: 10.3389/fmed.2020.543290
Blom EF, Haaf KT, de Koning HJ. Systematic review and meta-analysis of community- and choice-based health state utility values for lung cancer. Pharmacoeconomics. 2020;38(11):1187–200.
pubmed: 32754857
pmcid: 7547043
doi: 10.1007/s40273-020-00947-x
Foster E, Chen Z, Ofori-Asenso R, Norman R, Carney P, O’Brien TJ, et al. Comparisons of direct and indirect utilities in adult epilepsy populations: a systematic review. Epilepsia. 2019;60(12):2466–76.
pubmed: 31784994
doi: 10.1111/epi.16396
Haridoss M, Bagepally BS, Natarajan M. Health-related quality of life in rheumatoid arthritis: systematic review and meta-analysis of EuroQoL (EQ-5D) utility scores from Asia. Int J Rheum Dis. 2021;24(3):314–26.
pubmed: 33486900
doi: 10.1111/1756-185X.14066
Jiang M, Ma Y, Li M, Meng R, Ma A, Chen P. A comparison of self-reported and proxy-reported health utilities in children: a systematic review and meta-analysis. Health Qual Life Outcomes. 2021;19(1):45.
pubmed: 33546723
pmcid: 7866432
doi: 10.1186/s12955-021-01677-0
Landeiro F, Mughal S, Walsh K, Nye E, Morton J, Williams H, et al. Health-related quality of life in people with predementia Alzheimer’s disease, mild cognitive impairment or dementia measured with preference-based instruments: a systematic literature review. Alzheimers Res Ther. 2020;12(1):154.
pubmed: 33208190
pmcid: 7677851
doi: 10.1186/s13195-020-00723-1
Rebchuk AD, O’Neill ZR, Szefer EK, Hill MD, Field TS. Health utility weighting of the Modified Rankin Scale: a systematic review and meta-analysis. JAMA Netw Open. 2020;3(4): e203767.
pubmed: 32347948
pmcid: 7191324
doi: 10.1001/jamanetworkopen.2020.3767
Tran AD, Fogarty G, Nowak AK, Espinoza D, Rowbotham N, Stockler MR, et al. A systematic review and meta-analysis of utility estimates in melanoma. Br J Dermatol. 2018;178(2):384–93.
pubmed: 29077983
doi: 10.1111/bjd.16098
Yang Z, Li S, Wang X, Chen G. Health state utility values derived from EQ-5D in psoriatic patients: a systematic review and meta-analysis. J Dermatol Treat. 2020;9:1–8.
Golicki D, Jaśkowiak K, Wójcik A, Młyńczak K, Dobrowolska I, Gawrońska A, et al. EQ-5D-derived health state utility values in hematologic malignancies: a catalog of 796 utilities based on a systematic review. Value Health. 2020;23(7):953–68.
pubmed: 32762998
doi: 10.1016/j.jval.2020.04.1825
Paracha N, Thuresson PO, Moreno SG, MacGilchrist KS. Health state utility values in locally advanced and metastatic breast cancer by treatment line: a systematic review. Expert Rev Pharmacoecon Outcomes Res. 2016;16(5):549–59.
pubmed: 27574879
doi: 10.1080/14737167.2016.1222907
Cooper JT, Lloyd A, Sanchez JJG, Sörstadius E, Briggs A, McFarlane P. Health related quality of life utility weights for economic evaluation through different stages of chronic kidney disease: a systematic literature review. Health Qual Life Outcomes. 2020;18(1):310.
pubmed: 32957990
pmcid: 7507735
doi: 10.1186/s12955-020-01559-x
Aceituno D, Pennington M, Iruretagoyena B, Prina AM, McCrone P. Health state utility values in schizophrenia: a systematic review and meta-analysis. Value Health. 2020;23(9):1256–67.
pubmed: 32940244
doi: 10.1016/j.jval.2020.05.014
Li YK, Alolabi N, Kaur MN, Thoma A. A systematic review of utilities in hand surgery literature. J Hand Surg Am. 2015;40(5):997–1005.
pubmed: 25771479
doi: 10.1016/j.jhsa.2015.01.020
Magnus A, Isaranuwatchai W, Mihalopoulos C, Brown V, Carter R. A systematic review and meta-analysis of prostate cancer utility values of patients and partners between 2007 and 2016. MDM Policy Pract. 2019;4(1):2381468319852332.
Meregaglia M, Cairns J. A systematic literature review of health state utility values in head and neck cancer. Health Qual Life Outcomes. 2017;15(1):174.
pubmed: 28865475
pmcid: 5581467
doi: 10.1186/s12955-017-0748-z
Ó Céilleachair A, O'Mahony JF, O'Connor M, O'Leary J, Normand C, Martin C, et al. Health-related quality of life as measured by the EQ-5D in the prevention, screening and management of cervical disease: a systematic review. Qual Life Res. 2017;26(11):2885–97.
Khadka J, Kwon J, Petrou S, Lancsar E, Ratcliffe J. Mind the (inter-rater) gap. An investigation of self-reported versus proxy-reported assessments in the derivation of childhood utility values for economic evaluation: a systematic review. Soc Sci Med (1982). 2019;240:112543.
Kwon J, Kim SW, Ungar WJ, Tsiplova K, Madan J, Petrou S. A systematic review and meta-analysis of childhood health utilities. Med Decis Mak. 2018;38(3):277–305.
doi: 10.1177/0272989X17732990
Hatswell AJ, Burns D, Baio G, Wadelin F. Frequentist and Bayesian meta-regression of health state utilities for multiple myeloma incorporating systematic review and analysis of individual patient data. Health Econ. 2019;28(5):653–65.
pubmed: 30790379
doi: 10.1002/hec.3871
Moher D, Jadad AR, Nichol G, Penman M, Tugwell P, Walsh S. Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. Control Clin Trials. 1995;16(1):62–73.
pubmed: 7743790
doi: 10.1016/0197-2456(94)00031-W
Belcher BM, Rasmussen KE, Kemshaw MR, Zornes D. Defining and assessing research quality in a transdisciplinary context. Res Eval. 2016;25:1–17.
doi: 10.1093/reseval/rvv025
Oxman AD, Guyatt GH. Validation of an index of the quality of review articles. J Clin Epidemiol. 1991;44(11):1271–8.
pubmed: 1834807
doi: 10.1016/0895-4356(91)90160-B
Verhagen AP, de Vet HC, de Bie RA, Kessels AG, Boers M, Bouter LM, et al. The Delphi list: a criteria list for quality assessment of randomized clinical trials for conducting systematic reviews developed by Delphi consensus. J Clin Epidemiol. 1998;51(12):1235–41.
pubmed: 10086815
doi: 10.1016/S0895-4356(98)00131-0
Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials. 1996;17(1):1–12.
pubmed: 8721797
doi: 10.1016/0197-2456(95)00134-4
Whiting P, Rutjes AWS, Reitsma JB, Bossuyt PMM, Kleijnen J. The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Med Res Methodol. 2003;3(1):25.
pubmed: 14606960
pmcid: 305345
doi: 10.1186/1471-2288-3-25
Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011;155(8):529–36.
pubmed: 22007046
doi: 10.7326/0003-4819-155-8-201110180-00009
Centre for Reviews and Dissemination (CRD). Systematic reviews: CRD's guidance for undertaking reviews in health care. 3rd ed. York: Centre for Reviews and Dissemination, University of York Centre for Reviews and Dissemination, University of York; 2013.
Viswanathan M, Patnode CD, Berkman ND, Bass EB, Chang S, Hartling L, et al. AHRQ methods for effective health care assessing the risk of bias in systematic reviews of health care interventions. Methods guide for effectiveness and comparative effectiveness reviews. Rockville: Agency for Healthcare Research and Quality (US); 2008.
Viswanathan M, Patnode CD, Berkman ND, Bass EB, Chang S, Hartling L, et al. Recommendations for assessing the risk of bias in systematic reviews of health-care interventions. J Clin Epidemiol. 2018;97:26–34.
pubmed: 29248724
doi: 10.1016/j.jclinepi.2017.12.004
Armijo-Olivo S, Fuentes J, Ospina M, Saltaji H, Hartling L. Inconsistency in the items included in tools used in general health research and physical therapy to evaluate the methodological quality of randomized controlled trials: a descriptive analysis. BMC Med Res Methodol. 2013;17(13):116.
doi: 10.1186/1471-2288-13-116
Balshem H, Helfand M, Schünemann HJ, Oxman AD, Kunz R, Brozek J, et al. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol. 2011;64(4):401–6.
pubmed: 21208779
doi: 10.1016/j.jclinepi.2010.07.015
Kim DD, Do LA, Synnott PG, Lavelle TA, Prosser LA, Wong JB, et al. Developing criteria for health economic quality evaluation tool. Value Health. 2023;26(8):1225–34.
pubmed: 37068557
doi: 10.1016/j.jval.2023.04.004
Littlewood C, Ashton J, Chance-Larsen K, May S, Sturrock B. The quality of reporting might not reflect the quality of the study: implications for undertaking and appraising a systematic review. J Man Manip Ther. 2012;20(3):130–4.
pubmed: 23904751
pmcid: 3419569
doi: 10.1179/2042618611Y.0000000013
Jüni P, Witschi A, Bloch R, Egger M. The hazards of scoring the quality of clinical trials for meta-analysis. JAMA. 1999;282(11):1054–60.
pubmed: 10493204
doi: 10.1001/jama.282.11.1054
Nasa P, Jain R, Juneja D. Delphi methodology in healthcare research: how to decide its appropriateness. World J Methodol. 2021;11(4):116–29.
pubmed: 34322364
pmcid: 8299905
doi: 10.5662/wjm.v11.i4.116
Hsu C-C, Sandford BA. The Delphi technique: making sense of consensus. Pract Assess Res Eval. 2007;12:10.
Jones J, Hunter D. Consensus methods for medical and health services research. BMJ (Clin Res Ed). 1995;311(7001):376–80.
doi: 10.1136/bmj.311.7001.376
Murphy MK, Black NA, Lamping DL, McKee CM, Sanderson CF, Askham J, et al. Consensus development methods, and their use in clinical guideline development. Health Technol Assess. 1998;2(3):i–iv, 1–88.
Keeney S, Hasson F, McKenna H. Consulting the oracle: ten lessons from using the Delphi technique in nursing research. J Adv Nurs. 2006;53(2):205–12.
pubmed: 16422719
doi: 10.1111/j.1365-2648.2006.03716.x
Al-Jundi A, Sakka S. Critical appraisal of clinical research. J Clin Diagn Res. 2017;11(5):JE01–5.
Burls A. What is critical appraisal? 2014. http://www.bandolier.org.uk/painres/download/whatis/What_is_critical_appraisal.pdf . Accessed 5 Nov 2021.
Higgins JPT, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ (Clin Res Ed). 2011;343: d5928.
doi: 10.1136/bmj.d5928
Eiring Ø, Landmark BF, Aas E, Salkeld G, Nylenna M, Nytrøen K. What matters to patients? A systematic review of preferences for medication-associated outcomes in mental disorders. BMJ Open. 2015;5(4): e007848.
pubmed: 25854979
pmcid: 4390680
doi: 10.1136/bmjopen-2015-007848
Brazier J, Ara R, Azzabi I, Busschbach J, Chevrou-Séverac H, Crawford B, et al. Identification, review, and use of health state utilities in cost-effectiveness models: an ISPOR Good Practices for Outcomes Research Task Force report. Value Health. 2019;22(3):267–75.
pubmed: 30832964
doi: 10.1016/j.jval.2019.01.004
Shamliyan T, Kane RL, Dickinson S. A systematic review of tools used to assess the quality of observational studies that examine incidence or prevalence and risk factors for diseases. J Clin Epidemiol. 2010;63(10):1061–70.
pubmed: 20728045
doi: 10.1016/j.jclinepi.2010.04.014
Drummond MF, Sculpher MJ, Claxton K, Stoddart GL, Torrance GW, Drummond D. Methods for the economic evaluation of health care programmes. 4th ed. Oxford: Oxford University Press; 2015.
Oliver A. Distinguishing between experienced utility and remembered utility. Public Health Ethics. 2016;10(2):122–8.
doi: 10.1093/phe/phw014
Pieterse AH, Stiggelbout AM. What are values, utilities, and preferences? A clarification in the context of decision making in health care, and an exploration of measurement issues. In: Diefenbach MA, Miller-Halegoua S, Bowen DJ, editors. Handbook of health decision science. New York: Springer New York; 2016. p. 3–13.
Sterne JAC, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ (Clin Res Ed). 2016;355: i4919.
Sterne JAC, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ (Clin Res Ed). 2019;366: l4898.
Morgan DL, Krueger RA. When to use focus groups and why. Successful focus groups: advancing the state of the art. Thousand Oaks: Sage Publications, Inc.; 1993: p. 3–19.
Massey OT. A proposed model for the analysis and interpretation of focus groups in evaluation research. Eval Program Plann. 2011;34(1):21–8.
pubmed: 20655593
doi: 10.1016/j.evalprogplan.2010.06.003
Bernstein SJ, Laouri M, Hilborne LH, Leape LL, Kahan JP, Park RE, et al. Coronary angiography: a literature review and ratings of appropriateness and necessity. Santa Monica: RAND Corporation; 1992.
Culyer AJ. The dictionary of health economics. Cheltenham: Edward Elgar Publishing; 2005.
doi: 10.4337/9781845426736