Angiographic results of surgical or endovascular treatment of intracranial aneurysms: a systematic review and inter-observer reliability study.
Classification system
Digital subtraction angiography
Inter-rater reliability
Intracranial aneurysms
Journal
Neuroradiology
ISSN: 1432-1920
Titre abrégé: Neuroradiology
Pays: Germany
ID NLM: 1302751
Informations de publication
Date de publication:
Sep 2021
Sep 2021
Historique:
received:
05
11
2020
accepted:
15
02
2021
pubmed:
25
2
2021
medline:
30
9
2021
entrez:
24
2
2021
Statut:
ppublish
Résumé
Results of surgical or endovascular treatment of intracranial aneurysms are often assessed using angiography. A reliable method to report results irrespective of treatment modality is needed to enable comparisons. Our goals were to systematically review existing classification systems, and to propose a 3-point classification applicable to both treatments and assess its reliability. We conducted two systematic reviews on classification systems of angiographic results after clipping or coiling to select a simple 3-category scale that could apply to both treatments. We then circulated an electronic portfolio of angiograms of clipped (n=30) or coiled (n=30) aneurysms, and asked raters to evaluate the degree of occlusion using this scale. Raters were also asked to choose an appropriate follow-up management for each patient based on the degree of occlusion. Agreement was assessed using Krippendorff's α statistics (α The systematic reviews found 70 different grading scales with heterogeneous reliability (kappa values from 0.12 to 1.00). The 60-patient portfolio was independently evaluated by 19 raters of diverse backgrounds (neurosurgery, radiology, and neurology) and experience. There was substantial agreement (α A simple 3-point scale demonstrated sufficient reliability to be used in reporting aneurysm treatments or in evaluating treatment results in comparative randomized trials.
Identifiants
pubmed: 33625550
doi: 10.1007/s00234-021-02676-0
pii: 10.1007/s00234-021-02676-0
doi:
Types de publication
Journal Article
Systematic Review
Langues
eng
Sous-ensembles de citation
IM
Pagination
1511-1519Subventions
Organisme : Fonds de Recherche du Québec - Santé
ID : 271994
Informations de copyright
© 2021. The Author(s), under exclusive licence to Springer-Verlag GmbH, DE part of Springer Nature.
Références
Macdonald RL, Schweizer TA (2017) Spontaneous subarachnoid haemorrhage. Lancet 389(10069):655–666. https://doi.org/10.1016/S0140-6736(16)30668-7
doi: 10.1016/S0140-6736(16)30668-7
pubmed: 27637674
Johnston SC, Dowd CF, Higashida RT, Lawton MT, Duckwiler GR, Gress DR, CARAT Investigators (2008) Predictors of rehemorrhage after treatment of ruptured intracranial aneurysms: the Cerebral Aneurysm Rerupture After Treatment (CARAT) study. Stroke 39(1):120–125. https://doi.org/10.1161/STROKEAHA.107.495747
doi: 10.1161/STROKEAHA.107.495747
pubmed: 18048860
White PM, Lewis SC, Nahser H, Sellar RJ, Goddard T, Gholkar A, on behalf of the HELPS Trial Collaboration (2008) HydroCoil Endovascular Aneurysm Occlusion and Packing Study (HELPS trial): procedural safety and operator-assessed efficacy results. AJNR Am J Neuroradiol 29(2):217–223. https://doi.org/10.3174/ajnr.A0936
doi: 10.3174/ajnr.A0936
pubmed: 18184832
pmcid: 8119017
Darsaut TE, Findlay JM, Raymond J, Group CC (2011) The design of the Canadian UnRuptured Endovascular versus Surgery (CURES) trial. Can J Neurol Sci 38(2):236–241. https://doi.org/10.1017/s0317167100011392
doi: 10.1017/s0317167100011392
pubmed: 21320826
pmcid: 3528784
Raymond J, Roy D, White PM, Fiorella D, Chapot R, Bracard S, Kallmes DF, THE ICONE COLLABORATIVE GROUP (2008) A randomized trial comparing platinum and hydrogel-coated coils in patients prone to recurrence after endovascular treatment (the PRET trial). Interv Neuroradiol 14(1):73–83. https://doi.org/10.1177/159101990801400110
doi: 10.1177/159101990801400110
pubmed: 20557789
pmcid: 3313709
Becske T, Kallmes DF, Saatci I, McDougall CG, Szikora I, Lanzino G et al (2013) Pipeline for uncoilable or failed aneurysms: results from a multicenter clinical trial. Radiology 267(3):858–868. https://doi.org/10.1148/radiol.13120099
doi: 10.1148/radiol.13120099
pubmed: 23418004
pmcid: 23418004
Pierot L, Moret J, Barreau X, Szikora I, Herbreteau D, Turjman F, Holtmannspötter M, Januel AC, Costalat V, Fiehler J, Klisch J, Gauvrit JY, Weber W, Desal H, Velasco S, Liebig T, Stockx L, Berkefeld J, Molyneux A, Byrne J, Spelle L (2018) Safety and efficacy of aneurysm treatment with WEB in the cumulative population of three prospective, multicenter series. J Neurointerv Surg 10(6):553–559. https://doi.org/10.1136/neurintsurg-2017-013448
doi: 10.1136/neurintsurg-2017-013448
pubmed: 28965106
Darsaut TE, Chapot R, Raymond J (2020) Changing the rules of the game: the problem of surrogate angiographic outcomes in the evaluation of aneurysm treatments. AJNR Am J Neuroradiol 41:2174–2175. https://doi.org/10.3174/ajnr.A6825
doi: 10.3174/ajnr.A6825
pubmed: 33033042
Molyneux AJ, Kerr RS, Yu LM, Clarke M, Sneade M, Yarnold JA et al (2005) International subarachnoid aneurysm trial (ISAT) of neurosurgical clipping versus endovascular coiling in 2143 patients with ruptured intracranial aneurysms: a randomised comparison of effects on survival, dependency, seizures, rebleeding, subgroups, and aneurysm occlusion. Lancet 366(9488):809–817. https://doi.org/10.1016/S0140-6736(05)67214-5
doi: 10.1016/S0140-6736(05)67214-5
pubmed: 16139655
Burkhardt JK, Chua MHJ, Weiss M, Do ASS, Winkler EA, Lawton MT (2017) Risk of aneurysm residual regrowth, recurrence, and de novo aneurysm formation after microsurgical clip occlusion based on follow-up with catheter angiography. World Neurosurg 106:74–84. https://doi.org/10.1016/j.wneu.2017.06.110
doi: 10.1016/j.wneu.2017.06.110
pubmed: 28648910
Kotowski M, Farzin B, Fahed R, Guilbert F, Chagnon M, Darsaut TE, Daniel RT, Raymond J (2019) Residual cerebral aneurysms after microsurgical clipping: a new scale, an agreement study, and a systematic review of the literature. World Neurosurg 121:e302–e321. https://doi.org/10.1016/j.wneu.2018.09.100
doi: 10.1016/j.wneu.2018.09.100
pubmed: 30261387
Darsaut TE, Jack AS, Kerr RS, Raymond J (2013) International subarachnoid aneurysm trial - ISAT part II: study protocol for a randomized controlled trial. Trials 14:156. https://doi.org/10.1186/1745-6215-14-156
doi: 10.1186/1745-6215-14-156
pubmed: 23714335
pmcid: 3680206
Darsaut TE, Findlay JM, Magro E, Kotowski M, Roy D, Weill A, Bojanowski MW, Chaalala C, Iancu D, Lesiuk H, Sinclair J, Scholtes F, Martin D, Chow MM, O’Kelly CJ, Wong JH, Butcher K, Fox AJ, Arthur AS, Guilbert F, Tian L, Chagnon M, Nolet S, Gevry G, Raymond J (2017) Surgical clipping or endovascular coiling for unruptured intracranial aneurysms: a pragmatic randomised trial. J Neurol Neurosurg Psychiatry 88(8):663–668. https://doi.org/10.1136/jnnp-2016-315433
doi: 10.1136/jnnp-2016-315433
pubmed: 28634280
Darsaut TE, Desal H, Cognard C, Januel AC, Bourcier R, Boulouis G, Shiva Shankar JJ, Findlay JM, Rempel JL, Fahed R, Boccardi E, Valvassori L, Magro E, Gentric JC, Bojanowski MW, Chaalala C, Iancu D, Roy D, Weill A, Diouf A, Gevry G, Chagnon M, Raymond J (2020) Comprehensive aneurysm management (CAM): an all-inclusive care trial for unruptured intracranial aneurysms. World Neurosurg 141:e770–e777. https://doi.org/10.1016/j.wneu.2020.06.018
doi: 10.1016/j.wneu.2020.06.018
pubmed: 32526362
Cloft HJ, Kallmes DF (2011) Scaling back on scales with a scale of scales. AJNR Am J Neuroradiol 32(2):219–220. https://doi.org/10.3174/ajnr.A2432
doi: 10.3174/ajnr.A2432
pubmed: 21183620
pmcid: 7965709
Tollard E, Darsaut TE, Bing F, Guilbert F, Gevry G, Raymond J (2012) Outcomes of endovascular treatments of aneurysms: observer variability and implications for interpreting case series and planning randomized trials. AJNR Am J Neuroradiol 33(4):626–631. https://doi.org/10.3174/ajnr.A2848
doi: 10.3174/ajnr.A2848
pubmed: 22194386
pmcid: 8050430
Ernst M, Yoo AJ, Kriston L, Schonfeld MH, Vettorazzi E, Fiehler J (2015) Is visual evaluation of aneurysm coiling a reliable study end point? Systematic review and meta-analysis. Stroke 46(6):1574–1581. https://doi.org/10.1161/STROKEAHA.114.008513
doi: 10.1161/STROKEAHA.114.008513
pubmed: 25944331
Jamali S, Fahed R, Gentric JC, Letourneau-Guillon L, Raoult H, Bing F, Estrade L, Nguyen TN, Tollard E, Ferre JC, Iancu D, Naggara O, Chagnon M, Weill A, Roy D, Fox AJ, Kallmes DF, Raymond J (2016) Inter- and intrarater agreement on the outcome of endovascular treatment of aneurysms using MRA. AJNR Am J Neuroradiol 37(5):879–884. https://doi.org/10.3174/ajnr.A4609
doi: 10.3174/ajnr.A4609
pubmed: 26659336
pmcid: 7960328
Raymond J, Roy D, Bojanowski M, Moumdjian R, L’Esperance G (1997) Endovascular treatment of acutely ruptured and unruptured aneurysms of the basilar bifurcation. J Neurosurg 86(2):211–219. https://doi.org/10.3171/jns.1997.86.2.0211
doi: 10.3171/jns.1997.86.2.0211
pubmed: 9010422
Roy D, Milot G, Raymond J (2001) Endovascular treatment of unruptured aneurysms. Stroke 32(9):1998–2004. https://doi.org/10.1161/hs0901.095600
doi: 10.1161/hs0901.095600
pubmed: 11546888
Raymond J, Guilbert F, Weill A, Georganos SA, Juravsky L, Lambert A et al (2003) Long-term angiographic recurrences after selective endovascular treatment of aneurysms with detachable coils. Stroke 34(6):1398–1403. https://doi.org/10.1161/01.STR.0000073841.88563.E9
doi: 10.1161/01.STR.0000073841.88563.E9
pubmed: 12775880
pmcid: 12775880
Kottner J, Audige L, Brorson S, Donner A, Gajewski BJ, Hrobjartsson A et al (2011) Guidelines for reporting reliability and agreement studies (GRRAS) were proposed. J Clin Epidemiol 64(1):96–106. https://doi.org/10.1016/j.jclinepi.2010.03.002
doi: 10.1016/j.jclinepi.2010.03.002
pubmed: 21130355
Moher D, Liberati A, Tetzlaff J, Altman DG, Group P (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 6(7):e1000097. https://doi.org/10.1371/journal.pmed.1000097
doi: 10.1371/journal.pmed.1000097
pubmed: 2707599
pmcid: 2707599
Donner A, Rotondi MA (2010) Sample size requirements for interval estimation of the kappa statistic for interobserver agreement studies with a binary outcome and multiple raters. Int J Biostat 6(1):Article 31. https://doi.org/10.2202/1557-4679.1275
Fahed R, Darsaut TE, Farzin B, Chagnon M, Raymond J (2020) Measuring clinical uncertainty and equipoise by applying the agreement study methodology to patient management decisions. BMC Med Res Methodol 20(1):214. https://doi.org/10.1186/s12874-020-01095-8
doi: 10.1186/s12874-020-01095-8
pubmed: 32842953
pmcid: 7448326
Feinstein AR, Cicchetti DV (1990) High agreement but low kappa: I. The problems of two paradoxes. J Clin Epidemiol 43(6):543–549. https://doi.org/10.1016/0895-4356(90)90158-l
doi: 10.1016/0895-4356(90)90158-l
pubmed: 2348207
Cicchetti DV, Feinstein AR (1990) High agreement but low kappa: II. Resolving the paradoxes. J Clin Epidemiol 43(6):551–558. https://doi.org/10.1016/0895-4356(90)90159-m
doi: 10.1016/0895-4356(90)90159-m
pubmed: 2189948
Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33(1):159–174
pubmed: 843571
pmcid: 843571
McDougall CG, Johnston SC, Hetts SW, Gholkar A, Barnwell SL, Vazquez Suarez JC et al (2020) Five-year results of randomized bioactive versus bare metal coils in the treatment of intracranial aneurysms: the Matrix and Platinum Science (MAPS) Trial. J Neurointerv Surg:neurintsurg-2020-016906. https://doi.org/10.1136/neurintsurg-2020-016906
Byrt T, Bishop J, Carlin JB (1993) Bias, prevalence and kappa. J Clin Epidemiol 46(5):423–429. https://doi.org/10.1016/0895-4356(93)90018-v
doi: 10.1016/0895-4356(93)90018-v
pubmed: 8501467
pmcid: 8501467
Quarfoot A, Levine RA (2016) How robust are multirater interrater reliability indices to changes in frequency distribution? Am Stat 70(4):373–384. https://doi.org/10.1080/00031305.2016.1141708
doi: 10.1080/00031305.2016.1141708
Gwet KL (2014) The kappa coefficient and its paradoxes. In: handbook of inter-rater reliability: the definitive guide to measuring the extent of agreement among raters. 4th edn. Advanced Analytics, LLC, Gaithersburg, MD, pp 57–62