Inter-reader variability of SPECT MPI readings in low- and middle-income countries: Results from the IAEA-MPI Audit Project (I-MAP).
Aged
Cardiac-Gated Single-Photon Emission Computer-Assisted Tomography
Developing Countries
Exercise Test
Female
Geography
Heart
Heart Ventricles
Humans
International Cooperation
Male
Middle Aged
Myocardial Perfusion Imaging
/ methods
Observer Variation
Positron-Emission Tomography
/ methods
Poverty
Risk
Stroke Volume
Technetium
Tomography, Emission-Computed, Single-Photon
/ methods
Ventricular Dysfunction, Left
/ physiopathology
Ventricular Function, Left
CAD
Gated SPECT
Image interpretation
Myocardial ischemia and infarction
Journal
Journal of nuclear cardiology : official publication of the American Society of Nuclear Cardiology
ISSN: 1532-6551
Titre abrégé: J Nucl Cardiol
Pays: United States
ID NLM: 9423534
Informations de publication
Date de publication:
04 2020
04 2020
Historique:
received:
04
07
2018
accepted:
03
08
2018
pubmed:
1
9
2018
medline:
10
8
2021
entrez:
1
9
2018
Statut:
ppublish
Résumé
Consistency of results between different readers is an important issue in medical imaging, as it affects portability of results between institutions and may affect patient care. The International Atomic Energy Agency (IAEA) in pursuing its mission of fostering peaceful applications of nuclear technologies has supported several training activities in the field of nuclear cardiology (NC) and SPECT myocardial perfusion imaging (MPI) in particular. The aim of this study was to verify the outcome of those activities through an international clinical audit on MPI where participants were requested to report on studies distributed from a core lab. The study was run in two phases: in phase 1, SPECT MPI studies were distributed as raw data and full processing was requested as per local practice. In phase 2, images from studies pre-processed at the core lab were distributed. Data to be reported included summed stress score (SSS); summed rest score (SRS); summed difference score (SDS); left ventricular (LV) ejection fraction (EF) and end- diastolic volume (EDV). Qualitative appraisals included the assessment of perfusion and presence of ischemia, scar or mixed patterns, presence of transient ischemic dilation (TID), and risk for cardiac events (CE). Twenty-four previous trainees from low- and middle-income countries participated (core participants group) and their results were assessed for inter-observer variability in each of the two phases, and for changes between phases. The same evaluations were performed for a group of eleven international experts (experts group). Results were also compared between the groups. Expert readers showed an excellent level of agreement for all parameters in both phase 1 and 2. For core participants, the concordance of all parameters in phase 1 was rated as good to excellent. Two parameters which were re-evaluated in phase 2, namely SSS and SRS, showed an increased level of concordance, up to excellent in both cases. Reporting of categorical variables by expert readers remained almost unchanged between the two phases, while core participants showed an increase in phase 2. Finally, pooled LVEF values did not show a significant difference between core participants and experts. However, significant differences were found between LVEF values obtained using different software packages for cardiac analysis. In this study, inter-observer agreement was moderate-to-good for core group readers and good-to-excellent for expert readers. The quality of reporting is affected by the quality of processing. These results confirm the important role of the IAEA training activities in improving imaging in low- and middle-income countries.
Sections du résumé
BACKGROUND
Consistency of results between different readers is an important issue in medical imaging, as it affects portability of results between institutions and may affect patient care. The International Atomic Energy Agency (IAEA) in pursuing its mission of fostering peaceful applications of nuclear technologies has supported several training activities in the field of nuclear cardiology (NC) and SPECT myocardial perfusion imaging (MPI) in particular. The aim of this study was to verify the outcome of those activities through an international clinical audit on MPI where participants were requested to report on studies distributed from a core lab.
METHODS
The study was run in two phases: in phase 1, SPECT MPI studies were distributed as raw data and full processing was requested as per local practice. In phase 2, images from studies pre-processed at the core lab were distributed. Data to be reported included summed stress score (SSS); summed rest score (SRS); summed difference score (SDS); left ventricular (LV) ejection fraction (EF) and end- diastolic volume (EDV). Qualitative appraisals included the assessment of perfusion and presence of ischemia, scar or mixed patterns, presence of transient ischemic dilation (TID), and risk for cardiac events (CE). Twenty-four previous trainees from low- and middle-income countries participated (core participants group) and their results were assessed for inter-observer variability in each of the two phases, and for changes between phases. The same evaluations were performed for a group of eleven international experts (experts group). Results were also compared between the groups.
RESULTS
Expert readers showed an excellent level of agreement for all parameters in both phase 1 and 2. For core participants, the concordance of all parameters in phase 1 was rated as good to excellent. Two parameters which were re-evaluated in phase 2, namely SSS and SRS, showed an increased level of concordance, up to excellent in both cases. Reporting of categorical variables by expert readers remained almost unchanged between the two phases, while core participants showed an increase in phase 2. Finally, pooled LVEF values did not show a significant difference between core participants and experts. However, significant differences were found between LVEF values obtained using different software packages for cardiac analysis.
CONCLUSIONS
In this study, inter-observer agreement was moderate-to-good for core group readers and good-to-excellent for expert readers. The quality of reporting is affected by the quality of processing. These results confirm the important role of the IAEA training activities in improving imaging in low- and middle-income countries.
Identifiants
pubmed: 30168029
doi: 10.1007/s12350-018-1407-4
pii: 10.1007/s12350-018-1407-4
pmc: PMC7174263
doi:
Substances chimiques
Technetium
7440-26-8
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
465-478Investigateurs
M Beretta
(M)
N Better
(N)
S Bouyoucef
(S)
L O Cabrera Rodríguez
(LO)
G Chalal
(G)
C Cittanti
(C)
C Cruz
(C)
A Cuocolo
(A)
N Girotto
(N)
N T Huong
(NT)
S S Iqbal
(SS)
A Klaipetch
(A)
C Marcassa
(C)
E Milan
(E)
F Mut Bastos
(F)
Q Naïli
(Q)
D Nanayakkara
(D)
J Obaldo
(J)
T F Ouattara
(TF)
Faso Burkina
(F)
K M Padrón García
(KM)
A Peix
(A)
Y Peña
(Y)
N Y Poyraz
(NY)
M Prpic
(M)
A Rochela
(A)
D F Ruiz Castañeda
(DF)
R Sciagra
(R)
S Scotti
(S)
E Sereegotov
(E)
S Sestini
(S)
D Sobic Saranovic
(D)
J Spuler
(J)
T Thientunyakit
(T)
W Vangu
(W)
J Vitola
(J)
G Vuleta
(G)
Références
https://www.iaea.org/ . Last accessed 22 May 2018.
https://www.iaea.org/about/organizational-structure/department-of-nuclear-sciences-and-applications/division-of-human-health . Last accessed 22 May 2018.
https://www.iaea.org/topics/nuclear-medicine-and-diagnostic-imaging-section . Last accessed 22 May 2018.
https://sustainabledevelopment.un.org/sdg3 . Last accessed on 08 June 2018.
http://www.who.int/nmh/en/ . Last accessed 08 June 2018.
Pradeepa R, Prabhakaran D, Mohan V. Emerging economies and diabetes and cardiovascular disease. Diabetes Technol Ther. 2012;14(Suppl 1):S59-67. https://doi.org/10.1089/dia.2012.00 .
doi: 10.1089/dia.2012.00
pubmed: 22650226
Jaarsma C, Leiner T, Bekkers SC, Crijns HJ, Wildberger JE, Nagel E, Nelemans PJ. Schalla S Diagnostic performance of noninvasive myocardial perfusion imaging using single-photon emission computed tomography, cardiac magnetic resonance, and positron emission tomography imaging for the detection of obstructive coronary artery disease: A meta-analysis. J Am Coll Cardiol. 2012;59(19):1719-28.
doi: 10.1016/j.jacc.2011.12.040
Metz LD, Beattie M, Hom R, Redberg RF, Grady D. Fleischmann KE The prognostic value of normal exercise myocardial perfusion imaging and exercise echocardiography: A meta-analysis. J Am Coll Cardiol. 2007;49(2):227-37.
doi: 10.1016/j.jacc.2006.08.048
Hachamovitch R, Hayes SW, Friedman JD, Cohen I, Berman DS. Comparison of the short-term survival benefit associated with revascularization compared with medical therapy in patients with no prior coronary artery disease undergoing stress myocardial perfusion single photon emission computed tomography. Circulation. 2003;107(23):2900-7.
doi: 10.1161/01.CIR.0000072790.23090.41
Dondi M, Andreo P. Developing nuclear medicine in developing countries: IAEA’s possible mission. Eur J Nucl Med Mol Imaging. 2006;33:514-5.
doi: 10.1007/s00259-006-0065-y
Casas-Zamora JA, Kashyap R. The IAEA technical cooperation programme and nuclear medicine in the developing world: Objectives, trends, and contributions. Semin Nucl Med. 2013;43(3):172-80. https://doi.org/10.1053/j.semnuclmed.2012.11.007 .
doi: 10.1053/j.semnuclmed.2012.11.007
pubmed: 23561454
https://humanhealth.iaea.org/HHW/NuclearMedicine/NUMDAB/index.html . Last accessed 22 May 2018.
Cerqueira MD, Weissman NJ, Dilsizian V, Jacobs AK, Kaul S, Laskey WK, et al. Standardized myocardial segmentation and nomenclature for tomographic imaging of the heart. Circulation. 2002;105:539-42.
doi: 10.1161/hc0402.102975
IAEA Human Health Series No. 23 (Rev. 1) Nuclear Cardiology: Guidance on the Implementation of SPECT Myocardial Perfusion Imaging International Atomic Energy Agency, Vienna; 2016.
Armstrong RA. When to use the Bonferroni correction. Ophthalmic Physiol Opt. 2014;34(5):502-8. https://doi.org/10.1111/opo.12131 .
doi: 10.1111/opo.12131
pubmed: 24697967
Deyo RA, Diehr P, Patrick DL. Reproducibility and responsiveness of health status measures. Statistics and strategies for evaluation. Control Clin Trials. 1991;12(Suppl 4):142S-58S.
doi: 10.1016/S0197-2456(05)80019-4
Hallgren KA. Computing inter-rater reliability for observational data: An overview and tutorial. Tutor Quant Methods Psychol. 2012;8(1):23-34.
doi: 10.20982/tqmp.08.1.p023
Fleiss JL, Levin B, Paik MC. The measurement of interrater agreement. In: Fleiss JL, Levin B, Paik MC, editors. Statistical methods for rates and proportions. 3rd ed. New York: Wiley; 2003. p. 598-626.
doi: 10.1002/0471445428.ch18
Cicchetti DV. Guidelines, criteria and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychol Assess. 1994;6(4):284-90.
doi: 10.1037/1040-3590.6.4.284
Hartling L, Hamm M, Milne A, Vandermeer B, Lina Santaguida P, Ansari M, et al. Validity and inter-rater reliability testing of quality assessment instruments. Rockville, MD: Agency for Healthcare Research and Quality (US); 2012. Table B, Interpretation of Fleiss’ kappa (κ) (from Landis and Koch 1977).
Papadopoulou SL, Garcia-Garcia HM, Rossi A, Girasis C, Dharampal AS, et al. Reproducibility of computed tomography angiography data analysis using semiautomated plaque quantification software: Implications for the design of longitudinal studies. Int J Cardiovasc Imaging. 2013;29:1095-104. https://doi.org/10.1007/s10554-012-0167-5 .
doi: 10.1007/s10554-012-0167-5
pubmed: 23224377
Herzog C, Kerl JM, De Rosa S, Tekin T, Boehme E, Liem S, et al. Influence of observer experience and training on proficiency in coronary CT angiography interpretation. Eur J Radiol. 2013;82:1240-7.
doi: 10.1016/j.ejrad.2013.02.037
Taylor AJ, Patrick J, Abbara S, Berman DS, Halliburton SS, Hines JL, et al. Relationship between previous training and experience and results of the certification examination in cardiovascular computed tomography. JACC Cardiovasc Imaging. 2010;9:976-80.
doi: 10.1016/j.jcmg.2010.06.015
Chauvela C, Abergel E, Renault L, Chatellier G, Cohen I, Attane C, et al. Improving stress echocardiography accuracy for detecting left circumflex artery stenosis: A new echocardiographic sign? Arch Cardiovasc Dis. 2012;105:196-202.
doi: 10.1016/j.acvd.2012.02.008
Kataoka A, Scherrer-Crosbie M, Senior R, Gosselin G, Phaneuf D, Guzman G, et al. The value of core lab stress echocardiography interpretations: Observations from the ISCHEMIA Trial. Cardiovasc Ultrasound. 2015;13:47.
doi: 10.1186/s12947-015-0043-2
Knight DS, Schwaiger JP, Krupickova S, Davar J, Muthurangu V, Coghlan JG, et al. Accuracy and test-retest reproducibility of two-dimensional knowledge-based volumetric reconstruction of the right ventricle in pulmonary hypertension. J Am Soc Echocardiogr. 2015;28:989-98.
doi: 10.1016/j.echo.2015.02.020
Berman DS, Kang X, Gransar H, Gerlach J, Friedman JD, Hayes SW, et al. Quantitative assessment of myocardial perfusion abnormality on SPECT myocardial perfusion imaging is more reproducible than expert visual analysis. J Nucl Cardiol. 2009;16:45.
doi: 10.1007/s12350-008-9018-0
Nakajima K, Higuchi T, Taki J, Kawano M, Tonami N, et al. Accuracy of ventricular volume and ejection fraction measured by gated myocardial SPECT: Comparison of 4 software programs. J Nucl Med. 2001;42:1571-8.
pubmed: 11585875
Larghat AM, Maredia N, Biglands J, Greenwood JP, Ball SG, Jerosch-Herold M, et al. Reproducibility of first-pass cardiovascular magnetic resonance myocardial perfusion. J Magn Reson Imaging. 2013;37:865-74.
doi: 10.1002/jmri.23889
Meriki N, Izurieta A, Welsh A. Reproducibility of constituent time intervals of right and left fetal modified myocardial performance indices on pulsed Doppler echocardiography: A short report. Ultrasound Obstet Gynecol. 2012;39:654-8.
doi: 10.1002/uog.10049
Wackers FJ. Artifacts in planar and SPECT myocardial perfusion imaging. Am J Card Imaging. 1992;6:42-57.
pubmed: 10147614
Germano G, Kavanagh PB, Waechter P, Areeda J, Van Kriekinge S, Sharir T, et al. A new algorithm for the quantitation of myocardial perfusion SPECT. I: Technical principles and reproducibility. J Nucl Med. 2000;41:712-9.
pubmed: 10768574
Ljungberg M, Pretorius PH. SPECT/CT: An update on technological developments and clinical applications. Br J Radiol. 2018;91(1081):20160402. https://doi.org/10.1259/bjr.20160402 . Epub 2017 Jan 16.
Malek H, Yaghoobi N, Hedayati R. Artifacts in quantitative analysis of myocardial perfusion SPECT, using Cedars-Sinai QPS Software. J Nucl Cardiol. 2017;24(2):534-42. https://doi.org/10.1007/s12350-016-0726-6 Epub 2016 Nov 10.
doi: 10.1007/s12350-016-0726-6
pubmed: 27834038
Chrysanthou-Baustert I, Polycarpou I, Demetriadou O, Livieratos L, Lontos A, Antoniou A, et al. Characterization of attenuation and respiratory motion artifacts and their influence on SPECT MP image evaluation using a dynamic phantom assembly with variable cardiac defects. J Nucl Cardiol. 2017;24:698-707. https://doi.org/10.1007/s12350-015-0378-y Epub 2016 Feb 4.
doi: 10.1007/s12350-015-0378-y
pubmed: 26846369
Foley TA, Mankad SV, Anavekar NS, Bonnichsen CR, Morris MF, Miller TD, et al. Measuring left ventricular ejection fraction—Techniques and potential pitfalls. Eur Cardiol. 2012;8(2):108-14.
doi: 10.15420/ecr.2012.8.2.108
Steyn R, Boniaszczuk J, Geldenhuys T. Comparison of estimates of left ventricular ejection fraction obtained from gated blood pool imaging, different software packages and cameras. Cardiovasc J Afr. 2014;25:44-9.
doi: 10.5830/CVJA-2013-082