Accuracy and Interrater Reliability of Point-of-Care Ultrasonography Image Interpretation for Intussusception.
Journal
Pediatric emergency care
ISSN: 1535-1815
Titre abrégé: Pediatr Emerg Care
Pays: United States
ID NLM: 8507560
Informations de publication
Date de publication:
01 Sep 2022
01 Sep 2022
Historique:
entrez:
30
8
2022
pubmed:
31
8
2022
medline:
2
9
2022
Statut:
ppublish
Résumé
The aim of this study was to determine the accuracy and interrater reliability of (1) point-of-care ultrasound (POCUS) image interpretation for identification of intussusception and (2) reliability of secondary signs associated with intussusception among experts compared with novice POCUS reviewers. We conducted a planned secondary analysis of a prospective, convenience sample of children aged 3 months to 6 years who were evaluated with POCUS for intussusception across 17 international pediatric emergency departments between October 2018 and December 2020. A random sample of 100 POCUS examinations was reviewed by novice and expert POCUS reviewers. The primary outcome was identification of the presence or absence of intussusception. Secondary outcomes included intussusception size and the presence of trapped free fluid or echogenic foci. Accuracy was summarized using sensitivity and specificity, which were estimated via generalized mixed effects logistic regression. Interrater reliability was summarized via Light's κ statistics with bootstrapped standard errors (SEs). Accuracy and reliability of expert and novice POCUS reviewers were compared. Eighteen expert and 16 novice POCUS reviewers completed the reviews. The average expert sensitivity was 94.5% (95% confidence interval [CI], 88.6-97.5), and the specificity was 94.3% (95% CI, 90.3-96.7), significantly higher than the average novice sensitivity of 84.7% (95% CI, 74.3-91.4) and specificity of 80.4% (95% CI, 72.4, 86.7). κ was significantly greater for expert (0.679, SE 0.039) compared with novice POCUS reviewers (0.424, SE 0.044; difference 0.256, SE 0.033). For our secondary outcome measure of intussusception size, κ was significantly greater for experts (0.661, SE 0.038) compared with novices (0.397, SE 0.041; difference 0.264, SE 0.029). Interrater reliability was weak for expert and minimal for novice reviewers regarding the detection of trapped free fluid and echogenic foci. Expert POCUS reviewers demonstrate high accuracy and moderate interrater reliability when identifying intussusception via image interpretation and perform better than novice reviewers.
Identifiants
pubmed: 36040465
doi: 10.1097/PEC.0000000000002786
pii: 00006565-202209000-00006
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
442-447Informations de copyright
Copyright © 2022 Wolters Kluwer Health, Inc. All rights reserved.
Déclaration de conflit d'intérêts
Disclosure: The authors declare no conflict of interest.
Références
Sivitz AB, Cohen SG, Tejani C. Evaluation of acute appendicitis by pediatric emergency physician sonography. Ann Emerg Med . 2014;64:358–364.e4.
Garcia AM, Asad I, Tessaro MO, et al. A multi-institutional case series with review of point-of-care ultrasound to diagnose malrotation and midgut volvulus in the pediatric emergency department. Pediatr Emerg Care . 2019;35:443–447.
Malcom GE 3rd, Raio CC, Del Rios M, et al. Feasibility of emergency physician diagnosis of hypertrophic pyloric stenosis using point-of-care ultrasound: a multi-center case series. J Emerg Med . 2009;37:283–286.
Riera A, Hsiao AL, Langhan ML, et al. Diagnosis of intussusception by physician novice sonographers in the emergency department. Ann Emerg Med . 2012;60:264–268.
Bergmann KR, Arroyo AC, Tessaro MO, et al. Diagnostic accuracy of point-of-care ultrasound for intussusception: a multicenter, noninferiority study of paired diagnostic tests. Ann Emerg Med . 2021;78:606–615.
Arroyo AC, Zerzan J, Vazquez H, et al. Diagnostic accuracy of point-of-care ultrasound for intussusception performed by pediatric emergency medicine physicians. J Emerg Med . 2021;60:626–632.
Bhisitkul DM, Listernick R, Shkolnik A, et al. Clinical application of ultrasonography in the diagnosis of intussusception. J Pediatr . 1992;121:182–186.
Chang YJ, Hsia SH, Chao HC. Emergency medicine physicians performed ultrasound for pediatric intussusceptions. Biom J . 2013;36:175–178.
Tonson la Tour A, Desjardins MP, Gravel J. Evaluation of bedside sonography performed by emergency physicians to detect intussusception in children in the emergency department. Acad Emerg Med . 2021;28:866–872.
Chamberlain MC, Reid SR, Madhok M. Utilization of emergency ultrasound in pediatric emergency departments. Pediatr Emerg Care . 2011;27:628–632.
Cohen JS, Teach SJ, Chapman JI. Bedside ultrasound education in pediatric emergency medicine fellowship programs in the United States. Pediatr Emerg Care . 2012;28:845–850.
Marin JR, Zuckerbraun NS, Kahn JM. Use of emergency ultrasound in United States pediatric emergency medicine fellowship programs in 2011. J Ultrasound Med . 2012;31:1357–1363.
Ramirez-Schrempp D, Dorfman DH, Tien I, et al. Bedside ultrasound in pediatric emergency medicine fellowship programs in the United States: little formal training. Pediatr Emerg Care . 2008;24:664–667.
Cramer N, Cantwell L, Ong H, et al. Pediatric emergency medicine fellowship point-of-care ultrasound training in 2020. AEM Educ Train . 2021;5:e10643.
Hallgren KA. Computing inter-rater reliability for observational data: an overview and tutorial. Tutor Quant Methods Psychol . 2012;8:23–34.
Harris PA, Taylor R, Thielke R, et al. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform . 2009;42:377–381.
Dessie AS, Calhou AW, Gilbert GE, et al. 112 development and validation of a point-of-care ultrasound image quality assessment tool: the POCUS IQ scale. Ann Emerg Med . 2019;74:S45–S46.
Rodriguez G, Elo I. Intra-class correlation in random-effects models for binary data. Stata J . 2003;3:32–46.
Light RJ. Measures of response agreement for qualitative data: some generalizations and alternatives. Psychol Bull . 1971;76:365–377.
McHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb) . 2012;22:276–282.
Lin-Martore M, Kornblith AE, Kohn MA, et al. Diagnostic accuracy of point-of-care ultrasound for intussusception in children presenting to the emergency department: a systematic review and meta-analysis. West J Emerg Med . 2020;21:1008–1016.
Trigylidas TE, Hegenbarth MA, Patel L, et al. Pediatric emergency medicine point-of-care ultrasound for the diagnosis of intussusception. J Emerg Med . 2019;57:367–374.
Tsou PY, Wang YH, Ma YK, et al. Accuracy of point-of-care ultrasound and radiology-performed ultrasound for intussusception: a systematic review and meta-analysis. Am J Emerg Med . 2019;37:1760–1769.
Hom J, Kaplan C, Fowler S, et al. Evidence-based diagnostic test accuracy of history, physical examination, and imaging for intussusception: a systematic review and meta-analysis. Pediatr Emerg Care . 2022;38:e225–e230.
Lee JY, Kim JH, Choi SJ, et al. Point-of-care ultrasound may be useful for detecting pediatric intussusception at an early stage. BMC Pediatr . 2020;20:155.
Lam SHF, Wise A, Yenter C. Emergency bedside ultrasound for the diagnosis of pediatric intussusception: a retrospective review. World J Emerg Med . 2014;5:255–258.
Kim JH, Lee JY, Kwon JH, et al. Point-of-care ultrasound could streamline the emergency department workflow of clinically nonspecific intussusception. Pediatr Emerg Care . 2020;36:e90–e95.
Paek SH, Kim DK, Kwak YH, et al. Effectiveness of the implementation of pediatric intussusception clinical pathway: a pre- and postintervention trial. Medicine (Baltimore) . 2021;100:e27971.
Constantine E, Levine M, Abo A, et al. Core content for pediatric emergency medicine ultrasound fellowship training: a modified Delphi consensus study. AEM Educ Train . 2020;4:130–138.
Shefrin AE, Warkentine F, Constantine E, et al. Consensus core point-of-care ultrasound applications for pediatric emergency medicine training. AEM Educ Train . 2019;3:251–258.
Hsiao HJ, Wang CJ, Lee CC, et al. Point-of-care ultrasound may reduce misdiagnosis of pediatric intussusception. Front Pediatr . 2021;9:601492.
Vieira RL, Hsu D, Nagler J, et al. Pediatric emergency medicine fellow training in ultrasound: consensus educational guidelines. Acad Emerg Med . 2013;20:300–306.
Park BL, Rabiner JE, Tsung JW. Point-of-care ultrasound diagnosis of small bowel-small bowel vs ileocolic intussusception. Am J Emerg Med . 2019;37:1746–1750.
Zhang M, Zhou X, Hu Q, et al. Accurately distinguishing pediatric ileocolic intussusception from small-bowel intussusception using ultrasonography. J Pediatr Surg . 2021;56:721–726.
Abo AM, Alade KH, Rempell RG, et al. Credentialing pediatric emergency medicine faculty in point-of-care ultrasound: expert guidelines. Pediatr Emerg Care . 2021;37:e1687–e1694.
Bell C, Hall AK, Wagner N, et al. The ultrasound competency assessment tool (UCAT): development and evaluation of a novel competency-based assessment tool for point-of-care ultrasound. AEM Educ Train . 2021;5:e10520.