Interpretation Accuracy and Interrater Agreement of Pediatric Skin and Soft Tissue Point-of-Care Ultrasound Images among Residents and Faculty.
accuracy
emergency
interrater agreement
pediatrics
skin and soft tissue infections
ultrasonography
Journal
The Journal of emergency medicine
ISSN: 0736-4679
Titre abrégé: J Emerg Med
Pays: United States
ID NLM: 8412174
Informations de publication
Date de publication:
Mar 2020
Mar 2020
Historique:
received:
18
06
2019
revised:
14
10
2019
accepted:
18
10
2019
pubmed:
18
12
2019
medline:
8
6
2021
entrez:
18
12
2019
Statut:
ppublish
Résumé
Point-of-care ultrasound (POCUS) is commonly used to facilitate care in the emergency department. Acquired images are often reviewed by local experts for educational and quality assurance purposes. However, no published study has examined the accuracy and reliability of POCUS image interpretation by multiple reviewers. We studied the accuracy and interrater agreement among expert and trainee reviewers of prerecorded pediatric skin and soft tissue (SST) POCUS images. POCUS faculty and emergency medicine (EM) residents blindly reviewed deidentified pediatric SST POCUS images and indicated whether a drainable fluid collection was present, absent, or indeterminate. This was then compared with the gold standard based on discharge diagnoses and telephone follow-up. Images rated as indeterminate were excluded from the initial analysis. Sensitivity analysis assuming indeterminate answers were inaccurate was subsequently conducted. In phase 1, 6 pediatric EM POCUS directors reviewed 168 images. The overall accuracy was 79.7% (range 66.1-86.0%). The mean Cohen's kappa was 0.58 (range 0.24-0.84). Sensitivity analysis yielded an overall accuracy of 71.3% (range 56.5-76.9%) and a Cohen's kappa of 0.43 (range 0.20-0.59). In phase 2, 6 general EM POCUS faculty and 20 EM residents reviewed 120 images. The overall accuracy among residents was 72.2% (range 51.4-84.7%) and among faculty was 83.6% (range 77.9-88.8%). Sensitivity analysis yielded an overall resident accuracy of 63.0% (range 49.5-80.7) and an overall faculty accuracy of 73.9% (range 67.0-79.8%). Fleiss' kappa was 0.322 for residents and 0.461 for faculty. We found moderate accuracy and fair to good interrater agreement among POCUS faculty and EM residents reviewing pediatric SST POCUS images.
Sections du résumé
BACKGROUND
BACKGROUND
Point-of-care ultrasound (POCUS) is commonly used to facilitate care in the emergency department. Acquired images are often reviewed by local experts for educational and quality assurance purposes. However, no published study has examined the accuracy and reliability of POCUS image interpretation by multiple reviewers.
OBJECTIVES
OBJECTIVE
We studied the accuracy and interrater agreement among expert and trainee reviewers of prerecorded pediatric skin and soft tissue (SST) POCUS images.
METHODS
METHODS
POCUS faculty and emergency medicine (EM) residents blindly reviewed deidentified pediatric SST POCUS images and indicated whether a drainable fluid collection was present, absent, or indeterminate. This was then compared with the gold standard based on discharge diagnoses and telephone follow-up. Images rated as indeterminate were excluded from the initial analysis. Sensitivity analysis assuming indeterminate answers were inaccurate was subsequently conducted.
RESULTS
RESULTS
In phase 1, 6 pediatric EM POCUS directors reviewed 168 images. The overall accuracy was 79.7% (range 66.1-86.0%). The mean Cohen's kappa was 0.58 (range 0.24-0.84). Sensitivity analysis yielded an overall accuracy of 71.3% (range 56.5-76.9%) and a Cohen's kappa of 0.43 (range 0.20-0.59). In phase 2, 6 general EM POCUS faculty and 20 EM residents reviewed 120 images. The overall accuracy among residents was 72.2% (range 51.4-84.7%) and among faculty was 83.6% (range 77.9-88.8%). Sensitivity analysis yielded an overall resident accuracy of 63.0% (range 49.5-80.7) and an overall faculty accuracy of 73.9% (range 67.0-79.8%). Fleiss' kappa was 0.322 for residents and 0.461 for faculty.
CONCLUSIONS
CONCLUSIONS
We found moderate accuracy and fair to good interrater agreement among POCUS faculty and EM residents reviewing pediatric SST POCUS images.
Identifiants
pubmed: 31843323
pii: S0736-4679(19)30932-1
doi: 10.1016/j.jemermed.2019.10.028
pii:
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
457-463Informations de copyright
Copyright © 2019 Elsevier Inc. All rights reserved.