Automated Left Ventricular Dimension Assessment Using Artificial Intelligence Developed and Validated by a UK-Wide Collaborative.
consensus
echocardiography
hospital
left ventricle
machine learning
Journal
Circulation. Cardiovascular imaging
ISSN: 1942-0080
Titre abrégé: Circ Cardiovasc Imaging
Pays: United States
ID NLM: 101479935
Informations de publication
Date de publication:
05 2021
05 2021
Historique:
pubmed:
18
5
2021
medline:
22
9
2021
entrez:
17
5
2021
Statut:
ppublish
Résumé
requires training and validation to standards expected of humans. We developed an online platform and established the Unity Collaborative to build a dataset of expertise from 17 hospitals for training, validation, and standardization of such techniques. The training dataset consisted of 2056 individual frames drawn at random from 1265 parasternal long-axis video-loops of patients undergoing clinical echocardiography in 2015 to 2016. Nine experts labeled these images using our online platform. From this, we trained a convolutional neural network to identify keypoints. Subsequently, 13 experts labeled a validation dataset of the end-systolic and end-diastolic frame from 100 new video-loops, twice each. The 26-opinion consensus was used as the reference standard. The primary outcome was precision SD, the SD of the differences between AI measurement and expert consensus. In the validation dataset, the AI's precision SD for left ventricular internal dimension was 3.5 mm. For context, precision SD of individual expert measurements against the expert consensus was 4.4 mm. Intraclass correlation coefficient between AI and expert consensus was 0.926 (95% CI, 0.904-0.944), compared with 0.817 (0.778-0.954) between individual experts and expert consensus. For interventricular septum thickness, precision SD was 1.8 mm for AI (intraclass correlation coefficient, 0.809; 0.729-0.967), versus 2.0 mm for individuals (intraclass correlation coefficient, 0.641; 0.568-0.716). For posterior wall thickness, precision SD was 1.4 mm for AI (intraclass correlation coefficient, 0.535 [95% CI, 0.379-0.661]), versus 2.2 mm for individuals (0.366 [0.288-0.462]). We present all images and annotations. This highlights challenging cases, including poor image quality and tapered ventricles. Experts at multiple institutions successfully cooperated to build a collaborative AI. This performed as well as individual experts. Future echocardiographic AI research should use a consensus of experts as a reference. Our collaborative welcomes new partners who share our commitment to publish all methods, code, annotations, and results openly.
Sections du résumé
BACKGROUND
requires training and validation to standards expected of humans. We developed an online platform and established the Unity Collaborative to build a dataset of expertise from 17 hospitals for training, validation, and standardization of such techniques.
METHODS
The training dataset consisted of 2056 individual frames drawn at random from 1265 parasternal long-axis video-loops of patients undergoing clinical echocardiography in 2015 to 2016. Nine experts labeled these images using our online platform. From this, we trained a convolutional neural network to identify keypoints. Subsequently, 13 experts labeled a validation dataset of the end-systolic and end-diastolic frame from 100 new video-loops, twice each. The 26-opinion consensus was used as the reference standard. The primary outcome was precision SD, the SD of the differences between AI measurement and expert consensus.
RESULTS
In the validation dataset, the AI's precision SD for left ventricular internal dimension was 3.5 mm. For context, precision SD of individual expert measurements against the expert consensus was 4.4 mm. Intraclass correlation coefficient between AI and expert consensus was 0.926 (95% CI, 0.904-0.944), compared with 0.817 (0.778-0.954) between individual experts and expert consensus. For interventricular septum thickness, precision SD was 1.8 mm for AI (intraclass correlation coefficient, 0.809; 0.729-0.967), versus 2.0 mm for individuals (intraclass correlation coefficient, 0.641; 0.568-0.716). For posterior wall thickness, precision SD was 1.4 mm for AI (intraclass correlation coefficient, 0.535 [95% CI, 0.379-0.661]), versus 2.2 mm for individuals (0.366 [0.288-0.462]). We present all images and annotations. This highlights challenging cases, including poor image quality and tapered ventricles.
CONCLUSIONS
Experts at multiple institutions successfully cooperated to build a collaborative AI. This performed as well as individual experts. Future echocardiographic AI research should use a consensus of experts as a reference. Our collaborative welcomes new partners who share our commitment to publish all methods, code, annotations, and results openly.
Identifiants
pubmed: 33998247
doi: 10.1161/CIRCIMAGING.120.011951
pmc: PMC8136463
mid: EMS120446
doi:
Types de publication
Journal Article
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
e011951Subventions
Organisme : European Research Council
ID : 281524
Pays : International
Organisme : British Heart Foundation
ID : FS/12/12/29294
Pays : United Kingdom
Organisme : British Heart Foundation
ID : PG/19/78/34733
Pays : United Kingdom
Organisme : Wellcome Trust
ID : 212183/Z/18/Z
Pays : United Kingdom
Références
JAMA. 2016 Dec 13;316(22):2402-2410
pubmed: 27898976
Eur Heart J Cardiovasc Imaging. 2014 Nov;15(11):1188-93
pubmed: 25344557
Nat Biomed Eng. 2018 Oct;2(10):719-731
pubmed: 31015651
Lancet Digit Health. 2021 Jan;3(1):e20-e28
pubmed: 33735065
Eur J Echocardiogr. 2010 Mar;11(2):149-56
pubmed: 19959533
Transl Vis Sci Technol. 2020 Feb 12;9(2):7
pubmed: 32704413
Circulation. 2018 Oct 16;138(16):1623-1635
pubmed: 30354459
Nature. 2020 Apr;580(7802):252-256
pubmed: 32269341
Control Clin Trials. 1994 Oct;15(5):395-410
pubmed: 8001359
J Am Soc Echocardiogr. 2015 Jan;28(1):1-39.e14
pubmed: 25559473
IEEE Trans Med Imaging. 2019 Sep;38(9):2198-2210
pubmed: 30802851