External validation of the SORG 90-day and 1-year machine learning algorithms for survival in spinal metastatic disease.
External validation
Machine learning
Neurosurgery
Ninety-day
One year
Orthopedic surgery
Prediction
Prognosis
Spine metastasis
Spine surgery
Survival
Journal
The spine journal : official journal of the North American Spine Society
ISSN: 1878-1632
Titre abrégé: Spine J
Pays: United States
ID NLM: 101130732
Informations de publication
Date de publication:
01 2020
01 2020
Historique:
received:
08
04
2019
revised:
03
09
2019
accepted:
04
09
2019
pubmed:
11
9
2019
medline:
12
9
2020
entrez:
11
9
2019
Statut:
ppublish
Résumé
Preoperative survival estimation in spinal metastatic disease helps determine the appropriateness of invasive management. The SORG ML 90-day and 1-year machine learning algorithms for survival in spinal metastatic disease were previously developed in a single institutional sample but remain to be externally validated. The purpose of this study was to externally validate these algorithms in an independent population from another institution. Retrospective study at a large, tertiary care center. Patients 18 years or older who underwent surgery between 2003 and 2016. Ninety-day and 1-year mortality. Baseline characteristics of the validation cohort were compared to the developmental cohort for the SORG ML algorithms. Discrimination (c-statistic and receiver operating curve), calibration (calibration slope, intercept, calibration plot, and observed proportions by predicted risk groups), overall performance (Brier score), and decision curve analysis were used to assess the performance of the SORG ML algorithms in the validation cohort. Overall, 176 patients underwent surgery for spinal metastatic disease, of which 44 (22.7%) experienced 90-day mortality and 99 (56.2%) experienced 1-year mortality. The validation cohort differed significantly from the developmental cohort on primary tumor histology, metastatic tumor burden, previous systemic therapy, overall comorbidity burden, and preoperative laboratory characteristics. Despite these differences, the SORG ML algorithms generalized well to the validation cohort on discrimination (c-statistic 0.75-0.81 for 90-day mortality and 0.77-0.78 for 1-year mortality), calibration, Brier score, and decision curve analysis. Initial results from external validation of the SORG ML 90-day and 1-year algorithms for survival prediction in spinal metastatic disease suggest potential utility of these digital decision aids in clinical practice. Further studies are needed to validate or refute these algorithms in large patient samples from prospective, international, multi-institutional trials.
Sections du résumé
BACKGROUND CONTEXT
Preoperative survival estimation in spinal metastatic disease helps determine the appropriateness of invasive management. The SORG ML 90-day and 1-year machine learning algorithms for survival in spinal metastatic disease were previously developed in a single institutional sample but remain to be externally validated.
PURPOSE
The purpose of this study was to externally validate these algorithms in an independent population from another institution.
STUDY DESIGN/SETTING
Retrospective study at a large, tertiary care center.
PATIENT SAMPLE
Patients 18 years or older who underwent surgery between 2003 and 2016.
OUTCOME MEASURES
Ninety-day and 1-year mortality.
METHODS
Baseline characteristics of the validation cohort were compared to the developmental cohort for the SORG ML algorithms. Discrimination (c-statistic and receiver operating curve), calibration (calibration slope, intercept, calibration plot, and observed proportions by predicted risk groups), overall performance (Brier score), and decision curve analysis were used to assess the performance of the SORG ML algorithms in the validation cohort.
RESULTS
Overall, 176 patients underwent surgery for spinal metastatic disease, of which 44 (22.7%) experienced 90-day mortality and 99 (56.2%) experienced 1-year mortality. The validation cohort differed significantly from the developmental cohort on primary tumor histology, metastatic tumor burden, previous systemic therapy, overall comorbidity burden, and preoperative laboratory characteristics. Despite these differences, the SORG ML algorithms generalized well to the validation cohort on discrimination (c-statistic 0.75-0.81 for 90-day mortality and 0.77-0.78 for 1-year mortality), calibration, Brier score, and decision curve analysis.
CONCLUSION AND RELEVANCE
Initial results from external validation of the SORG ML 90-day and 1-year algorithms for survival prediction in spinal metastatic disease suggest potential utility of these digital decision aids in clinical practice. Further studies are needed to validate or refute these algorithms in large patient samples from prospective, international, multi-institutional trials.
Identifiants
pubmed: 31505303
pii: S1529-9430(19)30967-2
doi: 10.1016/j.spinee.2019.09.003
pii:
doi:
Types de publication
Journal Article
Validation Study
Langues
eng
Sous-ensembles de citation
IM
Pagination
14-21Informations de copyright
Copyright © 2019. Published by Elsevier Inc.