Derivation and External Validation of the Ottawa Bloodstream Infection Model for Acutely Ill Adults.
Bacteremia
Bloodstream infection
Predictive models
Temporal external validation
Journal
Journal of general internal medicine
ISSN: 1525-1497
Titre abrégé: J Gen Intern Med
Pays: United States
ID NLM: 8605834
Informations de publication
Date de publication:
18 Sep 2023
18 Sep 2023
Historique:
received:
12
06
2023
accepted:
30
08
2023
pubmed:
19
9
2023
medline:
19
9
2023
entrez:
18
9
2023
Statut:
aheadofprint
Résumé
Knowing the probability that patients have a bloodstream infection (BSI) could influence the ordering of blood cultures and interpretation of their preliminary results. Many previous BSI probability models have limited applicability and accuracy. This study used currently recommended modeling techniques and a large sample to derive and validate the Ottawa BSI Model. At a tertiary care teaching hospital, we retrieved a random sample of 4180 adults having blood cultures in our emergency department or during the initial 48 h of the encounter. Variable selection was based on clinical experience and a systematic review of previous model performance. Model performance was measured in a temporal external validation group of 4680 patients. A total of 327 derivation patients had a BSI (8.0%). BSI risk increased with increased number of culture sets (2 sets: adjusted odds ratio [aOR] 1.52 [1.10-2.11]; 3 sets: 1.99 [0.86-4.58]); with indwelling catheter (aOR 2.07 [1.34-3.20); with increasing temperature, heart rate, and neutrophil-lymphocyte ratio; and with decreasing systolic blood pressure, platelet count, urea-creatinine ratio, and estimated glomerular filtration rate. In the temporal external validation group, model discrimination was good (c-statistic 0.71 [0.69-0.74]) and calibration was very good (integrated calibration index .016 [.010-.024]). Exclusion of validation patients with acute SARS-CoV-2 infection improved discrimination slightly (c-statistic 0.73 [0.69-0.76]). The Ottawa BSI Model uses commonly available data to return an expected BSI probability for acutely ill patients. However, it cannot exclude BSI and its complexity requires computational assistance to use.
Sections du résumé
BACKGROUND
BACKGROUND
Knowing the probability that patients have a bloodstream infection (BSI) could influence the ordering of blood cultures and interpretation of their preliminary results. Many previous BSI probability models have limited applicability and accuracy. This study used currently recommended modeling techniques and a large sample to derive and validate the Ottawa BSI Model.
METHODS
METHODS
At a tertiary care teaching hospital, we retrieved a random sample of 4180 adults having blood cultures in our emergency department or during the initial 48 h of the encounter. Variable selection was based on clinical experience and a systematic review of previous model performance. Model performance was measured in a temporal external validation group of 4680 patients.
RESULTS
RESULTS
A total of 327 derivation patients had a BSI (8.0%). BSI risk increased with increased number of culture sets (2 sets: adjusted odds ratio [aOR] 1.52 [1.10-2.11]; 3 sets: 1.99 [0.86-4.58]); with indwelling catheter (aOR 2.07 [1.34-3.20); with increasing temperature, heart rate, and neutrophil-lymphocyte ratio; and with decreasing systolic blood pressure, platelet count, urea-creatinine ratio, and estimated glomerular filtration rate. In the temporal external validation group, model discrimination was good (c-statistic 0.71 [0.69-0.74]) and calibration was very good (integrated calibration index .016 [.010-.024]). Exclusion of validation patients with acute SARS-CoV-2 infection improved discrimination slightly (c-statistic 0.73 [0.69-0.76]).
CONCLUSIONS
CONCLUSIONS
The Ottawa BSI Model uses commonly available data to return an expected BSI probability for acutely ill patients. However, it cannot exclude BSI and its complexity requires computational assistance to use.
Identifiants
pubmed: 37723368
doi: 10.1007/s11606-023-08407-w
pii: 10.1007/s11606-023-08407-w
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Informations de copyright
© 2023. The Author(s), under exclusive licence to Society of General Internal Medicine.
Références
Bates DW, Goldman L, Lee TH. Contaminant blood cultures and resource utilization. The true consequences of false-positive results. JAMA 1991;265(3):365–369.
Eliakim-Raz N, Bates DW, Leibovici L. Predicting bacteraemia in validated models - a systematic review. Clinical Microbiology and Infection 2015;21(4):295-301.
doi: 10.1016/j.cmi.2015.01.023
pubmed: 25677625
Rodic S, Hryciw BN, Selim S et al. Concurrent external validation of bloodstream infection probability models. Clinical Microbiology and Infection. In press.
Wolff RF, Moons KGM, Riley RD et al. PROBAST: a tool to assess the risk of bias and applicability of prediction model studies. Ann Intern Med 2019;170(1):51-58.
doi: 10.7326/M18-1376
pubmed: 30596875
Riley RD, Ensor J, Snell KIE et al. Calculating the sample size required for developing a clinical prediction model. Br Med J 2020;368:m441.
doi: 10.1136/bmj.m441
Institute for Quality Management in Healthcare. Consensus practice recommendations - BACT - Blood cultures. Toronto: 2012
Steyerberg EW. Overfitting and optimism in prediction models. Clinical Prediction Models: a practical approach to development, validation, and updating. 2 ed. New York: Springer; 2019:95–112.
Cockerill FR, III, Wilson JW, Vetter EA et al. Optimal testing parameters for blood cultures. Clinical Infectious Diseases 2004;38(12):1724-30.
doi: 10.1086/421087
pubmed: 15227618
Jiang J, Liu R, Yu X et al. The neutrophil-lymphocyte count ratio as a diagnostic marker for bacteraemia: a systematic review and meta-analysis. The American Journal of Emergency Medicine 2019;37(8):1482-1489.
doi: 10.1016/j.ajem.2018.10.057
pubmed: 30413366
van Walraven C, Tuna M. The Network Relative Model Accuracy (NeRMA) score can quantify the relative accuracy of prediction models in concurrent external validations. Journal of Evaluation in Clinical Practice. In press.
Leibovici L, Greenshtain S, Cohen O, Mor F, Wysenbeek AJ. Bacteremia in febrile patients: a clinical model for diagnosis. Arch Intern Med 1991;151(9):1801-1806.
doi: 10.1001/archinte.1991.00400090089016
pubmed: 1888246
Levey AS, Coresh J, Greene T et al. Using standardized serum creatinine values in the modification of diet in renal disease study equation for estimating glomerular filtration rate. Ann Intern Med 2006;145(4):247-254.
doi: 10.7326/0003-4819-145-4-200608150-00004
pubmed: 16908915
van Walraven C, McCudden C, Austin PC. Laboratory test results differ significantly when they are not ordered: implications for imputing missing lab data. J Clin Epidemiol. In press.
Sauerbrei W, Meier-Hirmer C, Benner A, Royston P. Multivariable regression model building by using fractional polynomials: description of SAS, STATA and R programs. Computational Statistics and Data Analysis 2006;50(12):3464-3485.
doi: 10.1016/j.csda.2005.07.015
Steyerberg EW. Coding of categorical and continuous variables. Clinical Prediction Models: a practical approach to development, validation, and updating. 2 ed. New York: Springer; 2019:175–190.
Sullivan LM, Massaro JM, D’Agostino RB, Sr. Presentation of multivariate data for clinical use: the Framingham Study risk score functions. Stat Med 2004;23(10):1631-1660.
doi: 10.1002/sim.1742
pubmed: 15122742
Austin PC, Steyerberg EW. Graphical assessment of internal and external calibration of logistic regression models by using loess smoothers. Stat Med 2014;33(3):517-535.
doi: 10.1002/sim.5941
pubmed: 24002997
Efron B, Tibshirani RJ. Confidence intervals based on bootstrap percentiles. An introduction to the bootstrap. New York: Chapman&Hall; 1994:168–177.
Riley RD, Debray TPA, Collins GS et al. Minimum sample size for external validation of a clinical prediction model with a binary outcome. Stat Med 2021;40(19):4230-4251.
doi: 10.1002/sim.9025
pubmed: 34031906
Karvanen J. The statistical basis of laboratory data normalization. Drug information journal : DIJ / Drug Information Association 2003;37(1):101-107.
doi: 10.1177/009286150303700112
Colgan R, Nicolle LE, McGlone A, Hooton TM. Asymptomatic bacteriuria in adults. American Family Physician 2006;74(6):985-990.
pubmed: 17002033
Mozes B, Milatiner D, Block C, Blumstein Z, Halkin H. Inconsistency of a model aimed at predicting bacteremia in hospitalized patients. J Clin Epidemiol 1993;46(9):1035-1040.
doi: 10.1016/0895-4356(93)90171-V
pubmed: 8263576
Shapiro NI, Wolfe REM, Wright SB, Moore R, Bates DW. Who needs a blood culture? A prospectively derived and validated prediction rule. The Journal of Emergency Medicine 2008;35(3):255-264.
doi: 10.1016/j.jemermed.2008.04.001
pubmed: 18486413
Jessen MK, Mackenhauer J, Hvass AM et al. Prediction of bacteremia in the emergency department: an external validation of a clinical decision rule. Eur J Emerg Med 2016;23(1):44-49.
doi: 10.1097/MEJ.0000000000000203
pubmed: 25222426