Inter-rater agreement in pPOSSUM scores of geriatric trauma patients: a prospective evaluation.
Geriatric trauma care
Inter-rater agreement
Observer variation
Predicting tool
pPossum
Journal
Archives of orthopaedic and trauma surgery
ISSN: 1434-3916
Titre abrégé: Arch Orthop Trauma Surg
Pays: Germany
ID NLM: 9011043
Informations de publication
Date de publication:
Dec 2022
Dec 2022
Historique:
received:
20
06
2021
accepted:
21
11
2021
pubmed:
16
1
2022
medline:
28
10
2022
entrez:
15
1
2022
Statut:
ppublish
Résumé
Risk prediction models are widely used in the perioperative setting to identify high-risk patients who may benefit from additional care and to aid clinical decision-making. pPOSSUM is such a prediction model, however, little is known about the inter-rater agreement when scoring subjective parameters. This study assessed the inter-rater agreement between clinicians of different specialties and work-level when scoring 30 clinical case reports of geriatric hip fracture patients with pPOSSUM. Eighteen clinicians of the department of Surgery (three specialists, four residents), Anaesthesiology (four specialists, two residents) and Emergency Medicine (three specialists, two residents) who were familiar with the pPOSSUM scoring system were asked to calculate the scores. The kappa statistic and the statistical method of Fleiss were used to analyse inter-rater agreement. The response rate was 100%. Among surgeons, Anaesthesiologists and Emergency department doctors (ED), the overall mean kappa values were 0.42, 0.08 and 0.20, respectively. Among surgery, anaesthesiology and ED residents the overall mean kappa values were 0.21, 0.33 and 0.37, respectively. Within the department of Surgery, Anaesthesiology and Emergency Medicine the overall mean kappa values were 0.23, 0.12 and 0.22, respectively. An overall mean kappa value of 0.19 was seen among all specialists. All residents had an overall mean kappa value of 0.21 and all clinicians had an overall mean kappa value of 0.21. The overall inter-rater agreement of clinicians and interdisciplinary agreement when scoring geriatric hip fracture patients with pPOSSUM was low and prone to subjectivity in our study. A higher work-experience level did not lead to better agreement. When pPOSSUM is calculated without clinical assessment by the same clinician, caution is advised to prevent over-reliance on the pPOSSUM risk prediction model. III.
Identifiants
pubmed: 35031826
doi: 10.1007/s00402-021-04275-3
pii: 10.1007/s00402-021-04275-3
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
3869-3876Informations de copyright
© 2021. The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature.
Références
Eamer G, Al-Amoodi MJH, Holroyd-Leduc J, Rolfson DB, Warkentin LM, Khadaroo RG (2018) Review of risk assessment tools to predict morbidity and mortality in elderly surgical patients. Am J Surg 216:585
doi: 10.1016/j.amjsurg.2018.04.006
Copeland GP, Jones D, Walters M (1991) POSSUM: a scoring system for surgical audit. Br J Surg 78:355
doi: 10.1002/bjs.1800780327
Prytherch DR, Whiteley MS, Higgins B, Weaver PC, Prout WG, Powell SJ (1998) POSSUM and Portsmouth POSSUM for predicting mortality. Br J Surg 85:1217
doi: 10.1046/j.1365-2168.1998.00840.x
Whiteley MS, Prytherch DR, Higgins B, Weaver PC, Prout WG (1996) An evaluation of the POSSUM surgical scoring system. Br J Surg 83:812
doi: 10.1002/bjs.1800830628
Mohil RS, Bhatnagar D, Bahadur L, Dev DK, Magan M (2004) POSSUM and P-POSSUM for risk-adjusted audit of patients undergoing emergency laparotomy. Br J Surg 91:500
doi: 10.1002/bjs.4465
Tekkis PP, Kocher HM, Bentley AJE, Cullen PT, South LM, Trotter GA et al (2000) Operative mortality rates among surgeons: comparison of POSSUM and p-POSSUM scoring systems in gastrointestinal surgery. Dis Colon Rectum 43:1528
doi: 10.1007/BF02236732
Brunelli A, Fianchini A, Xiume F, Gesuita R, Mattei A, Carle F (1998) Evaluation of the POSSUM scoring system in lung surgery. Thorac Cardiovasc Surg 46:141
doi: 10.1055/s-2007-1010211
Kusen JQ, Schafroth B, Poblete B, van der Vet PCR, Link BC, Wijdicks FJG et al (2019) The implementation of a geriatric fracture centre for hip fractures to reduce mortality and morbidity: an observational study. Arch Orthop Trauma Surg 139:1705
doi: 10.1007/s00402-019-03229-0
Kusen JQ, van der Vet PCR, Wijdicks FJG, Link BC, Poblete B, van der Velde D et al (2020) Does preoperative hemodynamic preconditioning improve morbidity and mortality after traumatic hip fracture in geriatric patients? A retrospective cohort study. Arch Orthop Trauma Surg 141:1491
doi: 10.1007/s00402-020-03601-5
Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33:153
doi: 10.2307/2529310
Ethikkommission Nordwest-und Zentralschweiz (EKNZ). Approval number 2014-343. https://www.eknz.ch
Fleiss JL (1971) Measuring nominal scale agreement among many raters. Psychol Bull 76:382
doi: 10.1037/h0031619
Van Boxel GI, McLure S, Jones K, Jones G (2019) Inter-operator variability in pPOSSUM scores: a note of caution. Br J Hosp Med 80:343
doi: 10.12968/hmed.2019.80.6.343
Sadler ME, Yamamoto RT, Khurana L, Dallabrida SM (2017) The impact of rater training on clinical outcomes assessment data: a literature review. Int J Clin Trials 4:101
doi: 10.18203/2349-3259.ijct20173133
Marufu TC, Mannings A, Moppett IK (2015) Risk scoring models for predicting peri-operative morbidity and mortality in people with fragility hip fractures: qualitative systematic review. Injury 46:2325
doi: 10.1016/j.injury.2015.10.025
Boddaert J, Na N, Le Manach Y, Raux M, Cohen-Bittan J, Vallet H et al (2017) Prediction of postoperative mortality in elderly patients with hip fracture: are specific and geriatric scores better than general scores? Br J Anaesth 118:952
doi: 10.1093/bja/aex148
Johns WL, Strong B, Kates S, Patel NK (2020) POSSUM and P-POSSUM scoring in hip fracture mortalities. Geriatr Orthop Surg Rehabil. https://doi.org/10.1177/2151459320931674
doi: 10.1177/2151459320931674
pubmed: 32577320
pmcid: 7290268
Bonicoli E, Parchi P, Piolanti N, Andreani L, Niccolai F, Lisanti M (2014) Comparison of the POSSUM score and P-POSSUM score in patients with femoral neck fracture. Musculoskelet Surg 98:201
doi: 10.1007/s12306-013-0294-8
Jonsson MH, Bentzer P, Turkiewicz A, Hommel A (2018) Accuracy of the physiological and operative severity score for the enumeration of mortality and morbidity score and the Nottingham risk score in hip fracture patients in Sweden—a prospective observational study. Acta Anaesthesiol Scand 62:1057
doi: 10.1111/aas.13131
Hobson SA, Sutton CD, Garcea G, Thomas WM (2007) Prospective comparison of POSSUM and P-POSSUM with clinical assessment of mortality following emergency surgery. Acta Anaesthesiol Scand 51:94
doi: 10.1111/j.1399-6576.2006.01167.x
Dilaver NM, Gwilym BL, Preece R, Twine CP, Bosanquet DC (2020) Systematic review and narrative synthesis of surgeons’ perception of postoperative outcomes and risk. BJS Open 4:16
doi: 10.1002/bjs5.50233