Interobserver reproducibility of The Paris System of Reporting Urine Cytology on cytocentrifuged samples.
The Paris System of Reporting Urine Cytology
high grade urothelial carcinoma
interobserver reproducibility
urine cytology
Journal
Diagnostic cytopathology
ISSN: 1097-0339
Titre abrégé: Diagn Cytopathol
Pays: United States
ID NLM: 8506895
Informations de publication
Date de publication:
Nov 2020
Nov 2020
Historique:
received:
05
04
2020
accepted:
05
05
2020
pubmed:
17
6
2020
medline:
15
10
2021
entrez:
17
6
2020
Statut:
ppublish
Résumé
The Paris System of Reporting Urine Cytology aims to screen samples for HGUC and to provide a universally acceptable reporting format for urine cytology specimens. However, studies detailing the reproducibility of this system, especially on cyto-centrifuge preparations, are lacking. 149 voided urine samples received in Department of Pathology were reviewed independently by five cytopathologists. To estimate the overall agreement, Gwet's AC and AC1statistics between each pair of raters were used. To measure the random error component, polychoric correlations were used. To assess the systematic error, Friedman test was used. There was moderately good inter-rater agreement between the raters. Gwets AC2 ranged between 0.67 and 0.89 for the classification of the cases once the sample was found adequate for assessment, while the Gwet's AC1 ranged between 0.61 and 0.94 in assessing for adequacy. There were significant systematic differences between raters in their thresholds for the different categories as well as in differentiating between an adequate and inadequate sample (P value by Friedman test <.001). The association between pathologists was moderately high (polychoric correlations ranging from 0.67 to 0.93). In the majority (108 of 149, 72.5%) of the cases, the range of differences between raters were of one category or less, suggesting satisfactory agreement, but having large disagreements in minority. The interobserver reproducibility for the Paris System is moderately good, and is suitable for adoption. It is limited by the lack of agreement as to what constitutes an adequate specimen and differing threshold for categorizing the lesions in differing groups.
Sections du résumé
BACKGROUND
BACKGROUND
The Paris System of Reporting Urine Cytology aims to screen samples for HGUC and to provide a universally acceptable reporting format for urine cytology specimens. However, studies detailing the reproducibility of this system, especially on cyto-centrifuge preparations, are lacking.
METHODS
METHODS
149 voided urine samples received in Department of Pathology were reviewed independently by five cytopathologists. To estimate the overall agreement, Gwet's AC and AC1statistics between each pair of raters were used. To measure the random error component, polychoric correlations were used. To assess the systematic error, Friedman test was used.
RESULTS
RESULTS
There was moderately good inter-rater agreement between the raters. Gwets AC2 ranged between 0.67 and 0.89 for the classification of the cases once the sample was found adequate for assessment, while the Gwet's AC1 ranged between 0.61 and 0.94 in assessing for adequacy. There were significant systematic differences between raters in their thresholds for the different categories as well as in differentiating between an adequate and inadequate sample (P value by Friedman test <.001). The association between pathologists was moderately high (polychoric correlations ranging from 0.67 to 0.93). In the majority (108 of 149, 72.5%) of the cases, the range of differences between raters were of one category or less, suggesting satisfactory agreement, but having large disagreements in minority.
CONCLUSION
CONCLUSIONS
The interobserver reproducibility for the Paris System is moderately good, and is suitable for adoption. It is limited by the lack of agreement as to what constitutes an adequate specimen and differing threshold for categorizing the lesions in differing groups.
Types de publication
Journal Article
Observational Study
Langues
eng
Sous-ensembles de citation
IM
Pagination
979-985Informations de copyright
© 2020 Wiley Periodicals LLC.
Références
Cumberbatch MGK, Jubber I, Black PC, et al. Epidemiology of bladder cancer: a systematic review and contemporary update of risk factors in 2018. Eur Urol. 2018;74(6):784-795.
Barkan GA, Wojcik EM, Nayar R, et al. The Paris System for Reporting Urinary Cytology: the quest to develop a standardized terminology. Adv Anat Pathol. 2016;23(4):193-201.
Long T, Layfield LJ, Esebua M, Frazier SR, Giorgadze DT, Schmidt RL. Interobserver reproducibility of the Paris System for Reporting Urinary Cytology. Cytojournal. 2017;14:17.
Bakkar R, Mirocha J, Fan X, et al. Impact of the Paris System for Reporting Urine Cytopathology on predictive values of the equivocal diagnostic categories and interobserver agreement. Cytojournal. 2019;16:21.
Kurtycz DFI, Barkan GA, Pavelec DM, et al. Paris interobserver reproducibility study (PIRST). J Am Soc Cytopathol. 2018;7(4):174-184.
Rosenthal DL, Wojcik EM, Kurtycz DFI, eds. The Paris System for Reporting Urinary Cytology. 1st ed.Cham: Springer International Publishing AG; 2016.
Barkan GA, Wojcik EM, Nayar R, et al. The Paris System for Reporting Urinary Cytology: the quest to develop a standardized terminology. Acta Cytol. 2016;60(3):185-197.
Gwet KL. Handbook of Inter-Rater Reliability: The Definitive Guide to Measuring the Extent of Agreement among Raters. Gaithersburg, MD: STATAXIS Publishing Company; 2010:197.
Hripcsak G, Heitjan DF. Measuring agreement in medical informatics reliability studies. J Biomed Inform. 2002;35(2):99-110.
Chowdhury N, Pai MR, Lobo FD, Kini H, Varghese R. Interobserver variation in breast cancer grading: a statistical modeling approach. Anal Quant Cytol Histol. 2006;28(4):213-218.
R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing; 2019.
LoMartire R. rel: Reliability Coefficients. R package version 1.4.2. 2017. https://CRAN.R-project.org/package=rel.
Fox J. polycor: Polychoric and Polyserial Correlations. R package version 0.7-10. 2019. https://CRAN.R-project.org/package=polycor.
Richardson CJ, Pambuccian SE, Barkan GA. Split-sample comparison of urothelial cells in thin prep and cytospin preparations in urinary cytology: do we need to adjust The Paris System for Reporting Urinary Cytology criteria? Cancer Cytopathol. 2020;128(2):119-125.
Layfield LJ, Esebua M, Frazier SR, et al. Accuracy and reproducibility of nuclear/cytoplasmic ratio assessments in urinary cytology specimens. Diagn Cytopathol. 2017;45(2):107-112.
Zhang ML, Guo AX, vanden Bussche CJ. Morphologists overestimate the nuclear-to-cytoplasmic ratio. Cancer Cytopathol. 2016;124(9):669-677.
Rezaee N, Tabatabai ZL, Olson MT. Adequacy of voided urine specimens prepared by thin prep and evaluated using The Paris System for Reporting Urinary Cytology. J Am Soc Cytopathol. 2017;6(4):155-161.
Xing J, Qi Y, Monaco SE, Pantanowitz L. Determination of appropriate urine volume cutoff values for voided urine specimens to assess adequacy. J Am Soc Cytopathol. 2019;8(2):89-94.
Vandenbussche CJ, Rosenthal DL, Olson MT. Adequacy in voided urine cytology specimens: the role of volume and a repeat void upon predictive values for high-grade urothelial carcinoma. Cancer Cytopathol. 2016;124(3):174-180.
Maclure M, Willett WC. Misinterpretation and misuse of the kappa statistic. Am J Epidemiol. 1987;126(2):161-169.
Tran D, Dolgun A, Demirhan H. Weighted inter-rater agreement measures for ordinal outcomes. Commun Stat Simul Comput. 2020;49(4):989-1003.
Wongpakaran N, Wongpakaran T, Wedding D, Gwet KL. A comparison of Cohen's Kappa and Gwet's AC1 when calculating inter-rater reliability coefficients: a study conducted with personality disorder samples. BMC Med Res Methodol. 2013;13(1):61.
Zare S, Mirsadraei L, Reisian N, et al. A single institutional experience with The Paris System for Reporting Urinary Cytology. Am J Clin Pathol. 2018;150(2):162-167.
Rai S, Lali BS, Venkataramana CG, Philipose CS, Rao R, Prabhu GL. A quest for accuracy: evaluation of the Paris system in diagnosis of urothelial carcinomas. J Cytol. 2019;36(3):169-173.
Meilleroux J, Daniel G, Aziza J, et al. One year of experience using The Paris System for Reporting Urinary Cytology. Cancer Cytopathol. 2018;126(6):430-436.