Consistency checks to improve measurement with the Young Mania Rating Scale (YMRS).
Careless ratings
Consistency of measurement
Inconsistent ratings
YMRS
Young Mania Rating Scale
Journal
Journal of affective disorders
ISSN: 1573-2517
Titre abrégé: J Affect Disord
Pays: Netherlands
ID NLM: 7906073
Informations de publication
Date de publication:
15 01 2024
15 01 2024
Historique:
received:
31
05
2023
revised:
22
09
2023
accepted:
15
10
2023
medline:
23
11
2023
pubmed:
22
10
2023
entrez:
21
10
2023
Statut:
ppublish
Résumé
Mitigating rating inconsistency can improve measurement fidelity and detection of treatment response. The International Society for CNS Clinical Trials and Methodology convened an expert Working Group that developed logical consistency (LC) checks for ratings of the Young Mania Rating Scale (YMRS), which is widely used in studies of mood and bipolar disorders. LC and statistical outlier-response pattern checks (SC) were applied to 63,228 YMRS administrations from 14 clinical trials evaluating treatments for bipolar disorder. Checks were also applied to Monte Carlo-simulated data as a proxy for their use under conditions of inconsistency. 42 LC flags were developed, and four SC flags were created from the data set (n = 14). Almost 20 % of the rating administrations had at least one LC flag, 6.7 % had two or more, 1.7 % had three or more; 17.3 % percent of the administrations had at least one SC flag and 4.6 % percent had two or more. Overall, 31 % of administrations had at least one flag of any type, 12.1 % had two or more and 5.3 % had three or more. In acute antimanic treatment trials (n = 10) there were more flags of any type compared to relapse prevention trials (n = 4). Flagged ratings may represent less-common presentations assessed correctly. Using established methods, we illustrate development and application of consistency flags for YMRS ratings. Applying flags and mitigation during trials may improve the value of YMRS data, help focus attention on rater training, and improve reliability and validity of trial data.
Sections du résumé
BACKGROUND
Mitigating rating inconsistency can improve measurement fidelity and detection of treatment response.
METHODS
The International Society for CNS Clinical Trials and Methodology convened an expert Working Group that developed logical consistency (LC) checks for ratings of the Young Mania Rating Scale (YMRS), which is widely used in studies of mood and bipolar disorders. LC and statistical outlier-response pattern checks (SC) were applied to 63,228 YMRS administrations from 14 clinical trials evaluating treatments for bipolar disorder. Checks were also applied to Monte Carlo-simulated data as a proxy for their use under conditions of inconsistency.
RESULTS
42 LC flags were developed, and four SC flags were created from the data set (n = 14). Almost 20 % of the rating administrations had at least one LC flag, 6.7 % had two or more, 1.7 % had three or more; 17.3 % percent of the administrations had at least one SC flag and 4.6 % percent had two or more. Overall, 31 % of administrations had at least one flag of any type, 12.1 % had two or more and 5.3 % had three or more. In acute antimanic treatment trials (n = 10) there were more flags of any type compared to relapse prevention trials (n = 4).
LIMITATIONS
Flagged ratings may represent less-common presentations assessed correctly.
CONCLUSIONS
Using established methods, we illustrate development and application of consistency flags for YMRS ratings. Applying flags and mitigation during trials may improve the value of YMRS data, help focus attention on rater training, and improve reliability and validity of trial data.
Identifiants
pubmed: 37865349
pii: S0165-0327(23)01287-9
doi: 10.1016/j.jad.2023.10.098
pii:
doi:
Substances chimiques
Antimanic Agents
0
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
24-31Informations de copyright
Copyright © 2023 The Author(s). Published by Elsevier B.V. All rights reserved.
Déclaration de conflit d'intérêts
Declaration of competing interest The authors report the following possible conflicts of interest: Jonathan Rabinowitz has received research grant(s) support and/or travel support and/or speaker fees and/or consultant fees from Janssen (J&J), Eli Lilly, Pfizer, Clexio, Roche, Lundbeck, Pierre Fabre, Intra-cellular Therapies, Minerva and Takeda. Robert C. Young and Patricia Marino have no conflicts of interest to report. Janet B.W. Williams receives modest royalties from the Royal College of Psychiatrists for the licensing of the SIGMA and from MAPI for the use of the SIGH-D. Christian Yavorsky is a principal at Valis Bioscience. Jan Sedway and Chris Brady are full-time employees of WCG. Alan Kott is a full time Employee of Signant Health, LLC. Christopher Matteo is a full-time employee of TPS Global. Jenicka Engler is a full-time employee of Cronos/IQVIA. The five previously mentioned companies provide risk-based data monitoring that utilizes algorithms like those described in this paper to identify sources of potential error in clinical trials. Atul Mahableshwarkar, is a principal at ARM Pharma Consulting. Nanco Hefting is an employee and shareholder of H. Lundbeck A/S Lundbeck.