Development and Evaluation of a Pragmatic Measure of Adherence to Dialectical Behavior Therapy: The DBT Adherence Checklist for Individual Therapy.
Adherence
Dialectical behavior therapy
Fidelity
Psychometric
Journal
Administration and policy in mental health
ISSN: 1573-3289
Titre abrégé: Adm Policy Ment Health
Pays: United States
ID NLM: 8914574
Informations de publication
Date de publication:
09 2023
09 2023
Historique:
accepted:
08
05
2023
medline:
21
7
2023
pubmed:
7
6
2023
entrez:
7
6
2023
Statut:
ppublish
Résumé
This paper presents two studies conducted to develop and evaluate a new pragmatic measure of therapist adherence to Dialectical Behavior Therapy (DBT): the DBT Adherence Checklist for Individual Therapy (DBT AC-I). Study 1 used item response analysis to select items from the gold standard DBT Adherence Coding Scale (DBT ACS) using archival data from 1271 DBT sessions. Items were then iteratively refined based on feedback from 33 target end-users to ensure relevance, usability, and understandability. Study 2 examined the psychometric properties of the DBT AC-I as a therapist self-report and observer-rated measure in 100 sessions from 50 therapist-client dyads, while also evaluating predictors of therapist accuracy in self-rated adherence. When used as a therapist self-report measure, concordance between therapist and observer ratings was at least moderate (AC
Identifiants
pubmed: 37284966
doi: 10.1007/s10488-023-01274-x
pii: 10.1007/s10488-023-01274-x
pmc: PMC10246542
doi:
Types de publication
Journal Article
Research Support, N.I.H., Extramural
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
734-749Subventions
Organisme : Department of Health
ID : PB-PG-0906-10540
Pays : United Kingdom
Organisme : NIMH NIH HHS
ID : R56 MH117085
Pays : United States
Organisme : NIMH NIH HHS
ID : R01 MH034486
Pays : United States
Organisme : NIMH NIH HHS
ID : R01 MH090159
Pays : United States
Organisme : NIMH NIH HHS
ID : R34 MH082143
Pays : United States
Organisme : NIMH NIH HHS
ID : R34 MH106598
Pays : United States
Organisme : NIDA NIH HHS
ID : R01 DA014997
Pays : United States
Informations de copyright
© 2023. This is a U.S. Government work and not under copyright protection in the US; foreign copyright protection may apply.
Références
Altman, D. G. (1999). Practical statistics for medical research. Hall/CRC Press.
An, X., & Yung, Y. F. (2014). Item response theory: What it is and how you can use the IRT procedure to apply it. SAS Institute.
Bangor, A., Kortum, P. T., & Miller, J. T. (2008). An empirical evaluation of the system usability scale. International Journal of Human-Computer Interaction, 24, 574–594. https://doi.org/10.1002/jclp.20221
doi: 10.1002/jclp.20221
Becker-Haimes, E. M., Marcus, S. C., Klein, M. R., Schoenwald, S. K., Fugo, P. B., & Beidas, R. S. (2022). A randomized trial to identify accurate measurement methods for adherence to cognitive-behavioral therapy. Behavior Therapy. https://doi.org/10.1016/j.beth.2022.06.001
doi: 10.1016/j.beth.2022.06.001
pubmed: 36229116
Bentler, P. M., & Bonett, D. G. (1980). Significance tests and goodness-of-fit in the analysis of covariance structures. Psychological Bulletin, 88, 588–600. https://doi.org/10.1037/0033-2909.88.3.588
doi: 10.1037/0033-2909.88.3.588
Bohus, M., Kleindienst, N., Limberger, M. F., Stieglitz, R., & DomsallaWolf, M. M. (2009). The short version of the borderline symptom list (BSL-23): Development and initial data on psychometric properties. Psychopathology, 42, 32–39. https://doi.org/10.1159/000173701
doi: 10.1159/000173701
pubmed: 19023232
Brookman-Frazee, L., Stadnick, N. A., Lind, T., Roesch, S., Terrones, L., & Lau, A. S. (2021). Therapist-observer concordance in ratings of EBP strategy delivery: Challenges and targeted directions in pursuing pragmatic measurement in children’s mental health services. Administration and Policy in Mental Health Services Research, 48, 155–170. https://doi.org/10.1007/s10488-020-01054-x
doi: 10.1007/s10488-020-01054-x
Brosan, L., Reynolds, S., & Moore, R. G. (2008). Self-evaluation of cognitive therapy performance: Do therapists know how competent they are? Behavioural and Cognitive Psychotherapy, 36, 581–587. https://doi.org/10.1017/S1352465808004438
doi: 10.1017/S1352465808004438
Carmel, A., Rose, M., & Fruzzetti, A. (2014). Barriers and solutions to implementing dialectical behavior therapy in a public behavioral health system. Administration and Policy in Mental Health Services Research, 41, 608–614. https://doi.org/10.1007/s10488-013-0504-6
doi: 10.1007/s10488-013-0504-6
Chalker, S. A., Carmel, A., Atkins, D. C., Landes, S. J., Kerbrat, A. H., & Comtois, K. A. (2015). Examining challenging behaviors of clients with borderline personality disorder. Behaviour Research and Therapy, 75, 11–19. https://doi.org/10.1016/j.brat.2015.10.003
doi: 10.1016/j.brat.2015.10.003
pubmed: 26496225
pmcid: 4690778
Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, 284–290. https://doi.org/10.1037/1040-3590.6.4.284
doi: 10.1037/1040-3590.6.4.284
Cicchetti, D. V., & Feinstein, A. R. (1990). High agreement but low Kappa: II. Resolving the paradoxes. Journal of Clinical Epidemiology, 43, 551–558. https://doi.org/10.1016/0895-4356(90)90159-m
doi: 10.1016/0895-4356(90)90159-m
pubmed: 2189948
DeCou, C. R., Comtois, K. A., & Landes, S. J. (2019). Dialectical behavior therapy is effective for the treatment of suicidal behavior: A meta-analysis. Behavior Therapy, 50, 60–72. https://doi.org/10.1016/j.beth.2018.03.009
doi: 10.1016/j.beth.2018.03.009
pubmed: 30661567
DuBose, A. P., Botanov, Y., & Ivanoff, A. (2019). International implementation of dialectical behaviour therapy: The challenge of training therapists across cultures. In M. A. Swales (Ed.), The Oxford handbook of dialectical behaviour therapy (pp. 909–930). Oxford University Press.
Endicott, J., Spitzer, R. L., Fleiss, J. L., & Cohen, J. (1976). The global assessment scale. A procedure for measuring overall severity of psychiatric disturbance. Archives of General Psychiatry, 33, 766–771. https://doi.org/10.1001/archpsyc.1976.01770060086012
doi: 10.1001/archpsyc.1976.01770060086012
pubmed: 938196
Flynn, D., Joyce, M., Gillespie, C., Kells, M., Swales, M., & Weihrauch, M. (2020). Evaluating the national multisite implementation of dialectical behaviour therapy in a community setting: A mixed methods approach. BMC Psychiatry, 20, 235. https://doi.org/10.1186/s12888-020-02610-3
doi: 10.1186/s12888-020-02610-3
pubmed: 32410670
pmcid: 7227064
Gallop, R. J., Crits-Christoph, P., Muenz, L. R., & Tu, X. M. (2003). Determination and interpretation of the optimal operating point for ROC curves derived through generalized linear models. Understanding Statistics, 2, 219–242. https://doi.org/10.1207/S15328031US0204_01
doi: 10.1207/S15328031US0204_01
Gwet, K. L. (2008). Computing inter-rater reliability and its variance in the presence of high agreement. The British Journal of Mathematical and Statistical Psychology, 61, 29–48. https://doi.org/10.1348/000711006X126600
doi: 10.1348/000711006X126600
pubmed: 18482474
Harned, M. S., Gallop, R. J., Schmidt, S. C., & Korslund, K. E. (2022). The temporal relationships between therapist adherence and client outcomes in dialectical behavior therapy. Journal of Consulting and Clinical Psychology, 90, 272–281. https://doi.org/10.1037/ccp0000714
doi: 10.1037/ccp0000714
pubmed: 35084894
Harned, M. S., Korslund, K. E., Schmidt, S. C., & Gallop, R. J. (2021a). The Dialectical Behavior Therapy Adherence Coding Scale (DBT ACS): Psychometric properties. Psychological Assessment, 33, 552–561. https://doi.org/10.1037/pas0000999
doi: 10.1037/pas0000999
pubmed: 33764118
pmcid: 8363169
Harned, M. S., Schmidt, S. C., & Korslund, K. E. (2021b). Does adding the dialectical behavior therapy prolonged exposure (DBT PE) protocol for PTSD to DBT improve outcomes in public mental health agencies? A pilot nonrandomized effectiveness trial with benchmarking. Behavior Therapy, 52, 639–655. https://doi.org/10.1016/j.beth.2020.08.003
doi: 10.1016/j.beth.2020.08.003
pubmed: 33990239
Herschell, A. D., Lindhiem, O. J., Kogan, J. N., Celedonia, K. L., & Stein, B. D. (2014). Evaluation of an implementation initiative for embedding dialectical behavior therapy in community settings. Evaluation and Program Planning, 43, 55–63. https://doi.org/10.1016/j.evalprogplan.2013.10.007
doi: 10.1016/j.evalprogplan.2013.10.007
pubmed: 24333657
Hinkin, T. R. (1998). A brief tutorial on the development of measures for use in survey questionnaires. Organizational Research Methods, 1, 104–121. https://doi.org/10.1177/109442819800100106
doi: 10.1177/109442819800100106
Hogue, A., Dauber, S., Lichvar, E., Bobek, M., & Henderson, C. E. (2015). Validity of therapist self-report ratings of fidelity to evidence-based practices for adolescent behavior problems: Correspondence between therapists and observers. Administration and Policy in Mental Health Services Research, 42, 229–243. https://doi.org/10.1007/s10488-014-0548-2
doi: 10.1007/s10488-014-0548-2
Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55. https://doi.org/10.1080/10705519909540118
doi: 10.1080/10705519909540118
Hurlburt, M. S., Garland, A. F., Nguyen, K., & Brookman-Frazee, L. (2010). Child and family therapy process: Concordance of therapist and observational perspectives. Administration and Policy in Mental Health Services Research, 37, 230–244. https://doi.org/10.1007/s10488-009-0251-x
doi: 10.1007/s10488-009-0251-x
Linehan, M. M. (1993). Cognitive-behavioral treatment of borderline personality disorder. Guilford Press.
Linehan, M. M., & Korslund, K. E. (2003). Dialectical behavior therapy adherence coding scale manual. University of Washington.
Loades, M. E., & Myles, P. J. (2016). Does a therapist’s reflective ability predict the accuracy of their self-evaluation of competence in cognitive behavioural therapy? The Cognitive Behaviour Therapist, 9, 1–14. https://doi.org/10.1017/S1754470X16000027
doi: 10.1017/S1754470X16000027
Martino, S., Ball, S., Nich, C., Frankforter, T. L., & Carroll, K. M. (2009). Correspondence of motivational enhancement treatment integrity ratings among therapists, supervisors, and observers. Psychotherapy Research, 19, 181–193. https://doi.org/10.1080/10503300802688460
doi: 10.1080/10503300802688460
pubmed: 19396649
pmcid: 2782382
Navarro-Haro, M. V., Harned, M. S., Korslund, K. E., DuBose, A., Chen, T., Ivanoff, A., & Linehan, M. M. (2019). Predictors of adoption and reach following dialectical behavior therapy intensive training. Community Mental Health Journal, 55, 100–111. https://doi.org/10.1007/s10597-018-0254-8
doi: 10.1007/s10597-018-0254-8
pubmed: 29508180
Perepletchikova, F., Treat, T. A., & Kazdin, A. E. (2007). Treatment integrity in psychotherapy research: Analysis of the studies and examination of the associated factors. Journal of Consulting and Clinical Psychology, 75, 829–841. https://doi.org/10.1037/0022-006X.75.6.829
doi: 10.1037/0022-006X.75.6.829
pubmed: 18085901
Posner, K., Brown, G., StanleyBrentYershova, K. D. A. K. V., & Mann, J. J. (2011). The Columbia-Suicide Severity Rating Scale: Initial validity and internal consistency findings from three multisite studies with adolescents and adults. American Journal of Psychiatry, 168, 1266–1277. https://doi.org/10.1176/appi.ajp.2011.10111704
doi: 10.1176/appi.ajp.2011.10111704
pubmed: 22193671
Proctor, E. K., Landsverk, J., Aarons, G., Chambers, D., Glisson, C., & Mittman, B. (2009). Implementation research in mental health services: An emerging science with conceptual, methodological, and training challenges. Administration and Policy in Mental Health Services Research, 36, 24–34. https://doi.org/10.1007/s10488-008-0197-4
doi: 10.1007/s10488-008-0197-4
Reise, S. P., & Waller, N. G. (2009). Item response theory and clinical measurement. Annual Review of Clinical Psychology, 5, 27–48. https://doi.org/10.1146/annurev.clinpsy.032408.153553
doi: 10.1146/annurev.clinpsy.032408.153553
pubmed: 18976138
Schoenwald, S. K., & Garland, A. F. (2013). A review of treatment adherence measurement methods. Psychological Assessment, 25, 146–156. https://doi.org/10.1037/a0029715
doi: 10.1037/a0029715
pubmed: 22888981
Schoenwald, S. K., Garland, A. F., Chapman, J. E., Frazier, S. L., Sheidow, A. J., & Southam-Gerow, M. A. (2011). Toward the effective and efficient measurement of implementation fidelity. Administration and Policy in Mental Health Services Research, 38, 32–43. https://doi.org/10.1007/s10488-010-0321-0
doi: 10.1007/s10488-010-0321-0
Storebø, O. J., Stoffers-Winterling, J. M., Völlm, B. A., Kongerslev, M. T., Mattivi, J. T., Jørgensen, M. S., & Simonsen, E. (2020). Psychological therapies for people with borderline personality disorder. Cochrane Database of Systematic Reviews. https://doi.org/10.1002/14651858.CD012955.pub2
doi: 10.1002/14651858.CD012955.pub2
pubmed: 32681745
pmcid: 7388313
Swales, M. A., Taylor, B., & Hibbs, R. A. B. (2012). Implementing dialectical behaviour therapy: Programme survival in routine healthcare settings. Journal of Mental Health, 21, 548–555. https://doi.org/10.3109/09638237.2012.689435
doi: 10.3109/09638237.2012.689435
pubmed: 22958107
Taasoobshirazi, G., & Wang, S. (2016). The performance of the SRMR, RMSEA, CFI, and TLI: An examination of sample size, path size, and degrees of freedom. Journal of Applied Quantitative Methods, 11, 31–39.
Wiltsey Stirman, S., Gutner, C. A., Gamarra, J., Suvak, M. K., Vogt, D., & Resick, P. A. (2021). A novel approach to the assessment of fidelity to a cognitive behavioral therapy for PTSD using clinical worksheets: A proof of concept with cognitive processing therapy. Behavior Therapy, 52, 656–672. https://doi.org/10.1016/j.beth.2020.08.005
doi: 10.1016/j.beth.2020.08.005
Wolcott, M. D., & Lobczowski, N. G. (2021). Using cognitive interviews and think-aloud protocols to understand thought processes. Currents in Pharmacy Teaching and Learning, 13, 181–188. https://doi.org/10.1016/j.cptl.2020.09.005
doi: 10.1016/j.cptl.2020.09.005
pubmed: 33454077