Psychometric validation of the Laval developmental benchmarks scale for family medicine.
Criterion-referenced assessment
Family medicine
Validation
Journal
BMC medical education
ISSN: 1472-6920
Titre abrégé: BMC Med Educ
Pays: England
ID NLM: 101088679
Informations de publication
Date de publication:
27 Jun 2021
27 Jun 2021
Historique:
received:
28
07
2020
accepted:
17
06
2021
entrez:
28
6
2021
pubmed:
29
6
2021
medline:
30
6
2021
Statut:
epublish
Résumé
With the implementation of competency-based education in family medicine, there is a need for summative end-of-rotation assessments that are criterion-referenced rather than normative. Laval University's family residency program therefore developed the Laval Developmental Benchmarks Scale for Family Medicine (DBS-FM), based on competency milestones. This psychometric validation study investigates its internal structure and its relation to another variable, two sources of validity evidence. We used assessment data from a cohort of residents (n = 1432 assessments) and the Rasch Rating Scale Model to investigate its reliability, dimensionality, rating scale functioning, targeting of items to residents' competency levels, biases (differential item functioning), items hierarchy (adequacy of milestones ordering), and score responsiveness. Convergent validity was estimated by its correlation with the clinical rotation decision (pass, in difficulty/fail). The DBS-FM can be considered as a unidimensional scale with good reliability for non-extreme scores (.83). The correlation between expected and empirical items hierarchies was of .78, p < .0001.Year 2 residents achieved higher scores than year 1 residents. It was associated with the clinical rotation decision. Advancing its validation, this study found that the DBS-FM has a sound internal structure and demonstrates convergent validity.
Sections du résumé
BACKGROUND
BACKGROUND
With the implementation of competency-based education in family medicine, there is a need for summative end-of-rotation assessments that are criterion-referenced rather than normative. Laval University's family residency program therefore developed the Laval Developmental Benchmarks Scale for Family Medicine (DBS-FM), based on competency milestones. This psychometric validation study investigates its internal structure and its relation to another variable, two sources of validity evidence.
METHODS
METHODS
We used assessment data from a cohort of residents (n = 1432 assessments) and the Rasch Rating Scale Model to investigate its reliability, dimensionality, rating scale functioning, targeting of items to residents' competency levels, biases (differential item functioning), items hierarchy (adequacy of milestones ordering), and score responsiveness. Convergent validity was estimated by its correlation with the clinical rotation decision (pass, in difficulty/fail).
RESULTS
RESULTS
The DBS-FM can be considered as a unidimensional scale with good reliability for non-extreme scores (.83). The correlation between expected and empirical items hierarchies was of .78, p < .0001.Year 2 residents achieved higher scores than year 1 residents. It was associated with the clinical rotation decision.
CONCLUSION
CONCLUSIONS
Advancing its validation, this study found that the DBS-FM has a sound internal structure and demonstrates convergent validity.
Identifiants
pubmed: 34176475
doi: 10.1186/s12909-021-02797-3
pii: 10.1186/s12909-021-02797-3
pmc: PMC8237442
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
357Subventions
Organisme : Medical Council of Canada
ID : MCC-5/1718
Références
Acad Psychiatry. 2014 Jun;38(3):294-302
pubmed: 24743953
GMS J Med Educ. 2017 Nov 15;34(5):Doc63
pubmed: 29226231
Acad Med. 2017 Jun;92(6):809-819
pubmed: 28557947
Acad Med. 2007 Jun;82(6):542-7
pubmed: 17525536
J Grad Med Educ. 2012 Jun;4(2):220-6
pubmed: 23730445
J Surg Educ. 2017 Jan - Feb;74(1):68-73
pubmed: 27395399
Acad Med. 2002 May;77(5):361-7
pubmed: 12010689
Med Teach. 2017 May;39(5):494-504
pubmed: 28281837
Teach Learn Med. 2014;26(3):244-51
pubmed: 25010235
Med Teach. 2017 Jun;39(6):582-587
pubmed: 28598739
Fam Med. 2011 Feb;43(2):90-8
pubmed: 21344328
Fam Med. 2008 Sep;40(8):574-8
pubmed: 18988044
Arthritis Rheum. 2007 Dec 15;57(8):1358-62
pubmed: 18050173