Multi-level longitudinal learning curve regression models integrated with item difficulty metrics for deliberate practice of visual diagnosis: groundwork for adaptive learning.
Adaptive Learning
Electrocardiograms
Item-response theory
Learning curves
Multi-level modelling
Predictive analytics
Radiology
Statistical modelling
Journal
Advances in health sciences education : theory and practice
ISSN: 1573-1677
Titre abrégé: Adv Health Sci Educ Theory Pract
Pays: Netherlands
ID NLM: 9612021
Informations de publication
Date de publication:
08 2021
08 2021
Historique:
received:
21
07
2020
accepted:
07
01
2021
pubmed:
2
3
2021
medline:
26
10
2021
entrez:
1
3
2021
Statut:
ppublish
Résumé
Visual diagnosis of radiographs, histology and electrocardiograms lends itself to deliberate practice, facilitated by large online banks of cases. Which cases to supply to which learners in which order is still to be worked out, with there being considerable potential for adapting the learning. Advances in statistical modeling, based on an accumulating learning curve, offer methods for more effectively pairing learners with cases of known calibrations. Using demonstration radiograph and electrocardiogram datasets, the advantages of moving from traditional regression to multilevel methods for modeling growth in ability or performance are demonstrated, with a final step of integrating case-level item-response information based on diagnostic grouping. This produces more precise individual-level estimates that can eventually support learner adaptive case selection. The progressive increase in model sophistication is not simply statistical but rather brings the models into alignment with core learning principles including the importance of taking into account individual differences in baseline skill and learning rate as well as the differential interaction with cases of varying diagnosis and difficulty. The developed approach can thus give researchers and educators a better basis on which to anticipate learners' pathways and individually adapt their future learning.
Identifiants
pubmed: 33646468
doi: 10.1007/s10459-021-10027-0
pii: 10.1007/s10459-021-10027-0
doi:
Types de publication
Journal Article
Research Support, U.S. Gov't, Non-P.H.S.
Langues
eng
Sous-ensembles de citation
IM
Pagination
881-912Informations de copyright
© 2021. The Author(s), under exclusive licence to Springer Nature B.V. part of Springer Nature.
Références
Anderson, S. J., Hecker, K. G., Krigolson, O. E., & Jamniczky, H. A. (2018). A Reinforcement-Based Learning Paradigm Increases Anatomical Learning and Retention—A Neuroeducation Study. Frontiers in Human Neuroscience, 12, 38.
doi: 10.3389/fnhum.2018.00038
Bates, D., Maechler, M., Bolker, B, & Walker, S. Fitting linear mixed-effects models using lme4. Journal of Statistical Software. 2000. https://arxiv.org/abs/1406.5823v1
Bates D, Maechler M, Bolker B, Walker S. (2015). lme4: Linear Mixed-Effects Models Using Eigen and S4. R package version 1.1–10, URL http://CRAN.R-project.org/package=lme4 .
Bok, H. G., de Jong, L. H., O’Neill, T., Maxey, C., & Hecker, K. G. (2018). Validity evidence for programmatic assessment in competency-based education. Perspectives on medical education., 7(6), 362–372.
doi: 10.1007/s40037-018-0481-2
Bolsin, S., & Colson, M. (2000). The use of the Cusum Technique in the assessment of trainee competence in new procedures. International Journal for Quality in Health Care, 12(5), 433–438.
doi: 10.1093/intqhc/12.5.433
Boutis, K., Pecaric, M., Carrière, B., Stimec, J., Willan, A., Chan, J., & Pusic, M. (2019). The effect of testing and feedback on the forgetting curves for radiograph interpretation skills. Medical Teacher, 41(7), 756–764.
doi: 10.1080/0142159X.2019.1570098
Bryk, A. S., & Raudenbush, S. W. (1987). Application of hierarchical linear models to assessing change. Psychological Bulletin, 101, 147–158.
doi: 10.1037/0033-2909.101.1.147
Cepeda, N. J., Vul, E., Rohrer, D., Wixted, J. T., & Pashler, H. (2008). Spacing effects in learning: A temporal ridgeline of optimal retention. Psychological Science., 19(11), 1095–1102.
doi: 10.1111/j.1467-9280.2008.02209.x
Cepeda, N. J., Coburn, N., Rohrer, D., Wixted, J. T., Mozer, M. C., & Pashler, H. (2009). Optimizing distributed practice: Theoretical analysis and practical implications. Experimental Psychology., 56(4), 236–246.
doi: 10.1027/1618-3169.56.4.236
Chaiklin, S. (2003). The zone of proximal development in Vygotsky’s analysis of learning and instruction. In A. Kozulin, B. Gindis, V. Ageyev, & S. Miller (Eds.), Vygotsky’s educational theory in cultural context (First (pp. 39–64). Cambridge, England: Cambridge University Press.
doi: 10.1017/CBO9780511840975.004
Davis, A. L., Pecaric, M., Pusic, M. V., Smith, T., Shouldice, M., Brown, J., & Boutis, K. (2020). Deliberate practice as an educational method for learning to interpret the prepubescent female genital examination. Child Abuse and Neglect, 101, 104379.
doi: 10.1016/j.chiabu.2020.104379
De Boeck, P., et al. (2011). The Estimation of Item Response Models with the lmer Function from the lme4 Package in R. Journal of Statistical Software, 39(12), 1–28.
doi: 10.18637/jss.v039.i12
De Boeck, P., & Wilson, M. (2004). Explanatory item response models: A generalized linear and nonlinear approach. New York: Springer.
doi: 10.1007/978-1-4757-3990-9
Diederich, E., Thomas, L., Mahnken, J., & Lineberry, M. (2018). Pretest scores uniquely predict 1-year-delayed performance in a simulation-based mastery course for central line insertion. Simulation in Healthcare, 13(3), 163–167.
doi: 10.1097/SIH.0000000000000327
Doran, H., Bates, D., Bliese, P., & Dowling, M. (2007). Estimating the multilevel rasch model: with the lme4 package. Journal of Statistical Software, 20(2), 1–18.
doi: 10.18637/jss.v020.i02
Downing, S. M. (2003). Item response theory: applications of modern test theory in medical education. Medical education, 37(8), 739–745.
doi: 10.1046/j.1365-2923.2003.01587.x
Ericsson, K. A. (2004). Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Academic Medicine, 79(10), S70–S81.
doi: 10.1097/00001888-200410001-00022
Ericsson, K. A. (2015). Acquisition and maintenance of medical expertise. Academic Medicine, 90(11), 1471–1486.
doi: 10.1097/ACM.0000000000000939
Ertmer, P. A., & Newby, T. J. (1993). Behaviorism, cognitivism, constructivism: comparing critical features from an instructional design perspective. Performance Improvement Quarterly, 6(4), 50–72.
doi: 10.1111/j.1937-8327.1993.tb00605.x
Faraway, J. J. (2016). Linear models with R. Boca Raton: Chapman and Hall/CRC.
doi: 10.1201/b17144
Gelman, A., & Hill, J. (2007). Multilevel linear models: the basics. Data analysis using regression and multilevel/hierarchical models (pp. 251–278). Cambridge: Cambridge University Press.
Gelman, A., & Hill, J. (2007). Simulation of Probability Models and Statistical Inferences in Data Analysis Using Regression and Multilevel/Hierarchical Models (pp. 251–278). Cambridge: Cambridge University Press.
Guadagnoli, M., Morin, M., & Dubrowski, A. (2012). The application of the challenge point framework in medical education. Medical Education, 46, 447–453.
doi: 10.1111/j.1365-2923.2011.04210.x
Gulliksen, H. (1934). A rational equation of the learning curve based on Thorndike’s law of effect. The Journal of General Psychology, 11(2), 395–434.
doi: 10.1080/00221309.1934.9917847
Hatala, R., Gutman, J., Lineberry, M., Triola, M., & Pusic, M. (2019). How well is each learner learning? Validity investigation of a learning curve-based assessment approach for ECG interpretation. Advances in Health Sciences Education, 24(1), 45–63.
doi: 10.1007/s10459-018-9846-x
Jaber, M. Y., & Bonney, M. (1997). A comparative study of learning curves with forgetting. Applied Mathematical Modelling, 21(8), 523–531.
doi: 10.1016/S0307-904X(97)00055-3
Jonassen, D. (1991). Objectivism versus constructivism: Do we need a new philosophical paradigm? Educational Technology Research and Development, 39(3), 5–14.
doi: 10.1007/BF02296434
Kerfoot, B. P., Baker, H., Pangara, L., Agarwal, K., Taffet, G., Mechaber, A. J., et al. (2012). An online spaced-education game to teach and assess medical students. Academic Medicine, 87(10), 1443–1449.
doi: 10.1097/ACM.0b013e318267743a
Linacre, J. (1994). Sample size and item calibration stability. Rasch Measurement Transactions, 7(4), 328.
Lindsey, R. V., Shroyer, J. D., Pashler, H., & Mozer, M. C. (2014). Improving students’ long-term knowledge retention through personalized review. Psychological Science, 25(3), 639–647.
doi: 10.1177/0956797613504302
Park, O., & Lee, J. (2003). Adaptive instructional systems. Educational Technology Research and Development, 25, 651–684.
Pavlik, P. I., & Anderson, J. R. (2008). Using a model to compute the optimal schedule of practice. Journal of Experimental Psychology: Applied, 14(2), 101–117.
Pecaric, M., Boutis, K., Beckstead, J., & Pusic, M. (2017). A big data and learning analytics approach to process-level feedback in cognitive simulations. Academic Medicine, 92(2), 175–184.
doi: 10.1097/ACM.0000000000001234
Price, D. W., Swanson, D. B., Irons, M. B., & Hawkins, R. E. (2018 Sep 2). Longitudinal assessments in continuing specialty certification and lifelong learning. Medical Teacher, 40(9), 917–919.
doi: 10.1080/0142159X.2018.1471202
Pusic, M. V., Pecaric, M., & Boutis, K. (2011). How much practice is enough? Using learning curves to assess the deliberate practice of radiograph interpretation. Academic Medicine, 86, 731–736.
doi: 10.1097/ACM.0b013e3182178c3c
Pusic, M. V., Boutis, K., Hatala, R., & Cook, D. (2015). Learning curves in health professions education. Academic Medicine : Journal of the Association of American Medical Colleges, 90(8), 1034–1042.
doi: 10.1097/ACM.0000000000000681
Pusic, M. V., Boutis, K., Pecaric, M. R., Savenkov, O., Beckstead, J. W., & Jaber, M. Y. (2017). A primer on the statistical modelling of learning curves in health professions education. Advances in Health Sciences Education, 22(3), 741–759.
doi: 10.1007/s10459-016-9709-2
Pusic, M. V., Boutis, K., & McGaghie, W. C. (2018). Role of scientific theory in simulation education research. Simulation in Healthcare, 13(3S), S7-14.
doi: 10.1097/SIH.0000000000000282
R Core Team (2015). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. URL http://www.R-project.org/ .
Rijmen, F., Tuerlinckx, F., Boeck, P. D., & Kuppens, P. (2003). A nonlinear mixed model framework for item response theory. Psychological Methods, 8(2), 185–205.
doi: 10.1037/1082-989X.8.2.185
Robson, K., & Pevalin, D. J. (2016). Multilevel modeling in plain language. London: SAGE.
doi: 10.4135/9781473920712
Singer, J. D., & Willett, J. B. (2003). Doing data analysis with the multilevel model for change. Applied longitudinal data analysis: modeling change and event occurrence (pp. 75–137). Oxford: Oxford University Press.
doi: 10.1093/acprof:oso/9780195152968.003.0004
Thurstone, L. L. (1919). The learning curve equation. Psychological Review, 34, 278–286.
van der Linden, W. J. (2009). Constrained adaptive testing with shadow tests. Elements of adaptive testing (pp. 31–55). New York: Springer.
doi: 10.1007/978-0-387-85461-8_2
Versteeg, M., Hendriks, R. A., Thomas, A., Ommering, B. W. C., & Steendijk, P. (2020). Conceptualising spaced learning in health professions education: A scoping review. Medical Education., 54(3), 205–216.
doi: 10.1111/medu.14025
Wang, L., Zhang, Z., McArdle, J. J., & Salthouse, T. A. (2008). Investigating ceiling effects in longitudinal data analysis. Multivariate Behavioral Research, 43(3), 476–496.
doi: 10.1080/00273170802285941
Wolfe, J. M., Evans, K. K., Drew, T., Aizenman, A., & Josephs, E. (2016). How do radiologists use the human search engine? Radiation Protection Dosimetry, 169(1–4), 24–31.
doi: 10.1093/rpd/ncv501
Wood, G., BattAppelboam, J. A., Harris, A., & Wilson, M. R. (2013). Exploring the impact of expertise, clinical history, and visual search on electrocardiogram interpretation. Medical Decision Making, 34(1), 75–83.
doi: 10.1177/0272989X13492016