Development and validation of a multiple-choice questionnaire-based theoretical test in direct ophthalmoscopy.
Messick
direct ophthalmoscopy
education
multiple-choice questionnaire
theoretical test
Journal
Acta ophthalmologica
ISSN: 1755-3768
Titre abrégé: Acta Ophthalmol
Pays: England
ID NLM: 101468102
Informations de publication
Date de publication:
Nov 2019
Nov 2019
Historique:
received:
19
06
2018
accepted:
02
02
2019
pubmed:
1
3
2019
medline:
6
2
2020
entrez:
1
3
2019
Statut:
ppublish
Résumé
Direct ophthalmoscopy can reveal systemic, neurologic and ophthalmic conditions, but is poorly mastered among young physicians. A theoretical test is needed to measure effect of educational interventions. We developed and gathered validity evidence for a multiple-choice questionnaire (MCQ)-based theoretical test in direct ophthalmoscopy. The MCQ was developed by interviewing experts. Then, validity evidence was evaluated using Messick's validity framework. Content was ensured by inviting the experts to contribute in a Delphi-like process. Response process was ensured by piloting and by streamlining all instructions. Then, the test was taken by ophthalmologists and by medical students without experience in direct ophthalmoscopy. Results were used to evaluate internal structure (item quality analysis and internal consistency), relations to other variables (correlation of test scores to experience level) and consequences (establishment of pass-fail score and the consequences of its use). The first phase of the study yielded 100 MCQs. In second phase, we identified that 60 items fulfilled predefined relevance and item quality requirements. These items demonstrated very high internal consistency (Cronbach's alpha = 0.95), significantly discriminated medical students from specialists (p < 0.001, independent samples t-test) and the established pass-fail score of 50 (83%) correct answers resulted in no false positives (students passing) and no false negatives (specialists failing). A Decision study identified that sampling 15 items suffice for certification. We developed and validated an MCQ-based theoretical test in direct ophthalmoscopy that enables an evidence-based approach to measuring, evaluating and certifying the theoretical knowledge necessary for direct ophthalmoscopy.
Types de publication
Journal Article
Validation Study
Langues
eng
Sous-ensembles de citation
IM
Pagination
700-706Subventions
Organisme : Det Sundhedsvidenskabelige Fakultet, Københavns Universitet
Informations de copyright
© 2019 Acta Ophthalmologica Scandinavica Foundation. Published by John Wiley & Sons Ltd.
Références
Appleton JP & Nicholl DJ (2016): Comment on: ‘Direct ophthalmoscopy should be taught to undergraduate medical students’. Eye (Lond) 30: 327.
Bacon AS, Dart JK, Ficker LA, Matheson MM & Wright P (1993): Acanthamoeba keratitis. The value of early diagnosis. Ophthalmology 100: 1238-1243.
Bek T, Hjortdal J & La Cour M (2016): [Eye diseases], 2nd edn. Copenhagen: FADLs Forlag.
Bloch R & Norman G (2012): Generalizability theory for the perplexed: a practical introduction and guide: AMEE Guide No. 68. Med Teach 34: 960-992.
Borgersen NJ, Naur TMH, Sørensen SMD, Bjerrum F, Konge L, Subhi Y & Thomsen ASS (2018): Gathering validity evidence for surgical simulation: a systematic review. Ann Surg [ePub ahead of print 4/Jan/2018]. https://doi.org/10.1097/sla.0000000000002652
Bruce BB, Lamirel C, Wright DW, Ward A, Heilpern KL, Biousse V & Newman NJ (2011): Nonmydriatic ocular fundus photography in the emergency department. N Engl J Med 364: 387-389.
Case SM & Swanson DB (2001): Constructing written test questions for the basic and clinical sciences, 3rd edn. Philadelphia: National Board of Medical Examiners.
Chawla B, Hasan F, Azad R, Seth R, Upadhyay AD, Pathy S & Pandey PM (2016): Clinical presentation and survival of retinoblastoma in Indian children. Br J Ophthalmol 100: 172-178.
De Montbrun S, Satterthwaite L & Grantcharov TP (2015): Setting pass scores for assessment of technical performance by surgical trainees. Br J Surg 103: 300-306.
Downing SM (2004): Reliability: On the reproducibility of assessment data. Med Educ 38: 1006-1012.
Downing SM & Yudkowsky R (eds.) (2009): Assessment in health professions education, 1st edn. New York, NY: Routledge.
Eijk ES, Busschbach JJ, Timman R, Monteban HC, Vissers JM & van Meurs JC (2016): What made you wait so long? Delays in presentation of retinal detachment: knowledge is related to an attached macula Acta Ophthalmol 94: 434-440.
Fahmy P, Hamann S, Larsen M & Sjølie AK (2013): [Practical ophthalmology], 3rd edn. Copenhagen: Gads Forlag.
Georgouli T, Pountos I, Chang BY & Giannoudis PV (2011): Prevalence of ocular and orbital injuries in polytrauma patients. Eur J Trauma Emerg Surg 37: 135-140.
Goldenberg MG, Garbens A, Szasz P, Hauer T & Grantcharov TP (2017): Systematic review to establish absolute standards for technical performance in surgery. Br J Surg 104: 13-21.
Graham B, Regehr G & Wright JG (2003): Delphi as a method to establish consensus for diagnostic criteria. J Clin Epidemiol 56: 1150-1156.
Gupta RR & Lam WC (2006): Medical students’ self-confidence in performing direct ophthalmoscopy in clinical training. Can J Ophthalmol 41: 169-174.
Gurwin J, Revere KE, Niepold S, Bassett B, Mitchell R, Davidson S, DeLisser H & Binenbaum G (2018): A randomized controlled study of art observation training to improve medical student ophthalmology skills. Ophthalmology 125: 8-14.
Haladyna TM & Rodriguez MC (eds.) (2013): Developing and validating test items, 1st edn. New York, NY: Routledge.
Harper RA (2016): Basic Ophthalmology, essentials for medical students, 10th edn. San Francisco, CA: American Academy of Ophthalmology.
Hill SC, Jawais I & Amoaku W (2016): Response to: ‘Direct ophthalmoscopy should be taught to undergraduate medical students’. Eye (Lond) 30: 327-328.
Imonikhe RJ, Finer N, Gallagher K, Plant G, Bremner FD & Acheson JF (2016): Direct ophthalmoscopy should be taught to undergraduate medical students. Eye (Lond) 20: 497.
International Task Force on Ophthalmic Education of Medical Students (2006): Principles and guidelines of a curriculum for ophthalmic education of medical students. Klin Monbl Augenheilkd 223(Suppl 5): S1-S19.
Jensen JT, Savran MM, Møller AM, Vilmann P, Hornslet P & Konge L (2016): Development and validation of a theoretical test in non-anaesthesiologist-administered propofol sedation for gastrointestinal endoscopy. Scand J Gastroenterol 51: 872-879.
Jørgensen M, Konge L & Subhi Y (2018): Contrasting groups’ standard setting for consequences analysis in validity studies: reporting considerations. Adv Simul (Lond) 3: 5.
Kohls-Gatzoulis JA, Regehr G & Hutchison C (2004): Teaching cognitive skills improves learning in surgical skills courses: a blinded, prospective, randomized study. Can J Surg 47: 277-283.
Korndorffer JR, Kasten SJ & Downing SM (2010): A call for the utilization of consensus standards in the surgical education literature. Am J Surg 199: 99-104.
Kromann CB, Jensen ML & Ringsted C (2009): The effect of testing on skills learning. Med Educ 43: 21-27.
Leske MC, Heijl A, Hussein M, Bengtsson B, Hyman L & Komaroff E (2003): Factors for glaucoma progression and the effect of treatment: the early manifest glaucoma trial. Arch Ophthalmol 121: 48-56.
Mackay DD, Garza PS, Bruce BB, Newman NJ & Biousse V (2015): The demise of direct ophthalmoscopy: a modern clinical challenge. Neurol Clin Pract 5: 150-157.
Morad Y, Barkana Y, Avni I & Kozer E (2004): Fundus anomalies: what the pediatrician's eye can't see. Int J Qual Health Care 16: 363-365.
Nicholl DJ, Yap CP, Cahill V, Appleton J, Willetts E & Sturman S (2012): The TOS study: can we use our patients to help improve clinical assessment? J R Coll Physicians Edinb 42: 306-310.
Patil SG, Kotwal IA, Joshi U, Allurkar S, Thakur N & Aftab A (2016): Ophthalmological evaluation by a maxillofacial surgeon and an ophthalmologist in assessing the damage to the orbital contents in midfacial fractures: a prospective study. J Maxillofac Oral Surg 15: 328-335.
Purbrick RM & Chong NV (2015): Direct ophthalmoscopy should be taught to undergraduate medical students - no. Eye (Lond) 29: 990-991.
Rasmussen A, Brandi S, Fuchs J, Hansen LH, Lund-Andersen H, Sander B & Larsen M (2015): Visual outcomes in relation to time to treatment in neovascular age-related macular degeneration. Acta Ophthalmol 93: 616-620.
Ringsted CV, Pallisgaard J & Falck G (2010): [Physicians’ clinical skills after finishing internship]. Ugeskr Laeger 164: 3211-3215.
Savran MM, Clementsen PF, Annema JT, Minddal V, Larsen KR, Park YS & Konge L (2014): Development and validation of a theoretical test in endosonography for pulmonary diseases. Respiration 88: 67-73.
Savran MM, Hansen HJ, Petersen RH, Walker W, Schmid T, Bojsen SR & Konge L (2015): Development and validation of a theoretical test of proficiency for video-assisted thorascopic surgery (VATS) lobectomy. Surg Endosc 29: 2598-2604.
Schuwirth LW & van der Vleuten CP (2003): ABC of learning and teaching in medicine: written assessment. BMJ 326: 643-645.
Streiner DL & Norman GR (eds) (2008): Health measurement scales: a practical guide to their development and use, 4th edn. Oxford: Oxford University Press.
Thomsen AS, Subhi Y, Kiilgaard JF, la Cour M & Konge L (2015): Update on simulation-based surgical training and assessment in ophthalmology: a systematic review. Ophthalmology 122: 1111-1130.
Ting DS, Sim SS, Yau CW, Rosman M, Aw AT & Yeo IY (2016): Ophthalmology simulation for undergraduate and postgraduate clinical education. Int J Ophthalmol 9: 920-924.
Yusuf IH, Salmon JF & Patel CK (2015): Direct ophthalmoscopy should be taught to undergraduate medical students - yes. Eye (Lond) 29: 987-989.