Effects of the informed health choices secondary school intervention after 1 year: a prospective meta-analysis using individual participant data.
Humans
Adolescent
Prospective Studies
Choice Behavior
Randomized Controlled Trials as Topic
Time Factors
Uganda
Kenya
Adolescent Behavior
Health Knowledge, Attitudes, Practice
Female
Schools
Male
Health Education
/ methods
School Health Services
Thinking
Students
/ psychology
Rwanda
Child
Retention, Psychology
Adolescents
Critical health literacy
Critical thinking
Health education
Individual participant-level data meta-analysis
Prospective meta-analysis
Secondary school
Journal
Trials
ISSN: 1745-6215
Titre abrégé: Trials
Pays: England
ID NLM: 101263253
Informations de publication
Date de publication:
30 Oct 2024
30 Oct 2024
Historique:
received:
05
07
2024
accepted:
22
10
2024
medline:
31
10
2024
pubmed:
31
10
2024
entrez:
31
10
2024
Statut:
epublish
Résumé
Critical thinking about health choices is essential to avoid being misled by unreliable information and to use reliable information appropriately. The aim of this prospective meta-analysis was to synthesize the results of 1-year follow-up data from three cluster-randomized trials of an intervention designed to teach lower secondary school students to think critically about health choices. Only one other randomized trial has evaluated a school-based intervention to teach adolescents to think critically about health choices. That trial compared two teaching strategies to teach statistical reasoning. It did not assess long-term learning-retention. We conducted the trials in Kenya, Rwanda, and Uganda. The intervention included providing a 2-3-day teacher training workshop and digital resources for ten lessons. The intervention focused on nine key concepts. We did not intervene in control schools. The primary outcome was a passing score on a test (≥ 9 of 18 multiple-choice questions answered correctly). We performed random effects meta-analyses to estimate the overall intervention effects. We calculated learning retention as the test results in the intervention schools after 1 year relative to just after the intervention, adjusted for chance. Altogether, 244 schools (11,344 students) took part in the three trials. Follow-up data was collected for 8298 students (73%). The overall odds ratio for the primary outcome after 1 year was 3.6 (95% CI: 1.9-7.1; p = 0.0001) in favor of the intervention, whereas it was 5.5 (95% CI: 3.0-10.2) just after the intervention. This corresponds to 25.6% (95% CI: 21.1-30.0%) more students in the intervention schools passing the test after 1 year versus 33.3% (95% CI: 28.7-37.8%) just after the intervention. Overall, 2273 (52.6%) of 4324 students in intervention schools had a passing score after 1 year compared to 3397 (58.1%) of 5846 students just after the intervention, indicating 88.3% learning retention. One year after the intervention, we still found a positive effect on the ability of students to think critically about health choices, but 5.5% fewer students in the intervention schools had a passing score. The certainty of the evidence was also lower due to 27% of students being lost to follow-up. The protocol for this prospective meta-analysis was registered with PROSPERO May 31, 2022, ID 336580. The three randomized trials were registered in the Pan African Clinical Trial Registry February 15, 2022, PACTR202203880375077; April 5, 2022, PACTR20220488391731; and April 14, 2022, PACTR202204861458660.
Sections du résumé
BACKGROUND
BACKGROUND
Critical thinking about health choices is essential to avoid being misled by unreliable information and to use reliable information appropriately. The aim of this prospective meta-analysis was to synthesize the results of 1-year follow-up data from three cluster-randomized trials of an intervention designed to teach lower secondary school students to think critically about health choices. Only one other randomized trial has evaluated a school-based intervention to teach adolescents to think critically about health choices. That trial compared two teaching strategies to teach statistical reasoning. It did not assess long-term learning-retention.
METHODS
METHODS
We conducted the trials in Kenya, Rwanda, and Uganda. The intervention included providing a 2-3-day teacher training workshop and digital resources for ten lessons. The intervention focused on nine key concepts. We did not intervene in control schools. The primary outcome was a passing score on a test (≥ 9 of 18 multiple-choice questions answered correctly). We performed random effects meta-analyses to estimate the overall intervention effects. We calculated learning retention as the test results in the intervention schools after 1 year relative to just after the intervention, adjusted for chance.
RESULTS
RESULTS
Altogether, 244 schools (11,344 students) took part in the three trials. Follow-up data was collected for 8298 students (73%). The overall odds ratio for the primary outcome after 1 year was 3.6 (95% CI: 1.9-7.1; p = 0.0001) in favor of the intervention, whereas it was 5.5 (95% CI: 3.0-10.2) just after the intervention. This corresponds to 25.6% (95% CI: 21.1-30.0%) more students in the intervention schools passing the test after 1 year versus 33.3% (95% CI: 28.7-37.8%) just after the intervention. Overall, 2273 (52.6%) of 4324 students in intervention schools had a passing score after 1 year compared to 3397 (58.1%) of 5846 students just after the intervention, indicating 88.3% learning retention.
CONCLUSIONS
CONCLUSIONS
One year after the intervention, we still found a positive effect on the ability of students to think critically about health choices, but 5.5% fewer students in the intervention schools had a passing score. The certainty of the evidence was also lower due to 27% of students being lost to follow-up.
TRIAL REGISTRATION
BACKGROUND
The protocol for this prospective meta-analysis was registered with PROSPERO May 31, 2022, ID 336580. The three randomized trials were registered in the Pan African Clinical Trial Registry February 15, 2022, PACTR202203880375077; April 5, 2022, PACTR20220488391731; and April 14, 2022, PACTR202204861458660.
Identifiants
pubmed: 39478569
doi: 10.1186/s13063-024-08577-w
pii: 10.1186/s13063-024-08577-w
doi:
Types de publication
Journal Article
Meta-Analysis
Langues
eng
Sous-ensembles de citation
IM
Pagination
733Subventions
Organisme : Norges Forskningsråd
ID : 69006
Informations de copyright
© 2024. The Author(s).
Références
Oxman AD, Chalmers I, Dahlgren A. Key concepts for informed health choices: where’s the evidence? F1000Res. 2023;11:890.
pubmed: 37928808
pmcid: 10623542
doi: 10.12688/f1000research.123051.2
Cusack L, Del Mar CB, Chalmers I, Gibson E, Hoffmann TC. Educational interventions to improve people’s understanding of key concepts in assessing the effects of health interventions: a systematic review. Syst Rev. 2018;7(1):68.
pubmed: 29716639
pmcid: 5930693
doi: 10.1186/s13643-018-0719-4
Nordheim LV, Gundersen MW, Espehaug B, Guttersrud Ø, Flottorp S. Effects of school-based educational interventions for enhancing adolescents abilities in critical appraisal of health claims: a systematic review. PLoS One. 2016;11(8):e0161485.
pubmed: 27557129
pmcid: 4996462
doi: 10.1371/journal.pone.0161485
Rosenbaum SE, Moberg J, Chesire F, Mugisha M, Ssenyonga R, Ochieng M, et al. Teaching critical thinking about health information and choices in secondary schools: human-centred design of digital resources. F1000Res. 2024;12:481.
pubmed: 39246586
pmcid: 11377934
doi: 10.12688/f1000research.132580.2
Rosenbaum S, Moberg J, Oxman M, Oxman AD, Chesire F, Mugisha M, et al. Be Smart about your Health 2022. Available from: https://besmarthealth.org/ .
Agaba JJ, Chesire F, Mugisha M, Nandi P, Njue J, Nsangi A, et al. Prioritisation of Informed Health Choices (IHC) Key Concepts to be included in lower-secondary school resources: a consensus study. PLoS One. 2023;18(4):e0267422.
pubmed: 37027357
pmcid: 10081733
doi: 10.1371/journal.pone.0267422
Chesire F, Kaseje M, Ochieng M, Ngatia B, Mugisha M, Ssenyonga R, et al. Effects of the informed health choices secondary school intervention on the ability of students in Kenya to think critically about health choices: a cluster-randomized trial. J Evid Based Med. 2023;16(3):275–84.
pubmed: 37735827
doi: 10.1111/jebm.12556
Mugisha M, Nyirazinyoye L, Simbi CMC, Chesire F, Senyonga R, Oxman M, et al. Effects of the Informed Health Choices secondary school intervention on the ability of students in Rwanda to think critically about health choices: a cluster-randomized trial. J Evid Based Med. 2023;16:264–74.
pubmed: 37735809
doi: 10.1111/jebm.12551
Ssenyonga R, Oxman AD, Nakyejwe E, Chesire F, Mugisha M, Nsangi A, et al. Use of the informed health choices educational intervention to improve secondary students’ ability to think critically about health interventions in Uganda: a cluster-randomized trial. J Evid Based Med. 2023;16(3):285–93.
pubmed: 37725488
doi: 10.1111/jebm.12553
Dahlgren A, Semakula D, Chesire F, Oxman AD, Mugisha M, Nakyejwe E, et al. Critical thinking about treatment effects in Eastern Africa: development and Rasch analysis of an assessment tool. F1000Res. 2023;12:887.
doi: 10.12688/f1000research.132052.1
Nsangi A, Aranza D, Asiimwe R, Munaabi-Babigumira S, Nantongo J, Nordheim L, et al. Measuring lower secondary school students’ ability to assess claims about treatment effects: establishment of a standard for passing and mastery. BMJ Open. 2023;13:e066890.
pubmed: 36828652
pmcid: 9972413
doi: 10.1136/bmjopen-2022-066890
Chesire F, Mugisha M, Ssenyonga R, Rose CJ, Nsangi A, Kaseje M, et al. Effects of the Informed Health Choices secondary school intervention: a prospective meta-analysis. J Evid Based Med. 2023;16(3):321–31.
pubmed: 37735807
doi: 10.1111/jebm.12552
Seidler AL, Hunter KE, Cheyne S, Ghersi D, Berlin JA, Askie L. A guide to prospective meta-analysis. BMJ. 2019;367:l5342.
pubmed: 31597627
doi: 10.1136/bmj.l5342
Chesire F, Kaseje M, Ochieng M, Mugisha M, Ssenyonga R, Oxman M, et al. Effects of the Informed Health Choices secondary school intervention on the ability of lower secondary students in Kenya to think critically about health information and choices: Protocol for a cluster-randomized trial. IHC Working Paper. 2022. Available from: https://doi.org/10.5281/zenodo.6562940 .
Mugisha M, Nyirazinyoye L, Simbi CMC, Chesire F, Ssenyonga R, Oxman M, et al. Effects of Informed Health Choices secondary school resources on the ability of Rwandan students to think critically about health: protocol for a cluster-randomised trial. IHC Working Paper. 2022. Available from: https://doi.org/10.5281/zenodo.6562788 .
Ssenyonga R, Oxman AD, Nakyejwe E, Mugagga SK, Nsangi A, Semakula D, et al. Does the use of the Informed Health Choices secondary school resources improve critical thinking about the effects of health among secondary school students in Uganda? A cluster-randomised trial protocol. IHC Working Paper. 2022. Available from: https://doi.org/10.5281/zenodo.6560218 .
Oxman A, Mugisha M, Chesire F, Ssenyonga R, Rose CJ, Nsangi A, et al. Effects of using the Informed Health Choices secondary school resources: protocol for a prospective meta-analysis addendum. Zenodo; 2022. https://zenodo.org/doi/10.5281/zenodo.6597492 .
Mugisha M, Uwitonze AM, Chesire F, Senyonga R, Oxman M, Nsangi A, et al. Teaching critical thinking about health using digital technology in lower secondary schools in Rwanda: a qualitative context analysis. PLoS One. 2021;16(3):e0248773.
pubmed: 33750971
pmcid: 7984628
doi: 10.1371/journal.pone.0248773
Chesire F, Ochieng M, Mugisha M, Ssenyonga R, Oxman M, Nsangi A, et al. Contextualizing critical thinking about health using digital technology in secondary schools in Kenya: a qualitative analysis. Pilot Feasibility Stud. 2022;8(1):227.
pubmed: 36203201
pmcid: 9535840
doi: 10.1186/s40814-022-01183-0
Ssenyonga R, Sewankambo NK, Mugagga SK, Nakyejwe E, Chesire F, Mugisha M, et al. Learning to think critically about health using digital technology in Ugandan lower secondary schools: a contextual analysis. PLoS One. 2022;17(2):e0260367.
pubmed: 35108268
pmcid: 8809610
doi: 10.1371/journal.pone.0260367
Phillips AC, Lewis LK, McEvoy MP, Galipeau J, Glasziou P, Moher D, et al. Development and validation of the guideline for reporting evidence-based practice educational interventions and teaching (GREET). BMC Med Educ. 2016;16:237.
pubmed: 27599967
pmcid: 5011880
doi: 10.1186/s12909-016-0759-1
Oxman AD, Mugisha M, Chesire F, Ssenyonga R, Rose CJ, Nsangi A, et al. Effects of using the Informed Health Choices secondary school resources: protocol for a prospective meta-analysis. IHC Working Paper. 2022. https://doi.org/10.5281/zenodo.6597493 .
Lee DS. Training, wages, and sample selection: estimating sharp bounds on treatment effects. Rev Econ Stud. 2009;76(3):1071–102.
doi: 10.1111/j.1467-937X.2009.00536.x
Eldridge S, Campbell M, Campbell M, Drahota A, Giraudeau B, Reeves B, et al. Revised Cochrane risk of bias tool for randomized trials (RoB 2): additional considerations for cluster-randomized trials (RoB 2 CRT). Risk of bias tools. 2021. Available from: https://www.riskofbias.info/welcome/rob-2-0-tool/rob-2-for-cluster-randomized-trials .
Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011;64(4):383–94.
pubmed: 21195583
doi: 10.1016/j.jclinepi.2010.04.026
Zeng L, Brignardello-Petersen R, Hultcrantz M, Siemieniuk RAC, Santesso N, Traversy G, et al. GRADE guidelines 32: GRADE offers guidance on choosing targets of GRADE certainty of evidence ratings. J Clin Epidemiol. 2021;137:163–75.
pubmed: 33857619
doi: 10.1016/j.jclinepi.2021.03.026
Schandelmaier S, Briel M, Varadhan R, Schmid CH, Devasenapathy N, Hayward RA, et al. Development of the Instrument to assess the Credibility of Effect Modification Analyses (ICEMAN) in randomized controlled trials and meta-analyses. CMAJ. 2020;192(32):E901–6.
pubmed: 32778601
pmcid: 7829020
doi: 10.1503/cmaj.200077
Verdugo-Paiva F, Novillo F, Peña J, Ávila-Oliver C, Rada G. Screening (partial report). Update of: Educational interventions to improve people’s understanding of key concepts in assessing the effects of health interventions. Epistemonikos Foundation. 2023. Available from: https://doi.org/10.5281/zenodo.7542970 .
Jacque B, Koch-Weser S, Faux R, Meiri K. Addressing health literacy challenges with a cutting-edge infectious disease curriculum for the high school biology classroom. Health Educ Behav. 2016;43(1):43–53.
pubmed: 26194205
doi: 10.1177/1090198115596163
Chesire F, Kaseje M, Gisore V, Mugisha M, Ssenyonga R, Munthe-Kaas H, et al. The Informed Health Choices secondary school intervention to improve critical thinking about health choices in Kenya: a mixed methods process evaluation. Glob Health Sci Pract. 2024 (in press).
Mugisha M, Oxman AD, Nyirazinyoye L, Uwitonze A-M, Simbi CMC, Chesire F, et al. Process evaluation of teaching critical thinking about health using the Informed Health Choices intervention in Rwanda: a mixed methods study. Glob Health Sci Pract. 2024 (in press).
Ssenyonga R, Lewin S, Nakyejwe E, Chesire F, Mugisha M, Oxman M, et al. The Informed Health Choices secondary school intervention to teach Ugandan students to critically appraise health information and make informed choices: a process evaluation. Glob Health Sci Pract. 2024 (in press).
Zapfe L, Gross C. How do characteristics of educational systems shape educational inequalities? Results from a systematic review. Int J Educ Res. 2021;109:101837.
doi: 10.1016/j.ijer.2021.101837
Ombati V, Ombati M. Gender inequality in education in sub-Saharan Africa. JWEE. 2012;3–4:114–36.
Zickafoose A, Ilesanmi O, Diaz-Manrique M, Adeyemi AE, Walumbe B, Strong R, et al. Barriers and challenges affecting quality education (Sustainable Development Goal #4) in sub-Saharan Africa by 2030. Sustainability. 2024;16(7):2657.
doi: 10.3390/su16072657
Nowak JK. Gender inequality in education. Human. In: Daniela L, editor. Human, Technologies and Quality of Education. Rīga: University of Latvia; 2021. p. 424–33.
Nsangi A, Semakula D, Oxman AD, Austvoll-Dahlgren A, Oxman M, Rosenbaum S, et al. Effects of the Informed Health Choices primary school intervention on the ability of children in Uganda to assess the reliability of claims about treatment effects, 1-year follow-up: a cluster-randomised trial. Trials. 2020;21(1):27.
pubmed: 31907013
pmcid: 6945419
doi: 10.1186/s13063-019-3960-9
Nsangi A, Semakula D, Rosenbaum SE, Oxman AD, Oxman M, Morelli A, et al. Development of the informed health choices resources in four countries to teach primary school children to assess claims about treatment effects: a qualitative study employing a user-centred approach. Pilot Feasibility Stud. 2020;6:18.
pubmed: 32055405
pmcid: 7008535
doi: 10.1186/s40814-020-00565-6
Tutal Ö, Yazar T. Active learning improves academic achievement and learning retention in K-12 settings: a meta-analysis. J School Educ Tech. 2023;18(3). https://doi.org/10.26634/jsch.18.3.19288 .
Adesope OO, Trevisan DA, Sundararajan N. Rethinking the use of tests: a meta-analysis of practice testing. Rev Educ Res. 2017;87(3):659–701.
doi: 10.3102/0034654316689306
Custers E. Long-term retention of basic science knowledge: a review study. Adv Health Sci Educ Theory Pract. 2010;15(1):109–28.
pubmed: 18274876
doi: 10.1007/s10459-008-9101-y
Oxman AD, Nsangi A, Martinez Garcia L, Kaseje M, Samsó Jofra L, Semakula D, et al. The effects of teaching strategies on learning to think critically in primary and secondary schools: an overview of systematic reviews. F1000Res. 2024; in press.
Schneider S, Beege M, Nebel S, Rey GD. A meta-analysis of how signaling affects learning with media. Educ Res Rev. 2018;23:1–24.
doi: 10.1016/j.edurev.2017.11.001
Randolph JJ. Meta-analysis of the research on response cards: effects on test achievement, quiz achievement, participation, and off-task behavior. J Posit Behav Interv. 2007;9(2):113–28.
doi: 10.1177/10983007070090020201
Barnett SM, Ceci SJ. When and where do we apply what we learn?: A taxonomy for far transfer. Psychol Bull. 2002;128(4):612.
pubmed: 12081085
doi: 10.1037/0033-2909.128.4.612
Aronson JK, Barends E, Boruch R, Brennan M, Chalmers I, Chislett J, et al. Key concepts for making informed choices. Nature. 2019;572(7769):303–6.
pubmed: 31406318
doi: 10.1038/d41586-019-02407-9
Halpern DF. Teaching critical thinking for transfer across domains. Dispositions, skills, structure training, and metacognitive monitoring. Am Psychol. 1998;53(4):449–55.
pubmed: 9572008
doi: 10.1037/0003-066X.53.4.449
National Research Council. Education for life and work: developing transferable knowledge and skills in the 21st century. Pellegrino JW, Hilton ML, editors. Washington, DC: The National Academies Press; 2012. p. 256.
Cutler DM, Lleras-Muney A. Understanding differences in health behaviors by education. J Health Econ. 2010;29(1):1–28.
pubmed: 19963292
doi: 10.1016/j.jhealeco.2009.10.003
Zhao Y. What works may hurt: side effects in education. J Educ Change. 2017;18(1):1–19.
doi: 10.1007/s10833-016-9294-4
Oxman M, Chesire F, Mugisha M, Ssenyonga R, Ngatia B, Nsangi A, et al. Potential adverse effects of an educational intervention: development of a framework. F1000Res. 2024; in press.
Oxman M, Oxman AD, Fretheim A, Lewin S. Participants’ and investigators’ experiences and views of potential adverse effects of an educational intervention: protocol for a qualitative evidence synthesis. IHC Working Paper. 2023. Available from: https://doi.org/10.5281/zenodo.7681365 .
Roozenbeek J, Schneider CR, Dryhurst S, Kerr J, Freeman ALJ, Recchia G, et al. Susceptibility to misinformation about COVID-19 around the world. R Soc Open Sci. 2020;7(10):201199.
pubmed: 33204475
pmcid: 7657933
doi: 10.1098/rsos.201199
Swire-Thompson B, Lazer D. Public health and online misinformation: challenges and recommendations. Annu Rev Public Health. 2020;41:433–51.
pubmed: 31874069
doi: 10.1146/annurev-publhealth-040119-094127
Pierce H, Gibby AL, Forste R. Caregiver decision-making: household response to child illness in sub-Saharan Africa. Popul Res Policy Rev. 2016;35(5):581–97.
pubmed: 28794575
pmcid: 5546145
doi: 10.1007/s11113-016-9396-y
Oxman AD, Paulsen EJ. Who can you trust? A review of free online sources of “trustworthy” information about treatment effects for patients and the public. BMC Med Inform Decis Mak. 2019;19(1):35.
pubmed: 30786889
pmcid: 6381637
doi: 10.1186/s12911-019-0772-5
Suarez-Lledo V, Alvarez-Galvez J. Prevalence of health misinformation on social media: systematic review. J Med Internet Res. 2021;23(1):e17187.
pubmed: 33470931
pmcid: 7857950
doi: 10.2196/17187
do Borges Nascimento IJ, Pizarro AB, Almeida JM, Azzopardi-Muscat N, Gonçalves MA, Björklund M, et al. Infodemics and health misinformation: a systematic review of reviews. Bull World Health Organ. 2022;100(9):544–61.
doi: 10.2471/BLT.21.287654
Faubert B. A literature review of school practices to overcome school failure. OECD Education Working Papers. 2012;(68). Available from: https://doi.org/10.1787/5k9flcwwv9tk-en .
García E, Weiss E. Reducing and averting achievement gaps: key findings from the report ‘Education Inequalities at the School Starting Gate’ and comprehensive strategies to mitigate early skills gaps. Economic Policy Institute. 2017. Available from: http://files.eric.ed.gov/fulltext/ED587806.pdf .
Informed Health Choices. Translation and adaptation of educational resources 2024. Available from: https://www.informedhealthchoices.org/publications/#ta .
Bruner JS. The process of education. Boston: Harvard University Press; 2009.
doi: 10.2307/j.ctvk12qst