Reliability generalization meta-analysis of orthorexia nervosa using the ORTO-11/12/15/R scale in all populations and language versions.
Alpha
Internal consistency
Ortho-15
Orthorexia
Psychometric
Reliability
Validity
Journal
Journal of eating disorders
ISSN: 2050-2974
Titre abrégé: J Eat Disord
Pays: England
ID NLM: 101610672
Informations de publication
Date de publication:
19 Mar 2024
19 Mar 2024
Historique:
received:
20
10
2023
accepted:
13
03
2024
medline:
20
3
2024
pubmed:
20
3
2024
entrez:
20
3
2024
Statut:
epublish
Résumé
The ORTO scale was developed in 2004 as a self-report questionnaire to assess symptoms of orthorexia nervosa (ON). ON is an unhealthy preoccupation with eating healthy food. The scale aims to measure obsessive attitudes and behaviors related to the selection, purchase, preparation, and consumption of pure, healthy food. Since its development, the ORTO-15 has been adapted into several shorter versions. The objective was to conduct a reliability generalization meta-analysis of the ORTO scale and its variant versions in all populations and languages. A systematic literature search was conducted to identify studies reporting the internal consistency of ORTO. Random-effect models were used to evaluate summary statistics of reliability coefficients, weighting the coefficients by the inverse variance using the restricted maximum likelihood method. The heterogeneity among the reliability coefficients was evaluated and assessed using numerous statistical metrics. The tau (τ), tau Twenty-one studies (k = 21) involving 11,167 participants (n = 11,167) were analyzed. The overall effect estimate on internal consistency was 0.59 (95% CI 0.49-0.68), with a minimum reliability coefficient of 0.23 and a maximum reliability coefficient of 0.83. The heterogeneity statistics were found to have an I The ORTO scale has a relatively low pooled reliability coefficient. Alternative ON assessment tools with enhanced psychometric properties are needed. Clinicians should not base diagnoses or treatment decisions on ORTO alone. Comprehensive psychiatric assessment is essential for accurate ON evaluation. This review looked at the reliability of the ORTO scale and its shortened versions for assessing orthorexia nervosa (an unhealthy obsession with eating healthy foods). The researchers analyzed data from 21 previous studies involving over 11,000 participants. Results showed that the ORTO scale had relatively low reliability in consistently measuring orthorexia symptoms across studies. The summary reliability score was 0.59 on a 0 to 1 scale, with individual study scores ranging from 0.23 to 0.83. There was a significant inconsistency across the different study results. We concluded that the ORTO scale has low reliability overall for diagnosing orthorexia nervosa. New assessment tools with better measurement properties are needed. Clinicians should not rely solely on the ORTO scale, but should conduct a comprehensive psychological evaluation to properly assess for orthorexia.
Sections du résumé
BACKGROUND
BACKGROUND
The ORTO scale was developed in 2004 as a self-report questionnaire to assess symptoms of orthorexia nervosa (ON). ON is an unhealthy preoccupation with eating healthy food. The scale aims to measure obsessive attitudes and behaviors related to the selection, purchase, preparation, and consumption of pure, healthy food. Since its development, the ORTO-15 has been adapted into several shorter versions. The objective was to conduct a reliability generalization meta-analysis of the ORTO scale and its variant versions in all populations and languages.
METHODS
METHODS
A systematic literature search was conducted to identify studies reporting the internal consistency of ORTO. Random-effect models were used to evaluate summary statistics of reliability coefficients, weighting the coefficients by the inverse variance using the restricted maximum likelihood method. The heterogeneity among the reliability coefficients was evaluated and assessed using numerous statistical metrics. The tau (τ), tau
RESULTS
RESULTS
Twenty-one studies (k = 21) involving 11,167 participants (n = 11,167) were analyzed. The overall effect estimate on internal consistency was 0.59 (95% CI 0.49-0.68), with a minimum reliability coefficient of 0.23 and a maximum reliability coefficient of 0.83. The heterogeneity statistics were found to have an I
CONCLUSIONS
CONCLUSIONS
The ORTO scale has a relatively low pooled reliability coefficient. Alternative ON assessment tools with enhanced psychometric properties are needed. Clinicians should not base diagnoses or treatment decisions on ORTO alone. Comprehensive psychiatric assessment is essential for accurate ON evaluation.
This review looked at the reliability of the ORTO scale and its shortened versions for assessing orthorexia nervosa (an unhealthy obsession with eating healthy foods). The researchers analyzed data from 21 previous studies involving over 11,000 participants. Results showed that the ORTO scale had relatively low reliability in consistently measuring orthorexia symptoms across studies. The summary reliability score was 0.59 on a 0 to 1 scale, with individual study scores ranging from 0.23 to 0.83. There was a significant inconsistency across the different study results. We concluded that the ORTO scale has low reliability overall for diagnosing orthorexia nervosa. New assessment tools with better measurement properties are needed. Clinicians should not rely solely on the ORTO scale, but should conduct a comprehensive psychological evaluation to properly assess for orthorexia.
Autres résumés
Type: plain-language-summary
(eng)
This review looked at the reliability of the ORTO scale and its shortened versions for assessing orthorexia nervosa (an unhealthy obsession with eating healthy foods). The researchers analyzed data from 21 previous studies involving over 11,000 participants. Results showed that the ORTO scale had relatively low reliability in consistently measuring orthorexia symptoms across studies. The summary reliability score was 0.59 on a 0 to 1 scale, with individual study scores ranging from 0.23 to 0.83. There was a significant inconsistency across the different study results. We concluded that the ORTO scale has low reliability overall for diagnosing orthorexia nervosa. New assessment tools with better measurement properties are needed. Clinicians should not rely solely on the ORTO scale, but should conduct a comprehensive psychological evaluation to properly assess for orthorexia.
Identifiants
pubmed: 38504366
doi: 10.1186/s40337-024-00997-y
pii: 10.1186/s40337-024-00997-y
doi:
Types de publication
Journal Article
Langues
eng
Pagination
39Informations de copyright
© 2024. The Author(s).
Références
Arusoğlu G, Kabakçi E, Köksal G, Merdol TK. Orthorexia nervosa and adaptation of ORTO-11 into Turkish. Turk Psikiyatri Derg. 2008;19(3):283–91.
pubmed: 18791881
Bağci Bosi AT, Camur D, Güler C. Prevalence of orthorexia nervosa in resident medical doctors in the faculty of medicine (Ankara, Turkey). Appetite. 2007;49(3):661–6.
pubmed: 17586085
doi: 10.1016/j.appet.2007.04.007
Varga M, Dukay-Szabó S, Túry F, van Furth EF. Evidence and gaps in the literature on orthorexia nervosa. Eat Weight Disord. 2013;18(2):103–11.
pubmed: 23760837
doi: 10.1007/s40519-013-0026-y
Messer M, Liu C, McClure Z, Mond J, Tiffin C, Linardon J. Negative body image components as risk factors for orthorexia nervosa: Prospective findings. Appetite. 2022;178: 106280.
pubmed: 35988801
doi: 10.1016/j.appet.2022.106280
Donini LM, Marsili D, Graziani MP, Imbriale M, Cannella C. Orthorexia nervosa: a preliminary study with a proposal for diagnosis and an attempt to measure the dimension of the phenomenon. Eat Weight Disord. 2004;9(2):151–7.
pubmed: 15330084
doi: 10.1007/BF03325060
Fidan T, Ertekin V, Işikay S, Kirpinar I. Prevalence of orthorexia among medical students in Erzurum, Turkey. Compr Psychiatry. 2010;51(1):49–54.
pubmed: 19932826
doi: 10.1016/j.comppsych.2009.03.001
Varga M, Thege BK, Dukay-Szabó S, Túry F, van Furth EF. When eating healthy is not healthy: orthorexia nervosa and its measurement with the ORTO-15 in Hungary. BMC Psychiatry. 2014;14:59.
pubmed: 24581288
pmcid: 3943279
doi: 10.1186/1471-244X-14-59
Brytek-Matera A, Donini LM, Krupa M, Poggiogalle E, Hay P. Orthorexia nervosa and self-attitudinal aspects of body image in female and male university students. J Eat Disord. 2015;3:2.
pubmed: 25774296
pmcid: 4359442
doi: 10.1186/s40337-015-0038-2
Brytek-Matera A, Krupa M, Poggiogalle E, Donini LM. Adaptation of the ORTHO-15 test to Polish women and men. Eat Weight Disord. 2014;19(1):69–76.
pubmed: 24448996
doi: 10.1007/s40519-014-0100-0
Haddad C, Hallit R, Akel M, Honein K, Akiki M, Kheir N, Obeid S, Hallit S. Validation of the Arabic version of the ORTO-15 questionnaire in a sample of the Lebanese population. Eat Weight Disord. 2020;25(4):951–60.
pubmed: 31119588
doi: 10.1007/s40519-019-00710-y
Gonidakis F, Poulopoulou C, Michopoulos I, Varsou E. Validation of the Greek ORTO-15 questionnaire for the assessment of orthorexia nervosa and its relation to eating disorders symptomatology. Eat Weight Disord. 2021;26(8):2471–9.
pubmed: 33475990
doi: 10.1007/s40519-020-01080-6
Andreas S, Schedler K, Schulz H, Nutzinger DO. Evaluation of a German version of a brief diagnosis questionnaire of symptoms of orthorexia nervosa in patients with mental disorders (Ortho-10). Eat Weight Disord. 2018;23(1):75–85.
pubmed: 29294251
doi: 10.1007/s40519-017-0473-y
Parra-Fernandez ML, Rodríguez-Cano T, Onieva-Zafra MD, Perez-Haro MJ, Casero-Alonso V, Muñoz Camargo JC, Notario-Pacheco B. Adaptation and validation of the Spanish version of the ORTO-15 questionnaire for the diagnosis of orthorexia nervosa. PLoS ONE. 2018;13(1): e0190722.
pubmed: 29320545
pmcid: 5761893
doi: 10.1371/journal.pone.0190722
Niedzielski A, Kaźmierczak-Wojtaś N. Prevalence of Orthorexia nervosa and its diagnostic tools: a literature review. Int J Environ Res Public Health. 2021;18(10):5488.
pubmed: 34065506
pmcid: 8160773
doi: 10.3390/ijerph18105488
Opitz MC, Newman E, Alvarado Vázquez Mellado AS, Robertson MDA, Sharpe H. The psychometric properties of Orthorexia Nervosa assessment scales: a systematic review and reliability generalization. Appetite. 2020;155:104797.
pubmed: 32652100
doi: 10.1016/j.appet.2020.104797
Farchakh Y, Hallit S, Soufia M. Association between orthorexia nervosa, eating attitudes and anxiety among medical students in Lebanese universities: results of a cross-sectional study. Eat Weight Disord. 2019;24(4):683–91.
pubmed: 31183627
doi: 10.1007/s40519-019-00724-6
Plichta M, Jezewska-Zychowicz M. Orthorexic tendency and eating disorders symptoms in polish students: examining differences in eating behaviors. Nutrients. 2020;12(1):218.
pubmed: 31952161
pmcid: 7019551
doi: 10.3390/nu12010218
Tremelling K, Sandon L, Vega GL, McAdams CJ. Orthorexia nervosa and eating disorder symptoms in registered dietitian nutritionists in the United States. J Acad Nutr Diet. 2017;117(10):1612–7.
pubmed: 28624376
pmcid: 5623148
doi: 10.1016/j.jand.2017.05.001
Rogoza R, Donini LM. Introducing ORTO-R: a revision of ORTO-15: based on the re-assessment of original data. Eat Weight Disord. 2021;26(3):887–95.
pubmed: 32436165
doi: 10.1007/s40519-020-00924-5
Meule A, Holzapfel C, Brandl B, Greetfeld M, Hessler-Kaufmann JB, Skurk T, Quadflieg N, Schlegl S, Hauner H, Voderholzer U. Measuring orthorexia nervosa: a comparison of four self-report questionnaires. Appetite. 2020;146: 104512.
pubmed: 31707072
doi: 10.1016/j.appet.2019.104512
Roncero M, Barrada JR, Perpiñá C. Measuring orthorexia nervosa: psychometric limitations of the ORTO-15. Span J Psychol. 2017;20:E41.
pubmed: 28929989
doi: 10.1017/sjp.2017.36
Gkiouras K, Grammatikopoulou MG, Tsaliki T, Ntwali L, Nigdelis MP, Gerontidis A, Taousani E, Tzimos C, Rogoza R, Bogdanos DP, et al. Orthorexia nervosa: replication and validation of the ORTO questionnaires translated into Greek in a survey of 848 Greek individuals. Hormones (Athens). 2022;21(2):251–60.
pubmed: 35092606
doi: 10.1007/s42000-022-00351-4
Parra-Fernández ML, Onieva-Zafra MD, Fernández-Martínez E, Abreu-Sánchez A, Fernández-Muñoz JJ. Assessing the prevalence of orthorexia nervosa in a sample of university students using two different self-report measures. Int J Environ Res Public Health. 2019;16(14):2459.
pubmed: 31373283
pmcid: 6678205
doi: 10.3390/ijerph16142459
Rogoza R, Mhanna M, Gerges S, Donini LM, Obeid S, Hallit S. Validation of the Arabic version of the ORTO-R among a sample of Lebanese young adults. Eat Weight Disord. 2022;27(6):2073–80.
pubmed: 35092001
pmcid: 8799435
doi: 10.1007/s40519-021-01350-x
Lasserson TJ, Thomas J, Higgins JP. Starting a review. Cochr Handb Syst Rev Interv 2019:1–12.
Sánchez-Meca J, Marín-Martínez F, López-López JA, Núñez-Núñez RM, Rubio-Aparicio M, López-García JJ, López-Pina JA, Blázquez-Rincón DM, López-Ibáñez C, López-Nicolás R. Improving the reporting quality of reliability generalization meta-analyses: the REGEMA checklist. Res Synth Methods. 2021;12(4):516–36.
pubmed: 33742752
doi: 10.1002/jrsm.1487
Tavakol M, Dennick R. Making sense of Cronbach’s alpha. Int J Med Educ. 2011;2:53.
pubmed: 28029643
pmcid: 4205511
doi: 10.5116/ijme.4dfb.8dfd
Malkewitz CP, Schwall P, Meesters C, Hardt J. Estimating reliability: a comparison of Cronbach’s α, McDonald’s ωt and the greatest lower bound. Soc Sci Hum Open. 2023;7(1): 100368.
Alvarenga MS, Martins MC, Sato KS, Vargas SV, Philippi ST, Scagliusi FB. Orthorexia nervosa behavior in a sample of Brazilian dietitians assessed by the Portuguese version of ORTO-15. Eat Weight Disord. 2012;17(1):e29-35.
pubmed: 22751269
doi: 10.1007/BF03325325
Babeau C, Le Chevanton T, Julien-Sweerts S, Brochenin A, Donini LM, Fouques D. Structural validation of the ORTO-12-FR questionnaire among a French sample as a first attempt to assess orthorexia nervosa in France. Eat Weight Disord. 2020;25(6):1771–8.
pubmed: 31863296
doi: 10.1007/s40519-019-00835-0
Dell’Osso L, Abelli M, Carpita B, Massimetti G, Pini S, Rivetti L, Gorrasi F, Tognetti R, Ricca V, Carmassi C. Orthorexia nervosa in a sample of Italian university population. Riv Psichiatr. 2016;51(5):190–6.
pubmed: 27869905
Graham CC, Hare, K.E., Graham, C.C. and Hare, K.E.: Hadamard sets; 2013.
Hartung J, Knapp G. A refined method for the meta-analysis of controlled clinical trials with binary outcome. Stat Med. 2001;20(24):3875–89.
pubmed: 11782040
doi: 10.1002/sim.1009
Heiss S, Coffino JA, Hormes JM. What does the ORTO-15 measure? Assessing the construct validity of a common orthorexia nervosa questionnaire in a meat avoiding sample. Appetite. 2019;135:93–9.
pubmed: 30597192
doi: 10.1016/j.appet.2018.12.042
Hyrnik J, Janas-Kozik M, Stochel M, Jelonek I, Siwiec A, Rybakowski JK. The assessment of orthorexia nervosa among 1899 Polish adolescents using the ORTO-15 questionnaire. Int J Psychiatry Clin Pract. 2016;20(3):199–203.
pubmed: 27314473
doi: 10.1080/13651501.2016.1197271
Li WL, Tan SX, Ouyang RQ, Cui YF, Ma JR, Cheng C, Mu YJ, Zhang SW, Zheng L, Xiong P, et al. Translation and validation of the Chinese version of the orthorexia nervosa assessment questionnaires among college students. Eat Weight Disord. 2022;27(8):3389–98.
pubmed: 36071328
doi: 10.1007/s40519-022-01469-5
Missbach B, Hinterbuchinger B, Dreiseitl V, Zellhofer S, Kurz C, König J. When eating right, is measured wrong! A validation and critical examination of the ORTO-15 Questionnaire in German. PLoS ONE. 2015;10(8): e0135772.
pubmed: 26280449
pmcid: 4539204
doi: 10.1371/journal.pone.0135772
Mitrofanova E, Pummell E, Martinelli L, Petróczi A. Does ORTO-15 produce valid data for “orthorexia nervosa”? A mixed-method examination of participants’ interpretations of the fifteen test items. Eat Weight Disord. 2021;26(3):897–909.
pubmed: 32445114
doi: 10.1007/s40519-020-00919-2
Moller S, Apputhurai P, Knowles SR. Confirmatory factor analyses of the ORTO 15-, 11- and 9-item scales and recommendations for suggested cut-off scores. Eat Weight Disord. 2019;24(1):21–8.
pubmed: 29796780
doi: 10.1007/s40519-018-0515-0
Stochel M, Janas-Kozik M, Zejda J, Hyrnik J, Jelonek I, Siwiec A. Validation of ORTO-15 Questionnaire in the group of urban youth aged 15–21. Psychiatr Pol. 2015;49(1):119–34.
pubmed: 25844415
doi: 10.12740/PP/25962
Valente M, Syurina EV, Donini LM. Shedding light upon various tools to assess orthorexia nervosa: a critical literature review with a systematic search. Eat Weight Disord. 2019;24(4):671–82.
pubmed: 31228168
pmcid: 6647444
doi: 10.1007/s40519-019-00735-3
Vuillier L, Robertson S, Greville-Harris M. Orthorexic tendencies are linked with difficulties with emotion identification and regulation. J Eat Disord. 2020;8:15.
pubmed: 32337044
pmcid: 7178625
doi: 10.1186/s40337-020-00291-7
McHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb). 2012;22(3):276–82.
pubmed: 23092060
doi: 10.11613/BM.2012.031
McGuinness LA, Higgins JPT. Risk-of-bias VISualization (robvis): an R package and Shiny web app for visualizing risk-of-bias assessments. Res Synth Methods. 2021;12(1):55–61.
pubmed: 32336025
doi: 10.1002/jrsm.1411
Mokkink LB, Prinsen CA, Bouter LM, Vet HC, Terwee CB. The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) and how to select an outcome measurement instrument. Braz J Phys Ther. 2016;20(2):105–13.
pubmed: 26786084
pmcid: 4900032
doi: 10.1590/bjpt-rbf.2014.0143
Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med. 2002;21(11):1539–58.
pubmed: 12111919
doi: 10.1002/sim.1186
Biggerstaff BJ, Tweedie RL. Incorporating variability in estimates of heterogeneity in the random effects model in meta-analysis. Stat Med. 1997;16(7):753–68.
pubmed: 9131763
doi: 10.1002/(SICI)1097-0258(19970415)16:7<753::AID-SIM494>3.0.CO;2-G
Borenstein M, Higgins JP, Hedges LV, Rothstein HR. Basics of meta-analysis: I(2) is not an absolute measure of heterogeneity. Res Synth Methods. 2017;8(1):5–18.
pubmed: 28058794
doi: 10.1002/jrsm.1230
Migliavaca CB, Stein C, Colpani V, Barker TH, Ziegelmann PK, Munn Z, Falavigna M. Meta-analysis of prevalence: I(2) statistic and how to deal with heterogeneity. Res Synth Methods. 2022;13(3):363–7.
pubmed: 35088937
doi: 10.1002/jrsm.1547
Laliman V, Roïz J. Frequentist approach for detecting heterogeneity in meta-analysis pair-wise comparisons: enhanced Q-test use by using I
pubmed: 27201935
doi: 10.1016/j.jval.2014.08.1942
Barili F, Parolari A, Kappetein PA, Freemantle N. Statistical Primer: heterogeneity, random- or fixed-effects model analyses?†. Interact Cardiovasc Thorac Surg. 2018;27(3):317–21.
pubmed: 29868857
doi: 10.1093/icvts/ivy163
van Aert RCM, Wicherts JM, van Assen M. Publication bias examined in meta-analyses from psychology and medicine: a meta-meta-analysis. PLoS ONE. 2019;14(4): e0215052.
pubmed: 30978228
pmcid: 6461282
doi: 10.1371/journal.pone.0215052
Chen Z, Zhang G, Li J. Goodness-of-fit test for meta-analysis. Sci Rep. 2015;5:16983.
pubmed: 26592212
pmcid: 4655309
doi: 10.1038/srep16983
Mathur MB, VanderWeele TJ. Sensitivity analysis for publication bias in meta-analyses. J R Stat Soc Ser C Appl Stat. 2020;69(5):1091–119.
pubmed: 33132447
pmcid: 7590147
doi: 10.1111/rssc.12440
Schwarzer G, Schwarzer MG. Package ‘meta.’ R Found Stat Comput. 2012;9:27.
Viechtbauer W, Viechtbauer MW: Package ‘metafor’. The Comprehensive R Archive Network Package ‘metafor’ http://cran.r-project.org/web/packages/metafor/metafor.pdf 2015.
Ellis JL. A test can have multiple reliabilities. Psychometrika. 2021;86(4):869–76.
pubmed: 34498211
pmcid: 8636415
doi: 10.1007/s11336-021-09800-2
Yang FM, Kao ST. Item response theory for measurement validity. Shanghai Arch Psychiatry. 2014;26(3):171–7.
pubmed: 25114494
pmcid: 4118016
Mohamed Halim Z, Dickinson KM, Kemps E, Prichard I. Orthorexia nervosa: examining the Eating Habits Questionnaire’s reliability and validity, and its links to dietary adequacy among adult women. Public Health Nutr. 2020;23(10):1684–92.
pubmed: 32204747
pmcid: 10200377
doi: 10.1017/S1368980019004282
Chard CA, Hilzendegen C, Barthels F, Stroebele-Benschop N. Psychometric evaluation of the English version of the Düsseldorf Orthorexie Scale (DOS) and the prevalence of orthorexia nervosa among a US student sample. Eat Weight Disord. 2019;24(2):275–81.
pubmed: 30196526
doi: 10.1007/s40519-018-0570-6
Barthels F, Barrada JR, Roncero M. Orthorexia nervosa and healthy orthorexia as new eating styles. PLoS ONE. 2019;14(7): e0219609.
pubmed: 31291364
pmcid: 6619829
doi: 10.1371/journal.pone.0219609
Oberle CD, De Nadai AS, Madrid AL. Orthorexia nervosa inventory (ONI): development and validation of a new measure of orthorexic symptomatology. Eat Weight Disord. 2021;26(2):609–22.
pubmed: 32279201
doi: 10.1007/s40519-020-00896-6