Comparative evaluation of commercially available AI-based cephalometric tracing programs.
Accuracy
Artificial intelligence
Diagnosis
Lateral cephalometric
Orthodontics
Journal
BMC oral health
ISSN: 1472-6831
Titre abrégé: BMC Oral Health
Pays: England
ID NLM: 101088684
Informations de publication
Date de publication:
18 Oct 2024
18 Oct 2024
Historique:
received:
10
08
2024
accepted:
08
10
2024
medline:
19
10
2024
pubmed:
19
10
2024
entrez:
18
10
2024
Statut:
epublish
Résumé
Compare the accuracy and diagnostic concordance of three commercially available AI-based lateral cephalometric tracing software. Sixty-three lateral cephalometric radiographs were analyzed using semi-automatic (Dolphin Imaging Systems LLC) and AI-based software programs (WebCeph™, Cephio, and Ceppro DDH Inc.). Intra- and inter-observer reliability were assessed for human expert measurements, and repeated-measures one-way ANOVA was used to compare the AI and human expert measurements. The diagnostic performance was evaluated using sensitivity and specificity tests. Human expert reliability was excellent (ICC > 0.9) for most cephalometric parameters. Compared to human experts, significant differences were observed for all three AI-based cephalometric programs (WebCeph™ - 10 of 11, Cephio - 7 of 11, and Ceppro DDH Inc. - 7 of 11 cephalometric measurements). Variations exceeding two units were noted for most parameters, and differences in defining the sagittal and vertical skeletal patterns, dental, and soft tissue characteristics were observed. All three AI-based tracing programs showed inaccuracies compared to human expert measurements and lacked reliability in measuring key cephalometric parameters. Clinicians should exercise caution when relying solely on AI-based analyses for orthodontic treatment planning and assessment.
Identifiants
pubmed: 39425100
doi: 10.1186/s12903-024-05032-9
pii: 10.1186/s12903-024-05032-9
doi:
Types de publication
Journal Article
Comparative Study
Langues
eng
Sous-ensembles de citation
IM
Pagination
1241Informations de copyright
© 2024. The Author(s).
Références
Downs WB. Analysis of the Dentofacial Profile. Angle Orthod. 1956;26(4):191–212.
Tarazona B, Lucas-Dominguez R, Paredes-Gallardo V, Alonso-Arroyo A, Vidal-Infer A. The 100 most-cited articles in orthodontics: a bibliometric study. Angle Orthod. 2018;88(6):785–96.
pubmed: 30102086
pmcid: 8174067
doi: 10.2319/012418-65.1
Houston WJ. The analysis of errors in orthodontic measurements. Am J Orthod. 1983;83(5):382–90.
pubmed: 6573846
doi: 10.1016/0002-9416(83)90322-6
Prasad S, Denotti G, Farella M. Effect of prior knowledge about treatment on cephalometric measurements. J Orthod. 2022;49(4):388–93.
pubmed: 35465757
doi: 10.1177/14653125221094333
Popova T, Stocker T, Khazaei Y, Malenova Y, Wichelhaus A, Sabbagh H. Influence of growth structures and fixed appliances on automated cephalometric landmark recognition with a customized convolutional neural network. BMC Oral Health. 2023;23(1):274.
pubmed: 37165409
pmcid: 10173502
doi: 10.1186/s12903-023-02984-2
Tanikawa C, Oka A, Lim J, Lee C, Yamashiro T. Clinical applicability of automated cephalometric landmark identification: part II – number of images needed to re-learn various quality of images. Orthod Craniofac Res. 2021;24(S2):53–8.
pubmed: 34145974
doi: 10.1111/ocr.12511
Menezes LDS, Silva TP, Lima Dos Santos MA, Hughes MM, Mariano Souza SDR, Leite Ribeiro PM, Freitas PHL, Takeshita WM. Assessment of landmark detection in cephalometric radiographs with different conditions of brightness and contrast using the an artificial intelligence software. Dentomaxillofac Radiol. 2023;52(8):20230065.
pubmed: 37869886
doi: 10.1259/dmfr.20230065
Subramanian AK, Chen Y, Almalki A, Sivamurthy G, Kafle D. Cephalometric Analysis in Orthodontics using Artificial Intelligence-A Comprehensive Review. Biomed Res Int. 2022;2022:1880113.
pubmed: 35757486
pmcid: 9225851
doi: 10.1155/2022/1880113
Junaid N, Khan N, Ahmed N, Abbasi MS, Das G, Maqsood A, Ahmed AR, Marya A, Alam MK, Heboyan A. Development, Application, and performance of Artificial Intelligence in Cephalometric Landmark Identification and diagnosis: a systematic review. Healthc (Basel) 10(12) (2022).
Kiełczykowski M, Kamiński K, Perkowski K, Zadurska M, Czochrowska E. Application of Artificial Intelligence (AI) in a cephalometric analysis: a narrative review. Diagnostics (Basel) 13(16) (2023).
Chung EJ, Yang BE, Park IY, Yi S, On SW, Kim YH, Kang SH, Byun SH. Effectiveness of cone-beam computed tomography-generated cephalograms using artificial intelligence cephalometric analysis. Sci Rep. 2022;12(1):20585.
pubmed: 36446924
pmcid: 9708822
doi: 10.1038/s41598-022-25215-0
Hwang HW, Moon JH, Kim MG, Donatelli RE, Lee SJ. Evaluation of automated cephalometric analysis based on the latest deep learning method. Angle Orthod. 2021;91(3):329–35.
pubmed: 33434275
pmcid: 8084461
doi: 10.2319/021220-100.1
Schwendicke F, Chaurasia A, Arsiwala L, Lee JH, Elhennawy K, Jost-Brinkmann PG, Demarco F, Krois J. Deep learning for cephalometric landmark detection: systematic review and meta-analysis. Clin Oral Investig. 2021;25(7):4299–309.
pubmed: 34046742
pmcid: 8310492
doi: 10.1007/s00784-021-03990-w
Jeon S, Lee KC. Comparison of cephalometric measurements between conventional and automatic cephalometric analysis using convolutional neural network. Prog Orthod. 2021;22(1):14.
pubmed: 34056670
pmcid: 8165048
doi: 10.1186/s40510-021-00358-4
Cohen AM. Uncertainty in cephalometrics. Br J Orthod. 1984;11(1):44–8.
pubmed: 6584177
doi: 10.1179/bjo.11.1.44
El-Dawlatly M, Attia KH, Abdelghaffar AY, Mostafa YA. M. Abd El-Ghafour, Preciseness of artificial intelligence for lateral cephalometric measurements, Journal of Orofacial Orthopedics / Fortschritte der Kieferorthopädie (2023).
Durão APR, Morosolli A, Pittayapat P, Bolstad N, Ferreira AP, Jacobs R. Cephalometric landmark variability among orthodontists and dentomaxillofacial radiologists: a comparative study. Imaging Sci Dentistry. 2015;45(4):213–20.
doi: 10.5624/isd.2015.45.4.213
Trpkova B, Major P, Prasad N, Nebbe B. Cephalometric landmarks identification and reproducibility: a meta analysis. Am J Orthod Dentofac Orthop. 1997;112(2):165–70.
doi: 10.1016/S0889-5406(97)70242-7
Kim H-J, Kim KD, Kim D-H. Deep convolutional neural network-based skeletal classification of cephalometric image compared with automated-tracing software. Sci Rep. 2022;12(1):11659.
pubmed: 35804075
pmcid: 9270345
doi: 10.1038/s41598-022-15856-6
Baumrind S, Frantz RC. The reliability of head film measurements. 2. Conventional angular and linear measures. Am J Orthod. 1971;60(5):505–17.
pubmed: 5286677
doi: 10.1016/0002-9416(71)90116-3
Arık S, Ibragimov B, Xing L. Fully automated quantitative cephalometry using convolutional neural networks. J Med Imaging (Bellingham). 2017;4(1):014501.
pubmed: 28097213
doi: 10.1117/1.JMI.4.1.014501
Nishimoto S, Sotsuka Y, Kawai K, Ishise H, Kakibuchi M. Personal Computer-based Cephalometric Landmark Detection with Deep Learning, using cephalograms on the internet. J Craniofac Surg. 2019;30(1):91–5.
pubmed: 30439733
doi: 10.1097/SCS.0000000000004901
Celik E, Polat-Ozsoy O, Toygar TU, Memikoglu. Comparison of cephalometric measurements with digital versus conventional cephalometric analysis. Eur J Orthod. 2009;31(3):241–6.
pubmed: 19237509
doi: 10.1093/ejo/cjn105
Lagravère MO, Low C, Flores-Mir C, Chung R, Carey JP, Heo G, Major PW. Intraexaminer and interexaminer reliabilities of landmark identification on digitized lateral cephalograms and formatted 3-dimensional cone-beam computerized tomography images. Am J Orthod Dentofac Orthop. 2010;137(5):598–604.
doi: 10.1016/j.ajodo.2008.07.018
katyal BN. D, Evaluation of the accuracy and reliability of WebCeph-An artificial intelligence -based online software., APOS Trends Orthod (2022).
Mahto RK, Kafle D, Giri A, Luintel S, Karki A. Evaluation of fully automated cephalometric measurements obtained from web-based artificial intelligence driven platform. BMC Oral Health. 2022;22(1):132.
pubmed: 35440037
pmcid: 9020017
doi: 10.1186/s12903-022-02170-w
Kunz F, Stellzig-Eisenhauer A, Widmaier LM, Zeman F, Boldt J. Assessment of the quality of different commercialproviders using artificial intelligence for automated cephalometric analysis compared to human orthodontic experts.J Orofac Orthop. 2023 Aug 29. English. https://doi.org/10.1007/s00056-023-00491-1 . Epub ahead of print. PMID: 37642657.
Yassir YA, Salman AR, Nabbat SA. The accuracy and reliability of WebCeph for cephalometric analysis. J Taibah Univ Med Sci. 2022;17(1):57–66.
pubmed: 35140566
Silva TP, Pinheiro MCR, Freitas DQ, Gaêta-Araujo H, Oliveira-Santos C. Assessment of accuracy and reproducibility of cephalometric identification performed by 2 artificial intelligence-driven tracing applications and human examiners. Oral Surg Oral Med Oral Pathol Oral Radiol. 2024;137(4):431–40.
pubmed: 38365543
doi: 10.1016/j.oooo.2024.01.011
Danisman H. Artificial intelligence web-based cephalometric analysis platform: comparison with the computer assisted cephalometric method. Clin Invest Orthod. 2023;82(4):194–203.
doi: 10.1080/27705781.2023.2254537
Sayinsu K, Isik F, Trakyali G, Arun T. An evaluation of the errors in cephalometric measurements on scanned cephalometric images and conventional tracings. Eur J Orthod. 2007;29(1):105–8.
pubmed: 17290023
doi: 10.1093/ejo/cjl065
Park J-H, Hwang H-W, Moon J-H, Yu Y, Kim H, Her S-B, Srinivasan G, Aljanabi MNA, Donatelli RE, Lee S-J. Automated identification of cephalometric landmarks: part 1—Comparisons between the latest deep-learning methods YOLOV3 and SSD. Angle Orthod. 2019;89(6):903–9.
pubmed: 31282738
pmcid: 8109157
doi: 10.2319/022019-127.1
Bulatova G, Kusnoto B, Grace V, Tsay TP, Avenetti DM, Sanchez FJC. Assessment of automatic cephalometric landmark identification using artificial intelligence. Orthod Craniofac Res. 2021;24(S2):37–42.
pubmed: 34842346
doi: 10.1111/ocr.12542
Saifeldin H. Comparison between manual lateral cephalometric analysis and Artificial Intelligence Driven platforms. Ain Shams Dent J. 2023;29:1–5.
doi: 10.21608/asdj.2022.171778.1152
Cheng R. A survey: comparison between Convolutional Neural Network and YOLO in image identification. J Phys: Conf Ser. 2020;1453(1):012139.
Rousseau A-J, Geubbelmans M, Valkenborg D, Burzykowski T. Explainable artificial intelligence. Am J Orthod Dentofac Orthop. 2024;165(4):491–4.
doi: 10.1016/j.ajodo.2024.01.006
Markus AF, Kors JA, Rijnbeek PR. The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategies. J Biomed Inform. 2021;113:103655.
pubmed: 33309898
doi: 10.1016/j.jbi.2020.103655
Lee J, Bae SR, Noh HK. Commercial artificial intelligence lateral cephalometric analysis: part 2-effects of human examiners on artificial intelligence performance, a pilot study. J Clin Pediatr Dent. 2023;47(6):130–41.
pubmed: 37997244
Panesar S, Zhao A, Hollensbe E, Wong A, Bhamidipalli SS, Eckert G, Dutra V, Turkkahraman H. Precision and Accuracy Assessment of cephalometric analyses performed by Deep Learning Artificial Intelligence with and without human augmentation. Appl Sci. 2023;13(12):6921.
doi: 10.3390/app13126921
Yu HJ, Cho SR, Kim MJ, Kim WH, Kim JW, Choi J. Automated skeletal classification with lateral cephalometry based on Artificial Intelligence. J Dent Res. 2020;99(3):249–56.
pubmed: 31977286
doi: 10.1177/0022034520901715
Ueda A, Tussie C, Kim S, Kuwajima Y, Matsumoto S, Kim G, Satoh K, Nagai S. Classification of Maxillofacial morphology by Artificial Intelligence using cephalometric analysis measurements. Diagnostics (Basel) 13(13) (2023).
Nucera R, Lo Giudice A, Bellocchio M, Spinuzza P, Caprioglio A, Cordasco G. Diagnostic concordance between skeletal cephalometrics, radiograph-based soft-tissue cephalometrics, and photograph-based soft-tissue cephalometrics. Eur J Orthod. 2017;39(4):352–7.
pubmed: 27932406
Aksakallı S, Yılancı H, Görükmez E, Ramoğlu S. Reliability Assessment of Orthodontic Apps for Cephalometrics. Turk J Orthod. 2016;29(4):98–102.
pubmed: 30112482
pmcid: 6007603
doi: 10.5152/TurkJOrthod.2016.1618