Applied Deep Learning in Plastic Surgery: Classifying Rhinoplasty With a Mobile App.
Journal
The Journal of craniofacial surgery
ISSN: 1536-3732
Titre abrégé: J Craniofac Surg
Pays: United States
ID NLM: 9010410
Informations de publication
Date de publication:
Historique:
pubmed:
22
10
2019
medline:
14
3
2020
entrez:
22
10
2019
Statut:
ppublish
Résumé
Advances in deep learning (DL) have been transformative in computer vision and natural language processing, as well as in healthcare. The authors present a novel application of DL to plastic surgery. Here, the authors describe and demonstrate the mobile deployment of a deep neural network that predicts rhinoplasty status, assess model accuracy compared to surgeons, and describe future directions for such applications in plastic surgery. A deep convolutional neural network ("RhinoNet") was developed to classify rhinoplasty images using only pixels and rhinoplasty status labels ("before"/"after") as inputs. RhinoNet was trained using a dataset of 22,686 before and after photos which were collected from publicly available sites. Network classification was compared to that of plastic surgery attendings and residents on 2269 previously-unseen test-set images. RhinoNet correctly predicted rhinoplasty status in 85% of the test-set images. Sensitivity and specificity of model predictions were 0.840 (0.79-0.89) and 0.826 (0.77-0.88), respectively; the corresponding values for expert consensus predictions were 0.814 (0.76-0.87) and 0.867 (0.82-0.91). RhinoNet and humans performed with effectively equivalent accuracy in this classification task. The authors describe the development of DL applications to identify the presence of superficial surgical procedures solely from images and labels. DL is especially well suited for unstructured, high-fidelity visual and auditory data that does not lend itself to classical statistical analysis, and may be deployed as mobile applications for potentially unbridled use, so the authors expect DL to play a key role in many areas of plastic surgery.
Sections du résumé
BACKGROUND
BACKGROUND
Advances in deep learning (DL) have been transformative in computer vision and natural language processing, as well as in healthcare. The authors present a novel application of DL to plastic surgery. Here, the authors describe and demonstrate the mobile deployment of a deep neural network that predicts rhinoplasty status, assess model accuracy compared to surgeons, and describe future directions for such applications in plastic surgery.
METHODS
METHODS
A deep convolutional neural network ("RhinoNet") was developed to classify rhinoplasty images using only pixels and rhinoplasty status labels ("before"/"after") as inputs. RhinoNet was trained using a dataset of 22,686 before and after photos which were collected from publicly available sites. Network classification was compared to that of plastic surgery attendings and residents on 2269 previously-unseen test-set images.
RESULTS
RESULTS
RhinoNet correctly predicted rhinoplasty status in 85% of the test-set images. Sensitivity and specificity of model predictions were 0.840 (0.79-0.89) and 0.826 (0.77-0.88), respectively; the corresponding values for expert consensus predictions were 0.814 (0.76-0.87) and 0.867 (0.82-0.91). RhinoNet and humans performed with effectively equivalent accuracy in this classification task.
CONCLUSION
CONCLUSIONS
The authors describe the development of DL applications to identify the presence of superficial surgical procedures solely from images and labels. DL is especially well suited for unstructured, high-fidelity visual and auditory data that does not lend itself to classical statistical analysis, and may be deployed as mobile applications for potentially unbridled use, so the authors expect DL to play a key role in many areas of plastic surgery.
Identifiants
pubmed: 31633665
doi: 10.1097/SCS.0000000000005905
pii: 00001665-202001000-00027
doi:
Types de publication
Journal Article
Langues
eng
Pagination
102-106Références
Esteva A, Kuprel B, Novoa RA, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 2017; 542:115–118.
Sirinukunwattana K, Raza SEA, Tsang YW, et al. Locality sensitive deep learning for detection and classification of nuclei in routine colon cancer histology images. IEEE Trans Med Imaging 2016; 35:1196–1206.
Albarqouni S, Baur C, Achilles F, et al. AggNet: deep learning from crowds for mitosis detection in breast cancer histology images. IEEE Trans Med Imaging 2016; 35:1313–1321.
Bar Y, Diamant I, Wolf L, et al. Chest pathology detection using deep learning with non-medical training. In: Proceedings of Biomedical Imaging (ISBI), 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI). Brooklyn, NY: 2015, 294–297.
Rajpurkar P, Irvin J, Zhu K, et al. Chexnet: Radiologist-level pneumonia detection on chest x-rays with deep learning 2017. Available at: https://arxiv.org/abs/1711.05225. Accessed July 1, 2018
Nasr-esfahani E, Samavi S, Karimi N, et al. Melanoma detection by analysis of clinical images using convolutional neural network. Conf Proc IEEE Eng Med Biol Soc 2016; 2016:1373–1376.
Lecun Y, Bengio Y, Hinton G. Deep learning. Nature 2015; 521:436–444.
Deng J, Dong W, Socher R, et al. Imagenet: a large-scale hierarchical image database. Comp Vis Pattern Recognit 2009; 248–255.
Sandler M, Howard A, Zhu M, et al. MobileNetV2: inverted residuals and linear bottlenecks. Proc IEEE Conf Comp Vis Pattern Recognit 2018; 4510–4520.
2017 Plastic Surgery Statistics. American Society of Plastic Surgeons. Available at: https://www.plasticsurgery.org/documents/News/Statistics/2017/plastic-surgery-statistics-report-2017.pdf. Accessed December 1, 2018.
Real Self. 2019. Available at: https://www.realself.com. Accessed July 1, 2018.
Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. In: Proceedings of the 25th International Conference on Neural Information Processing Systems, Volume 1. Eds: Pereira F, Burges CJC, Bottou L, Weinberger KQ. Red Hook, NY: Curran Associates: 2012,1097–1105.
Kingma DP, Ba J. Adam: Adam: A method for stochastic optimization. Published as a conference paper at the 3rd international conference for learning representations. Presented at San Diego, CA; May 7–9, 2015.
Peng R D. Reproducible research in computational science. Science 2011; 334:1226–1227.
Desimone RD. RhinoplastyNet. GitHub repository. Available at: https://github.com/desimone/RhinoplastyNet. Accessed July 1, 2018.
Desimone RD. RhinoNet. Version 1.1. Apple App Store. Available at: https://itunes.apple.com/us/app/rhinonet/id1405109080?mt=8. Accessed July 1, 2018.
Agresti A, Min Y. Simple improved confidence intervals for comparing matched proportions. Stat Med 2005; 24:729–740.
Gu W, Pepe MS. Estimating the capacity for improvement in risk prediction with a marker. Biostatistics 2009; 10:172–186.
Stock C, Hielscher T. DTComPair: comparison of binary diagnostic tests in a paired study design. R package version 1.0.3. 2014. URL: Available at: http://CRAN.R-project.org/package=DTComPair. Accessed July 1, 2018.
Zhou XH, McClish DK, Obuchowski NA. Statistical Methods in Diagnostic Medicine. 2nd ed. Hoboken, NJ: John Wiley & Sons, 2011:57–103.
Safran T, Viezel-mathieu A, Corban J, et al. Machine learning and melanoma: the future of screening. J Am Acad Dermatol 2018; 78:620–621.
Gunes H, Piccardi M, Jan T. Automated classification of female facial beauty by image analysis and supervised learning. Proc SPIE 2004; 5308:968–978.
Gunes H, Piccardi M. Assessing facial beauty through proportion analysis by image processing and supervised learning. Int J Hum Comp Studies 2006; 64:1184–1199.
Mendoza CS, Safdar N, Okada K, et al. Personalized assessment of craniosynostosis via statistical shape modeling. Med Image Anal 2014; 18:635–646.
Valente DS, Silveira eifler L, Carvalho LA, et al. Telemedicine and plastic surgery: a pilot study. Plast Surg Int 2015; 2015:187505.
Vyas KS, Hambrick HR, Shakir A, et al. A systematic review of the use of telemedicine in plastic and reconstructive surgery and dermatology. Ann Plast Surg 2017; 78:736–768.
Chandawarkar AA, Gould DJ, Stevens WG. Insta-grated plastic surgery residencies: the rise of social media use by trainees and responsible guidelines for use. Aesthet Surg J 2018; 38:1145–1152.