A general deep learning model for bird detection in high-resolution airborne imagery.
airborne monitoring
bird detection
computer vision
deep learning
unoccupied aerial vehicle
Journal
Ecological applications : a publication of the Ecological Society of America
ISSN: 1051-0761
Titre abrégé: Ecol Appl
Pays: United States
ID NLM: 9889808
Informations de publication
Date de publication:
12 2022
12 2022
Historique:
revised:
09
02
2022
received:
09
09
2021
accepted:
22
03
2022
pubmed:
17
6
2022
medline:
3
12
2022
entrez:
16
6
2022
Statut:
ppublish
Résumé
Advances in artificial intelligence for computer vision hold great promise for increasing the scales at which ecological systems can be studied. The distribution and behavior of individuals is central to ecology, and computer vision using deep neural networks can learn to detect individual objects in imagery. However, developing supervised models for ecological monitoring is challenging because it requires large amounts of human-labeled training data, requires advanced technical expertise and computational infrastructure, and is prone to overfitting. This limits application across space and time. One solution is developing generalized models that can be applied across species and ecosystems. Using over 250,000 annotations from 13 projects from around the world, we develop a general bird detection model that achieves over 65% recall and 50% precision on novel aerial data without any local training despite differences in species, habitat, and imaging methodology. Fine-tuning this model with only 1000 local annotations increases these values to an average of 84% recall and 69% precision by building on the general features learned from other data sources. Retraining from the general model improves local predictions even when moderately large annotation sets are available and makes model training faster and more stable. Our results demonstrate that general models for detecting broad classes of organisms using airborne imagery are achievable. These models can reduce the effort, expertise, and computational resources necessary for automating the detection of individual organisms across large scales, helping to transform the scale of data collection in ecology and the questions that can be addressed.
Types de publication
Journal Article
Research Support, U.S. Gov't, Non-P.H.S.
Research Support, Non-U.S. Gov't
Langues
eng
Sous-ensembles de citation
IM
Pagination
e2694Informations de copyright
© 2022 The Ecological Society of America.
Références
Afán, I., M. Máñez, and R. Díaz-Delgado. 2018. “Drone Monitoring of Breeding Waterbird Populations: The Case of the Glossy Ibis.” Drones 2: 42.
Ahumada, J. A., E. Fegraus, T. Birch, N. Flores, R. Kays, T. G. O'Brien, J. Palmer, et al. 2020. “Wildlife Insights: A Platform to Maximize the Potential of Camera Trap and Other Passive Sensor Wildlife Data for the Planet.” Environmental Conservation 47: 1-6.
Beery, S., G. Wu, V. Rathod, R. Votel, and J. Huang. 2020. “Context R-CNN: Long Term Temporal Context for per-Camera Object Detection.” 13075-13085.
Beijbom, O., P. J. Edmunds, C. Roelfsema, J. Smith, D. I. Kline, B. P. Neal, M. J. Dunlap, et al. 2015. “Towards Automated Annotation of Benthic Survey Images: Variability of Human Experts and Operational Modes of Automation.” PLoS One 10: e0130312.
Berger-Wolf, T. Y., D. I. Rubenstein, C. V. Stewart, J. A. Holmberg, J. Parham, S. Menon, J. Crall, J. Van Oast, E. Kiciman, and L. Joppa. 2017. “Wildbook: Crowdsourcing, Computer Vision, and Data Science for Conservation.” arXiv:1710.08880 [cs].
Bondi, E., D. Dey, A. Kapoor, J. Piavis, S. Shah, F. Fang, B. Dilkina, et al. 2018. “AirSim-W: A Simulation Environment for Wildlife Conservation with UAVs.” In Proceedings of the 1st ACM SIGCAS Conference on Computing and Sustainable Societies 1-12. New York, NY: Association for Computing Machinery.
Bowley, C., M. Mattingly, A. Barnas, S. Ellis-Felege, and T. Desell. 2018. “Detecting Wildlife in Unmanned Aerial Systems Imagery Using Convolutional Neural Networks Trained with an Automated Feedback Loop.” In Computational Science - ICCS 2018, edited by Y. Shi, H. Fu, Y. Tian, V. V. Krzhizhanovskaya, M. H. Lees, J. Dongarra, and P. M. A. Sloot, 69-82. Cham: Springer International Publishing.
Chabot, D., C. Dillon, and C. Francis. 2018. “An Approach for Using off-the-Shelf Object-Based Image Analysis Software to Detect and Count Birds in Large Volumes of Aerial Imagery.” Avian Conservation and Ecology 13: 15.
Crall, J. P., C. V. Stewart, T. Y. Berger-Wolf, D. I. Rubenstein, and S. R. Sundaresan. 2013. “HotSpotter - Patterned Species Instance Recognition.” In 2013 IEEE Workshop on Applications of Computer Vision (WACV) 230-7.
Dulava, S., W. T. Bean, and O. M. W. Richmond. 2015. “Environmental Reviews and Case Studies: Applications of Unmanned Aircraft Systems (UAS) for Waterbird Surveys.” Environmental Practice 17: 201-10.
Graves, A., M. G. Bellemare, J. Menick, R. Munos, and K. Kavukcuoglu. 2017. “Automated Curriculum Learning for Neural Networks.” In Proceedings of the 34th International Conference on Machine Learning, edited by D. Precup, and Y. W. Teh, Vol 70: 1311-20.
Gregory, R. D., and A. van Strien. 2010. “Wild Bird Indicators: Using Composite Population Trends of Birds as Measures of Environmental Health.” Ornithological Science 9: 3-22.
Groom, G., M. Stjernholm, R. D. Nielsen, A. Fleetwood, and I. K. Petersen. 2013. “Remote Sensing Image Data and Automated Analysis to Describe Marine Bird Distributions and Abundances.” Ecological Informatics 14: 2-8.
Hayes, M. C., P. C. Gray, G. Harris, W. C. Sedgwick, V. D. Crawford, N. Chazal, S. Crofts, and D. W. Johnston. 2021. “Drones and Deep Learning Produce Accurate and Efficient Monitoring of Large-Scale Seabird Colonies.” Ornithological Applications 123(3): 1.
Kawaguchi, K., L. P. Kaelbling, and Y. Bengio. 2020. “Generalization in Deep Learning.” arXiv:1710.05468 [cs, stat].
Kellenberger, B., D. Marcos, and D. Tuia. 2018. “Detecting Mammals in UAV Images: Best Practices to Address a Substantially Imbalanced Dataset with Deep Learning.” Remote Sensing of Environment 216: 139-53.
Kellenberger, B., D. Tuia, and D. Morris. 2020. “AIDE: Accelerating Image-Based Ecological Surveys with Interactive Machine Learning.” Methods in Ecology and Evolution 11: 1716-27.
Kim, S., and M. Kim. 2020. “Learning of Counting Crowded Birds of Various Scales Via Novel Density Activation Maps.” IEEE Access 8: 155296-305.
LaRue, M. A., S. Stapleton, and M. Anderson. 2017. “Feasibility of Using High-Resolution Satellite Imagery to Assess Vertebrate Wildlife Populations.” Conservation Biology 31: 213-20.
Lin, T.-Y., P. Goyal, R. Girshick, K. He, and P. Dollar. 2017. “Focal Loss for Dense Object Detection.” In Proceedings of the IEEE international conference on computer vision, 2980-8.
Liu, Y., V. Shah, A. Borowicz, M. Wethington, N. Strycker, S. Forrest, H. Lynch, and H. Singh. 2020. “Towards Efficient Machine Learning Methods for Penguin Counting in Unmanned Aerial System Imagery.” In 2020 IEEE/OES Autonomous Underwater Vehicles Symposium (AUV) 1-7.
McKellar, A. E., N. G. Shephard, and D. Chabot. 2021. “Dual Visible-Thermal Camera Approach Facilitates Drone Surveys of Colonial Marshbirds.” Remote Sensing in Ecology and Conservation 7: 214-26.
Moreland, E. E., M. F. Cameron, R. P. Angliss, and P. L. Boveng. 2015. “Evaluation of a Ship-Based Unoccupied Aircraft System (UAS) for Surveys of Spotted and Ribbon Seals in the Bering Sea Pack Ice.” Journal of Unmanned Vehicle Systems 3: 114-22.
Pan, S. J., and Q. Yang. 2010. “A Survey on Transfer Learning.” IEEE Transactions on Knowledge and Data Engineering 22: 1345-59.
Pfeifer, C., M.-C. Rümmler, and O. Mustafa. 2021. “Assessing Colonies of Antarctic Shags by Unmanned Aerial Vehicle (UAV) at South Shetland Islands, Antarctica.” Antarctic Science 33: 133-49.
Reintsma, K. M., P. C. McGowan, C. Callahan, T. Collier, D. Gray, J. D. Sullivan, and D. J. Prosser. 2018. “Preliminary Evaluation of Behavioral Response of Nesting Waterbirds to Small Unmanned Aircraft Flight.” Waterbirds 41: 326-31.
Torney, C. J., D. J. Lloyd-Jones, M. Chevallier, D. C. Moyer, H. T. Maliti, M. Mwita, E. M. Kohi, and G. C. Hopcraft. 2019. “A Comparison of Deep Learning and Citizen Science Techniques for Counting Wildlife in Aerial Survey Images.” Methods in Ecology and Evolution 10: 779-87.
Van Horn, G., O. Mac Aodha, Y. Song, Y. Cui, C. Sun, A. Shepard, H. Adam, P. Perona, and S. Belongie. 2018. “The iNaturalist Species Classification and Detection Dataset.” In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition 8769-78. Salt Lake City, UT: IEEE.
Weinstein, B. 2021. “weecology/BirdDetector: Paper Submission (1.1).” Zenodo. https://doi.org/10.5281/zenodo.5156926.
Weinstein, B. G. 2018. “A Computer Vision for Animal Ecology.” Journal of Animal Ecology 87: 533-45.
Weinstein, B., D. Fang, H. Senyondo, E. White, and D. Munshi. 2021. “weecology/DeepForest: Pytorch release (1.0.0).” Zenodo. https://doi.org/10.5281/zenodo.4904184.
Weinstein, B., L. Garner, V. R. Saccomanno, A. Steinkraus, A. Ortega, K. Brush, G. Yenni, et al. 2021. “A Global Model of Bird Detection in High Resolution Airborne Images Using Computer Vision.” https://doi.org/10.5281/zenodo.5033174.
Weinstein, B. G., S. Marconi, S. Bohlman, A. Zare, and E. White. 2019. “Individual Tree-Crown Detection in RGB Imagery Using Semi-Supervised Deep Learning Neural Networks.” Remote Sensing 11: 1309. https://doi.org/10.3390/rs11111309.
Weinstein, B. G., S. Marconi, M. Aubry-Kientz, G. Vincent, H. Senyondo, and E. P. White. 2020. “DeepForest: A Python Package for RGB Deep Learning Tree Crown Delineation.” Methods in Ecology and Evolution 11: 1743-51.
Weissensteiner, M. H., J. W. Poelstra, and J. B. W. Wolf. 2015. “Low-Budget Ready-to-Fly Unmanned Aerial Vehicles: An Effective Tool for Evaluating the Nesting Status of Canopy-Breeding Bird Species.” Journal of Avian Biology 46: 425-30.
Willi, M., R. T. Pitman, A. W. Cardoso, C. Locke, A. Swanson, A. Boyer, M. Veldthuis, and L. Fortson. 2019. “Identifying Animal Species In Camera Trap Images Using Deep Learning and Citizen Science.” Methods in Ecology and Evolution 10: 80-91.
Zoph, B., E. D. Cubuk, G. Ghiasi, T.-Y. Lin, J. Shlens, and Q. V. Le. 2019. “Learning Data Augmentation Strategies for Object Detection.” arXiv:1906.11172 [cs].