SIGNIFICANCE deep learning based platform to fight illicit trafficking of Cultural Heritage goods.
Journal
Scientific reports
ISSN: 2045-2322
Titre abrégé: Sci Rep
Pays: England
ID NLM: 101563288
Informations de publication
Date de publication:
02 Jul 2024
02 Jul 2024
Historique:
received:
03
08
2023
accepted:
25
06
2024
medline:
3
7
2024
pubmed:
3
7
2024
entrez:
2
7
2024
Statut:
epublish
Résumé
The illicit traffic of cultural goods remains a persistent global challenge, despite the proliferation of comprehensive legislative frameworks developed to address and prevent cultural property crimes. Online platforms, especially social media and e-commerce, have facilitated illegal trade and pose significant challenges for law enforcement agencies. To address this issue, the European project SIGNIFICANCE was born, with the aim of combating illicit traffic of Cultural Heritage (CH) goods. This paper presents the outcomes of the project, introducing a user-friendly platform that employs Artificial Intelligence (AI) and Deep learning (DL) to prevent and combat illicit activities. The platform enables authorities to identify, track, and block illegal activities in the online domain, thereby aiding successful prosecutions of criminal networks. Moreover, it incorporates an ontology-based approach, providing comprehensive information on the cultural significance, provenance, and legal status of identified artefacts. This enables users to access valuable contextual information during the scraping and classification phases, facilitating informed decision-making and targeted actions. To accomplish these objectives, computationally intensive tasks are executed on the HPC CyClone infrastructure, optimizing computing resources, time, and cost efficiency. Notably, the infrastructure supports algorithm modelling and training, as well as web, dark web and social media scraping and data classification. Preliminary results indicate a 10-15% increase in the identification of illicit artifacts, demonstrating the platform's effectiveness in enhancing law enforcement capabilities.
Identifiants
pubmed: 38956250
doi: 10.1038/s41598-024-65885-6
pii: 10.1038/s41598-024-65885-6
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
15081Informations de copyright
© 2024. The Author(s).
Références
Visconti, A. et al. Criminal offences against cultural heritage within the italian legal framework after law 9 march 2022, n. 22 (2023).
Blake, J. International cultural heritage law (OUP Oxford, 2015).
Schloenhardt, A., Calderoni, F., Lelliott, J. & Weißer, B. UN Convention against Transnational Organized Crime: A Commentary (Oxford University Press, 2023).
Lamotte, K. R. Unesco: Convention on the protection of the underwater cultural heritage. Int. Leg. Mater. 41, 37–56 (2002).
doi: 10.1017/S0020782900009153
of Europe, C. & de l’Europe, C. Council of europe convention on offences relating to cultural property (nicosia, 19 may 2017). Uniform Law Rev. 23, 656–691 (2018).
Brodie, N. Stolen history: Looting and illicit trade. Museum Int. 55, 10–22 (2003).
doi: 10.1111/j.1350-0775.2003.00432.x
Campbell, P. B. The illicit antiquities trade as a transnational criminal network: Characterizing and anticipating trafficking of cultural heritage. Int. J. Cult. Prop. 20, 113–153 (2013).
doi: 10.1017/S0940739113000015
Brodie, N. et al. Why there is still an illicit trade in cultural objects and what we can do about it. J. Field Archaeol. 47, 117–130 (2022).
doi: 10.1080/00934690.2021.1996979
Abate, D. et al. Significance. stop illicit heritage trafficking with artificial intelligence. Int. Arch. Photogramm. Remote. Sens. Spat. Inf. Sci. 43, 729–736 (2022).
doi: 10.5194/isprs-archives-XLIII-B2-2022-729-2022
Abate, D. et al. Artificial intelligence to fight illicit trafficking of cultural property. Int. Arch. Photogramm. Remote. Sens. Spat. Inf. Sci. 48, 3–10 (2023).
doi: 10.5194/isprs-archives-XLVIII-M-2-2023-3-2023
Ruggeri, F., Lagioia, F., Lippi, M. & Torroni, P. Detecting and explaining unfairness in consumer contracts through memory networks. Artif. Intell. Law 30, 59–92 (2022).
doi: 10.1007/s10506-021-09288-2
Zaccagnino, C. C. & Malandrino, D. Analysis of touch gestures for online child protection: Techno-regulation and intelligent safeguards. Multimed. Tools Appl. 80, 15803–15824 (2021).
doi: 10.1007/s11042-020-10446-y
Guarino, A., Malandrino, D. & Zaccagnino, R. An automatic mechanism to provide privacy awareness and control over unwittingly dissemination of online private information. Comput. Netw. 202, 108614 (2022).
doi: 10.1016/j.comnet.2021.108614
Pansoni, S., Tiribelli, S., Frontoni, E. & Giovanola, B. Design of an ethical framework for artificial intelligence in cultural heritage. In In 2023 IEEE International Symposium on Ethics in Engineering, Science, and Technology (ETHICS), 1–5 (IEEE, 2023).
Pansoni, S., Tiribelli, S., Frontoni, E. & Giovanola, B. Artificial intelligence and cultural heritage: Design and assessment of an ethical framework. In The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, pp. 1149–1155 (2023).
Hurtut, T., Gousseau, Y., Cheriet, F. & Schmitt, F. Artistic line-drawings retrieval based on the pictorial content. J. Comput. Cult. Herit. (JOCCH) 4, 1–23 (2011).
doi: 10.1145/2001416.2001419
Makridis, M. & Daras, P. Automatic classification of archaeological pottery sherds. J. Comput. Cult. Heritage (JOCCH) 5, 1–21 (2013).
Can, G., Odobez, J.-M. & Gatica-Perez, D. Evaluating shape representations for maya glyph classification. J. Comput. Cult. Heritage (JOCCH) 9, 1–26 (2016).
doi: 10.1145/2905369
Hu, R., Odobez, J.-M. & Gatica-Perez, D. Extracting maya glyphs from degraded ancient documents via image segmentation. J. Comput. Cult. Heritage (JOCCH) 10, 1–23 (2017).
doi: 10.1145/2996859
Bebis, G. et al. Advances in Visual Computing: 7th International Symposium, ISVC 2011, Las Vegas, NV, USA, September 26–28, 2011. Proceedings, Part I, vol. 6938 (Springer, 2011).
Mathias, M., Martinovic, A., Weissenberg, J., Haegler, S. & Van Gool, L. Automatic architectural style recognition. Int. Arch. Photogramm. Remote. Sens. Spat. Inf. Sci. 38, 171–176 (2012).
doi: 10.5194/isprsarchives-XXXVIII-5-W16-171-2011
Chu, W.-T. & Tsai, M.-H. Visual pattern discovery for architecture image classification and product image search. In Proceedings of the 2nd ACM International Conference on Multimedia Retrieval, 1–8 (2012).
Goel, A., Juneja, M. & Jawahar, C. Are buildings only instances? Exploration in architectural style categories. In Proceedings of the Eighth Indian Conference on Computer Vision, Graphics and Image Processing, 1–8 (2012).
Oses, N. & Dornaika, F. Image-based delineation of built heritage masonry for automatic classification. In Image Analysis and Recognition: 10th International Conference, ICIAR 2013, Póvoa do Varzim, Portugal, June 26–28, 2013. Proceedings 10, 782–789 (Springer, 2013).
Zhang, L. et al. Recognizing architecture styles by hierarchical sparse coding of blocklets. Inf. Sci. 254, 141–154 (2014).
doi: 10.1016/j.ins.2013.08.020
Xu, Z., Tao, D., Zhang, Y., Wu, J. & Tsoi, A. C. Architectural style classification using multinomial latent logistic regression. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part I 13, 600–615 (Springer, 2014).
Amato, G., Falchi, F. & Gennaro, C. Fast image classification for monument recognition. J. Comput. Cult. Herit. (JOCCH) 8, 1–25 (2015).
doi: 10.1145/2724727
Chen, H. et al. A deep learning cnn architecture applied in smart near-infrared analysis of water pollution for agricultural irrigation resources. Agric. Water Manag. 240, 106303 (2020).
doi: 10.1016/j.agwat.2020.106303
Pandit, V., Schmitt, M., Cummins, N. & Schuller, B. I see it in your eyes: Training the shallowest-possible cnn to recognise emotions and pain from muted web-assisted in-the-wild video-chats in real-time. Inf. Process. Manag. 57, 102347 (2020).
doi: 10.1016/j.ipm.2020.102347
Li, C., Bao, Z., Li, L. & Zhao, Z. Exploring temporal representations by leveraging attention-based bidirectional lstm-rnns for multi-modal emotion recognition. Inf. Process. Manag. 57, 102185 (2020).
doi: 10.1016/j.ipm.2019.102185
Ćosović, M. & Janković, R. Cnn classification of the cultural heritage images. In 2020 19th International Symposium INFOTEH-JAHORINA (INFOTEH), 1–6 (IEEE, 2020).
Kulkarni, U., Meena, S., Gurlahosur, S. V. & Mudengudi, U. Classification of cultural heritage sites using transfer learning. In 2019 IEEE fifth international conference on multimedia big data (BigMM), 391–397 (IEEE, 2019).
Llamas, J., M. Lerones, P., Medina, R., Zalama, E. & Gómez-García-Bermejo, J. Classification of architectural heritage images using deep learning techniques. Appl. Sci. 7, 992 (2017).
Fan, T., Wang, H. & Deng, S. Intangible cultural heritage image classification with multimodal attention and hierarchical fusion. Expert Syst. Appl. 120555 (2023).
Winterbottom, T., Leone, A. & Al Moubayed, N. A deep learning approach to fight illicit trafficking of antiquities using artefact instance classification. Sci. Rep. 12, 13468 (2022).
doi: 10.1038/s41598-022-15965-2
pubmed: 35931710
pmcid: 9356139
Amato, G., Falchi, F. & Vadicamo, L. Visual recognition of ancient inscriptions using convolutional neural network and fisher vector. J. Comput. Cult. Heritage (JOCCH) 9, 1–24 (2016).
doi: 10.1145/2964911
Guber, T. A translational approach to portable ontologies. Knowl. Acquis. 5(2), 199–229 (1993).
doi: 10.1006/knac.1993.1008
Guarino, N. Formal ontology in information systems: Proceedings of the first international conference (fois’98), June 6–8, Trento, Italy (vol. 46) (IOS press, 1998).
Silla, . F. A. A., C. N. A survey of hierarchical classification across different application domains. Data Min. Knowl. Discov. 22, 31–72 (2022).
Huang, K. S., J. & Zabih, R. An automatic hierarchical image classification scheme. 219–228 (ACM, 1998).
Simonyan, K. & Zisserman, A. Very deep convolutional networks for large-scale image recognition. https://doi.org/10.48550/ARXIV.1409.1556 (2014).
Russakovsky, O. et al. Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. 115, 211–252. https://doi.org/10.1007/s11263-015-0816-y (2015).
doi: 10.1007/s11263-015-0816-y