Metis: a python-based user interface to collect expert feedback for generative chemistry models.
De novo drug design
Human-in-the-loop
Machine learning
Preference learning
User interface
Journal
Journal of cheminformatics
ISSN: 1758-2946
Titre abrégé: J Cheminform
Pays: England
ID NLM: 101516718
Informations de publication
Date de publication:
14 Aug 2024
14 Aug 2024
Historique:
received:
15
05
2024
accepted:
02
08
2024
medline:
15
8
2024
pubmed:
15
8
2024
entrez:
14
8
2024
Statut:
epublish
Résumé
One challenge that current de novo drug design models face is a disparity between the user's expectations and the actual output of the model in practical applications. Tailoring models to better align with chemists' implicit knowledge, expectation and preferences is key to overcoming this obstacle effectively. While interest in preference-based and human-in-the-loop machine learning in chemistry is continuously increasing, no tool currently exists that enables the collection of standardized and chemistry-specific feedback. Metis is a Python-based open-source graphical user interface (GUI), designed to solve this and enable the collection of chemists' detailed feedback on molecular structures. The GUI enables chemists to explore and evaluate molecules, offering a user-friendly interface for annotating preferences and specifying desired or undesired structural features. By providing chemists the opportunity to give detailed feedback, allows researchers to capture more efficiently the chemist's implicit knowledge and preferences. This knowledge is crucial to align the chemist's idea with the de novo design agents. The GUI aims to enhance this collaboration between the human and the "machine" by providing an intuitive platform where chemists can interactively provide feedback on molecular structures, aiding in preference learning and refining de novo design strategies. Metis integrates with the existing de novo framework REINVENT, creating a closed-loop system where human expertise can continuously inform and refine the generative models.Scientific contributionWe introduce a novel Graphical User Interface, that allows chemists/researchers to give detailed feedback on substructures and properties of small molecules. This tool can be used to learn the preferences of chemists in order to align de novo drug design models with the chemist's ideas. The GUI can be customized to fit different needs and projects and enables direct integration into de novo REINVENT runs. We believe that Metis can facilitate the discussion and development of novel ways to integrate human feedback that goes beyond binary decisions of liking or disliking a molecule.
Identifiants
pubmed: 39143631
doi: 10.1186/s13321-024-00892-3
pii: 10.1186/s13321-024-00892-3
doi:
Types de publication
Journal Article
Langues
eng
Pagination
100Subventions
Organisme : National Academic Infrastructure for Supercomputing in Sweden
ID : 2022-06725
Organisme : Horizon 2020
ID : 956832
Organisme : UKRI Turing AI World-Leading Researcher Fellowship
ID : EP/W002973/
Informations de copyright
© 2024. The Author(s).
Références
Schneider G (2018) Automating drug discovery. Nat Rev Drug Discov 17:97–113
doi: 10.1038/nrd.2017.232
pubmed: 29242609
Korshunova M, Huang N, Capuzzi S, Radchenko DS, Savych O, Moroz YS, Wells CI, Willson TM, Tropsha A, Isayev O (2022) Generative and reinforcement learning approaches for the automated de novo design of bioactive compounds. Commun Chem 5:129
doi: 10.1038/s42004-022-00733-0
pubmed: 36697952
pmcid: 9814657
Svensson HG, Tyrchan C, Engkvist O, Chehreghani MH (2023) Utilizing Reinforcement learning for de novo drug design. arXiv preprint.
Mnih V, Kavukcuoglu K, Silver D, Graves A, Antonoglou I, Wierstra D, Riedmiller MA (2013) Playing atari with deep reinforcement learning. arXiv preprint. arxiv:1312.5602
Silver D, Huang A, Maddison CJ, Guez A, Sifre L, Van Den Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M et al (2016) Mastering the game of go with deep neural networks and tree search. Nature 529:484–489
doi: 10.1038/nature16961
pubmed: 26819042
Vinyals O, Babuschkin I, Czarnecki WM, Mathieu M, Dudzik A, Chung J, Choi DH, Powell R, Ewalds T, Georgiev P et al (2019) Grandmaster level in starCraft II using multi-agent reinforcement learning. Nature 575:350–354
doi: 10.1038/s41586-019-1724-z
pubmed: 31666705
Amodei D, Olah C, Steinhardt J, Christiano P, Schulman J, Mané D (2016) Concrete problems in AI safety. arXiv preprint. arXiv:1606.06565
Skalse J, Howe N, Krasheninnikov D, Krueger D (2022) Defining and characterizing reward gaming. Adv Neural Info Process Syst 35:9460–9471
Lee K, Smith L, Abbeel P (2021) Pebble: Feedback-efficient interactive reinforcement learning via relabeling experience and unsupervised pre-training. arXiv preprint. arXiv:2106.05091
Mosqueira-Rey E, Hernández-Pereira E, Alonso-Ríos D, Bobes-Bascarán J, Fernández-Leal Á (2023) Human-in-the-loop machine learning: a state of the art. Artif Intell Rev 56:3005–3054
doi: 10.1007/s10462-022-10246-w
Hussein A, Gaber MM, Elyan E, Jayne C (2017) Imitation learning: a survey of learning methods. ACM Comput Surv (CSUR) 50:1–35
doi: 10.1145/3054912
Torabi F, Warnell G, Stone P (2018) Behavioral cloning from observation. arXiv preprint. arXiv:1805.01954
Arora S, Doshi P (2021) A survey of inverse reinforcement learning: challenges methods and progress. Artif Intell 297
Rafailov R, Sharma A, Mitchell E, Manning CD, Ermon S, Finn C (2024) Direct preference optimization: your language model is secretly a reward model. Adv Neural Info Process Syst 36
Christiano PF, Leike J, Brown T, Martic M, Legg S, Amodei D (2017) Deep reinforcement learning from human preferences. Adv Neural Info Process Syst 30
Meyers J, Fabian B, De Brown N (2021) Novo molecular design and generative models. Drug Discov Today 26:2707–2715
doi: 10.1016/j.drudis.2021.05.019
pubmed: 34082136
Choung O-H, Vianello R, Segler M, Stiefl N, Jiménez-Luna J (2023) Extracting medicinal chemistry intuition via preference machine learning. Nat Commun 14:6651
doi: 10.1038/s41467-023-42242-1
pubmed: 37907461
pmcid: 10618272
Sundin I, Voronov A, Xiao H, Papadopoulos K, Bjerrum EJ, Heinonen M, Patronov A, Kaski S, Engkvist O (2022) Human-in-the-loop assisted de novo molecular design. J Cheminf 14:86
doi: 10.1186/s13321-022-00667-8
Blaschke T, Arús-Pous J, Chen H, Margreitter C, Tyrchan C, Engkvist O, Papadopoulos K, Patronov A (2020) REINVENT 2.0: an AI tool for de novo drug design. J Chem Info Model 60:5918–5922
doi: 10.1021/acs.jcim.0c00915
Pedregosa F et al (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825–2830
RDKit: Open-Source Cheminformatics Software (2021) https://www.rdkit.org .
Riniker S, Landrum GA (2013) Similarity maps -a visualization strategy for molecular fingerprints and machine-learning methods. J Cheminf 5:1–7
doi: 10.1186/1758-2946-5-43
Bjerrum EJ, Palunas K, Menke J (2024) Python-Based Interactive RDKit Molecule Editing with rdEditor. chemRxiv preprint , https://doi.org/10.26434/chemrxiv-2024-jfhmw
Qt for Python Team. PySide2: Python bindings for the Qt cross-platform application and UI framework. https://www.pyside.org . Accessed 08 Apr 2024
Loeffler HH, He J, Tibo A, Janet JP, Voronov A, Mervin LH, Engkvist O (2024) Reinvent 4: modern AI-driven generative molecule design. J Cheminf 16:20
doi: 10.1186/s13321-024-00812-5
Bjerrum EJ, Bachorz RA, Bitton A, Choung O-h, Chen Y, Esposito C, Ha SV, Poehlmann A (2023) Scikit-mol brings cheminformatics to scikit-learn. chemRxiv preprint.