Deep dynamic modeling with just two time points: Can we still allow for individual trajectories?
deep learning
latent representation
longitudinal data
similarity
small data
Journal
Biometrical journal. Biometrische Zeitschrift
ISSN: 1521-4036
Titre abrégé: Biom J
Pays: Germany
ID NLM: 7708048
Informations de publication
Date de publication:
Dec 2022
Dec 2022
Historique:
revised:
20
12
2021
received:
30
11
2020
accepted:
10
02
2022
pubmed:
7
4
2022
medline:
7
4
2022
entrez:
6
4
2022
Statut:
ppublish
Résumé
Longitudinal biomedical data are often characterized by a sparse time grid and individual-specific development patterns. Specifically, in epidemiological cohort studies and clinical registries we are facing the question of what can be learned from the data in an early phase of the study, when only a baseline characterization and one follow-up measurement are available. Inspired by recent advances that allow to combine deep learning with dynamic modeling, we investigate whether such approaches can be useful for uncovering complex structure, in particular for an extreme small data setting with only two observations time points for each individual. Irregular spacing in time could then be used to gain more information on individual dynamics by leveraging similarity of individuals. We provide a brief overview of how variational autoencoders (VAEs), as a deep learning approach, can be linked to ordinary differential equations (ODEs) for dynamic modeling, and then specifically investigate the feasibility of such an approach that infers individual-specific latent trajectories by including regularity assumptions and individuals' similarity. We also provide a description of this deep learning approach as a filtering task to give a statistical perspective. Using simulated data, we show to what extent the approach can recover individual trajectories from ODE systems with two and four unknown parameters and infer groups of individuals with similar trajectories, and where it breaks down. The results show that such dynamic deep learning approaches can be useful even in extreme small data settings, but need to be carefully adapted.
Identifiants
pubmed: 35384018
doi: 10.1002/bimj.202000366
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
1426-1445Subventions
Organisme : Albert-Ludwigs-Universität Freiburg
ID : Berta-Ottenstein clinician scientist program
Organisme : Deutsche Forschungsgemeinschaft
ID : 322977937/GRK2344
Informations de copyright
© 2022 The Authors. Biometrical Journal published by Wiley-VCH GmbH.
Références
Bezanson, J., Edelman, A., Karpinski, S., & Shah, V. B. (2017). Julia: A fresh approach to numerical computing. SIAM Review, 59, 65-98.
Blei, D. M., Kucukelbir, A., & McAuliffe, J. D. (2017). Variational inference: A review for statisticians. Journal of the American Statistical Association, 112, 859-877.
Bölcskei, H., Grohs, P., Kutyniok, G., & Petersen, P. (2019). Optimal approximation with sparsely connected deep neural networks. SIAM Journal on Mathematics of Data Science, 1, 8-45.
Breiman, L. (2001). Statistical modeling: The two cultures (with comments and a rejoinder by the author). Statistical Science, 16, 199-231.
Brouwer, E. D., Simm, J., Arany, A., & Moreau, Y. (2019). GRU-ODE-Bayes: Continuous modeling of sporadically-observed time series. In Advances in Neural Information Processing Systems.
Cappé, O., Moulines, E., & Rydén, T. (2006). Inference in hidden Markov models. Springer Science & Business Media.
Chen, T. Q., Rubanova, Y., Bettencourt, J., & Duvenaud, D. (2018). Neural ordinary differential equations. In Advances in Neural Information Processing Systems.
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society Series B (Methodological), 39, 1-38.
DeVore, R. A. (1998). Nonlinear approximation. Acta Numerica, 7, 51-150.
Fortuin, V., Baranchuk, D., Rätsch, G., & Mandt, S. (2019). GP-VAE: Deep probabilistic time series imputation. arXiv preprint: https://arxiv.org/abs/1907.04155
Gregor, K., Papamakarios, G., Besse, F., Buesing, L., & Weber, T. (2018). Temporal difference variational auto-encoder. arXiv preprint: https://arxiv.org/abs/1806.03107
Gribonval, R., Kutyniok, G., Nielsen, M., & Voigtlaender, F. (2019). Approximation spaces of deep neural networks. arXiv preprint: https://arxiv.org/abs/1905.01208
Gühring, I., Raslan, M., & Kutyniok, G. (2020). Expressivity of deep neural networks. arXiv preprint: https://arxiv.org/abs/2007.04759
Harvey, A. C. (1989). Forecasting Structural time series models and the Kalman filter. Cambridge University Press.
Hess, M., Lenz, S., Blätte, T. J., Bullinger, L., & Binder, H. (2017). Partitioned learning of deep Boltzmann machines for SNP data. Bioinformatics, 33, 3173-3180.
Innes, M., Edelman, A., Fischer, K., Rackauckas, C., Saba, E., Shah, V. B., & Tebbutt, W. (2019). A differentiable programming system to bridge machine learning and scientific computing. arXiv preprint: https://arxiv.org/abs/1907.07587
Kingma, D. P., & Ba, J. (2015). Adam: A method for stochastic optimization. In Y. Bengio & Y. LeCun (Eds.), 3rd International Conference on Learning Representations (ICLR), Conference Track Proceedings.
Kingma, D. P., & Welling, M. (2014). Auto-encoding variational Bayes. In Y. Bengio & Y. LeCun (Eds.), 2nd International Conference on Learning Representations (ICLR), Conference Track Proceedings.
Kingma, D. P., & Welling, M. (2019). An introduction to variational autoencoders. Foundations and Trends in Machine Learning, 12, 307-392.
Loader, C. (1999). Local regression and likelihood. Statistics and Computing. Springer-Verlag.
Mazzone, E. S., Mayhew, A., Montes, J., Ramsey, D., Fanelli, L., Young, S. D., Salazar, R., Sanctis, R. D., Pasternak, A., Glanzman, A., Coratti, G., Civitello, M., Forcina, N., Gee, R., Duong, T., Pane, M., Scoto, M., Pera, M. C., Messina, S., …, Mercuri, E. (2017). Revised upper limb module for spinal muscular atrophy: Development of a new module. Muscle Nerve, 55, 869-874.
Nußberger, J., Boesel, F., Lenz, S., Binder, H., & Hess, M. (2021). Synthetic observations from deep generative models and binary omics data with limited sample size. Briefings in Bioinformatics, 22(4), https://doi.org/10.1093/bib/bbaa226
Ormerod, J. T., & Wand, M. P. (2010). Explaining variational approximations. American Statistician, 64, 140-153.
Pechmann, A., König, K., Bernert, G., Schachtrup, K., Schara, U., Schorling, D., Schwersenz, I., Stein, S., Tassoni, A., Vogt, S., Walter, M. C., Lochmüller, H., & Kirschner, J. (2019). SMArtCARE-A platform to collect real-life outcome data of patients with spinal muscular atrophy. Orphanet Journal of Rare Diseases, 14, Article No. 18.
Rackauckas, C., Innes, M., Ma, Y., Bettencourt, J., White, L., & Dixit, V. (2019). DiffEqFlux.jl-A Julia library for neural differential equations. arXiv preprint: https://arxiv.org/abs/1902.02376
Rackauckas, C., Ma, Y., Martensen, J., Warner, C., Zubov, K., Supekar, R., Skinner, D., & Ramadhan, A. (2020). Universal differential equations for scientific machine learning. arXiv preprint: https://arxiv.org/abs/2001.04385
Rubanova, Y., Chen, T. Q., & Duvenaud, D. (2019). Latent ODEs for irregularly-sampled time series. arXiv preprint: https://arxiv.org/abs/1907.03907
Rumelhart, D., Hinton, G., & Williams, R. (1986). Learning representations by back-propagating errors. Nature, 323, 533-536.
Schorling, D. C., Pechmann, A., & Kirschner, J. (2020). Advances in treatment of spinal muscular atrophy-New phenotypes, new challenges, new implications for care. Journal of Neuromuscular Diseases, 7, 1-13.
Tutz, G., & Binder, H. (2005). Localized classification. Statistics and Computing, 15, 155-166.
West, M., & Harrison, P. (1989). Bayesian forecasting and dynamic models. Springer Science & Business Media.
Yıldız, C., Heinonen, M., & Lähdesmäki, H. (2019). ODE2$^2$VAE: Deep generative second order ODEs with Bayesian neural networks. arXiv preprint: https://arxiv.org/abs/1905.10994