The power of quantum neural networks.
Journal
Nature computational science
ISSN: 2662-8457
Titre abrégé: Nat Comput Sci
Pays: United States
ID NLM: 101775476
Informations de publication
Date de publication:
Jun 2021
Jun 2021
Historique:
received:
20
11
2020
accepted:
14
05
2021
medline:
1
6
2021
pubmed:
1
6
2021
entrez:
13
1
2024
Statut:
ppublish
Résumé
It is unknown whether near-term quantum computers are advantageous for machine learning tasks. In this work we address this question by trying to understand how powerful and trainable quantum machine learning models are in relation to popular classical neural networks. We propose the effective dimension-a measure that captures these qualities-and prove that it can be used to assess any statistical model's ability to generalize on new data. Crucially, the effective dimension is a data-dependent measure that depends on the Fisher information, which allows us to gauge the ability of a model to train. We demonstrate numerically that a class of quantum neural networks is able to achieve a considerably better effective dimension than comparable feedforward networks and train faster, suggesting an advantage for quantum machine learning, which we verify on real quantum hardware.
Identifiants
pubmed: 38217237
doi: 10.1038/s43588-021-00084-1
pii: 10.1038/s43588-021-00084-1
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
403-409Informations de copyright
© 2021. The Author(s), under exclusive licence to Springer Nature America, Inc.
Références
Goodfellow, I., Bengio, Y. & Courville, A. Deep Learning (MIT Press, 2016); http://www.deeplearningbook.org
Baldi, P. & Vershynin, R. The capacity of feedforward neural networks. Neural Networks 116, 288–311 (2019).
doi: 10.1016/j.neunet.2019.04.009
Dziugaite, G. K. & Roy, D. M. Computing nonvacuous generalization bounds for deep (stochastic) neural networks with many more parameters than training data. In Proc. 33rd Conference on Uncertainty in Artificial Intelligence (UAI, 2017).
Schuld, M. Supervised Learning with Quantum Computers (Springer, 2018).
Zoufal, C., Lucchi, A. & Woerner, S. Quantum generative adversarial networks for learning and loading random distributions. npj Quant. Inf. 5, 1–9 (2019).
Romero, J., Olson, J. P. & Aspuru-Guzik, A. Quantum autoencoders for efficient compression of quantum data. Quant. Sci. Technol. 2, 045001 (2017).
doi: 10.1088/2058-9565/aa8072
Dunjko, V. & Briegel, H. J. Machine learning & artificial intelligence in the quantum domain: a review of recent progress. Rep. Prog. Phys. 81, 074001 (2018).
doi: 10.1088/1361-6633/aab406
Ciliberto, C. et al. Quantum machine learning: a classical perspective. Proc. Roy. Soc. A 474, 20170551 (2018).
doi: 10.1098/rspa.2017.0551
Killoran, N. et al. Continuous-variable quantum neural networks. Phys. Rev. Res. 1, 033063 (2019).
doi: 10.1103/PhysRevResearch.1.033063
Schuld, M., Sinayskiy, I. & Petruccione, F. The quest for a quantum neural network. Quant. Inf. Proc. 13, 2567–2586 (2014).
doi: 10.1007/s11128-014-0809-8
Farhi, E. & Neven, H. Classification with quantum neural networks on near term processors. Quant. Rev. Lett. 1, 2 (2020).
Aaronson, S. Read the fine print. Nat. Phys. 11, 291–293 (2015).
doi: 10.1038/nphys3272
Vapnik, V. The Nature of Statistical Learning Theory Vol. 8, 1–15 (Springer, 2000).
Vapnik, V. N. & Chervonenkis, A. Y. On the uniform convergence of relative frequencies of events to their probabilities. Theory Probab. Appl. 16, 264–280 (1971).
doi: 10.1137/1116025
Sontag, E. D Neural Networks and Machine Learning 69–95 (Springer, 1998).
Vapnik, V., Levin, E. & Cun, Y. L. Measuring the VC-dimension of a learning machine. Neural Comput. 6, 851–876 (1994).
doi: 10.1162/neco.1994.6.5.851
Neyshabur, B., Bhojanapalli, S., McAllester, D. & Srebro, N. Exploring generalization in deep learning. In Advances in Neural Information Processing Systems 30, 5947–5956 (NIPS, 2017).
Arora, S., Ge, R., Neyshabur, B. & Zhang, Y. Stronger generalization bounds for deep nets via a compression approach. In Proc. 35th International Conference on Machine Learning Vol. 80, 254–263 (PMLR, 2018); http://proceedings.mlr.press/v80/arora18b.html
Wright, L. G. & McMahon, P. L. The capacity of quantum neural networks. In Conference on Lasers and Electro-Optics JM4G.5 (Optical Society of America, 2020); http://www.osapublishing.org/abstract.cfm?URI=CLEO_QELS-2020-JM4G.5
Du, Y., Hsieh, M.-H., Liu, T. & Tao, D. Expressive power of parametrized quantum circuits. Phys. Rev. Res. 2, 033125 (2020).
doi: 10.1103/PhysRevResearch.2.033125
Huang, H.-Y. et al. Power of data in quantum machine learning. Nat. Commun. 12, 2631 (2021).
doi: 10.1038/s41467-021-22539-9
Berezniuk, O., Figalli, A., Ghigliazza, R. & Musaelian, K. A scale-dependent notion of effective dimension. Preprint at https://arxiv.org/abs/2001.10872 (2020).
Rissanen, J. J. Fisher information and stochastic complexity. IEEE Trans. Inf. Theory 42, 40–47 (1996).
doi: 10.1109/18.481776
Cover, T. M. & Thomas, J. A. Elements of Information Theory (Wiley, 2006).
Nakaji, K. & Yamamoto, N. Expressibility of the alternating layered ansatz for quantum computation. Quantum 5, 434 (2021).
doi: 10.22331/q-2021-04-19-434
Holmes, Z., Sharma, K., Cerezo, M. & Coles, P. J. Connecting ansatz expressibility to gradient magnitudes and barren plateaus. Preprint at https://arxiv.org/abs/2101.02138 (2021).
McClean, J. R., Boixo, S., Smelyanskiy, V. N., Babbush, R. & Neven, H. Barren plateaus in quantum neural network training landscapes. Nat. Commun. 9, 1–6 (2018).
doi: 10.1038/s41467-018-07090-4
Wang, S. et al. Noise-induced barren plateaus in variational quantum algorithms. Preprint at https://arxiv.org/abs/2007.14384 (2020).
Cerezo, M., Sone, A., Volkoff, T., Cincio, L. & Coles, P. J. Cost function dependent barren plateaus in shallow parametrized quantum circuits. Nat. Commun. 12, 1791 (2021).
doi: 10.1038/s41467-021-21728-w
Verdon, G. et al. Learning to learn with quantum neural networks via classical neural networks. Preprint at https://arxiv.org/abs/1907.05415 (2019).
Volkoff, T. & Coles, P. J. Large gradients via correlation in random parameterized quantum circuits. Quant. Sci. Technol. 6, 025008 (2021).
doi: 10.1088/2058-9565/abd891
Skolik, A., McClean, J. R., Mohseni, M., van der Smagt, P. & Leib, M. Layerwise learning for quantum neural networks. Quant. Mach. Intell. 3, 5 (2021).
doi: 10.1007/s42484-020-00036-4
Huembeli, P. & Dauphin, A. Characterizing the loss landscape of variational quantum circuits. Quant. Sci. Technol. 6, 025011 (2021).
doi: 10.1088/2058-9565/abdbc9
Bishop, C. Exact calculation of the Hessian matrix for the multilayer perceptron. Neural Comput. 4, 494–501 (1992).
LeCun, Y. A., Bottou, L., Orr, G. B. & Müller, K.-R. Efficient BackProp 9–48 (Springer, 2012); https://doi.org/10.1007/978-3-642-35289-8_3
Cerezo, M. & Coles, P. J. Higher order derivatives of quantum neural networks with barren plateaus. Quant. Sci. Technol. 6, 035006 (2021).
doi: 10.1088/2058-9565/abf51a
Kunstner, F., Hennig, P. & Balles, L. Limitations of the empirical Fisher approximation for natural gradient descent. In Advances in Neural Information Processing Systems 32 4156–4167 (NIPS, 2019); http://papers.nips.cc/paper/limitations-of-fisher-approximation
Karakida, R., Akaho, S. & Amari, S.-I. Universal statistics of Fisher information in deep neural networks: mean field approach. In Proc. Machine Learning Research Vol. 89, 1032–1041 (PMLR, 2019); http://proceedings.mlr.press/v89/karakida19a.html
Schuld, M., Bocharov, A., Svore, K. M. & Wiebe, N. Circuit-centric quantum classifiers. Phys. Rev. A 101, 032308 (2020).
doi: 10.1103/PhysRevA.101.032308
Schuld, M., Sweke, R. & Meyer, J. J. Effect of data encoding on the expressive power of variational quantum-machine-learning models. Phys. Rev. A 103, 032430 (2021).
doi: 10.1103/PhysRevA.103.032430
Lloyd, S., Schuld, M., Ijaz, A., Izaac, J. & Killoran, N. Quantum embeddings for machine learning. Preprint at https://arxiv.org/abs/2001.03622 (2020).
Cong, I., Choi, S. & Lukin, M. D. Quantum convolutional neural networks. Nat. Phys. 15, 1273–1278 (2019).
doi: 10.1038/s41567-019-0648-8
Amari, S.-I. Natural gradient works efficiently in learning. Neural Comput. 10, 251–276 (1998).
doi: 10.1162/089976698300017746
Liang, T., Poggio, T., Rakhlin, A. & Stokes, J. Fisher–Rao metric, geometry, and complexity of neural networks. In Proc. Machine Learning Research Vol. 89, 888–896 (PMLR, 2019); http://proceedings.mlr.press/v89/liang19a.html
Neyshabur, B., Salakhutdinov, R. R. & Srebro, N. Path-SGD: path-normalized optimization in deep neural networks. In Advances in Neural Information Processing Systems 28, 2422–2430 (NIPS, 2015).
Neyshabur, B., Tomioka, R. & Srebro, N. Norm-based capacity control in neural networks. In Proc. Machine Learning Research Vol. 40, 1376–1401 (PMLR, 2015); http://proceedings.mlr.press/v40/Neyshabur15.html
Bartlett, P. L., Foster, D. J. & Telgarsky, M. J. Spectrally-normalized margin bounds for neural networks. In Advances in Neural Information Processing Systems 30, 6240–6249 (NIPS, 2017); http://papers.nips.cc/paper/7204-spectrally-normalized
Rissanen, J. J. Fisher information and stochastic complexity. IEEE Trans. Inf. Theory 42, 40–47 (1996).
doi: 10.1109/18.481776
Marrero, C. O., Kieferová, M. & Wiebe, N. Entanglement induced barren plateaus. Preprint at https://arxiv.org/abs/2010.15968 (2020).
Havlíček, V. et al. Supervised learning with quantum-enhanced feature spaces. Nature 567, 209–212 (2019).
doi: 10.1038/s41586-019-0980-2
Sim, S., Johnson, P. D. & Aspuru-Guzik, A. Expressibility and entangling capability of parameterized quantum circuits for hybrid quantum-classical algorithms. Adv. Quant. Technol. 2, 1900070 (2019).
doi: 10.1002/qute.201900070
Jia, Z. & Su, H. Information-theoretic local minima characterization and regularization. In Proc. 37th International Conference on Machine Learning Vol. 119, 4773–4783 (PMLR, 2020); http://proceedings.mlr.press/v119/jia20a.html
Virmaux, A. & Scaman, K. Lipschitz regularity of deep neural networks: analysis and efficient estimation. In Advances in Neural Information Processing Systems 31, 3835–3844 (NIPS, 2018); http://papers.nips.cc/paper/lipschitz-regularity-of-deep-neural-networks
Sweke, R. et al. Stochastic gradient descent for hybrid quantum-classical optimization. Quantum 4, 314 (2020).
doi: 10.22331/q-2020-08-31-314
Dua, D. & Graff, C. UCI Machine Learning Repository (2017); http://archive.ics.uci.edu/ml
Abbas, A. et al. amyami187/effective_dimension: The Effective Dimension Code (Zenodo, 2021); https://doi.org/10.5281/zenodo.4732830