Photonic neuromorphic accelerators for event-based imaging flow cytometry.
Journal
Scientific reports
ISSN: 2045-2322
Titre abrégé: Sci Rep
Pays: England
ID NLM: 101563288
Informations de publication
Date de publication:
15 10 2024
15 10 2024
Historique:
received:
26
04
2024
accepted:
05
10
2024
medline:
16
10
2024
pubmed:
16
10
2024
entrez:
15
10
2024
Statut:
epublish
Résumé
In this work, we present experimental results of a high-speed label-free imaging cytometry system that seamlessly merges the high-capturing rate and data sparsity of an event-based CMOS camera with lightweight photonic neuromorphic processing. This combination offers high classification accuracy and a massive reduction in the number of trainable parameters of the digital machine-learning back-end. The event-based camera is capable of capturing 1 Gevents/sec, where events correspond to pixel contrast changes, similar to the retina's ganglion cell function. The photonic neuromorphic accelerator is based on a hardware-friendly passive optical spectrum slicing technique that is able to extract meaningful features from the generated spike-trains using a purely analogue version of the convolutional operation. The experimental scenario comprises the discrimination of artificial polymethyl methacrylate calibrated beads, having different diameters, flowing at a mean speed of 0.1 m/sec. Classification accuracy, using only lightweight digital machine-learning schemes has topped at 98.2%. On the other hand, by experimentally pre-processing the raw spike data through the proposed photonic neuromorphic spectrum slicer at a rate of 3 × 10
Identifiants
pubmed: 39406898
doi: 10.1038/s41598-024-75667-9
pii: 10.1038/s41598-024-75667-9
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
24179Subventions
Organisme : EC | Horizon 2020 Framework Programme (EU Framework Programme for Research and Innovation H2020)
ID : 871330
Organisme : EC | Horizon 2020 Framework Programme (EU Framework Programme for Research and Innovation H2020)
ID : 101070195
Informations de copyright
© 2024. The Author(s).
Références
Barteneva, N. S., Fasler-Kan, E. & Vorobjev, I. A. Imaging flow cytometry: coping with heterogeneity in biological systems. J. Histochem. Cytochem. 60, 723–733 (2012).
doi: 10.1369/0022155412453052
Gouda, M. et al. Improving the classification accuracy in label-free flow cytometry using event-based vision and simple logistic regression. IEEE J. Sel. Top. Quantum Electron. 29, 1–8 (2023).
doi: 10.1109/JSTQE.2023.3244040
Blasi, T. et al. Label-free cell cycle analysis for high-throughput imaging flow cytometry. Nat. Commun. 7, 10256 (2016).
doi: 10.1038/ncomms10256
Rane, A. S., Rutkauskaite, J., deMello, A. & Stavrakis, S. High-throughput multi-parametric imaging flow cytometry. Chem 3, 588–602 (2017).
doi: 10.1016/j.chempr.2017.08.005
Zhang, F. et al. Intelligent image de-blurring for imaging flow cytometry. Cytometry A 95, 549–554 (2019).
doi: 10.1002/cyto.a.23771
Isozaki, A. et al. Intelligent image-activated cell sorting 2.0. Lab. Chip 20, 2263–2273 (2020).
doi: 10.1039/D0LC00080A
Vembadi, A., Menachery, A. & Qasaimeh, M. A. Cell cytometry: review and perspective on biotechnological advances. Front. Bioeng. Biotechnol. 7, 147 (2019).
doi: 10.3389/fbioe.2019.00147
Rožanc, J., Finšgar, M. & Maver, U. Progressive use of multispectral imaging flow cytometry in various research areas. The Analyst 146, 4985–5007 (2021).
doi: 10.1039/D1AN00788B
Doan, M. et al. Diagnostic potential of imaging flow cytometry. Trends Biotechnol. 36, 649–652 (2018).
doi: 10.1016/j.tibtech.2017.12.008
Rodrigues, M. A. et al. The in vitro micronucleus assay using imaging flow cytometry and deep learning. Npj Syst. Biol. Appl. 7, 20 (2021).
doi: 10.1038/s41540-021-00179-5
Lau, A. K. S., Shum, H. C., Wong, K. K. Y. & Tsia, K. K. Optofluidic time-stretch imaging—an emerging tool for high-throughput imaging flow cytometry. Lab. Chip 16, 1743–1756 (2016).
doi: 10.1039/C5LC01458A
Tsilikas, I. et al. Time-Stretched Imaging Flow Cytometry and Photonic Neuromorphic Processing for Particle Classification. in 2023 Conference on Lasers and Electro-Optics Europe & European Quantum Electronics Conference (CLEO/Europe-EQEC) 1–1 (IEEE, Munich, Germany, 2023). https://doi.org/10.1109/CLEO/Europe-EQEC57999.2023.10232590 .
Coddington, I., Newbury, N. & Swann, W. Dual-comb spectroscopy. Optica 3, 414 (2016).
doi: 10.1364/OPTICA.3.000414
Mizuno, T. et al. Optical image amplification in dual-comb microscopy. Sci. Rep. 10, 8338 (2020).
doi: 10.1038/s41598-020-64927-z
Lei, C. et al. High-throughput imaging flow cytometry by optofluidic time-stretch microscopy. Nat. Protoc. 13, 1603–1631 (2018).
doi: 10.1038/s41596-018-0008-7
Kiester, A. S., Ibey, B. L., Coker, Z. N., Pakhomov, A. G. & Bixler, J. N. Strobe photography mapping of cell membrane potential with nanosecond resolution. Bioelectrochemistry 142, 107929 (2021).
doi: 10.1016/j.bioelechem.2021.107929
Perot, E., de Tournemire, P., Nitti, D., Masci, J. & Sironi, A. Learning to Detect Objects with a 1 Megapixel Event Camera. in Advances in Neural Information Processing Systems vol. 33 16639–16652 (Curran Associates, Inc., 2020).
Abreu, S., Gouda, M., Lugnan, A. & Bienstman, P. Flow cytometry with event-based vision and spiking neuromorphic hardware. in 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) 4139–4147 (IEEE, Vancouver, BC, Canada, 2023). https://doi.org/10.1109/CVPRW59228.2023.00435 .
Tsilikas, I. et al. Neuromorphic Camera Assisted High-Flow Imaging Cytometry for Particle Classification. in 2023 Conference on Lasers and Electro-Optics Europe & European Quantum Electronics Conference (CLEO/Europe-EQEC) 1–1 (IEEE, Munich, Germany, 2023). https://doi.org/10.1109/CLEO/Europe-EQEC57999.2023.10232061 .
Zhang, Z. et al. Work in Progress: Neuromorphic Cytometry, High-throughput Event-based flow Flow-Imaging. in 2022 8th International Conference on Event-Based Control, Communication, and Signal Processing (EBCCSP) 1–5 (IEEE, Krakow, Poland, 2022). https://doi.org/10.1109/EBCCSP56922.2022.9845595 .
Tsirigotis, A., Tsilikas, I., Sozos, K., Bogris, A. & Mesaritakis, C. Photonic Neuromorphic Accelerator Combined with an Event-Based Neuromorphic Camera for High-Speed Object Classification. in 2023 Conference on Lasers and Electro-Optics Europe & European Quantum Electronics Conference (CLEO/Europe-EQEC) 1–1 (IEEE, Munich, Germany, 2023). https://doi.org/10.1109/CLEO/Europe-EQEC57999.2023.10232077 .
Haessig, G., Berthelon, X., Ieng, S.-H. & Benosman, R. A spiking neural network model of depth from defocus for event-based neuromorphic vision. Sci. Rep. 9, 3744 (2019).
doi: 10.1038/s41598-019-40064-0
Tsirigotis, A. et al. Unconventional integrated photonic accelerators for high-throughput convolutional neural networks. Intell. Comput. 2, 0032 (2023).
doi: 10.34133/icomputing.0032
Lugnan, A., Gooskens, E., Vatin, J., Dambre, J. & Bienstman, P. Machine learning issues and opportunities in ultrafast particle classification for label-free microflow cytometry. Sci. Rep. 10, 20724 (2020).
doi: 10.1038/s41598-020-77765-w
TensorFlow. https://www.tensorflow.org/ .
Kingma, D. P. & Ba, J. Adam: A Method for Stochastic Optimization. Preprint at https://doi.org/10.48550/arXiv.1412.6980 (2017).
Dalal, N. & Triggs, B. Histograms of Oriented Gradients for Human Detection. in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05) vol. 1 886–893 (IEEE, San Diego, CA, USA, 2005).
Li, Y. et al. Deep cytometry: deep learning with real-time inference in cell sorting and flow cytometry. Sci. Rep. 9, 11088 (2019).
doi: 10.1038/s41598-019-47193-6
Yu, H., Yang, L. T., Zhang, Q., Armstrong, D. & Deen, M. J. Convolutional neural networks for medical image analysis: State-of-the-art, comparisons, improvement and perspectives. Neurocomputing 444, 92–110 (2021).
doi: 10.1016/j.neucom.2020.04.157
Sozos, K. et al. High-speed photonic neuromorphic computing using recurrent optical spectrum slicing neural networks. Commun. Eng. 1, 1–10 (2022).
doi: 10.1038/s44172-022-00024-5
Ferrera, M. et al. On-chip CMOS-compatible all-optical integrator. Nat. Commun. 1, 29 (2010).
doi: 10.1038/ncomms1028
Akiba, T., Sano, S., Yanase, T., Ohta, T. & Koyama, M. Optuna: A Next-generation Hyperparameter Optimization Framework. in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining 2623–2631 (Association for Computing Machinery, New York, NY, USA, 2019). https://doi.org/10.1145/3292500.3330701 .
Gallego, G. et al. Event-based vision: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 44, 154–180 (2022).
doi: 10.1109/TPAMI.2020.3008413
Xu, X. et al. 11 TOPS photonic convolutional accelerator for optical neural networks. Nature 589, 44–51 (2021).
doi: 10.1038/s41586-020-03063-0
Feldmann, J. et al. Parallel convolutional processing using an integrated photonic tensor core. Nature 589, 52–58 (2021).
doi: 10.1038/s41586-020-03070-1
Keras: Deep Learning for humans. https://keras.io/ .
TechPowerUp. TechPowerUp https://www.techpowerup.com/gpuz/ (2024).