Non-contact imaging of peripheral hemodynamics during cognitive and psychological stressors.
Acoustic Stimulation
Adult
Blood Volume
Datasets as Topic
Face
Female
Fingers
Hand
Heart Rate
/ physiology
Hemodynamics
Hemoglobins
/ analysis
Humans
Male
Mathematics
Melanins
/ analysis
Photic Stimulation
Photoplethysmography
/ methods
Stress, Psychological
/ diagnostic imaging
Thinking
Vasomotor System
/ physiology
Young Adult
Journal
Scientific reports
ISSN: 2045-2322
Titre abrégé: Sci Rep
Pays: England
ID NLM: 101563288
Informations de publication
Date de publication:
02 07 2020
02 07 2020
Historique:
received:
02
02
2020
accepted:
26
05
2020
entrez:
4
7
2020
pubmed:
4
7
2020
medline:
18
12
2020
Statut:
epublish
Résumé
Peripheral hemodynamics, measured via the blood volume pulse and vasomotion, provide a valuable way of monitoring physiological state. Camera imaging-based systems can be used to measure these peripheral signals without contact with the body, at distances of multiple meters. While researchers have paid attention to non-contact imaging photoplethysmography, the study of peripheral hemodynamics and the effect of autonomic nervous system activity on these signals has received less attention. Using a method, based on a tissue-like model of the skin, we extract melanin [Formula: see text] and hemoglobin [Formula: see text] concentrations from videos of the hand and face and show that significant decreases in peripheral pulse signal power (by 36% ± 29%) and vasomotion signal power (by 50% ± 26%) occur during periods of cognitive and psychological stress. Via three experiments we show that similar results are achieved across different stimuli and regions of skin (face and hand). While changes in peripheral pulse and vasomotion power were significant the changes in pulse rate variability were less consistent across subjects and tasks.
Identifiants
pubmed: 32616832
doi: 10.1038/s41598-020-67647-6
pii: 10.1038/s41598-020-67647-6
pmc: PMC7331808
doi:
Substances chimiques
Hemoglobins
0
Melanins
0
Types de publication
Journal Article
Validation Study
Langues
eng
Sous-ensembles de citation
IM
Pagination
10884Références
Allen, J. Photoplethysmography and its application in clinical physiological measurement. Physiol. Meas. 28, R1 (2007).
doi: 10.1088/0967-3334/28/3/R01
Wilson, G. M. & Sasse, M. A. Do users always know what’s good for them? Utilising physiological responses to assess media quality. In People and computers XIV—Usability or else!, 327–339 (Springer, Berlin, 2000).
Scheirer, J., Fernandez, R., Klein, J. & Picard, R. W. Frustrating the user on purpose: A step toward building an affective computer. Interact. Comput. 14, 93–118 (2002).
doi: 10.1016/S0953-5438(01)00059-5
McDuff, D. & Kapoor, A. Visceral machines: Risk-aversion in reinforcement learning with intrinsic physiological rewards. In International Conference on Learning Representations (2019).
Fredrickson, L. & Levenson, R. W. Positive emotions speed recovery from the cardiovascular sequelae of negative emotions. Cogn. Emot. 12, 191–220 (1998).
doi: 10.1080/026999398379718
Gross, J. J. Emotion regulation: Affective, cognitive, and social consequences. Psychophysiology 39, 281–291 (2002).
doi: 10.1017/S0048577201393198
Intaglietta, M. Vasomotion and flowmotion: Physiological mechanisms and clinical evidence. Vascular Medicine Review 101–112, (1990).
Nilsson, H. & Aalkjær, C. Vasomotion: Mechanisms and physiological importance. Mol. Interv. 3, 79 (2003).
doi: 10.1124/mi.3.2.79
Stansberry, K. B. et al. Impaired peripheral vasomotion in diabetes. Diabetes Care 19, 715–721 (1996).
doi: 10.2337/diacare.19.7.715
McDuff, D., Estepp, J. R., Piasecki, A. M. & Blackford, E. B. A survey of remote optical photoplethysmographic imaging methods. In 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 6398–6404 (IEEE, New York, 2015).
McDuff, D. Using non-contact imaging photoplethysmography to recover diurnal patterns in heart rate. In 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 6830–6833 (IEEE, New York, 2019).
Verkruysse, W., Svaasand, L. O. & Nelson, J. S. Remote plethysmographic imaging using ambient light. Opt. Express 16, 21434–21445 (2008).
doi: 10.1364/OE.16.021434
Poh, M.-Z., McDuff, D. & Picard, R. W. Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Opt. Express 18, 10762–10774 (2010).
doi: 10.1364/OE.18.010762
Poh, M.-Z., McDuff, D. & Picard, R. W. Advancements in noncontact, multiparameter physiological measurements using a webcam. IEEE Trans. Biomed. Eng. 58, 7–11 (2011).
doi: 10.1109/TBME.2010.2086456
Wang, W., Den Brinker, A., Stuijk, S. & De Haan, G. Algorithmic Principles of Remote-PPG. IEEE Trans. Biomed. Eng.PP, 1–12, https://doi.org/10.1109/TBME.2016.2609282 (2016).
Nowara, E. M., Marks, T. K., Mansour, H. & Veeraraghavany, A. Sparseppg: Towards driver monitoring using camera-based vital signs estimation in near-infrared. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 1353–135309 (IEEE, New York, 2018).
Chen, W. & McDuff, D. Deepphys: Video-based physiological measurement using convolutional attention networks. arXiv preprint arXiv:1805.07888 (2018).
McDuff, D., Gontarek, S. & Picard, R. Improvements in remote cardio-pulmonary measurement using a five band digital camera. IEEE Trans. Biomed. Eng.61, 2593–2601 (2014).
doi: 10.1109/TBME.2014.2323695
Bousefsaf, F., Maaoui, C. & Pruski, A. Remote assessment of the heart rate variability to detect mental stress. In 2013 7th International Conference on Pervasive Computing Technologies for Healthcare and Workshops, 348–351 (IEEE, New York, 2013).
McDuff, D., Gontarek, S. & Picard, R. Remote measurement of cognitive stress via heart rate variability. In Engineering in Medicine and Biology Society (EMBC), 2014 Annual International Conference of the IEEE, 2957–2960 (IEEE, New York, 2014).
McDuff, D., Hernandez, J., Gontarek, S. & Picard, R. W. Cogcam: Contact-free measurement of cognitive stress during computer tasks with a digital camera. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, 4000–4004 (ACM, 2016).
Puri, C., Olson, L., Pavlidis, I., Levine, J. & Starren, J. Stresscam: non-contact measurement of users’ emotional states through thermal imaging. In CHI’05 extended abstracts on Human factors in computing systems, 1725–1728 (ACM, 2005).
Nishidate, I. et al. Noninvasive imaging of human skin hemodynamics using a digital red-green-blue camera. J. Biomed. Opt. 16, 086012 (2011).
doi: 10.1117/1.3613929
McDuff, D., Blackford, E. B., Estepp, J. R. & Nishidate, I. A fast non-contact imaging photoplethysmography method using a tissue-like model. In Optical Diagnostics and Sensing XVIII: Toward Point-of-Care Diagnostics, vol. 10501, 105010Q (International Society for Optics and Photonics, 2018).
Takano, C. & Ohta, Y. Heart rate measurement based on a time-lapse image. Med. Eng. Phys. 29, 853–857 (2007).
doi: 10.1016/j.medengphy.2006.09.006
Scully, C. et al. Physiological parameter monitoring from optical recordings with a mobile phone. IEEE Trans. Biomed. Eng. 59, 303–306 (2012).
doi: 10.1109/TBME.2011.2163157
Tarassenko, L. et al. Non-contact video-based vital sign monitoring using ambient light and auto-regressive models. Physiol. Meas. 35, 807 (2014).
doi: 10.1088/0967-3334/35/5/807
Shao, D. et al. Noncontact monitoring breathing pattern, exhalation flow rate and pulse transit time. IEEE Trans. Biomed. Eng. 61, 2760 (2014).
doi: 10.1109/TBME.2014.2327024
Tarvainen, M. P. et al. An advanced detrending method with application to hrv analysis. IEEE Trans. Biomed. Eng. 49, 172–175 (2002).
doi: 10.1109/10.979357
Nishidate, I. et al. Visualizing of skin chromophore concentrations by use of rgb images. Opt. Lett. 33, 2263–2265 (2008).
doi: 10.1364/OL.33.002263
Sun, Y. & Thakor, N. Photoplethysmography revisited: from contact to noncontact, from point to imaging. IEEE Trans. Biomed. Eng. 63, 463–477 (2015).
doi: 10.1109/TBME.2015.2476337
Hjortskov, N. et al. The effect of mental stress on heart rate variability and blood pressure during computer work. Eur. J. Appl. Physiol. 92, 84–89 (2004).
doi: 10.1007/s00421-004-1055-z
Moses, Z. B., Luecken, L. J. & Eason, J. C. Measuring task-related changes in heart rate variability. In 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 644–647 (IEEE, New York, 2007).
Bousefsaf, F., Maaoui, C. & Pruski, A. Remote detection of mental workload changes using cardiac parameters assessed with a low-cost webcam. Comput. Biol. Med. 53, 154–163 (2014).
doi: 10.1016/j.compbiomed.2014.07.014
Elgendi, M. et al. Frequency analysis of photoplethysmogram and its derivatives. Comput. Methods Prog. Biomed. 122, 503–512 (2015).
doi: 10.1016/j.cmpb.2015.09.021
McDuff, D., Gontarek, S. & Picard, R. Remote detection of photoplethysmographic systolic and diastolic peaks using a digital camera. IEEE Trans. Biomed. Eng. 61, 2948–2954 (2014).
doi: 10.1109/TBME.2014.2340991
Lyu, Y. et al. Measuring photoplethysmogram-based stress-induced vascular response index to assess cognitive load and stress. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, 857–866 (ACM, 2015).
Liu, C., Torralba, A., Freeman, W. T., Durand, F. & Adelson, E. H. Motion magnification. ACM Trans. Graph. 24, 519–526 (2005).
doi: 10.1145/1073204.1073223
Wu, H.-Y. et al. Eulerian video magnification for revealing subtle changes in the world. ACM Trans. Graph. 31, 65 (2012).
doi: 10.1145/2185520.2185561
Wadhwa, N., Rubinstein, M., Durand, F. & Freeman, W. T. Phase-based video motion processing. ACM Trans. Graph. 32, 80 (2013).
doi: 10.1145/2461912.2461966
Oh, T.-H. et al. Learning-based video motion magnification. In Proceedings of the European Conference on Computer Vision (ECCV) 633–648, (2018).
Chen, W. & McDuff, D. Deepmag: Source specific motion magnification using gradient ascent. arXiv preprint arXiv:1808.03338 (2018).
Billman, G. E. The lf/hf ratio does not accurately measure cardiac sympatho-vagal balance. Front. Physiol. 4, 26 (2013).
pubmed: 23431279
pmcid: 3576706
Sun, Y., Hu, S., Azorin-Peris, V., Kalawsky, R. & Greenwald, S. Noncontact imaging photoplethysmography to effectively access pulse rate variability. J. Biomed. Opt. 18, 061205 (2013).
doi: 10.1117/1.JBO.18.6.061205
Jacques, S. L. & McAuliffe, D. J. The melanosome: Threshold temperature for explosive vaporization and internal absorption coefficient during pulsed laser irradiation. Photochem. Photobiol. 53, 769–775 (1991).
doi: 10.1111/j.1751-1097.1991.tb09891.x
Nishidate, I. et al. Rgb camera-based noncontact imaging of plethysmogram and spontaneous low-frequency oscillation in skin perfusion before and during psychological stress. In Optical Diagnostics and Sensing XIX: Toward Point-of-Care Diagnostics, vol. 10885, 1088507 (International Society for Optics and Photonics, 2019).
Poh, M.-Z., Swenson, N. C. & Picard, R. W. A wearable sensor for unobtrusive, long-term assessment of electrodermal activity. IEEE Trans. Biomed. Eng. 57, 1243–1252 (2010).
doi: 10.1109/TBME.2009.2038487
Sackler, A., Weltman, A., Bradshaw, M. & Jurtshuk, P. Endocrine changes due to auditory stress. Eur. J. Endocrinol. 31, 405–418 (1959).
doi: 10.1530/acta.0.XXXI0405
Rojahn, J. & Gerhards, F. Subjective stress sensitivity and physiological responses to an aversive auditory stimulus in migraine and control subjects. J. Behav. Med. 9, 203–212 (1986).
doi: 10.1007/BF00848478
Estepp, J. R., Blackford, E. B. & Meier, C. M. Recovering pulse rate during motion artifact with a multi-imager array for non-contact imaging photoplethysmography. In 2014 IEEE International Conference on Systems, Man and Cybernetics (SMC), 1462–1469 (IEEE, New York, 2014).
Fitzpatrick, T. B. The validity and practicality of sun-reactive skin types i through vi. Arch. Dermatol. 124, 869–871 (1988).
doi: 10.1001/archderm.1988.01670060015008
de Haan, G. & van Leest, A. Improved motion robustness of remote-ppg by using the blood volume pulse signature. Physiol. Meas. 35, 1913 (2014).
doi: 10.1088/0967-3334/35/9/1913