Prolonged exposure to mixed reality alters task performance in the unmediated environment.
Augmented reality
Manual pointing
Sensorimotor recalibration
Virtual reality
Journal
Scientific reports
ISSN: 2045-2322
Titre abrégé: Sci Rep
Pays: England
ID NLM: 101563288
Informations de publication
Date de publication:
15 08 2024
15 08 2024
Historique:
received:
04
10
2023
accepted:
31
07
2024
medline:
16
8
2024
pubmed:
16
8
2024
entrez:
15
8
2024
Statut:
epublish
Résumé
The popularity of mixed reality (MR) technologies, including virtual (VR) and augmented (AR) reality, have advanced many training and skill development applications. If successful, these technologies could be valuable for high-impact professional training, like medical operations or sports, where the physical resources could be limited or inaccessible. Despite MR's potential, it is still unclear whether repeatedly performing a task in MR would affect performance in the same or related tasks in the physical environment. To investigate this issue, participants executed a series of visually-guided manual pointing movements in the physical world before and after spending one hour in VR or AR performing similar movements. Results showed that, due to the MR headsets' intrinsic perceptual geometry, movements executed in VR were shorter and movements executed in AR were longer than the veridical Euclidean distance. Crucially, the sensorimotor bias in MR conditions also manifested in the subsequent post-test pointing task; participants transferring from VR initially undershoot whereas those from AR overshoot the target in the physical environment. These findings call for careful consideration of MR-based training because the exposure to MR may perturb the sensorimotor processes in the physical environment and negatively impact performance accuracy and transfer of training from MR to UR.
Identifiants
pubmed: 39147910
doi: 10.1038/s41598-024-69116-w
pii: 10.1038/s41598-024-69116-w
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
18938Informations de copyright
© 2024. The Author(s).
Références
Milgram, P. & Kishino, F. A taxonomy of mixed reality visual displays. IEICE Trans. Inf. Syst. 77, 1321–1329 (1994).
Hwang, G.-J., Chang, C.-C. & Chien, S.-Y. A motivational model-based virtual reality approach to prompting learners’ sense of presence, learning achievements, and higher-order thinking in professional safety training. Br. J. Educ. Technol. 53, 1343–1360 (2022).
doi: 10.1111/bjet.13196
Zikas, P. et al. Virtual reality medical training for COVID-19 swab testing and proper handling of personal protective equipment: Development and usability. Front. Virtual Real. 2, 175 (2022).
doi: 10.3389/frvir.2021.740197
van Biemen, T., Müller, D. & Mann, D. L. Virtual reality as a representative training environment for football referees. Hum. Mov. Sci. 89, 103091 (2023).
doi: 10.1016/j.humov.2023.103091
pubmed: 37084551
Batmaz, A. U., Barrera Machuca, M. D., Sun, J., & Stuerzlinger, W. The effect of the vergence-accommodation conflict on virtual hand pointing in immersive displays. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems 1–15 (2022).
Kelly, J. W. Distance perception in virtual reality: A meta-analysis of the effect of head-mounted display characteristics. IEEE Trans. Vis. Comput. Graph. 1, 1–13. https://doi.org/10.1109/TVCG.2022.3196606 (2022).
doi: 10.1109/TVCG.2022.3196606
Harris, D. J., Buckingham, G., Wilson, M. R. & Vine, S. J. Virtually the same? How impaired sensory information in virtual reality may disrupt vision for action. Exp. Brain Res. 237, 2761–2766 (2019).
doi: 10.1007/s00221-019-05642-8
pubmed: 31485708
pmcid: 6794235
Renner, R. S., Velichkovsky, B. M. & Helmert, J. R. The perception of egocentric distances in virtual environments—A review. ACM Comput. Surv. CSUR 46, 1–40 (2013).
doi: 10.1145/2543581.2543590
Gagnon, H. C. et al. Estimating distances in action space in augmented reality. ACM Trans. Appl. Percept. TAP 18, 1–16 (2021).
doi: 10.1145/3449067
Gagnon, H. C., et al. The effect of feedback on estimates of reaching ability in virtual reality. In 2021 IEEE Virtual Reality and 3D User Interfaces (VR) 798–806 (IEEE, 2021).
Wang, X. M. et al. The geometry of vergence-accommodation conflict in mixed reality systems. Virtual Real. 28, 1 (2024).
doi: 10.1007/s10055-024-00991-4
Elliott, D., Helsen, W. F. & Chua, R. A century later: Woodworth’s (1899) two-component model of goal-directed aiming. Psychol. Bull. 127, 342 (2001).
doi: 10.1037/0033-2909.127.3.342
pubmed: 11393300
Wolpert, D. M., Ghahramani, Z. & Jordan, M. I. An internal model for sensorimotor integration. Science 269, 1880–1882 (1995).
doi: 10.1126/science.7569931
pubmed: 7569931
Ebrahimi, E., et al. Effects of visual and proprioceptive information in visuo-motor calibration during a closed-loop physical reach task in immersive virtual environments. In Proceedings of the ACM Symposium on Applied Perception, 103–110 (2014).
Ebrahimi, E., Altenhoff, B. M., Pagano, C. C., & Babu, S. V. Carryover effects of calibration to visual and proprioceptive information on near field distance judgments in 3d user interaction. In 2015 IEEE Symposium on 3D User Interfaces (3DUI), 97–104 (IEEE, 2015).
Kohm, K., Babu, S. V., Pagano, C. & Robb, A. Objects may be farther than they appear: depth compression diminishes over time with repeated calibration in virtual reality. IEEE Trans. Vis. Comput. Graph. 28, 3907–3916 (2022).
doi: 10.1109/TVCG.2022.3203112
pubmed: 36048992
Mohler, B. J., Creem-Regehr, S. H., & Thompson, W. B. The influence of feedback on egocentric distance judgments in real and virtual environments. In Proceedings of the 3rd Symposium on Applied Perception in Graphics and Visualization, 9–14 (2006).
Krakauer, J. W., Hadjiosif, A. M., Xu, J., Wong, A. L. & Haith, A. M. Motor learning. Compr Physiol 9, 613–663 (2019).
doi: 10.1002/cphy.c170043
pubmed: 30873583
Wang, X. M. & Troje, N. F. Relating visual and pictorial space: Binocular disparity for distance, motion parallax for direction. Vis. Cogn. 31, 107–125 (2023).
doi: 10.1080/13506285.2023.2203528
Wang, X. M., & Troje, N. F. Relating visual and pictorial space: Integration of binocular disparity and motion parallax. J. Vis. (under review). https://doi.org/10.31234/osf.io/cnkvq .
Fernandez-Ruiz, J., Wong, W., Armstrong, I. T. & Flanagan, J. R. Relation between reaction time and reach errors during visuomotor adaptation. Behav. Brain Res. 219, 8–14 (2011).
doi: 10.1016/j.bbr.2010.11.060
pubmed: 21138745
Adams, H. et al. Locomotive recalibration and prism adaptation of children and teens in immersive virtual environments. IEEE Trans. Vis. Comput. Graph. 24, 1408–1417 (2018).
doi: 10.1109/TVCG.2018.2794072
pubmed: 29543159
Mohler, B. J. et al. Calibration of locomotion resulting from visual motion in a treadmill-based virtual environment. ACM Trans. Appl. Percept. TAP 4, 4 (2007).
doi: 10.1145/1227134.1227138
Solini, H. M., Bhargava, A., & Pagano, C. C. Transfer of calibration in virtual reality to both real and virtual environments. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 1943–1947 (SAGE Publications Sage CA: Los Angeles, CA, 2019).
Rieser, J. J., Pick, H. L., Ashmead, D. H. & Garing, A. E. Calibration of human locomotion and models of perceptual-motor organization. J. Exp. Psychol. Hum. Percept. Perform. 21, 480 (1995).
doi: 10.1037/0096-1523.21.3.480
pubmed: 7790829
Warren, W. H., Kay, B. A., Zosh, W. D., Duchon, A. P. & Sahuc, S. Optic flow is used to control human walking. Nat. Neurosci. 4, 213–216 (2001).
doi: 10.1038/84054
pubmed: 11175884
Wright, W. G. et al. Sensorimotor recalibration in virtual environments. Virtual Real. Phys. Mot. Rehabil. 1, 71–94 (2014).
doi: 10.1007/978-1-4939-0968-1_5
Wang, X. M. & Welsh, T. N. TAT-HUM: Trajectory analysis toolkit for human movements in Python. Behav. Res. Methods https://doi.org/10.3758/s13428-024-02378-4 (2024).
doi: 10.3758/s13428-024-02378-4
pubmed: 38689154
Elliott, D., Hansen, S. & Grierson, L. E. Optimising speed and energy expenditure in accurate visually directed upper limb movements. Ergonomics 52, 438–447 (2009).
doi: 10.1080/00140130802707717
pubmed: 19401895
Elliott, D., Hansen, S., Mendoza, J. & Tremblay, L. Learning to optimize speed, accuracy, and energy expenditure: A framework for understanding speed-accuracy relations in goal-directed aiming. J. Mot. Behav. 36, 339–351 (2004).
doi: 10.3200/JMBR.36.3.339-351
pubmed: 15262629
Bootsma, R. J., Marteniuk, R. G., MacKenzie, C. L. & Zaal, F. T. The speed-accuracy trade-off in manual prehension: Effects of movement amplitude, object size and object width on kinematic characteristics. Exp. Brain Res. 98, 535–541 (1994).
doi: 10.1007/BF00233990
pubmed: 8056073
McIntosh, R. D., Mon-Williams, M. & Tresilian, J. R. Grasping at laws: Speed-accuracy trade-offs in manual prehension. J. Exp. Psychol. Hum. Percept. Perform. 44, 1022 (2018).
doi: 10.1037/xhp0000512
pubmed: 29697991
Eadie, A., Gray, L., Carlin, P. & Mon-Williams, M. Modelling adaptation effects in vergence and accommodation after exposure to a simulated virtual reality stimulus. Ophthalmic Physiol. Opt. 20, 242–251 (2000).
doi: 10.1046/j.1475-1313.2000.00499.x
pubmed: 10897346
Hung, G. K. Adaptation model of accommodation and vergence. Ophthalmic Physiol. Opt. 12, 319–326 (1992).
doi: 10.1111/j.1475-1313.1992.tb00404.x
pubmed: 1454369
Hung, G. K., Ciuffreda, K. J. & Rosenfield, M. Proximal contribution to a linear static model of accommodation and vergence. Ophthalmic Physiol. Opt. 16, 31–41 (1996).
doi: 10.1046/j.1475-1313.1996.95001107.x
pubmed: 8729564
Hung, G. K. & Semmlow, J. L. Static behavior of accommodation and vergence: computer simulation of an interactive dual-feedback system. IEEE Trans. Biomed. Eng. 1, 439–447 (1980).
doi: 10.1109/TBME.1980.326752
Singh, G., Ellis, S. R. & Swan, J. E. The effect of focal distance, age, and brightness on near-field augmented reality depth matching. IEEE Trans. Vis. Comput. Graph. 26, 1385–1398 (2018).
doi: 10.1109/TVCG.2018.2869729
pubmed: 30222576
Swan, J. E., Singh, G. & Ellis, S. R. Matching and reaching depth judgments with real and augmented reality targets. IEEE Trans. Vis. Comput. Graph. 21, 1289–1298 (2015).
doi: 10.1109/TVCG.2015.2459895
pubmed: 26340777
Wang, X. M., Nitsche, M., Resch, G., Mazalek, A., & Welsh, T. N. Mixed reality alters motor planning and control. Behav. Brain Res. (under review). https://doi.org/10.31234/osf.io/pxv5t .
Mostefa, M., El Boudadi, L. K., Loukil, A., Mohamed, K., & Amine, D. Design of mobile robot teleoperation system based on virtual reality. In 2015 3rd International Conference on Control, Engineering & Information Technology (CEIT), 1–6 (IEEE, 2015).
Toet, A., Kuling, I. A., Krom, B. N. & Van Erp, J. B. Toward enhanced teleoperation through embodiment. Front. Robot. AI 7, 14 (2020).
doi: 10.3389/frobt.2020.00014
pubmed: 33501183
pmcid: 7805894
Shin, M., Lee, S., Song, S. W. & Chung, D. Enhancement of perceived body ownership in virtual reality-based teleoperation may backfire in the execution of high-risk tasks. Comput. Hum. Behav. 115, 106605 (2021).
doi: 10.1016/j.chb.2020.106605
Faul, F., Erdfelder, E., Buchner, A. & Lang, A.-G. Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behav. Res. Methods 41, 1149–1160 (2009).
doi: 10.3758/BRM.41.4.1149
pubmed: 19897823
R. Giner-Sorolla, et al., Power to detect what? Considerations for planning and evaluating sample size. Personal. Soc. Psychol. Rev. 10888683241228328 (2019).