Abstract
This article presents a software platform to design serious games for the rehabilitation of severe memory loss by means of virtual reality (VR). In particular, the focus is on retrograde amnesia, a condition affecting patient's quality of life usually after brain stroke. Currently, the standard rehabilitation process includes showing pictures of familiar environments to help memory recover, while the proposed rehabilitation solution aims at developing patient-specific serious games for memory loss starting from 3D scanning of familiar environments. The Occipital Structure sensor and the Skanect application have been used for the virtualization of the real objects and the environment. Instead of following the traditional approach to design a video game during which the game logic is specifically developed for a virtual scene and the software code is not meant to be recombined, a modular procedure has been designed using Unity to interface the virtual objects of each acquired environment without modifying the game logic. In addition, the developed solution makes available a set of software modules for patient's monitoring and data management to automatically generate medical reports, which can be easily connected to each new serious game. A test has been performed to assess the main features of the VR platform and its usability recruiting ten testers. Medical personnel evaluated positively the platform, and they highlighted the importance of objective data to improve the ecological validity of the cognitive rehabilitation for retrograde amnesia.
Issue Section:
Special Issue: Highlights of CIE 2021
References
1.
Chin
, J. H.
, and Vora
, N.
, 2014
, “The Global Burden of Neurologic Diseases
,” Neurology
, 83
(4
), pp. 349
–351
. 2.
European Commission Directorate-General for Economic and Financial Affairs
, 2015
, “The 2015 Ageing Report. Economic and Budgetary Projections for the 28 EU Member States (2013–2060).
”3.
Riva
, G.
, Mancuso
, V.
, Cavedoni
, S.
, and Stramba-Badiale
, C.
, 2020
, “Virtual Reality in Neurorehabilitation: A Review of Its Effects on Multiple Cognitive Domains
,” Expert Rev. Med. Devices
, 17
(10
), pp. 1035
–1061
. 4.
Chellappan
, K.
, Mohsin
, N. K.
, Ali
, B. M.
, and Islam
, S. H.
, and S
, M.
, 2012
, “Post-Stroke Brain Memory Assessment Framework
,” 2012 IEEE-EMBS Conference on Biomedical Engineering and Sciences
, Langkawi, Malaysia
, IEEE, pp. 189
–194
. 5.
Schrijvers
, E. M. C.
, Schürmann
, B.
, Koudstaal
, P. J.
, van den Bussche
, H.
, Van Duijn
, C. M.
, Hentschel
, F.
, Heun
, R.
, et al, 2012
, “Genome-Wide Association Study of Vascular Dementia
,” Stroke
, 43
(2
), pp. 315
–319
. 6.
Burgess
, N.
, and Bisby
, J. A.
, 2021
, “Spatial Memory | Neuroscience | Britannica,” https://www.britannica.com/science/spatial-memory, Accessed July 25, 2021.7.
Cantu
, R. C.
, 2001
, “Posttraumatic Retrograde and Anterograde Amnesia: Pathophysiology and Implications in Grading and Safe Return to Play
,” J. Athl. Train.
, 36
(3
), p. 244
.8.
Descloux
, V.
, Bellmann
, A.
, and Maurer
, R.
, 2015
, “Assessment of Topographical Disorientation: First Application of New Tests and Case Report
,” Appl. Neuropsychol. Adult
, 22
(5
), pp. 373
–380
. 9.
Ruggiero
, G.
, Frassinetti
, F.
, Iavarone
, A.
, and Iachini
, T.
, 2014
, “The Lost Ability to Find the Way: Topographical Disorientation After a Left Brain Lesion
,” Neuropsychol.
, 28
(1
), pp. 147
–160
. 10.
Maeshima
, S.
, and Osawa
, A.
, 2021
, Stroke
, Vol. 6
, Exon Publications
, pp. 111
–120
. 11.
Takahashi
, N.
, Kawamura
, M.
, Tanihara
, N.
, Sato
, M.
, Kobayashi
, Y.
, Takahashi
, N.
, Kawamura
, M.
, Tanihara
, N.
, Sato
, M.
, and Kobayashi
, Y.
, 2020
, “Case Study on Focal Retrograde Amnesia Due to Limbic Encephalitis
,” World J. Neurosci.
, 10
(2
), pp. 91
–100
. 12.
Backhaus
, S. L.
, and Durand-Sanchez
, A.
, 2019
, “Traumatic Brain Injury Rehabilitation Case Study,” Physician's Field Guide to Neuropsychology
, Springer
, New York
, pp. 435
–456
.13.
Kwan
, D.
, Kurczek
, J.
, and Rosenbaum
, R. S.
, 2016
, “Specific, Personally Meaningful Cues Can Benefit Episodic Prospection in Medial Temporal Lobe Amnesia
,” Br. J. Clin. Psychol.
, 55
(2
), pp. 137
–153
. 14.
Lanzoni
, D.
, Vitali
, A.
, Regazzoni
, D.
, and Rizzi
, C.
, 2021
, “A Method to Develop Virtual Reality Platforms for the Medical Rehabilitation of Severe Memory Loss After Brain Stroke
,” International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
, p. V002T02A082
.15.
Maidenbaum
, S.
, Patel
, A.
, Stein
, E.
, and Jacobs
, J.
, 2019
, “Spatial Memory Rehabilitation in Virtual Reality—Extending Findings From Epilepsy Patients to the General Population
,” 2019 International Conference on Virtual Rehabilitation (ICVR)
, IEEE
, July
, pp. 1
–7
.16.
Das Nair
, R.
, Lincoln
, N. B.
, Ftizsimmons
, D.
, Brain
, N.
, Montgomery
, A.
, Bradshaw
, L.
, Drummond
, A.
, et al, 2015
, “Rehabilitation of Memory Following Brain Injury (ReMemBrIn): Study Protocol for a Randomised Controlled Trial
,” Trials
, 16
(1
), pp. 1
–9
. 17.
Bouwmeester
, L.
, van de Wege
, A.
, Haaxma
, R.
, and Snoek
, J. W.
, 2015
, “Rehabilitation in a Complex Case of Topographical Disorientation
,” Neuropsychol. Rehabil.
, 25
(1
), pp. 1
–14
. 18.
Roberts
, C. M.
, Spitz
, G.
, Mundy
, M.
, and Ponsford
, J. L.
, 2019
, “Retrograde Autobiographical Memory From PTA Emergence to Six-Month Follow-Up in Moderate to Severe Traumatic Brain Injury
,” J. Neuropsychiatry Clin. Neurosci.
, 31
(2
), pp. 112
–122
. 19.
Musso
, M. W.
, Gouvier
, W. D.
, and Barker
, A. A.
, 2021
, “Ecological Validity | Psychology | Britannica,” https://www.britannica.com/science/ecological-validity, Accessed August 3, 2021.20.
Withiel
, T. D.
, Stolwyk
, R. J.
, Ponsford
, J. L.
, Cadilhac
, D. A.
, and Wong
, D.
, 2020
, “Effectiveness of a Manualised Group Training Intervention for Memory Dysfunction Following Stroke: A Series of Single Case Studies
,” Disabil. Rehabil.
, 42
(21
), pp. 3033
–3042
. 21.
López
, C. E.
, Cunningham
, J.
, Ashour
, O.
, and Tucker
, C. S.
, 2020
, “Deep Reinforcement Learning for Procedural Content Generation of 3D Virtual Environments
,” ASME J. Comput. Inf. Sci. Eng.
, 20
(5
), p. 051005
. 22.
Coburn
, J. Q.
, Freeman
, I.
, and Salmon
, J. L.
, 2017
, “A Review of the Capabilities of Current Low-Cost Virtual Reality Technology and Its Potential to Enhance the Design Process
,” ASME J. Comput. Inf. Sci. Eng.
, 17
(3
), p. 031013
. 23.
Bartesaghi
, S.
, Colombo
, G.
, and Morone
, S.
, 2015
, “Spatial Augmented Reality and Simulations to Improve Abdominal Aortic Aneurysm Diagnosis and Monitoring
,” Comput. Aided Des. Appl.
, 12
(6
), pp. 803
–810
. 24.
Colombo
, G.
, Facoetti
, G.
, Rizzi
, C.
, and Vitali
, A.
, 2016
, “Mixed Reality to Design Lower Limb Prosthesis
,” Comput. Aided Des. Appl.
, 13
(6
), pp. 799
–807
. 25.
Romagnoli
, C.
, Bordegoni
, M.
, and Ferrise
, F.
, 2016
, “A Multimodal Virtual Environment Based on Haptic Interfaces for Upper-Limb Rehabilitation
,” International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
, American Society of Mechanical Engineers
, p. V01BT02A023
.26.
Micaroni
, L.
, Carulli
, M.
, Ferrise
, F.
, Gallace
, A.
, and Bordegoni
, M.
, 2019
, “An Olfactory Display to Study the Integration of Vision and Olfaction in a Virtual Reality Environment
,” ASME J. Comput. Inf. Sci. Eng.
, 19
(3
), p. 031015
. 27.
Parsons
, T. D.
, and Barnett
, M.
, 2017
, “Validity of a Newly Developed Measure of Memory: Feasibility Study of the Virtual Environment Grocery Store
,” J. Alzheimer’s Dis.
, 59
(4
), pp. 1227
–1235
. 28.
Ingman
, M.
, Virtanen
, J.-P.
, Vaaja
, M. T.
, and Hyyppä
, H.
, 2020
, “A Comparison of Low-Cost Sensor Systems in Automatic Cloud-Based Indoor 3D Modeling
,” Remote Sens.
, 12
(16
), p. 2624
. 29.
Meade
, M. E.
, Meade
, J. G.
, Sauzeon
, H.
, and Fernandes
, M. A.
, 2019
, “Active Navigation in Virtual Environments Benefits Spatial Memory in Older Adults
,” Brain Sci.
, 9
(3
), p. 47
. 30.
Maggio
, M. G.
, Latella
, D.
, Maresca
, G.
, Sciarrone
, F.
, Manuli
, A.
, Naro
, A.
, De Luca
, R.
, and Calabrò
, R. S.
, 2019
, “Virtual Reality and Cognitive Rehabilitation in People With Stroke: An Overview
,” J. Neurosci. Nurs.
, 51
(2
), pp. 101
–105
. 31.
Lecavalier
, N. C.
, Ouellet
, É
, Boller
, B.
, and Belleville
, S.
, 2018
, “Use of Immersive Virtual Reality to Assess Episodic Memory: A Validation Study in Older Adults
,” Neuropsychol. Rehabil.
, 30
(3
), pp. 462
–480
. 32.
Kalantari
, M.
, and Nechifor
, M.
, 2016
, “3D Indoor Surveying—A Low Cost Approach
,” Survey Review
, 49
(353
), pp. 93
–98
. 33.
Dai
, A.
, Chang
, A. X.
, Savva
, M.
, Halber
, M.
, Funkhouser
, T.
, and Niessner
, M.
, 2017
, “ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes
,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
, pp. 5828
–5839
.34.
Vitali
, A.
, Regazzoni
, D.
, Rizzi
, C.
, and Spajani
, A.
, 2021
, “VR Serious Games for Neuro-Cognitive Rehabilitation of Patients With Severe Memory Loss
,” Comput. Des. Appl.
, 18
(6
), pp. 1233
–1246
. 35.
Serino
, S.
, and Repetto
, C.
, 2018
, “New Trends in Episodic Memory Assessment: Immersive 360° Ecological Videos
,” Front. Psychol.
, 9
, p. 1878
. 36.
Winet
, Y. K.
, Tu
, Y.
, Choshen-Hillel
, S.
, and Fishbach
, A.
, 2020
, “Social Exploration: When People Deviate From Options Explored by Others
,” J. Pers. Soc. Psychol.
Copyright © 2021 by ASME
You do not currently have access to this content.
