This document summarizes a master thesis project trying to bring a new solution to hemiplegia rehabilitation, one of the numerous consequences of strokes. A hemiplegic patients observe paralysis on one side of their body, and as so, loses autonomy and their quality of life decreases. In this study, we decided to only focus on the hand rehabilitation aspect. However, there is a clear tendency in stroke patients to stop training regularly when returning home from the hospital and the first part of their rehabilitation is over. They often experience demotivation, having the feeling that they will never get back to a fully autonomous person ever again and tend to put their training aside, especially when they do not see clear and visible results anymore. This is also due to the supervised training becoming sparser. All of this results in patients stagnating or even worse, regressing. Thus, we decided to offer a motivating solution for hand rehabilitation at home through gamification.
Background: Upper limb functional deficits are common after stroke and result from motor weakness, ataxia, spasticity, spatial neglect, and poor stamina. Past studies employing a range of commercial gaming systems to deliver rehabilitation to stroke patients provided short-term efficacy but have not yet demonstrated whether or not those games are acceptable, that is, motivational, comfortable, and engaging, which are all necessary for potential adoption and use by patients.
Objective: The goal of the study was to assess the acceptability of a smartphone-based augmented reality game as a means of delivering stroke rehabilitation for patients with upper limb motor function loss.
Methods: Patients aged 50 to 70 years, all of whom experienced motor deficits after acute ischemic stroke, participated in 3 optional therapy sessions using augmented reality therapeutic gaming over the course of 1 week, targeting deficits in upper extremity strength and range of motion. After completion of the game, we administered a 16-item questionnaire to the patients to assess the game’s acceptability; 8 questions were answered by rating on a scale from 1 (very negative experience) to 5 (very positive experience); 8 questions were qualitative.
Results: Patients (n=5) completed a total of 23 out of 45 scheduled augmented reality game sessions, with patient fatigue as the primary factor for uncompleted sessions. Each patient consented to 9 potential game sessions and completed a mean of 4.6 (SE 1.3) games. Of the 5 patients, 4 (80%) completed the questionnaire at the end of their final gaming session. Of note, patients were motivated to continue to the end of a given gaming session (mean 4.25, 95% CI 3.31-5.19), to try other game-based therapies (mean 3.75, 95% CI 2.81-4.69), to do another session (mean 3.50, 95% CI 2.93-4.07), and to perform other daily rehabilitation exercises (mean 3.25, 95% CI 2.76-3.74). In addition, participants gave mean scores of 4.00 (95% CI 2.87-5.13) for overall experience; 4.25 (95% CI 3.31-5.19) for comfort; 3.25 (95% CI 2.31-4.19) for finding the study fun, enjoyable, and engaging; and 3.50 (95% CI 2.52-4.48) for believing the technology could help them reach their rehabilitation goals. For each of the 4 patients, their reported scores were statistically significantly higher than those generated by a random sampling of values (patient 1: P=.04; patient 2: P=.04; patient 4: P=.004; patient 5: P=.04).
Conclusions: Based on the questionnaire scores, the patients with upper limb motor deficits following stroke who participated in our case study found our augmented reality game motivating, comfortable, engaging, and tolerable. Improvements in augmented reality technology motivated by this case study may one day allow patients to work with improved versions of this therapy independently in their own home. We therefore anticipate that smartphone-based augmented reality gaming systems may eventually provide useful postdischarge self-treatment as a supplement to professional therapy for patients with upper limb deficiencies from stroke.
Stroke induces a variety of functional impairments, as well as pain and other ailments, depending on its type and location . Common deficits associated with ischemic stroke include motor function, spatial neglect, and psychological changes . Motor function deficits after stroke often include partial or total loss of function of the upper or lower limbs on a given side, with associated muscle weakness, poor stamina, lack of muscle control, and even paralysis . These deficits impact the patient’s independent lifestyle and decrease their performance of activities of daily living . According to the National Institute of Neurological Disorders and Stroke, the most important part of rehabilitation programs is “carefully directed, well-focused, repetitive practice .”
Patients who engage in rigorous, time-intensive, and challenging therapeutic exercises after ischemic stroke tend to experience greater functional recovery, while if ignored or insufficiently treated, impairments may remain [4,5]. The dosage of motor skill practice correlates to the extent of motor recovery following a stroke . In addition, the type of therapy delivered relative to patient’s impairment determines outcomes after therapy. For example, for those who have upper limb motor impairment, best therapeutic practice modifies the prescribed exercises as the patient’s symptoms evolve [5,6]. Regrettably, patients report their experiences of conventional repetitive stroke rehabilitation therapies as tedious and difficult to hold their interest, which conflicts with the fact that patient motivation is often required to obtain good clinical outcomes [7–10].
Rehabilitation doctors and medical staff, therefore, face a significant problem: how can they provide high intensity therapy in large quantities for upper limb impairments with this seemingly intrinsic motivational deficit? Especially problematic are patient’s therapeutic needs after their discharge from the hospital—their therapeutic needs still exist, but medical staff have substantially reduced access to the patient to provide targeted care. Given the difficulty of this problem, an insufficient percentage of patients regain the full functional potential of their upper limb after ischemic stroke . This regrettable outcome motivates an ongoing search for new therapeutic approaches that provide acceptable (motivational, comfortable, and engaging) experiences, hence, effective therapy, especially at the patient’s home.
Use of commercial augmented reality devices has found recent application in stroke rehabilitation using existing expensive commercial headsets [4,6–17]. However, there are few studies that assay the acceptability of augmented reality gaming system–based patient rehabilitation after stroke [10,12,17–19], and then, only in a cursory fashion. For example, 30 patients recovering from stroke were surveyed for their opinions on game-based rehabilitation, and the researchers concluded that though games for patients recovering from stroke existed, they were primarily designed for efficacy, not entertainment ; they suggest investing in a single, affordable gaming platform for patient rehabilitation after stroke that also focuses on entertainment and provides diverse gaming content . Augmented reality technology and an upper-limb assistive device were tested on 3 individuals recovering from stroke for 6 weeks, and the study reported that both the user and therapist believed that their augmented reality environment was user friendly due to the lightness of the assistive devices and the simplicity of set-up . Finally, a study of 4 patients recovering from stroke who were exposed to several gaming platforms reported that manually adjusting the difficulty of games to provide a challenge and creating games with deeper story lines helped the patients stay motivated to perform their gaming exercises . To the best of our knowledge, our case study is the first of its kind that analyzes the opinions of patients recovering from stroke regarding the problems of current augmented reality–specific game-based rehabilitation systems to provides insight into future designs of augmented reality game-based stroke rehabilitation systems. Augmented reality, provided by one of a variety of device designs, represents one such approach. Augmented reality projects a live camera view of a user’s environment and computer-generated objects with a variety of properties—movement and sound, typically. As an example, Pokémon Go, a smartphone-based augmented reality game, has had documented success sustaining the interest of users for extended periods of time while consistently increasing their physical activity , making augmented reality a prime candidate for facilitating otherwise tedious therapy.
Since patient motivation often drives a larger dosage of rehabilitation therapy, hence, improved clinical outcomes [20,21], we hypothesized that augmented reality deployed on a relatively inexpensive and readily available platform—a smartphone—could provide a motivational, comfortable, and engaging rehabilitation experience. To test this hypothesis, we first developed a candidate rehabilitation game on a smartphone that could encourage a patient’s hand motions through use of simple visual cues with a custom-made app. We then asked patients with acute upper-motor stroke to use this system and report their experiences via a questionnaire that assayed the acceptability of the game in terms of motivation to continue to play, comfort, and engagement.
This acceptability study was conducted at Harborview Medical Center in Seattle, Washington from November 2018 to March 2019. Inpatients who were recovering from an acute ischemic stroke participated and provided consent. These patients had impaired strength as determined by physical and occupational therapists. To be included in the study, they had to have at least antigravity strength in deltoid or biceps muscles as well as the ability to perform internal and external shoulder rotations. All patients in this study had a Medical Research Council manual muscle score of 3 or 4 in the affected limb.
We designed and built an augmented reality game using Unity (Unity Technologies) that is deployable on any modern smartphone with a camera (Table 1 and Figure 1). The game presents users with a view of an augmented reality dolphin swimming under the ocean with the task of capturing fish and feeding turtles, worn on the hand associated with the upper-limb deficit (Multimedia Appendix 1). To experience the game, patients wore an augmented reality headset, which did not obscure the camera mounted on the phone, and a custom device on their hand. We used two headsets—the Google Daydream headset, which required us to remove the front panel that held the phone in place, and the Merge augmented reality/virtual reality headset, which did not require any modification (Figure 1). The game also required users to place the hand associated with their motor deficits within a padded box that replaced their hand as seen in augmented reality with a dolphin (Figure 1). Finally, we required the user to look at a complex landscape through their headset while wearing the padded box and while playing the game. Instead of holding the phone, the headset supported the phone for the user. We built customized controllers with different interior sizes that changed the effective grip strength of the controller; this was important because our patients’ ability to hold the controllers varied. Viewing the complex landscape through the augmented reality system caused our software to create a seascape that contained a turtle, fish, and other underwater flora and fauna (Multimedia Appendix 1). Successful placement of the dolphin over a fish allowed the dolphin to capture the fish. Placement of the dolphin plus fish over the turtle allowed the user to feed the turtle, thereby winning points.
Notably, we used the TeamViewer (TeamViewer AG) app to project the screen view of the patient from the phone to a laptop, so we could see the patient’s view with, however, the complex landscape was also projected in the background, so we could check the viewer’s alignment with the landscape while they played (Figure 1).
Set-up of the game, to ensure that system function was verified, occurred prior to patients using the system. Patients followed verbal directions and instructions from study staff on how to use the system, facilitated by demonstration of the game using the TeamViewer app. Examples of directions included how to start the game, the actions required to pick up the fish, and how to colocate the dolphin plus fish with the turtle for point accumulation. Some patients required physical assistance to adjust the view of the environment. Examples of physical assistance included moving the patient’s chair or wheelchair closer or farther away from the images recognized by the camera (Figure 1).
Rehabilitation Medicine is Changing: Use Tech to Keep Up
The needs of patients are continually evolving just as the aging population continues to grow. Advancements in neurological rehabilitation help top facilities keep the best talent and optimize outcomes in the face of increasing stroke events.
A delicate balance
For neurorehabilitation therapy, there is a delicate balance between available resources and their ever-increasing demand. As demographics change and the global population ages, the healthcare system faces an even heavier economic burden. Experts estimate that stroke rates in Europe will increase by 30% by the year 20501. Improved acute care translates to a growing need for rehab. Limited time with a therapist and a shrinking work force translates to a significant gap in rehabilitation needs versus the availability of care.
A growing senior population
It is a well-known fact that in the coming years, the majority of the world’s population will be advanced in age.2 Many aging individuals will experience health complications such as neurological or cardiovascular diseases that require rehabilitative care.3 As acute medical care and survival rates improve, so does the urgent need for rehabilitation. If current rehabilitation practices do not change, hospitals and other medical facilities will likely struggle to accommodate their patients.
Limited Therapist Time
Reports show that even in top European rehab facilities, only a few hours a day are devoted to hands-on care.4 While intensity and repetition have been shown to produce the best clinical outcomes in neurological or physical rehab programs, the majority of a patient’s time in the hospital is spent idle. To maximize therapy time, a change in how rehabilitation is administered would likely benefit patients and providers tremendously. One such change includes technology-assisted training.
Emerging Trends in Biotech
Robotic rehabilitation has been shown to be as effective, if not more effective than conventional care5,6. In addition to facilitating more intense and thorough rehab for patients, this technology confers benefits such as:
Empowerment – by giving real-time feedback and promoting autonomy, technology helps patients heal themselves. The internet has also led to increased patient knowledge — a significant boon when handled appropriately by health care professionals.
Telemedicine – patients can connect to the best doctors through remote care, allowing them to heal from home. National healthcare systems have successfully reduced the length of inpatient rehabilitation via alternatives like telerehabilitation so that patients can continue their training after discharge.
Gamification – increases patient engagement in rehab practice through play. Not only for pediatric patients, games and virtual reality can help older patients remain motivated to complete rehab programs.
Body sensors – provide real-time, accurate and digital measurements for feedback and optimal care. Incorporating body sensors, virtual reality and gamification can provide an immersive therapy experience with digital precision.
Exoskeletons and prosthetics – enable movement assistance that stabilizes patients and helps them to walk and to complete daily life activities when they would never have been able to achieve this before.
As rehabilitation facilities incorporate new technologies, patient care will become more efficient. Technology enables hospitals to better meet the needs of a growing senior population while preventing therapist burnout in therapists — ultimately making world-class care a reality.
1 Norrving B, Barrick J, Davalos A, et al. Action Plan for Stroke in Europe 2018-2030. Eur Stroke J. 2018;3(4):309–336. doi:10.1177/2396987318808719 2 Beard JR, Officer A, de Carvalho IA, et al. The World report on ageing and health: a policy framework for healthy ageing. Lancet. 2016;387(10033):2145–2154. doi:10.1016/S0140-6736(15)00516-4 3 Béjot Y, Bailly H, Graber M, Garnier L, Laville A, Dubourget L, Mielle N, Chevalier C, Durier J, Giroud M. Impact of the Ageing Population on the Burden of Stroke: The Dijon Stroke Registry. Neuroepidemiology. 2019;52(1-2):78-85. doi: 10.1159/000492820. Epub 2019 Jan 2. PubMed PMID: 30602168. 4 De Wit L, Putman K, Dejaeger E, Baert I, Berman P, Bogaerts K, Brinkmann N, Connell L, Feys H, Jenni W, Kaske C, Lesaffre E, Leys M, Lincoln N, Louckx F, Schuback B, Schupp W, Smith B, De Weerdt W. Use of time by stroke patients: a comparison of four European rehabilitation centers. Stroke. 2005 Sep;36(9):1977-83. doi: 10.1161/01.STR.0000177871.59003.e3. Epub 2005 Aug 4. PubMed PMID: 16081860. 5 Mehrholz, J., S. Thomas, C. Werner, J. Kugler, M. Pohl and B. Elsner (2017). “Electromechanical-Assisted Training for Walking after Stroke (Update).” Cochrane Database Syst Rev 5: Cd006185. 6 Mehrholz, J., M. Pohl, T. Platz, J. Kugler and B. Elsner (2018). “Electromechanical and robot-assisted arm training for improving activities of daily living, arm function, and arm muscle strength after stroke.” Cochrane Database Syst Rev 9: CD006876.
Stroke is one of the leading causes of disability with over three-quarters of patients experiencing an upper limb impairment varying in severity. Early, intense, and frequent physical rehabilitation is important for quicker recovery of the upper limbs and the prevention of further deterioration of their upper limb impairment. Rehabilitation begins almost immediately at the hospital. Once released from the hospital it is intended that patients continue their rehabilitation program at home supported by a community stroke team. However, there are two main barriers to rehabilitation continuing effectively at this stage. The first is limited contact with a physiotherapist or occupational therapist to guide and support an intensive rehabilitation programme. The second is that conventional rehabilitation is tough to maintain immediately after stroke due to fatigue, lack of concentration, depression and other effects. Stroke patients can find exercises monotonous and tiring, and a lack of motivation can result in patients failing to engage fully with their treatment. Lack of participation in prescribed rehabilitation exercises may affect recovery or cause deterioration of mobility.
This thesis examines the hypothesis that upper limb stroke rehabilitation can be made more accessible and enjoyable through the use of modern commercial virtual reality (VR) hardware, with personalised models of user hand motion adapted to user capability over time, and VR games with tasks that utilise natural hand gestures as input controls to execute personalised physical rehabilitation exercises. To support the investigation of this hypothesis a novel adaptive, gamebased, virtual reality (VR) rehabilitation system has been designed and developed for self-managed rehabilitation. Hands are tracked using a Leap Motion Controller, with hand movements and gestures used as in input controller for VR tasks. A user-centred design methodology was adopted, and the final version of the system was evolved through several versions and iterative testing and feedback through trials with able-bodied testers, stroke survivor volunteers, and practising clinicians.
A key finding of the research was that an adapted form of Fitts’s law, that models difficulty of reaching and touching objects in 3D interaction spaces, could be used to profile movement capability for able-bodied people and stroke patients vii in upper arm VR stroke rehabilitation. It was also found that even when Fitts’s law was less effective, that the statistics of the regression quality were still informative in profiling users. Fitts law regression statistics along with information on task performance (such as percentage of hits) could be used to adapt task difficulty or advising rest. Further, it was found that multiple regression could provide better movement capability profiles with a modified form of Fitts law to account for varying degrees of difficulty due to the angles of motion in 3D space. In addition, a novel approach was developed which profiled sectors of the 3D VR interaction space separately, rather than treat movement through the whole space as being equally difficult. This approach accounts for some stroke patients having more difficulty moving in some directions than others, e.g. up and left. Results demonstrate that this has potential but may need to be investigated further with stroke patients and with larger numbers of people.
The VR system that utilised the movement capability model was evolved over time with a user-centred design methodology, with input from able-bodied people, stroke patients, and clinicians. A final longitudinal study investigated the suitability of three bespoke games, the usability of the system over a longer time, and the effectiveness of the movement profiler and adaptive system. Throughout this experiment, the system provided informative user movement profile variations that could identify unique movement behaviour traits in individuals. Results showed that user performance varied over time and the adaptive system proved effective in changing the difficulty of the tasks for individuals over multiple sessions. The VR rehabilitation games incorporated enhanced gameplay and feedback, and users expressed enjoyment with the interactive experience. Throughout all of the experiments, users enjoyed wearing a VR headset, preferring it over a standard PC monitor. Most users subjectively felt that they were more effective in completing tasks within VR, and results from experiments provided empirical evidence to support this view. Results within this thesis support the proposal that an appropriately designed, adaptive gamebased VR system can provide an accessible, personalised and enjoyable rehabilitation system that can motivate more regular rehabilitation participation and promote improved motor function.
Proprioception or body awareness is an essential sense that aids in the neural control of movement. Proprioceptive impairments are commonly found in people with neurological conditions such as stroke and Parkinson’s disease. Such impairments are known to impact the patient’s quality of life. Robot-aided proprioceptive training has been proposed and tested to improve sensorimotor performance. However, such robot-aided exercises are implemented similar to many physical rehabilitation exercises, requiring task-specific and repetitive movements from patients. Monotonous nature of such repetitive exercises can result in reduced patient motivation, thereby, impacting treatment adherence and therapy gains. Gamification of exercises can make physical rehabilitation more engaging and rewarding. In this work, we discuss our ongoing efforts to develop a game that can accompany a robot-aided wrist proprioceptive training exercise.
Proprioception, the sense of body awareness, is essential for normal motor function. Proprioceptive deficits are common in neurological conditions [Coupar et al. 2012; Konczak et al. 2009]. Such deficits cause a decline in precision of goal-directed movements, and altered postural and spinal reflexes resulting in balance and gait problems [Rothwell et al. 1982]. Proprioceptive training is an intervention aiming to improve proprioceptive function [Aman et al. 2015]. Previous work has established the efficacy of a robot-aided proprioceptive training using WristBot [Elangovan et al. 2017, 2018, 2019]. The WristBot (Figure 1. Left) is a three degrees-of-freedom (3-DoF) exoskeleton robot that allows full range of motion (ROM), delivers precise haptic, position, and velocity stimuli at the wrist, and accurately encodes wrist position across time. Additional details about the WristBot can found in [Cappello et al. 2015].
Nevertheless, while the WristBot has demonstrated its efficacy, it shares a limitation that is often encountered in rehabilitation settings. In a clinical setting, patients are often required to perform task-specific and repetitive movements [Kwakkel et al. 1999]. Initial patient enthusiasm to complete such activities rapidly declines as a result of the monotonous nature of movements. Patient engagement can be improved by complementing therapy with a virtual environment (VE). Prior research has shown that users have favored exercises complemented with a VE rather than conventional approaches [Hoffman et al. 2014]. Thus, our project objective is to turn these tedious movements into an interactive VE experience.
2 GAMIFICATION OF PROPRIOCEPTIVE TRAINING
Gamification process accounted for two key considerations: (1) the game should foster patient motivation and attention (2) and be clinically meaningful. To address these objectives, we reviewed the literature on game development [Bond 2014; Fullerton 2018] and identified four essential components: (1) Variability, (2) Feedback, (3) Rewards, and (4) a Compelling Purpose. The user will be gradually exposed to increasing levels of difficulty, which will likely reduce user frustrations. The user will receive meaningful feedback on concurrent metrics (e.g., Optimal ROM), as well as on previous treatment sessions. During game progress, the user will be alerted about deviations from the target movement requirements. Achievement badges will be rewarded to the user upon reaching therapy milestones, such as target ROM. Lastly, to encourage game completion, we establish an interesting backstory and a meaningful character arc for our virtual avatars. The developed game will be adaptable based on the user’s current clinical status, thus, making the game clinically meaningful. The clinician will have the ability to prescribe exercises based on user needs such as 1 DoF vs 3 DoF movements, continuous vs discrete movements, and strength training vs mobility training. WristBot will provide supportive forces aiding the user to achieve therapy milestones.
Gamified exercise is being developed using the Unity Game Engine, Python and libraries which interface with the WristBot. The game closely resembles an endless runner type game (Figure 1. Right) and utilizes the WrsitBot’s 3-DoF functionality to interact with the VE. Wrist flexion, extension, and abduction can be used to traverse their environment. The remaining 3 movements will allow interactions with their VE in unique ways, such as opening/closing doors, crouching, and pulling levers. In the VE, coins are strategically placed to maximize and improve the use of available ROM. Upon contact with either a wall or obstacle, visual feedback will be provided in the form of avatar damage and coin deduction. Consequently, users achieve improved mobility.
In Python, the connection between Unity and the WristBot library is managed through the use of a local WebSocket, a protocol for two-way communication over a single Transmission Control Protocol (TCP) connection [Fette and Melnikov 2011]. Through the WebSocket, reciprocal data are transferred between the WristBot and Unity. For example, wrist kinematic data will be streamed to the game while game progress is being relayed to the WristBot library. Game progress data will be utilized to compute and deliver haptic feedback to the user. Haptic feedback provided in the form of haptic assistance will aid users to improve their available ROM, while haptic resistance will improve muscle strength within the desired ROM. The clinical motive of the game is to transition the user from use of haptic assistance to resistance during game play. WristBot will adapt haptic feedback based on time spent and progress achieved in game play.
3 USABILITY TESTING
Usability testing will be conducted to ensure proper game usage by the clinical population and healthcare professionals. Specifically, the usability testing will evaluate areas such as 1) ease of game play, 2) game efficiency, and 3) user engagement. We will test the assumptions in each of these areas are accurately depicted in game development and met during game play. For example, we expect online visual feedback of deviations from target to help user focus on achieving the movement requirements. The users will be asked to verify the benefits of visual feedback in modifying their movements. Similarly, other assumptions such as performance badges and coins as rewards, and increase in difficulty levels will be evaluated. A common pitfall of usability studies involving physical rehabilitation setting is not recruiting from the representative population, most notably elderly population [Laver et al. 2017] as age has been shown to interfere with interactions in VE [Meldrum et al. 2012]. Therefore, to ensure our game is intuitive, we will recruit representative users from our patient populations.
This project was supported by National Science Foundation Partnerships For Innovation Technology Translation Award to Jürgen Konczak (1919036). Christopher Curry was supported by National Research Trainee-Understanding the Brain: Graduate Training Program in Sensory Science: Optimizing the Information Available for Mind and Brain (1734815).
Joshua E Aman, Naveen Elangovan, I Yeh, Jürgen Konczak, et al. 2015. The effectiveness of proprioceptive training for improving motor function: a systematic review. Frontiers in human neuroscience 8 (2015), 1075.
Jeremy Gibson Bond. 2014. Introduction to Game Design, Prototyping, and Development: From Concept to Playable Game with Unity and C. Addison-Wesley Professional.
Leonardo Cappello, Naveen Elangovan, Sara Contu, Sanaz Khosravani, Jürgen Konczak, and Lorenzo Masia. 2015. Robot-aided assessment of wrist proprioception. Frontiers in human neuroscience 9 (2015), 198.
Fiona Coupar, Alex Pollock, Phil Rowe, Christopher Weir, and Peter Langhorne. 2012. Predictors of upper limb recovery after stroke: a systematic review and meta-analysis. Clinical rehabilitation 26, 4 (2012), 291–313.
Naveen Elangovan, Leonardo Cappello, Lorenzo Masia, Joshua Aman, and Jürgen Konczak. 2017. A robot-aided visuo-motor training that improves proprioception and spatial accuracy of untrained movement. Scientific reports 7, 1 (2017), 17054.
Naveen Elangovan, Paul Tuite, and Jürgen Konczak. 2018. Somatosensory training improves proprioception and untrained motor function in Parkinson’s disease. Frontiers in neurology 9(2018), 1053.
Naveen Elangovan, I-Ling Yeh, Jessica Holst-Wolf, and Jürgen Konczak. 2019. A robot-assisted sensorimotor training program can improve proprioception and motor function in stroke survivors. In 2019 IEEE 16th International Conference on Rehabilitation Robotics (ICORR). IEEE, 660–664.
I. Fette and A. Melnikov. 2011. The WebSocket Protocol. Technical Report.
Tracy Fullerton. 2018. Game design workshop: a playcentric approach to creating innovative games. AK Peters/CRC Press.
Hunter G Hoffman, Walter J Meyer III, Maribel Ramirez, Linda Roberts, Eric J Seibel, Barbara Atzori, Sam R Sharar, and David R Patterson. 2014. Feasibility of articulated arm mounted Oculus Rift Virtual Reality goggles for adjunctive pain control during occupational therapy in pediatric burn patients. Cyberpsychology, Behavior, and Social Networking 17, 6(2014), 397–401.
Jürgen Konczak, Daniel M Corcos, Fay Horak, Howard Poizner, Mark Shapiro, Paul Tuite, Jens Volkmann, and Matthias Maschke. 2009. Proprioception and motor control in Parkinson’s disease. Journal of motor behavior 41, 6 (2009), 543–552.
Gert Kwakkel, Boudewijn J Kollen, and Robert C Wagenaar. 1999. Therapy impact on functional recovery in stroke rehabilitation: a critical review of the literature. Physiotherapy 85, 7 (1999), 377–391.
Kate E Laver, Belinda Lange, Stacey George, Judith E Deutsch, Gustavo Saposnik, and Maria Crotty. 2017. Virtual reality for stroke rehabilitation. Cochrane database of systematic reviews11 (2017).
Dara Meldrum, Aine Glennon, Susan Herdman, Deirdre Murray, and Rory McConn-Walsh. 2012. Virtual reality rehabilitation of balance: assessment of the usability of the Nintendo Wii® Fit Plus. Disability and rehabilitation: assistive technology 7, 3 (2012), 205–210.
JC Rothwell, MM Traub, BL Day, JA Obeso, PK Thomas, and CD Marsden. 1982. Manual motor performance in a deafferented man. Brain 105, 3 (1982), 515–542.
Background: Robot-assisted therapy has become a promising technology in the field of rehabilitation for poststroke patients with motor disorders. Motivation during the rehabilitation process is a top priority for most stroke survivors. With current advancements in technology there has been the introduction of virtual reality (VR), augmented reality (AR), customizable games, or a combination thereof, that aid robotic therapy in retaining, or increasing the interests of, patients so they keep performing their exercises. However, there are gaps in the evidence regarding the transition from clinical rehabilitation to home-based therapy which calls for an updated synthesis of the literature that showcases this trend. The present review proposes a categorization of these studies according to technologies used, and details research in both upper limb and lower limb applications.
Objective: The goal of this work was to review the practices and technologies implemented in the rehabilitation of poststroke patients. It aims to assess the effectiveness of exoskeleton robotics in conjunction with any of the three technologies (VR, AR, or gamification) in improving activity and participation in poststroke survivors.
Methods: A systematic search of the literature on exoskeleton robotics applied with any of the three technologies of interest (VR, AR, or gamification) was performed in the following databases: MEDLINE, EMBASE, Science Direct & The Cochrane Library. Exoskeleton-based studies that did not include any VR, AR or gamification elements were excluded, but publications from the years 2010 to 2017 were included. Results in the form of improvements in the patients’ condition were also recorded and taken into consideration in determining the effectiveness of any of the therapies on the patients.
Results: Thirty studies were identified based on the inclusion criteria, and this included randomized controlled trials as well as exploratory research pieces. There were a total of about 385 participants across the various studies. The use of technologies such as VR-, AR-, or gamification-based exoskeletons could fill the transition from the clinic to a home-based setting. Our analysis showed that there were general improvements in the motor function of patients using the novel interfacing techniques with exoskeletons. This categorization of studies helps with understanding the scope of rehabilitation therapies that can be successfully arranged for home-based rehabilitation.
Conclusions: Future studies are necessary to explore various types of customizable games required to retain or increase the motivation of patients going through the individual therapies.
Stroke refers to a sudden, often catastrophic neurological event that can lead to long-term adult disability. The American Heart Association (AHA) is responsible for providing up-to-date statistics related to heart disease and stroke. According to Benjamin et al , the AHA released a 2017 statistics report on heart disease and stroke that stated that approximately 795,000 stroke episodes occur in the US each year. With current advancements in medical technology there has been a decrease in the rate of stroke incidents, but it can still cause paralysis and muscle weakness. Such impairments can result in motor deficits that disturb a stroke survivor’s capacity to live independently.
There are several reasons for stroke occurrence, which could be related to an increased risk of a collection of symptoms caused by disorders affecting the brain (eg, dementia) . Various rehabilitation techniques have been used in the area of rehabilitation-based interactive technology to assist patients in recovering from impairments, and those techniques come under the umbrella of conventional therapy, exoskeleton or robot-aided therapy, virtual reality (VR) or augmented reality (AR) therapy, games-based therapy, or a combination of any of these. These forms of therapy can be done either in the clinic or in an in-home setting. In addition to these, there is a new technology known as telerehabilitation  that leverages the use of VR in home settings by providing patients access to real-time rehabilitation services through the internet while they sit at home.
One of the most effective techniques is robot-aided therapy, which has been gradually increasing in use primarily because patients may consider traditional rehabilitation therapy to be tiring and exhaustive. This may decrease their motivation and cohesion to the treatment, thus resulting in only minor improvement in the health of poststroke patients [4–6]. Various experimental evidence suggests that robot-assisted (or exoskeleton) rehabilitation has been effective in keeping patients motivated and interested in treatment for both upper or lower limb impairments [7,8]. With advancements in technology, there has also been an uptake of VR, AR, and Gamification for the purposes of rehabilitation , along with robotic rehabilitation [10,11], primarily to increase engagement, immersion and motivation on behalf of the patient. Both Colombo et al and Alankus et al [12,13] concluded and showed the positive effect of exoskeleton robots and games in poststroke rehabilitation. Wearable devices such as exoskeletons can also relay real-time feedback for any VR-based interactions .
Apart from these studies, Housman et al  showed user satisfaction survey results in which 90% of participants agreed to the fact that robot- or games-assisted therapies were less confusing, and improvements were very easy to track compared to traditional or conventional therapies. Further, it is thought that gamification can increase repetition, engagement, and range of care within the context of rehabilitation [16,17]. Games are not only useful for the field of rehabilitation, but they are also considered to be highly impactful and relevant in other medical and health fields. Russoniello et al  conducted a randomized controlled trial (RCT) study in which the effects of video games on stress-related disorders were tested, with the conclusion being that games were beneficial for their prevention and treatment. In another study, children who had cerebral palsy made use of a game (EyeToy) which was able to improve their upper extremity functions over time .
We present ongoing work to develop a virtual reality environment for the cognitive rehabilitation of patients as a part of their recovery from a stroke. A stroke causes damage to the brain and problem solving, memory and task sequencing are commonly affected. The brain can recover to some extent, however, and stroke patients have to relearn to carry out activities of daily learning. We have created an application called VIRTUE to enable such activities to be practiced using immersive virtual reality. Gamification techniques enhance the motivation of patients such as by making the level of difficulty of a task increase over time. The design and implementation of VIRTUE is presented together with the results of a small acceptability study.
The use of gamification in higher education context has become popular in recent years with one aim of enhancing learning motivation, yet, it is unknown how physiotherapy students perceive gamified education experience. Using gamification together with multi-media patient case studies, this study explored whether and how gamified education motivated physiotherapy students’ learning. It also investigated how other factors such as class design and mechanics affected gamified experience.
Six case studies in the subject Neurological Physiotherapy were transformed from paper-based cases to multi-media cases built by iSpring suite 8.1. Simulated, real or animated clients were used. Gamification mechanics such as leaderboards, scoring and prioritisation were embedded in the case studies. These gamified case studies were used in classes with Year-3 students enrolled in this subject. After taking these classes, 10 students participated in two focus groups and 32 students responded to a survey to share their experiences and perceptions on this pedagogy.
Results showed that students perceived gamified education as motivating since this satisfied their competence and social needs and enhanced their self-efficacy. In addition, authentic patient videos, class activities that allowed conflict resolution and reflection, and the use of leaderboards were enablers in this gamified experience.
Future gamified education in physiotherapy can provide authentic experience through class designs and gamification mechanics to foster learning motivation. A suggested mapping of gamified lessons for physiotherapy education is provided based on the results of this study.
Learning is an inherently human activity that involves many complex active and interactive processes. Motivation appears to be a key driver to both initial and ongoing learning, as well as improved learning outcomes [1, 2, 3, 4]. Gamification, or the use of game elements in non-game contexts , promotes achievement, challenge, goal, competition and collaboration to learning , which in turn motivates learners . Gamification is thought to enhance motivation and engagement through three levels of processes: cognitive, psychological/emotional and social [8, 9]. At the cognitive level, learners experience processes such as problem-solving and decision-making . At the psychological/emotional level, learners’ positive emotions (e.g. feeling competent) with certain experiences would wire into their memories to enhance further learning of similar experiences [11, 12]. At the social level, interactions with other learners facilitate knowledge constructions . Gamified education should be structured to promote these processes.
To promote the aforementioned processes, better conceptualisations of gamification are needed. Gamification mechanics are often classified by reward or process-tracking types; namely leaderboards, badges, points (or scores), feedback and prizes [7, 9]. Some educational gamification systems use one type of mechanics while others use a mix-and-match approach. Pedersen and Poulsen  found that feedback and points showed an increase in positive outcomes in terms of learning motivations, while other mechanics warrant further investigations. In addition, it is important to differentiate between game-based learning, gamification and serious game. Game-based learning is the use of games (digital or non-digital) as learning tools , while gamification does not necessarily include a game but embed game elements (such as competitions) in learning tasks . The term serious game is sometimes used interchangeably with game-based learning as it applies to any game with a purpose other than pleasure; here learning fits into this rationale [8, 15, 16]. The focus of this study is on gamification rather than game-based learning and serious game.
Gamification has been applied across different disciplines in higher education, such as computer science, mathematics, language and health education [17, 18, 19]. Currently, there is a lack of literature describing or studying gamification in physiotherapy education. In a recent systematic review on gamification in health care education by Wang, DeMaria , only two out of 48 reviewed studies included physiotherapy students as participants. This paucity warrants investigation in the use of gamification in physiotherapy education given its reported benefits on learning.[…]
Millions of individuals around the world suffer from motor impairment or disability, yet effective, engaging, and cost-effective therapeutic solutions are still lacking. In this work, we propose a game for hand rehabilitation that leverages the therapeutic aspects of music for motor rehabilitation, incorporates the power of gamification to improve adherence to medical treatment, and uses the versatility of devices such as the Leap Motion Controller to track users’ movements. The main characteristics of the game as well as future research directions are outlined.
A stroke is a life-changing event that may end up as a disability, with repercussions on the patient’s quality of life. Stroke rehabilitation therapies are helpful to regain some of the patient’s lost functionality. However, in practice stroke patients may suffer from a gradual loss of motivation. Gamified systems are used to increase user motivation, hence, gamified elements have been implemented into stroke rehabilitation therapies in order to improve patients’ engagement and adherence. This review work focuses on selecting and analyzing developed and validated gamified stroke rehabilitation systems published between 2009 and 2017 to identify the most important features of these systems. After extensive research, 32 articles have met the selection criteria, resulting in a total of 28 unique works. The works were analyzed and a total of 20 features were identified. The features are explained, making emphasis on the works that implement them extensively. Finally, a classification of features based on objectives is proposed, which was used to identify the relationships between features and implementation gaps. It was found that there is a tendency to develop low-cost solutions as in-home therapy systems; to include automated features; provide a diversity of games and use of simple interaction devices. This review allowed the definition of the opportunities for future research direction such as systems addressing the three rehabilitation areas; data analytics to make decisions; motivational content identification based on automatic engagement detection and emotion recognition; and alert systems for patient´s safety.
Brain stroke is a life-changing disease that can have fatal consequences. Stroke survivors may end up with long-term disabilities. These disabilities will depend on the damaged part of the brain and the body functions related to it. Older adults are the population with the highest risk of suffering a stroke and ending up with a disability. This makes of stroke the leading cause of adult disability worldwide [1-4].
Stroke rehabilitation therapy has proven to be useful in helping the patient to regain some of his lost functionality [5-8]. In traditional rehabilitation programs, when the rehabilitation in the hospital is completed, the patients return to their homes, where they should continue with more rehabilitation activities. However, the patient’s adherence is reduced at home. The two main causes for this are: the lack of available resources and tools to sustain training for longer periods; and, a diminishing motivation as repetitive exercises are perceived as tedious and boring [9-12]. Gamified rehabilitation systems have proven to be useful to improve motor and cognitive function and additionally as a tool to motivate patients to adhere to the therapy programme [13-22].
This study focuses on gamified systems dedicated for stroke patients’ upper limb rehabilitation for in-home use. The objectives of this study are: 1) provide a literature review of the developed and tested gamified systems for in-home stroke rehabilitation, between 2009 and 2017; 2) identify and explain the most used features of these systems; 3) provide a simple way to classify the features, in order to identify the relationships between them and the gaps of their implementations. A total of 32 articles have met the selection criteria, which resulted in a total of 28 unique works. From analysis of these studies, a total of 20 features were identified. The remaining of this paper is structured as follows. Section 2 describes the methodology used to find the reviewed works and the database to be used, as well as the selection criteria applied to select research works. Section 3 presents the results of the analysis, with the emphasis on the importance of each feature and the works that implemented them to higher extents. An analytical point of view is discussed in Section 4, where an objective-based classification is proposed, the relationships between the features are presented and additionally, the gaps in the current systems are identified. Finally, Section 5 is dedicated for the conclusion and the future research perspectives.[…]