• In this article, we presented a newly develop sensor glove as an engaging rehabilitation method for individuals with hand dysfunctions.
• The developed system will enable the therapists to monitor the patient’s progress through the player’s score in the game that is recorded in a progressive log.
• The preliminary results supported that the exercise training using the sensor glove is repetitious, functional, and easy to follow and comply with.
Rehabilitation programs that focus on motor recovery of the upper limb require long-term commitment from the clinicians/therapists, require one-to-one caring, and are usually labor-intensive.
Purpose of the Study
To contribute to this area, we have developed a sensored hand glove integrated with a computer game (Flappy Bird) to engage patients playing a game where the subject’s single/multiple fingers are involved, representing fine motor skill occupational therapeutic exercises.
We described the sensored rehab glove, its hardware design, electrical and electronic design and instrumentation, software design, and pilot testing results.
Experimental results supported that the developed rehab glove system can be effectively used to engage a patient playing a computer game (or a mobile phone game) that can record the data (ie, game score, finger flexion/extension angle, time spent in a therapeutic session, etc.) and put it in a format that could be easily read by a therapist or displayed to the therapists/patients in different graph formats.
We introduced a sensored rehab glove for home-based therapy. The exercise training using the glove is repetitious, functional, and easy to follow and comply with.
Background: Upper limb functional deficits are common after stroke and result from motor weakness, ataxia, spasticity, spatial neglect, and poor stamina. Past studies employing a range of commercial gaming systems to deliver rehabilitation to stroke patients provided short-term efficacy but have not yet demonstrated whether or not those games are acceptable, that is, motivational, comfortable, and engaging, which are all necessary for potential adoption and use by patients.
Objective: The goal of the study was to assess the acceptability of a smartphone-based augmented reality game as a means of delivering stroke rehabilitation for patients with upper limb motor function loss.
Methods: Patients aged 50 to 70 years, all of whom experienced motor deficits after acute ischemic stroke, participated in 3 optional therapy sessions using augmented reality therapeutic gaming over the course of 1 week, targeting deficits in upper extremity strength and range of motion. After completion of the game, we administered a 16-item questionnaire to the patients to assess the game’s acceptability; 8 questions were answered by rating on a scale from 1 (very negative experience) to 5 (very positive experience); 8 questions were qualitative.
Results: Patients (n=5) completed a total of 23 out of 45 scheduled augmented reality game sessions, with patient fatigue as the primary factor for uncompleted sessions. Each patient consented to 9 potential game sessions and completed a mean of 4.6 (SE 1.3) games. Of the 5 patients, 4 (80%) completed the questionnaire at the end of their final gaming session. Of note, patients were motivated to continue to the end of a given gaming session (mean 4.25, 95% CI 3.31-5.19), to try other game-based therapies (mean 3.75, 95% CI 2.81-4.69), to do another session (mean 3.50, 95% CI 2.93-4.07), and to perform other daily rehabilitation exercises (mean 3.25, 95% CI 2.76-3.74). In addition, participants gave mean scores of 4.00 (95% CI 2.87-5.13) for overall experience; 4.25 (95% CI 3.31-5.19) for comfort; 3.25 (95% CI 2.31-4.19) for finding the study fun, enjoyable, and engaging; and 3.50 (95% CI 2.52-4.48) for believing the technology could help them reach their rehabilitation goals. For each of the 4 patients, their reported scores were statistically significantly higher than those generated by a random sampling of values (patient 1: P=.04; patient 2: P=.04; patient 4: P=.004; patient 5: P=.04).
Conclusions: Based on the questionnaire scores, the patients with upper limb motor deficits following stroke who participated in our case study found our augmented reality game motivating, comfortable, engaging, and tolerable. Improvements in augmented reality technology motivated by this case study may one day allow patients to work with improved versions of this therapy independently in their own home. We therefore anticipate that smartphone-based augmented reality gaming systems may eventually provide useful postdischarge self-treatment as a supplement to professional therapy for patients with upper limb deficiencies from stroke.
Stroke induces a variety of functional impairments, as well as pain and other ailments, depending on its type and location . Common deficits associated with ischemic stroke include motor function, spatial neglect, and psychological changes . Motor function deficits after stroke often include partial or total loss of function of the upper or lower limbs on a given side, with associated muscle weakness, poor stamina, lack of muscle control, and even paralysis . These deficits impact the patient’s independent lifestyle and decrease their performance of activities of daily living . According to the National Institute of Neurological Disorders and Stroke, the most important part of rehabilitation programs is “carefully directed, well-focused, repetitive practice .”
Patients who engage in rigorous, time-intensive, and challenging therapeutic exercises after ischemic stroke tend to experience greater functional recovery, while if ignored or insufficiently treated, impairments may remain [4,5]. The dosage of motor skill practice correlates to the extent of motor recovery following a stroke . In addition, the type of therapy delivered relative to patient’s impairment determines outcomes after therapy. For example, for those who have upper limb motor impairment, best therapeutic practice modifies the prescribed exercises as the patient’s symptoms evolve [5,6]. Regrettably, patients report their experiences of conventional repetitive stroke rehabilitation therapies as tedious and difficult to hold their interest, which conflicts with the fact that patient motivation is often required to obtain good clinical outcomes [7–10].
Rehabilitation doctors and medical staff, therefore, face a significant problem: how can they provide high intensity therapy in large quantities for upper limb impairments with this seemingly intrinsic motivational deficit? Especially problematic are patient’s therapeutic needs after their discharge from the hospital—their therapeutic needs still exist, but medical staff have substantially reduced access to the patient to provide targeted care. Given the difficulty of this problem, an insufficient percentage of patients regain the full functional potential of their upper limb after ischemic stroke . This regrettable outcome motivates an ongoing search for new therapeutic approaches that provide acceptable (motivational, comfortable, and engaging) experiences, hence, effective therapy, especially at the patient’s home.
Use of commercial augmented reality devices has found recent application in stroke rehabilitation using existing expensive commercial headsets [4,6–17]. However, there are few studies that assay the acceptability of augmented reality gaming system–based patient rehabilitation after stroke [10,12,17–19], and then, only in a cursory fashion. For example, 30 patients recovering from stroke were surveyed for their opinions on game-based rehabilitation, and the researchers concluded that though games for patients recovering from stroke existed, they were primarily designed for efficacy, not entertainment ; they suggest investing in a single, affordable gaming platform for patient rehabilitation after stroke that also focuses on entertainment and provides diverse gaming content . Augmented reality technology and an upper-limb assistive device were tested on 3 individuals recovering from stroke for 6 weeks, and the study reported that both the user and therapist believed that their augmented reality environment was user friendly due to the lightness of the assistive devices and the simplicity of set-up . Finally, a study of 4 patients recovering from stroke who were exposed to several gaming platforms reported that manually adjusting the difficulty of games to provide a challenge and creating games with deeper story lines helped the patients stay motivated to perform their gaming exercises . To the best of our knowledge, our case study is the first of its kind that analyzes the opinions of patients recovering from stroke regarding the problems of current augmented reality–specific game-based rehabilitation systems to provides insight into future designs of augmented reality game-based stroke rehabilitation systems. Augmented reality, provided by one of a variety of device designs, represents one such approach. Augmented reality projects a live camera view of a user’s environment and computer-generated objects with a variety of properties—movement and sound, typically. As an example, Pokémon Go, a smartphone-based augmented reality game, has had documented success sustaining the interest of users for extended periods of time while consistently increasing their physical activity , making augmented reality a prime candidate for facilitating otherwise tedious therapy.
Since patient motivation often drives a larger dosage of rehabilitation therapy, hence, improved clinical outcomes [20,21], we hypothesized that augmented reality deployed on a relatively inexpensive and readily available platform—a smartphone—could provide a motivational, comfortable, and engaging rehabilitation experience. To test this hypothesis, we first developed a candidate rehabilitation game on a smartphone that could encourage a patient’s hand motions through use of simple visual cues with a custom-made app. We then asked patients with acute upper-motor stroke to use this system and report their experiences via a questionnaire that assayed the acceptability of the game in terms of motivation to continue to play, comfort, and engagement.
This acceptability study was conducted at Harborview Medical Center in Seattle, Washington from November 2018 to March 2019. Inpatients who were recovering from an acute ischemic stroke participated and provided consent. These patients had impaired strength as determined by physical and occupational therapists. To be included in the study, they had to have at least antigravity strength in deltoid or biceps muscles as well as the ability to perform internal and external shoulder rotations. All patients in this study had a Medical Research Council manual muscle score of 3 or 4 in the affected limb.
We designed and built an augmented reality game using Unity (Unity Technologies) that is deployable on any modern smartphone with a camera (Table 1 and Figure 1). The game presents users with a view of an augmented reality dolphin swimming under the ocean with the task of capturing fish and feeding turtles, worn on the hand associated with the upper-limb deficit (Multimedia Appendix 1). To experience the game, patients wore an augmented reality headset, which did not obscure the camera mounted on the phone, and a custom device on their hand. We used two headsets—the Google Daydream headset, which required us to remove the front panel that held the phone in place, and the Merge augmented reality/virtual reality headset, which did not require any modification (Figure 1). The game also required users to place the hand associated with their motor deficits within a padded box that replaced their hand as seen in augmented reality with a dolphin (Figure 1). Finally, we required the user to look at a complex landscape through their headset while wearing the padded box and while playing the game. Instead of holding the phone, the headset supported the phone for the user. We built customized controllers with different interior sizes that changed the effective grip strength of the controller; this was important because our patients’ ability to hold the controllers varied. Viewing the complex landscape through the augmented reality system caused our software to create a seascape that contained a turtle, fish, and other underwater flora and fauna (Multimedia Appendix 1). Successful placement of the dolphin over a fish allowed the dolphin to capture the fish. Placement of the dolphin plus fish over the turtle allowed the user to feed the turtle, thereby winning points.
Notably, we used the TeamViewer (TeamViewer AG) app to project the screen view of the patient from the phone to a laptop, so we could see the patient’s view with, however, the complex landscape was also projected in the background, so we could check the viewer’s alignment with the landscape while they played (Figure 1).
Set-up of the game, to ensure that system function was verified, occurred prior to patients using the system. Patients followed verbal directions and instructions from study staff on how to use the system, facilitated by demonstration of the game using the TeamViewer app. Examples of directions included how to start the game, the actions required to pick up the fish, and how to colocate the dolphin plus fish with the turtle for point accumulation. Some patients required physical assistance to adjust the view of the environment. Examples of physical assistance included moving the patient’s chair or wheelchair closer or farther away from the images recognized by the camera (Figure 1).
Tired of using dumbbells for rehabilitation following distal radius fractures? Looking for new interventions to increase client engagement? Look no further than your patient’s smartphone! Incorporate it into exercise routines to help your patients regain wrist balance and to provide proprioceptive input.
Evidence Supports Proprioceptive Activities
Emerging evidence supports the use of proprioceptive activities for distal radius fracture rehabilitation.1A cross-sectional study involving females treated operatively and non-operatively for a distal radius fracture found that participants had significantly less joint position sense in comparison to study controls.2 The proprioceptive limitations correlated highly with functional impairment on the Patient Rated Wrist Evaluation.3
By addressing proprioceptive deficits while encouraging functional wrist range of motion, smartphone applications complement a traditional hand therapy program for individuals requiring skilled therapy following a distal radius fracture.
Some games to consider:
Chopper Lite – Action packed side-scrolling helicopter game where a tilt of the screen flies the chopper.
Labyrinth – Classic labyrinth game in which you must guide a ball through a labyrinth by moving your device.
Tilt Maze Lite – Maze game where a tilt of your device helps a marble through a maze toward the exit. Use different mazes to test wrist balance and timing. The game stores the player’s best time for each maze so patients can track their performance as their wrist heals.
Water Slide Extreme – Unique water slide game featuring tight corners and huge loops that you must navigate by twisting or leaning your device.
Snail Mail – Kart-style racing game in which the player controls a racing snail on a mission to collect packages and deliver them to the farthest reaches of the universe while dodging obstacles such as laser towers, slugs, asteroids, and salt.
The clinician should consider using smartphones as an intervention following distal radius fractures. Skilled hand therapists can assist with appropriate postural mechanics and provide guidelines for the amount of time a patient should devote to gaming.
Rehabilitation at Your Fingertips
Certain smartphone applications can be used to address client-specific deficits, decrease functional concerns, and achieve client-centered goals. Incorporating smartphone gaming in hand therapy may provide motivation and convenience to your clients.
Kristin Valdes, OTD, OT, CHT, is the owner and director of Hand Works Therapy in Sarasota, Florida. She is an associate professor at Gannon University and has been in private practice for over twenty years, specializing in the treatment and rehabilitation of the upper extremity. Her clinical expertise includes treatment of the hand, wrist, elbow, shoulder, splinting, and arthritis. Dr. Valdes has published articles in the Journal of Hand Therapy, Hand, Journal of Manipulative and Physiological Therapeutics, and the Journal of Physical Therapy Science. She serves as a member of the Board of the American Society of Hand Therapists and American Association of Hand Surgery.Dr. Valdes has published chapters on hand, elbow, and shoulder fractures and dislocations in Advanced Concepts of Hand Pathology and Surgery. She is an assistant editor of the Journal of Hand Therapy. Dr. Valdes received her OTD degree with a specialization in hand rehabilitation from Rocky Mountain University of Health Care Professions in Provo, Utah. Dr Valdes is member of the Guatemala Healing Hands Medical Mission.
Algar, L., & Valdes, K. (2014). Using smartphone applications as hand therapy interventions. Journal of Hand Therapy, 27(3), 254–257. doi:10.1016/j.jht.2013.12.009
Karagiannopoulos, C., Sitler, M., Michlovitz, S., & Tierney, R. (2014a). A Descriptive Study on Wrist and Hand Sensori-Motor Impairment and Function Following Distal Radius Fracture Intervention. Journal of Hand Therapy, 27(3), e2–e3. doi:10.1016/j.jht.2013.08.006
Game-based exercise is effective for improving strength and motor function in stroke patients undergoing rehabilitation, and it creates fun and motivation for exercise.
We investigated the effect of game-based exercise on hand strength, motor function, and compliance in stroke patients.
Fifty stroke patients were randomly divided into experimental and control groups. The experimental group performed a game-based hand resistance exercise. This exercise was divided into isotonic and isometric types and was performed 30 min/day, 5 days/week, for 6 weeks with 70% of the 1-repetition maximum. In contrast, the control group was given a traditional manual exercise by the occupational therapist, and the type of exercise and time involved were the same as those in the experimental group. The primary outcome measure was hand strength test measured using a dynamometer. Secondary outcome measures were manual function tests (MFT) and hand function tests using box and block test (BBT). Subject-based reports of motivation, fun, pain/fatigue evaluated on 0 to 10 numeric rating scales were compared between groups.
After training, hand strength, MFT and BBT was improved in the experimental group compared to the control group (P < 0.001, both). Subject-based reports of motivation and fun was significantly greater in the experimental group than the control group (P < 0.001, both), except to pain/fatigue (P = 0.728).
In conclusion, we demonstrated that game-based exercise is more effective than manual exercise in improving muscle strength, motor function, and compliance in stroke patients.
This marks the second consecutive year that the company has received the VirtualTech award, according to a company announcement.
“The Smart Board for Home NextGen is the epitome of the 2019 SFDW Awards theme, and we’re humbled to have won this year after receiving Honorable Mention in the VirtualTech category last year for our Smart Glove for Home,” says Scott Kim, co-founder and CEO of NEOFECT USA, in the release.
“We took every aspect of the patient experience into account when redesigning the Smart Board for Home NextGen,” Kim adds.
“For example, stroke patients’ grip is often weak, so we re-engineered the handle to be more secure. We developed more interactive virtual reality games, like tennis, so patients can have more variety, and also created a dual-player option.”
SFDW is an international design competition that honors projects encouraging thought leadership in design, focusing on “Where Innovation Meets Social Responsibility.”
The awards celebrate and recognize exemplary work in all fields of design, including architecture, interior design, industrial design, communication design, and user experience design.
Twenty-four winning projects and 11 honorable mentions were selected by a jury comprised of professionals—including executives from Lyft, Google, Microsoft, and Fitbit—who reviewed submissions from a pool of applicants from the USA and Europe. Each winning project was judged based on impact, singularity, inclusiveness, social responsibility, ease of use, visual appeal, and feasibility.
Award winners from leading design firms, in-house teams, and creative individuals were honored recently during a ceremony that took place at Pier 27 in San Francisco, the release explains.
“We are extremely excited the San Francisco Design Week Awards returned this year,” states SFDW Executive Director Dawn Zidonis.
“As with last year, the quality of the many entries exceeded our expectations. Congratulations to this year’s outstanding and diverse winners, including NEOFECT.”
NEOFECT has redesigned its Smart Board for Home in reply to feedback from patients recovering from stroke and other musculoskeletal conditions and neurological disorders.
The new Smart Board for Home NextGen includes a smaller surface to help patients use it at home more easily, a redesigned handle to better stabilize the user’s hand and arm, and updated gamified software.
The board size has been reduced from 42 inches to 32 inches so it can fit on most tables. To accommodate the weakened grip of many stroke patients, the redesigned handle includes more straps to better stabilize the user’s arm, ensure appropriate measurement for the post-game metrics, and provide a more secure, comfortable experience, according to the company in a media release.
“We took patient feedback and completely revamped the Smart Board for Home NextGen,” says Scott Kim, co-founder and CEO of San Francisco-based NEOFECT USA.
“This new model still has all the fun, measurable qualities patients can use at home, but now we’ve reduced even more barriers so that people of all abilities can gain back function in their hands and upper arms.”
Patients play games on the Smart Board for Home NextGen by placing their forearm in a cradle and moving their arm across the board. All movements are virtually mimicked on a Bluetooth-connected screen in real time. The gamified software also features an updated AI-powered algorithm to curate a more customized experience for each patient.
The Smart Board for Home NextGen games mimic real-world motions to rehabilitate users’ upper arms and shoulders, including new games like “Air Hawk” and “Tennis.”
Additionally, NEOFECT is developing a dual-player game for patients to use at home, which will be available in summer 2019.
BACKGROUND: Virtual reality and interactive video games could decrease the demands on the time of the therapists. However, the cost of a virtual reality system and the requirement for technical support limits the availability of these systems. Commercial exergames are not specifically designed for therapeutic use, most patients with hemiplegic stroke are either too weak to play the games or develop undesirable compensatory movements.
AIM: To develop Kinect2Scratch games and compare the effects of training with therapist-based training on upper extremity (UE) function of patients with chronic stroke.
DESIGN: A randomized controlled single-blinded trial.
SETTING: An outpatient rehabilitation clinic of a tertiary hospital.
POPULATIONS: Thirty-three patients with chronic hemiplegic stroke.
METHODS: We developed 8 Kinect2Scratch games. The participants were randomly assigned to either a Kinect2Scratch game group or a therapist-based training group. The training comprised 24 sessions of 30 minutes over 12 weeks. The primary outcome measure was the Fugl-Meyer UE scale and the secondary outcome measures were the Wolf Motor Function Test and Motor Activity Log. Patients were assessed at baseline, after intervention, and at the 3-month follow-up. We used the Pittsburgh participation scale (PPS) to assess the participation level of patients at each training session and an accelerometer to assess the activity counts of the affected UE of patients was used at the 12th and 24th training sessions.
RESULTS: Seventeen patients were assigned to the Kinect2Scratch group and 16 were assigned to the therapist-based training group. There were no differences between the two groups for any of the outcome measures post-intervention and at the 3-month follow-up (all p>.05). The level of participation was higher in the Kinect2Scratch group than in the therapist-based training group (PPS 5.25vs. 5.00, p=0.112). The total activity counts of the affected UE was significantly higher in the Kinect2Scratch group than in the therapist-based training group (p<.001).
CONCLUSIONS: Kinect2Scratch game training was feasible, with effects similar to those of therapist-based training on UE function of patients with chronic stroke.
In a world in which our brains are almost constantly overstimulated, many of us may find it challenging to stay focused for extended periods. Researchers from the University of Cambridge in the United Kingdom have now developed an app that trains the mind to maintain concentration.
Research suggests that a newly developed brain training app may improve our concentration and other cognitive skills.
Many, if not most, of us spend our days rapidly switching between competing tasks. We call this “multitasking,” and take pride in how efficient we are in dealing with multiple problems at the same time.
However, multitasking requires that we quickly redirect our focus from one activity to another and then back again, which, in time, can have a detrimental effect on our ability to concentrate.
“We’ve all experienced coming home from work feeling that we’ve been busy all day but unsure what we actually did,” says Prof. Barbara Sahakian from the Department of Psychiatry at the University of Cambridge.
“Most of us spend our time answering emails, looking at text messages, searching social media, trying to multitask. But, instead of getting a lot done, we sometimes struggle to complete even a single task and fail to achieve our goal for the day,” she adds, noting that we may even find it difficult to stay focused on pleasant, relaxing activities, such as watching TV.
Yet, she continues, “For complex tasks, we need to get in the ‘flow’ and stay focused.” So, how can we re-teach our minds to stay focused?
Prof. Sahakian and colleagues believe that they may have found an effective and uncomplicated solution to this problem.
The research team has developed a brain training app called “Decoder,” which can help users improve their concentration, memory, and numerical skills.
The scientists have recently conducted a study to test the effectiveness of their new app, and they now report their results in the journal Frontiers in Behavioral Neuroscience.
An app that improves concentration
In the study, Prof. Sahakian and team worked with a cohort of 75 young and healthy adult participants. The trial spanned 4 weeks, and all the participants took a special test measuring their concentration skills at both the beginning and the end of the study.
As part of the trial, the researchers divided the participants into three groups. They asked one group to play the new Decoder training game, while the second group had to play Bingo, and the third group received no game to play.
Those in the first two groups played their respective games during eight 1-hour sessions over the 4 weeks, and they did so under the researchers’ supervision.
At the end of the trial period, the researchers found that the participants who had played Decoder demonstrated better attention skills than both the participants who had played Bingo and those who had played no game at all.
The researchers state that these improvements were “significant” and comparable to the effects of medication that doctors prescribe for the treatment of attention-impairing conditions, such as attention deficit hyperactivity disorder (ADHD).
App could help with ADHD
In the next step of the trial, Prof. Sahakian and team wanted to test whether Decoder could boost concentration without negatively affecting a person’s ability to shift their attention effectively from one task to another.
To do so, they asked participants who had used Decoder and Bingo to take the Trail Making Test (TMT), which assesses individuals’ attention-shifting capacity. The researchers found that Decoder players performed better on the TMT than Bingo players.
Finally, participants who played Decoder reported higher rates of enjoyment while participating in this activity, as well as stronger motivation and better alertness throughout all their sessions.
“Many people tell me that they have trouble focusing their attention. Decoder should help them improve their ability to do this,” says Prof. Sahakian.
“In addition to healthy people, we hope that the game will be beneficial for patients who have impairments in attention, including those with ADHD or traumatic brain injury. We plan to start a study with traumatic brain injury patients this year,” the researcher also notes.
An ‘evidence-based game’
Cambridge Enterprise recently licensed the new game to app developer Peak, who specialize in the release of brain training apps. Peak have adapted Decoder for the iPad platform, and the game is now available from the App Store as part of the Peak Brain Training package.
George Savulich, another of the current study’s authors, notes that, unlike other apps that claim to train the brain but do not necessarily deliver on their promise, he and his colleagues based the development of Decoder on hard scientific evidence.
“Many brain training apps on the market are not supported by rigorous scientific evidence. Our evidence-based game is developed interactively […]. The level of difficulty is matched to the individual player, and participants enjoy the challenge of the cognitive training.”
“Peak’s version of Decoder is even more challenging than our original test game, so it will allow players to continue to gain even larger benefits in performance over time,” Prof. Sahakian adds.
“By licensing our game, we hope it can reach a wide audience who are able to benefit by improving their attention,” she says.
Recovering after a stroke isn’t easy, but Neofect is here to help patients track their rehabilitation progress with an innovative wearable solution.
At CES 2019, the company exhibited its Rapael Smart Glove, a high-tech rehab device that helps stroke patients improve their hand movements. The device also syncs with an app, where patients can play rehabilitation games and track milestones.
Neofect didn’t disclose a price for the Rapael Smart Glove, but customers can go on the company’s website to buy it. The Rapael Smart Glove is also available for clinics that need stroke rehabilitation equipment.
Using the Rapael Smart Glove is very easy: Gently slide on the device, connect to the Rapael App with a smartphone or tablet, and play a variety of rehabilitation games. The app’s fun games include virtual tennis matches and house painting, and they’re available in different levels to balance challenge and motivation. Plus, the Rapael App collects practice data for patients, so they can track their hand recovery progress.
With the Rapael Smart Glove, patients can practice hand exercises and improve dexterity over time. An advantage of the Rapael Smart Glove is that it can help stroke patients who might not have immediate access to hospitals or physical therapy facilities, so they can work on their hand movements without leaving home.
“We aim to help patients all around the world including, but not limited to, those unable to receive appropriate treatment due to economic or geographic reasons,” says Neofect’s website. “By providing rehab training products and services that are available anytime and anywhere, we are committed to improving patient’s rehab experiences and quality of life.”
More than six million people worldwide die each year from strokes. Every two seconds, someone, somewhere is having one. Not all strokes are fatal, of course. In fact, 80 per cent of stroke victims survive, though many experience one or more serious lingering effects, including paralysis and cognitive and motor impairment. When a stroke occurs, areas of the brain are deprived of oxygen and neural pathways can become damaged. The good news is that the brain is a resourceful organ, and thanks to neural plasticity, it may be possible to relearn forgotten abilities through rehabilitation—targeted repetitive exercises—that helps the neurons re-organize themselves and allows the victim to regain function. The problem is that rehab is hard, and painful, and according to Regan David Petrie, some 69 per cent of stroke patients don’t get the recommended level of rehab activities. This is why the master student at Victoria University of Wellington has been developing an augmented reality (AR) mobile game, an “exergame,” whose purpose is to engage and reward stroke victims in order to keep them engaged in their therapy.
NZ Fauna AR
Petrie’s game was designed using Google’s Tango Augmented Reality platform prior to the search giant switching support to its newer, more consumer-oriented ARCore system. As the game’s player observes his or her surroundings through a mobile device, virtual 3D objects appear to set the scene and with which the player can interact.
(Photo: Petrie, et al)
The game, still under development, is called NZ Fauna AR. As its name implies, it’s designed for stroke victims of New Zealand, leveraging their love of the country’s forests to provide a calming and enjoyable context in which play can occur. Fizzy, a virtual Rowi kiwi, is the AR star of the current iteration of the game.
(Photo: Petrie, et al)
Players gather blueberries and feed them to Fizzy by performing sit-to-stand exercises, an important form of therapy for stroke victims. The most basic actions of the game are:
• standing up to throw berries to Fizzy
• sitting down to collect more berries from an AR bucket on the floor.
There are game controller buttons with interactive elements, but, says Petrie’s thesis, “The game was designed to incorporate minimal touch interactions—this was driven by the interaction model which was comprised of natural physical movements,” that is, standing up and sitting down.
Petrie has plans for at least two other versions of the game:
• Biggie the Tuatara, focusing on stepping exercises
Petrie’s thesis describes the design and goals of the app, and usability testing he completed for NZ Fauna AR.
Petrie began with a round of preliminary testing with three neurological physiotherapists who helped him refine the gameplay to strengthen the rehab it provided. The three endorsed the idea and game, with one saying, “This is what we need to get people to enjoy therapy.”
Petrie tested NZ Fauna AR with a cohort of five stroke victims in two phases. The five subjects were selected as “a user base that represented an audience with a wide range of cognitive and physiological abilities.” Gameplay lasted for 10-15 minutes, followed by a questionnaire in which they recorded their reactions to the experience.
A second test was conducted with two of the five subjects, who used a second version of the game prototype in which Petrie had fixed some flaws revealed in the first round. One of the subjects still had trouble understanding the game, while the other one loved the changes. One reaction to the game was particularly touching, and revealing: “Caroline passionately illustrated that taking the focus off her is important and that her being the caregiver (instead of the care receiver) and coherently helping someone else (the kiwi) was delightful and made the experience meaningful to her.” One can imagine that such a perspective shift would be welcome for anyone sick of being the subject of chronic care.
(Source: Petrie, et al)
Petrie will no doubt continue to refine his game, not least because he’ll probably want to migrate to Google’s latest AR platform. He also hopes to add a multiplayer mode of some sort so stroke patients can do their rehab together, with the added social element making it even more fun.
As to the efficacy of AR for promoting rehabilitation in stroke victims and others who require repetitive exercise to regain lost function, it would seem that the game designers involved will need to capture some Pokémon Go/Super Mario/Angry Birds magic to keep players—the patients—engaged. That’s a challenge for any game developer, but NZ Fauna AR is certainly an interesting and potentially life-changing use of AR.