Impairment of upper extremity function is a common outcome following stroke, to the detriment of lifestyle and employment opportunities. Yet, access to treatment may be limited due to geographical and transportation constraints, especially for those living in rural areas. While stroke rates are higher in these areas, stroke survivors in these regions of the country have substantially less access to clinical therapy. Home therapy could offer an important alternative to clinical treatment, but the inherent isolation and the monotony of self-directed training can greatly reduce compliance.
We developed a 3D, networked multi-user Virtual Environment for Rehabilitative Gaming Exercises (VERGE) system for home therapy. Within this environment, stroke survivors can interact with therapists and/or fellow stroke survivors in the same virtual space even though they may be physically remote. Each user’s own movement controls an avatar through kinematic measurements made with a low-cost, Kinect™ device. The system was explicitly designed to train movements important to rehabilitation and to provide real-time feedback of performance to users and clinicians. To obtain user feedback about the system, 15 stroke survivors with chronic upper extremity hemiparesis participated in a multisession pilot evaluation study, consisting of a three-week intervention in a laboratory setting. For each week, the participant performed three one-hour training sessions with one of three modalities: 1) VERGE system, 2) an existing virtual reality environment based on Alice in Wonderland (AWVR), or 3) a home exercise program (HEP).
Over 85% of the subjects found the VERGE system to be an effective means of promoting repetitive practice of arm movement. Arm displacement averaged 350 m for each VERGE training session. Arm displacement was not significantly less when using VERGE than when using AWVR or HEP. Participants were split on preference for VERGE, AWVR or HEP. Importantly, almost all subjects indicated a willingness to perform the training for at least 2–3 days per week at home.
Multi-user VR environments hold promise for home therapy, although the importance of reducing complexity of operation for the user in the VR system must be emphasized. A modified version of the VERGE system is currently being used in a home therapy study.
Chronic upper extremity impairment is all too common among the more than 7 million stroke survivors in the U.S. . These impairments have disabling effects on all facets of life, including self-care, employment, and leisure activities. Repetitive practice of movement, such as arm movement, is thought to improve outcomes for stroke survivors [2, 3, 4], but access to the clinic for therapy is often limited by geography or lack of transportation. While almost 50 million Americans live in rural areas, 90% of physical and occupational therapists live in major urban areas . Per capita ratios of therapists to overall population are 50% larger in urban as compared to rural regions of the country . Rates of stroke in these rural areas, however, exceed those of major urban areas [7, 8, 9]. Thus, a large number of stroke survivors have limited access to skilled treatment. Data from 21 states found that only 30% of stroke survivors received outpatient rehabilitation, a much lower percentage than that recommended by clinical practice guidelines . Declines seen following discharge from inpatient rehabilitation are undoubtedly exacerbated by limited access to clinical therapy .
Disparity in quality of care has been recognized in the acute treatment of stroke for a number of years. This situation has led to the development of telemedicine to extend expert care to individuals during the initial hours and days following the stroke, advance site-independent treatment, and create models of care in rural areas [12, 13, 14]. Therapy options after this acute period, however, generally remain limited for stroke survivors in rural areas. Akin to the telemedicine efforts, telerehabilitation treatments have been proposed. However, telerehabilitation interactions are typically limited to off-line monitoring by the therapist [8, 9, 15], phone calls between a therapist and client [16, 17], or videoconferencing [18, 19, 20]. While systems allowing more direct interaction have been proposed, the hardware cost and complexity limit applicability for home-based therapy [21, 22, 23]. Hence, the therapist is relegated to the role of observer and the intimacy of a clinical therapy session is lost. Therapy options are substantially restricted, as is the available feedback.
Recently, multiple investigators have been exploring means of improving home-based therapy through the development of systems or serious games which permit multiple, simultaneous users [24, 25, 26, 27, 28, 29, 30]. These efforts have proposed the inclusion of multiple users as a means to overcome resistance to home-based therapy that may result due to isolation or lack of engagement. Indeed, studies have observed a preference for multi-user vs, single-user therapy when utilizing these systems [26, 29]. However, these systems have largely been limited to control of a one-dimensional or two-dimensional space and both users remain in the same physical location (e.g., side by side). One team of researchers did develop a framework for supporting distant users (such as a therapist in the hospital and a stroke survivor in their home), but game control was limited to one or two dimensions [31, 32].
Here, we describe the development of a fully three-dimensional (3D) virtual reality environment (VRE) for home-based therapy in which multiple, remote users can interact in real time. In this Virtual Environment for Rehabilitative Gaming Exercises (VERGE) system , movement of the user is mapped to corresponding movement of an avatar to foster a sense of presence in and engagement with the VRE. The 3D environment encompasses aspects of clinical therapy, such as transport of objects or movement of the hand into specified regions of the upper extremity workspace. Although the importance of 3D movements in VR environments is a topic of debate [34, 35], movements tested in environments with lesser degrees-of-freedom (DOF) are often very limited and dictated by a one DOF robot. These movements differ substantially from the types of movements normally seen in 3D reaching movements [4, 36]. The network architecture of the system allows users to be located remotely from each other, such as a stroke survivor in their home, a therapist in a clinic, or a stroke survivor’s friend or relative living in another city or state. The virtual nature of the environment allows even very limited movements in the physical world to have successful functional outcomes in the virtual world, thereby offering a sense of accomplishment and motivation for successive attempts. Additionally, task difficulty can easily be modified in order to maintain the proper level of challenge, which is important for motor learning in general  and rehabilitation in particular .
We developed and performed preliminary testing of the VERGE system to gauge user response in comparison to two other therapy modalities that could be used for home therapy: an existing virtual reality system based on the Alice in Wonderland story (AWVR)  and a home exercise program (HEP). Fifteen stroke survivors completed three, one-hour therapy sessions per week with each of the three therapy modalities (9 sessions total). We hypothesized that the use of the VERGE system would not decrease the amount of arm movement promoted, in comparison with the AWVR and HEP modalities. We further expected that users’ self-described engagement would be greatest for the VERGE system due to the presence of a partner.
At its core, VERGE consists of a 3D VRE in which avatars interact with virtual objects. To date, we have created two such VREs, one depicting a dining room and the other a kitchen. The scenes were created in Maya (Autodesk Inc., San Rafael, CA) and imported into Unity 3D (Unity 4.5, Unity Technologies, San Francisco, CA), the software platform controlling VERGE. The VREs are rich in detail in order to provide depth cues . Thus, depth can be conveyed without the need for stereovision, such as that provided by head mounted displays (HMDs). We have found that HMDs can be difficult for stroke survivors to use due to the limited field-of-view and, especially, involuntary coupling between neck and arm motion [41, 42]. The latter may lead to complications with moving the arm while keeping the head steady.
The avatars were created from a custom skeleton in Maya (Autodesk Inc., San Rafael, CA), which was rigged to an existing mesh of the “casual young man” 3D model, purchased and modified for our project (Fig. 1). We created the custom skeleton to match the topology of the existing character while corresponding to the skeletal joint naming convention in Unity 3D. The skeleton (and thus avatar) is animated according to joint angle data captured with a Kinect™ I optical tracker (Microsoft Corp., Redmont, WA). The 3D motion data from the Kinect™ are transmitted to the Unity code through UDP to drive the movement of the avatar in the virtual environment.
To rehabilitate individuals with impaired upper-limb function, we have designed and developed a robot guided rehabilitation scheme. A humanoid robot, NAO was used for this purpose. NAO has 25 degrees of freedom. With its sensors and actuators, it can walk forward and backward, can sit down and stand up, can wave his hand, can speak to the audience, can feel the touch sensation, and can recognize the person he is meeting. All these qualities have made NAO a perfect coach to guide the subjects to perform rehabilitation exercises. To demonstrate rehabilitation exercises with NAO, a library of recommended rehabilitation exercises involving shoulder (i.e., abduction/adduction, vertical flexion/extension, and internal/external rotation), and elbow (i.e., flexion/extension) joint movements was formed in Choregraphe (graphical programming interface). In experiments, NAO was maneuvered to instruct and demonstrate the exercises from the NRL. A complex ‘touch and play’ game was also developed where NAO plays with the subject that represents a multi-joint movement’s exercise. To develop the proposed tele-rehabilitation scheme, kinematic model of human upper-extremity was developed based modified Denavit-Hartenberg notations. A complete geometric solution was developed to find a unique inverse kinematic solution of human upper-extremity from the Kinect data. In tele-rehabilitation scheme, a therapist can remotely tele-operate the NAO in real-time to instruct and demonstrate subjects different arm movement exercises. Kinect sensor was used in this scheme to get tele-operator’s kinematics data. Experiments results reveals that NAO can be tele-operated successfully to instruct and demonstrate subjects to perform different arm movement exercises. A control algorithm was developed in MATLAB for the proposed robot guided supervised rehabilitation scheme. Experimental results show that the NAO and Kinect sensor can effectively be used to supervise and guide the subjects in performing active rehabilitation exercises for shoulder and elbow joint movements.
Assad-Uz-Zaman, Md, “Design and Development of a Robot Guided Rehabilitation Scheme for Upper Extremity Rehabilitation” (2017). Theses and Dissertations. 1578. https://dc.uwm.edu/etd/1578
Lack of motivation during physical rehabilitation is a very common problem that worsens the efficacy of rehabilitation, decreasing the recovery rates of the patient. We suggest a gamified upper-limb rehabilitation that incorporates adaptive gameplay and difficulty so as to overcome that issue, emerging as a support tool for physical therapy professionals. The presence of difficulty adjustment in the game allows a higher motivation level for the patients by preserving the trade off between keeping the difficulty low enough to avoid frustration, but high enough to promote motivation and engagement. This rehabilitation game is a home-based system that allows the patient to exercise at home, due to its Kinect-based portable setup. The game aims to increase the motivation of the patients and thus the speed of their recovery. To accomplish that goal, it is key to potentiate a full immersion into the therapeutic activity. Thus gamification elements, gameplay design and adaptive difficulty are explored and incorporated into the concept.
We designed this study to prove the efficacy of the low-cost Kinect-based virtual rehabilitation (VR) system for upper limb recovery among patients with subacute stroke.
A double-blind, randomized, sham-controlled trial was performed. A total of 23 subjects with subacute stroke (<3 months) were allocated to sham (n = 11) and real VR group (n = 12). Both groups participated in a daily 30-minute occupational therapy for upper limb recovery for 10 consecutive weekdays. Subjects received an additional daily 30-minute Kinect-based or sham VR. Assessment was performed before the VR, immediately and 1 month after the last session of VR. Fugl-Meyer Assessment (FMA) (primary outcome) and other secondary functional outcomes were measured. Accelerometers were used to measure hemiparetic upper limb movements during the therapy.
FMA immediately after last VR session was not different between the sham (46.8 ± 16.0) and the real VR group (49.4 ± 14.2) (P = .937 in intention to treat analysis). Significant differences of total activity counts (TAC) were found in hemiparetic upper limb during the therapy between groups (F2,26 = 4.43; P = .22). Real VR group (107,926 ± 68,874) showed significantly more TACs compared with the sham VR group (46,686 ± 25,814) but there was no statistical significance between real VR and control (64,575 ± 27,533).
Low-cost Kinect-based upper limb rehabilitation system was not more efficacious compared with sham VR. However, the compliance in VR was good and VR system induced more arm motion than control and similar activity compared with the conventional therapy, which suggests its utility as an adjuvant additional therapy during inpatient stroke rehabilitation.
Therapeutic benefits of Kinect-based virtual reality (VR) game training in rehabilitation encourage its use to improve motor function.
To assess the effects of Kinect-based VR training on motor recovery of the upper extremity and functional outcomes in patients with chronic stroke.
In this randomized controlled trial, group A received 20 sessions of physical therapy (PT) + 20 sessions of Kinect-based VR training and group B received only 20 sessions of PT. Clinical outcome measures were assessed at baseline and at the end of the treatments. Primary outcome measures that assess stroke patients’ motor function included upper extremity (UE) Fugl-Meyer Assessment (FMA). Secondary outcome measures were Brunnstrom Recovery Stages (BRS), Modified Ashworth Scale (MAS), Box and Block test (BBT), Motricity index (MI), and active range of motion (AROM) measurement.
Statistically significant improvements in game scores (p < 0.05) were observed in group A. In within-group analysis, there were statistically significant improvements in all clinical outcome measures except for the BRS-hand, MAS-distal, and MAS-hand in group A; MAS-(proximal, distal, hand) and BRS-(UE, hand) in group B compared with baseline values. Differences from baseline of FMA, MI, and AROM (except adduction of shoulder and extension of elbow) were greater in group A (p < 0.05).
To conclude, our results suggest that the adjunct use of Kinect-based VR training may contribute to the improvement of UE motor function and AROM in chronic stroke patients. Further studies with a larger number of subjects with longer follow-up periods are needed to establish its effectiveness in neurorehabilitation.
The proportion of rehabilitation doctors and patients mismatch is very grim in the context of social aging. The Family Rehabilitation System captures the profound information of the trainer’s movements through the kinect bone tracing technique, allowing the doctor to remotely master the patient’s training progress. With the help of computers and the Internet, the patient can consult a physician, while the physician can remotely guide and launch the training “prescription” through the Internet according to the training effect. Patients can have rehabilitated training at home. The results of the test showed that the system has a positive effect on the rehabilitation of the patient.
The number of patients with motor dysfunction caused by hemiplegia and stroke increased. In order to promote better recovery of their body muscles, patients are still required to perform rehabilitation exercises in the community or family after the treatment of discharge. However, there are still some difficulties in community rehabilitation for patients with motor dysfunction:
(1) The number of therapists on-site services is scarce and expensive;
(2) In the absence of standard and systematic action guidance, the patients ‘ own training is not only the science is not high and the effect is limited.
(3) Patients need to be trained in special environments such as rehabilitation centers, and wearing complex training equipment is inconvenient for them.
The family rehabilitation system collects the depth information of the trainer’s movements through the Kinect skeletal tracking technique; With the help of computers and the Internet, patients can consult physicians, and the doctor through the Internet remote guidance and open training action “prescription” according to the training effect so that patients at home can be rehabilitation training. This liberated the physician’s labour force and formed a network community that was closely linked to the hospital and regularly received “training prescriptions” to improve patient rehabilitation. […]
In the modern scenario of neurological rehabilitation, which requires affordable solutions oriented toward promoting home training, the Institute of Industrial Technologies and Automation (ITIA) of the Italian National Research Council (CNR) developed a line of prototypal devices for the rehabilitation of the upper limb, called “Arm.” Arm devices were conceived to promote rehabilitation at affordable prices by capturing all the main features of the state-of-the-art devices. In fact, Arm devices focus on the main features requested by a robot therapist: mechanical adaptation to the patient, ranging from passive motion to high transparency, assist-as-needed and resistive modalities; proper use of sensors for performance monitoring; easy-to-use, modular, and adaptable design. These desirable features are combined with low-cost, additive manufacturing procedures, with the purpose of meeting the requirements coming from research on neuro-motor rehabilitation and motor control and coupling them with the recent breakthrough innovations in design and manufacturing.
The use of robotic devices for upper-limb neuro-motor rehabilitation is usual practice in clinical centers. In respect to conventional therapies, robots allow to increase training intensity and help patients to promote their active contribution. Furthermore, robots can act as measurers of patients’ performances and adapt their interaction modalities to the emerging needs during the rehabilitation course. Robots like ARMin, MIT Manus, Armeo Spring, Braccio di Ferro, represent the state of the art devices for rehabilitation of the upper-limb and for promoting motor recovery. According to the available assessments and studies in the literature, their efficacy is slightly/moderately higher than the one of conventional therapies. Furthermore, robots are used in research to learn more about physiological and pathological motor control and neuromuscular diseases. Unfortunately, while being the state of the art devices for neuro-motor stimulation and training, such robots are very expensive and not compliant to user-friendly requirements that are needed for semi-autonomous home use. Consequently, they can be used only in clinical environments, under the supervision of medical personnel. Furthermore, sanitary costs related to rehabilitation are increasing and clinical centers can hardly support their burden. The possibility of delocalizing rehabilitation from clinical centers opens the chance for training performed in home environment, with time and costs savings for both the sanitary system and patients. In this scenario, which requires affordable solutions oriented toward promoting home training, the Institute of Industrial Technologies and Automation (ITIA) of the Italian National Research Council (CNR) developed a line of prototypal devices for the rehabilitation of the upper-limb, called -Arm. Arm devices were conceived to test the possibility of promoting rehabilitation at affordable prices but capturing all the main features of the state of the art devices. In fact, Arm devices focus on the main features requested by a robot therapist: mechanical adaptation to the patient, ranging from passive motion to high transparency, assist-as-needed and resistive modalities; proper use of sensors for performance monitoring; easy-to-use, modular and adaptable design. These desirable features are combined with low-cost, additive manufacturing procedures, with the purpose of meeting the requirements coming from research on neuro-motor rehabilitation and motor control and coupling them with the recent breakthrough innovations in design and manufacturing. Arm devices cover both clinical and home-oriented training and are designed for adaptation to patients with different motor impairment.
The Arm prototypes are:
• LINarm: linear device, freely orientable in space, suitable for functional movements. It features a variable stiffness actuation, allowing to adapt the mechanical behavior of the device to patients’ needs. Functional Electrical Stimulation, simple Virtual Environments and a Patient Model, gathering data from integrated sensors and modulating the level of assistance, are integrated in the set-up. The LINarm++ Echord++ Project ended in October 2016 and guided the development of a second, more refined prototype, enhancing the original concept.
• PLANarm: planar device, freely orientable in space, suitable for planar functional movements. The state of the art planar robots used in literature for motor control and motor learning research inspired PLANarm. It features a variable stiffness actuation, allowing adapting the mechanical behavior of the device depending on patients’ needs.
• DUALarm: Low-Cost device for bimanual rehabilitation, exploiting the capability of the less affected limb to provide rehabilitation to the more affected limb. DUALarm is completely realized in 3D printing technology and aims at being an easy-to-use, low-cost, open-source project. Currently, reaching movements can be trained, but the device is conceived to be suitable for training of other functional gestures.
• LIGHTarm: Exoskeleton for the rehabilitation of the upper-limb, designed in two versions: LIGHTarm, not actuated, and conceived to support the weight of the impaired limb. The mechanical design includes high backdrivability, focusing on shoulder rhythm and elbow singular configurations.
• VIRTUALarm: Kinect One-based platform for motor monitoring, including body and limb tracking and a biomechanical evaluation of the performance in relation to databases of healthy subjects. Assessments include range of motion, motion dynamics, effort, motor control indexes, body segments barycenter tracking.
Purpose: To investigate the effectiveness and feasibility of Kinect-based upper
extremity rehabilitation on functional performance in chronic stroke survivors. Methods: This was a single cohort pre-post test study. Participants (N=10; mean age =
62.5 ± 9.06) engaged in Kinect-based training three times a week for four to five weeks
in a university laboratory. To simulate a clinic to home transfer condition,
individualized guidance was given to participants at the initial three sessions followed
by independent usage. Outcomes included Fugl-Meyer assessment of upper extremity,
Wolf Motor Function Test, Stroke Impact Scale, Confidence of Arm and Hand
Movement and Active Range of Motion. Participant experience was assessed using a
structured questionnaire and a semi-structured interview. Results. Improvement was found in Fugl-Meyer assessment scores (p=0.001), Wolf
Motor Function Test, (p=0.008), Active Range of Motion (p<0.05) and Stroke Impact
Scale-Hand function (p=0.016). Clinically important differences were found in FuglMeyer
assessment scores (Δ= 5.70 ± 3.47) and Wolf Motor Function Test (Δ Time= –
4.45 ± 6.02; ∆ Functional Ability Scores= 0.29 ± 0.31). All participants could use the
system independently and recognized the importance of exercise individualization by
the therapist. Conclusions. The Kinect-based UE rehabilitation provided clinically important
functional improvements to our study participants.
Stroke is the leading cause of long-term adult disability in the United States .
More than a half of survivors continue suffering from upper-limb hemiparesis poststroke with only 5% of people recovering their full arm function . The persistent
upper-limb dysfunction significantly impairs motor performance, and results in a
serious decline in functional ability as well as quality of life . Intensive and repeated
practice with the paretic arm appears necessary to enhance arm recovery and facilitate
neural reorganization [4-7]. Nevertheless, the healthcare system provides limited
amounts and duration of therapy, making it difficult for stroke survivors to achieve
maximal arm recovery before discharge from outpatient rehabilitation or home care
[8,9]. Therefore, identifying novel modalities that are accessible and affordable to the
general public while allowing continued practice of the arm is imperative for improving
long-term upper-limb outcomes after stroke.
One potential approach is the use of low-cost virtual reality (VR)-based systems,
for example, the Microsoft Kinect system. The Kinect is a vision-based motion
capturing system that can detect gesture and movements of the body through its RGA
camera and depth sensors. It allows users to interact with the VR-based system without
holding or wearing specialized equipment or markers for tracking. Users can play
games or practice exercises using natural movements while observing the performance of their virtual avatars shown in real-time on the computer screen. Through this interactive observation and feedback, stroke survivors can correct their movements towards more normal patterns. Furthermore, the Kinect is small and portable, thus enabling stroke survivors to practice exercises in a familiar and private environment. […]
Exercise-based rehabilitation for chronic conditions such as cardiovascular disease, diabetes, and chronic obstructive pulmonary disease, constitutes a key element in reducing patient symptoms and improving health status and quality of life. However, group exercise in rehabilitation programmes faces several challenges imposed by the diversified needs of their participants. In this direction, we propose a novel computer-assisted system enhanced with sensors such as Kinect cameras and wristband heart rate monitors, aiming to support the trainer in adapting the exercise programme on-the-fly, according to identified requirements. The proposed system design facilitates maximal tailoring of the exercise programme towards the most beneficial and enjoyable execution of exercises for patient groups. This work contributes in the design of the next-generation of computerised systems in exercise-based rehabilitation.
Vanhees L, Geladas N, Hansen D et al (2012) Importance of characteristics and modalities of physical activity and exercise in the management of cardiovascular health in individuals with cardiovascular risk factors: recommendations from the EACPR. Part II. Eur J Prev Cardiol 19:1005–33. https://doi.org/10.1177/1741826711430926
Thompson PD, Arena R, Riebe D, Pescatello LS (2013) ACSM’s new preparticipation health screening recommendations from ACSM’s guidelines for exercise testing and prescription, 9th (edn). Curr Sports Med Rep 12:215–217. https://doi.org/10.1249/JSR.0b013e31829a68cf
Chatzitofis A, Zarpalas D, Filos D et al (2017) Technological module for unsupervised, personalized cardiac rehabilitation exercising. In: 2017 IEEE 41st annual computer software and applications conference (COMPSAC). https://doi.org/10.1109/COMPSAC.2017.230
Compernolle S, Vandelanotte C, Cardon G et al (2015) Effectiveness of a web-based, computer-tailored, pedometer-based physical activity intervention for adults: a cluster randomized controlled trial. J Med Internet Res 17:e38. https://doi.org/10.2196/jmir.3402CrossRefGoogle Scholar
Triantafyllidis AK, Koutkias VG, Chouvarda I, Maglaveras N (2014) Development and usability of a personalized sensor-based system for pervasive healthcare. In: 2014 36th Annual international conference of the IEEE engineering in medicine and biology society EMBC. https://doi.org/10.1109/EMBC.2014.6945146
Exergames provide a challenging opportunity for home-based training and evaluation of postural control in the elderly population, but affordable sensor technology and algorithms for assessment of whole body movement patterns in the home environment are yet to be developed.
The aim of the present study was to evaluate the use of Kinect, a commonly available video game sensor, for capturing and analyzing whole body movement patterns.
Healthy adults (n=20) played a weight shifting exergame under five different conditions with varying amplitudes and speed of sway movement, while 3D positions of ten body segments were recorded in the frontal plane using Kinect and a Vicon 3D camera system. Principal Component Analysis (PCA) was used to extract and compare movement patterns and the variance in individual body segment positions explained by these patterns. Using the identified patterns, balance outcome measures based on spatiotemporal sway characteristics were computed.
The results showed that both Vicon and Kinect capture >90% variance of all body segment movements within three PCs. Kinect-derived movement patterns were found to explain variance in trunk movements accurately, yet explained variance in hand and foot segments was underestimated and overestimated respectively by as much as 30%. Differences between both systems with respect to balance outcome measures range 0.3–64.3%.
The results imply that Kinect provides the unique possibility of quantifying balance ability while performing complex tasks in an exergame environment.