In the therapist-centered rehabilitation program, the experienced therapists can observe emotional changes of stroke patients and make corresponding decisions on their intervention strategies. Likewise, robotic-assisted stroke rehabilitation systems will be more appreciated if they can also perceive emotional states of the stroke patients and enhance their engagements by exploring emotion-based dynamic difficulty adjustments. Nevertheless, few research have addressed this issue. A two-phase pilot study with anxiety as the target emotion state was conducted in this article. In phase I, the motor performances and the physiological responses to the stroke subject’s anxiety with high, medium, and low intensities were statistically analyzed, and anxiety models with three intensities were offline developed using support vector machine–based classifiers. In phase II, anxiety-based closed-loop robot-aided training task adaptation and its impacts on patient–robot interaction engagements were explored. As a comparison, a performance-based robotic behavior adaptation was also implemented. Experimental results with 12 recruited stroke patients conducted on the Barrett WAMTM manipulator verified that the rehabilitation robot can implicitly recognize the anxiety intensities of the stroke survivors and the anxiety-based real-time robotic behavior adaptation shows more engagements in the human–robot interactions.
Task-oriented repetitive rehabilitation training is becoming the state-of-the-art therapy approach for poststroke patients. These therapy tasks are traditionally implemented by physical therapists. In recent years, there is an increasing interest in using robotic devices to help providing motor rehabilitation therapy.1Compared with the therapist-centered therapy, robot-assisted stroke rehabilitation can not only provide a variety of highly repetitive movements and training protocols for stroke patients, but also offer objective measurements of stroke patients’ functional improvements.
Stroke patients’ active engagements in rehabilitation training have been shown to be a very positive factor to the success of rehabilitation.2 Early rehabilitation robots are able to provide active assistance to stroke patients, but do not take into account individual properties, spontaneous intentions, or voluntary efforts of that particular person. These problems were addressed by integrating the patients into the sensorimotor control loop. By recognizing the patients’ active motor abilities or movement intentions, the human-in-the-loop rehabilitation robotic systems are able to optimize participation and support the patients only as little as needed.3 However, stroke patients’ active involvements in the existing rehabilitation robotic systems are mostly considered from biomechanical and bioelectrical viewpoints, where the patients’ active force/position signals4,5 or electrical activities of the brain and the muscles6,7 were recorded.
In the therapist-centered program, the therapists who work with the stroke patients can not only perceive the patients’ active motor involvements, but also continuously monitor the patients’ emotion changes in order to make appropriate decisions on their intervention strategies. The stroke patients are particularly vulnerable to anxiety and frustration, which requires to plan tasks at an appropriate level of difficulty. Likewise, robotic-assisted stroke rehabilitation training systems will be more appreciated if they can perceive the stroke patients’ emotion changes and make emotion-based dynamic difficulty adjustment. Offering insights into the patients’ emotional changes and adapting emotion-based behavioral interventions are known as another critical factor to successful stroke rehabilitation.8 Nonetheless, very few research on robot-assisted stroke rehabilitation are specifically addressed how to automatically recognize and respond to the emotion changes of the stroke survivors. One possible reason is that there are some difficulties in perceiving the stroke patient’s emotion states.
There are several modalities such as facial expression,9 vocal intonation,10 body gesture,11 and physiology12 that can be utilized to recognize the emotion states of individuals in human–robot/computer interaction. Nevertheless, the patients with chronic stroke are often characterized by dull facial expression, severe aphasia, and limb motor dysfunction. These vulnerabilities place limits on observational, conversational, and limb methodologies to recognize the stroke survivors’ emotional states. Physiology-based measurements are far more robust against these difficulties because they are noninvasive and further the psycho-physiological signals can be continuously available without the stroke patient’s active cooperation. Besides, evidences show that the transition from one emotion state to another is accompanied by dynamic shifts in indicators of autonomous and central nervous system activity.13,14
In this article, anxiety, which can be easily evoked by training tasks with different difficulties in clinical rehabilitation therapy, was chosen as the target emotion state. The primary focuses of the current research were firstly to offline evaluate anxiety with high, medium, and low intensities and then to carry out real-time anxiety-based robot-aided rehabilitation training task adaptation.
The block diagram of the anxiety detection and subsequent anxiety-based robot-aided training task adaptation system are shown in Figure 1. It consists of two consecutive phases: offline anxiety modeling (phase I) and online anxiety-based robot-aided training task adaptation (phase II). In phase I, the features, from the physiological and the motor performances recordings of the stroke subjects under anxiety with high, medium, and low intensities, were firstly extracted and then subjected to analysis of variance (ANOVA)-based statistical analyses to obtain the features with significant differences among three anxiety intensities. Anxiety with three intensities was further offline evaluated using support vector machine (SVM)–based anxiety classifier, in which the features with significant differences were adopted as inputs while the self-reported questionnaires as outputs. In phase II, robot-aided rehabilitation training tasks were online adapted to the recognized intensities of anxiety from the stroke subjects. Further, to demonstrate the effect of introducing anxiety of stroke patients into robot-assisted stroke rehabilitation, the impacts of anxiety-based robot-aided behavior adaptation on the stroke patient’s engagements were explored using the performance-based robot-aided training task adaptation as a comparison. The details on the enrolled subjects are given in the “Subjects” section while the experimental system setup is depicted in the “Experimental setup” section. “Phase I: Offline anxiety modeling” section demonstrates the offline modeling of the anxiety with high, medium, and low intensities (phase I). The online anxiety-based robot-aided rehabilitation training task adaptation and its impacts on the stroke patient’s engagements are shown in the “Phase II: Online anxiety-based robot-aided rehabilitation training task adaptation” section (phase II).
The stroke patients, with upper extremity motor impairments and similar Brunnstrom Recovery Scale (BRS) evaluation scores, were recruited as representative of hemiparesis participants. Participants were excluded from the study if they had severe neurological disorder, senile dementia, or cognitive intact.15 Twelve stroke participants (mean age: 53.6 years, 7 males, 5 females, mean stroke time: 12.6 months, 6 right-sided hemiplegia, 6 left-sided hemiplegia, 10 stage-4 and 2 stage-3 BRS scores of upper extremity, and 9 stage-4 and 3 stage-3 BRS scores of hand) were recruited, and all were received motor rehabilitation therapy at the Rehabilitation Medicine Center of the Nanjing Tongren Hospital of China. Before the tasks began, ethical approval was obtained from the Medical Ethics Committee of the Nanjing Tongren Hospital of China, and all subjects were informed about the procedure and that they would be video-recorded and photo-taken during the experiment. All subjects gave written informed consent concerning the use of their video footage and questionnaire data for further analysis. Of the 12 stroke participants, one stroke subject (upper extremity and hand BRS scores were both stage-3) was not able to complete phase I experiments, and the rest also took part in phase II closed-loop experiments.