During robot-aided rehabilitation exercises, monotonous, and repetitive actions can, to the subject, feel tedious and tiring, so improving the subject’s motivation and active participation in the training is very important. A novel robot-aided upper limb rehabilitation training system, based on multimodal feedback, is proposed in this investigation. To increase the subject’s interest and participation, a friendly graphical user interface and diversiform game-based rehabilitation training tasks incorporating multimodal feedback are designed, to provide the subject with colorful and engaging motor training. During this training, appropriate visual, auditory, and tactile feedback is employed to improve the subject’s motivation via multi-sensory incentives relevant to the training performance. This approach is similar to methods applied by physiotherapists to keep the subject focused on motor training tasks. The experimental results verify the effectiveness of the designed multimodal feedback strategy in promoting the subject’s participation and motivation.
Introduction
Strokes are caused by acute cerebrovascular disease in one of the cerebral hemispheres, usually associating with impairment of the motor functions and other functional disabilities. Hemiplegic is the most common outcome of a stroke (Guo et al., 2017). According to the statistics of the “high-risk population screening and intervention program for strokes” (Wang et al., 2018), the prevalence rate of this type of brain impairment increased from 1.89% in 2012 to 2.19% in 2016. By this calculation, it can be inferred that the number of stroke patients aged 40 and above in China has now reached 12.42 million. Approximately 80–90% of stroke patients suffer from some form of dysfunction, of which the incidence of upper limb dysfunction is as high as 80% (Fernandes et al., 2018; Sun et al., 2018). This disease has a slow recovery rate and is accompanied by varying degrees of dysfunction that affect patients’ lives and imposes a considerable burden on their families (Michael et al., 2005; Edwards et al., 2006). Stroke patients are eager to obtain systematical rehabilitation treatment and return to healthy lives.
Medical theory and clinical practice have proved that the central nervous system of the human brain has a high degree of plasticity (Sano and Ishii, 1978). Neuronal plasticity (Kwakkel et al., 2008) opens up many possibilities for the rehabilitation of hemiplegic patients. According to the theory, the “model integration” of the cerebral cortex functional areas is achieved by inputting regular motions, as the repeated movements can improve motor coordination. The motion of muscles and joints also provides a large number of stimulations to the central nervous system of the brain, which can effectively prevent limb paralysis and muscle atrophy. However, conventional hand-in-hand rehabilitation has many drawbacks, such as a limited number of therapists, high treatment costs, long duration, and tiring training processes. Furthermore, the lack of accurate, objective evaluation mechanisms and real-time feedback of training statuses are urgent problems to be solved, which to some extent stunt the progress of treatment.
To provide immediate and appropriate treatment for stroke patients, many research institutes throughout the world have adopted robot technology (Ploughman and Corbett, 2004) and virtual reality (Saposnik et al., 2016) to help stroke patients perform rehabilitation training tasks. Substantial progress has been made, such as MIT-Manus (Volpe et al., 2000, 2003), GENTLE/S (Schmidt et al., 2004), and RUPERT (Balasubramanian et al., 2008). To make the rehabilitation process more interesting, domestic and international researchers are integrating toys and games into the design of robot-assisted rehabilitation systems. Experiments show that the application of toys and games invokes in patients feelings of pleasure, competition, and other emotions. This can encourage patients to more actively participate in rehabilitation training, resulting in longer training periods and better training results (Fluet et al., 2012; Bank et al., 2018; Avola et al., 2019). Ustinova et al. (2010) and Li et al. (2014) explored the influence of a virtual environment on arm stretching exercises, in which participants were asked to perform simulations of certain daily activities, such as gardening, shopping, and washing clothes. The results showed that patients had a stronger sense of active participation and clearer treatment objectives. Weiss et al. (2004) provided users with a virtual participatory environment, constructing various game scenes and adding corresponding auditory feedback, which enabled patients to maintain long-term interest. In addition, motion estimation (Jurgen et al., 2007), force feedback (Gorsic et al., 2017) and electrical stimulation (Berenpas et al., 2019; Li et al., 2019) are employed in rehabilitation training studies. However, current research usually applies only audio-visual or relatively limited feedback, which struggles to satisfy the requirements of personalized and intelligent rehabilitation training programs (Carignan et al., 2005; Shin et al., 2014).
To provide a more effective robot-aided rehabilitation training, a novel game-based training task with multi-modal feedback strategy is proposed to develop a more humanized training system. The rehabilitation system designed on the basis of the proposed method provides the subject with multi-sensory (visual, auditory, and tactile) feedback. During motor training, a variety of feedback is employed to help patients enjoy training, improving their motivation and active participation in the tasks.
Materials and Methods
Background
Clinical studies demonstrate that functional motor training plays a key role in the recovery of the central nervous system after a stroke. In general, rehabilitation training methods are primarily divided into four types: passive, assisted-active, active, and resistance. Patients experiencing paralysis and spasms are unable to make any active movements, thus passive training is suitable. Robot-assisted passive movement can enhance motor proprioception, stimulate flexion and extension reflexes, and gradually increase the range of motions available to joints. In assisted-active training, the rehabilitation robot limits abnormal motions and provides appropriate real-time assistance to the patient as they sense the state of their limbs. In active training processes, patients do not need auxiliary force or external resistance, and the entire training exercise is completed by them actively contracting their upper limb muscles, which can stimulate their active training consciousness and helps them maintain control of their nervous system. To maximize the motor function of the affected limbs, further muscle exercise and resistance training are usually adopted to enhance muscle strength and motor coordination. In our investigation, the designed rehabilitation system is based on the proposed game-based training tasks, and multi-modal feedback provides the subject with active training results.
Rehabilitation Training System Setup
In our previous research, a robot-aided upper limb rehabilitation system was constructed, which primarily included the whole arm manipulator (WAM), arm support device, self-developed three-dimensional (3D) force sensor, and controlling personal computer (PC) (Pan et al., 2017, 2019). The WAM works in a large workspace with four rotational degrees of freedom, and the self-developed 3D force sensor is installed at the endpoint of the WAM to measure the interactive force for use in some of the designed control algorithms, as shown in Figure 1. During operation, four driver motor angles can be measured to detect the position of every joint in real time, and the control torque can be set to provide joint control. For detailed information about the hardware and software characteristics of the constructed motion rehabilitation training system, please refer to Pan et al. (2017, 2019). The framework of the rehabilitation control system in this study, which incorporates multimodal feedback, is presented in Figure 2. The subject selects the appropriate game-based training task via a graphical user interface (GUI) to commence the training. The subject controls the end position of the robot arm determining the movement of the mouse, which they operate with their upper limbs, to conduct the game-based motion rehabilitation training. During the training, multimodal feedback is applied to the subject according to the motor performance, to improve the subject’s training motivation and participation level.
Figure 1. WAM rehabilitation robot.
[…]
TBI Rehabilitation 