Posts Tagged augmented reality

[THESIS] AUGMENTED REALITY SYSTEM FOR REHABILITATION: NEW APPROACH BASED ON HUMAN INTERACTION AND BIOFEEDBACK – Full Text PDF

Abstract

Rehabilitation is the process of training for someone in order to recover or improve their lost functions caused by neurological deficits. The upper limb rehabilitation system provides relearning of motor skills that are lost due to any neurological injuries via motor rehabilitation training. The process of motor rehabilitation is a form of motor learning via practice or experience. It requires thorough understanding and examination of neural processes involved in producing movement and learning as well as the medical aspects that may affect the central nervous system (CNS) or peripheral nervous system (PNS) in order to develop an effective treatment system. Although there are numerous rehabilitation systems which have been proposed in literatures, a low cost upper limb rehabilitation system that maximizes the functional recovery by stimulating the neural plasticity is not widely available. This is due to lack of motivation during rehabilitation training, lack of real time biofeedback information with complete database, the requirement of one to one attention between physiotherapist and patient, the technique to stimulate human neural plasticity.

Therefore, the main objective of this thesis is to develop a novel low cost rehabilitation system that helps recovery not only from loss of physical functions, but also from loss of cognitive functions to fulfill the aforementioned gaps via multimodal technologies such as augmented reality (AR), computer vision and signal processing. In order to fulfill such ambitious objectives, the following contributions have been implemented.

Firstly, since improvements in physical functions are targeted, the Rehabilitation system with Biofeedback simulation (RehaBio) is developed. The system enhances user’s motivation via game based therapeutic exercises and biofeedback. For this, AR based therapeutic games are developed to provide eye-hand coordination with inspiration in motivation via immediate audio and visual feedback. All the exercises in RehaBio are developed in a safe training environment for paralyzed patients. In addition to that, realtime biofeedback simulation is developed and integrated to serve in two ways: (1) from the patient’s point of view, the biofeedback simulation motivates the user to execute the movements since it will animate the different muscles in different colors, and (2) from the therapist’s point of view, the muscle simulations and EMG threshold level can be evaluated as patient’s muscle performance throughout the rehabilitation process.

Secondly, a new technique that stimulates the human neural plasticity is proposed. This is a virtual human arm (VHA) model that driven by proposed continuous joint angle prediction in real time based on human biological signal, Electromyogram (EMG). The VHA model simulation aims to create the illusion environment in Augmented Realitybased Illusion System (ARIS).

Finally, a complete novel upper limb rehabilitation system, Augmented Reality-based Illusion System (ARIS) is developed. The system incorporates some of the developments in RehaBio and real time VHA model to develop the illusion environment. By conducting the rehabilitation training with ARIS, user’s neural plasticity will be stimulated to reestablish the neural pathways and synapses that are able to control mobility. This is achieved via an illusion concept where an illusion scene is created in AR environment to remove the impaired real arm virtually and replace it with VHA model to be perceived as part of the user’s own body. The job of the VHA model in ARIS is when the real arm cannot perform the required task, it will take over the job of the real one and will let the user perceive the sense that the user is still able to perform the reaching movement by their own effort to the destination point. Integration with AR based therapeutic exercises and motivated immediate intrinsic and extrinsic feedback in ARIS leads to serve as a novel upper limb rehabilitation system in a clinical setting.

The usability tests and verification process of the proposed systems are conducted and provided with very encouraging results. Furthermore, the developments have been demonstrated to the clinical experts in the rehabilitation field at Port Kembla Hospital. The feedback from the professionals is very positive for both the RehaBio and ARIS systems and they have been recommended to be used in the clinical setting for paralyzed patients.

Download Full Text PDF

, , , , , , , , ,

Leave a comment

[THESIS] AUGMENTED REALITY SYSTEM FOR REHABILITATION: NEW APPROACH BASED ON HUMAN INTERACTION AND BIOFEEDBACK – Full Text PDF

Abstract

Rehabilitation is the process of training for someone in order to recover or improve their lost functions caused by neurological deficits. The upper limb rehabilitation system provides relearning of motor skills that are lost due to any neurological injuries via motor rehabilitation training. The process of motor rehabilitation is a form of motor learning via practice or experience. It requires thorough understanding and examination of neural processes involved in producing movement and learning as well as the medical aspects that may affect the central nervous system (CNS) or peripheral nervous system (PNS) in order to develop an effective treatment system. Although there are numerous rehabilitation systems which have been proposed in literatures, a low cost upper limb rehabilitation system that maximizes the functional recovery by stimulating the neural plasticity is not widely available. This is due to lack of motivation during rehabilitation training, lack of real time biofeedback information with complete database, the requirement of one to one attention between physiotherapist and patient, the technique to stimulate human neural plasticity. Therefore, the main objective of this thesis is to develop a novel low cost rehabilitation system that helps recovery not only from loss of physical functions, but also from loss of cognitive functions to fulfill the aforementioned gaps via multimodal technologies such as augmented reality (AR), computer vision and signal processing. In order to fulfill such ambitious objectives, the following contributions have been implemented. Firstly, since improvements in physical functions are targeted, the Rehabilitation system with Biofeedback simulation (RehaBio) is developed. The system enhances user’s motivation via game based therapeutic exercises and biofeedback. For this, AR based therapeutic games are developed to provide eye-hand coordination with inspiration in motivation via immediate audio and visual feedback. All the exercises in RehaBio are developed in a safe training environment for paralyzed patients. In addition to that, realtime biofeedback simulation is developed and integrated to serve in two ways: (1) from the patient’s point of view, the biofeedback simulation motivates the user to execute the movements since it will animate the different muscles in different colors, and (2) from the therapist’s point of view, the muscle simulations and EMG threshold level can be evaluated as patient’s muscle performance throughout the rehabilitation process. Secondly, a new technique that stimulates the human neural plasticity is proposed. This is a virtual human arm (VHA) model that driven by proposed continuous joint angle prediction in real time based on human biological signal, Electromyogram (EMG). The VHA model simulation aims to create the illusion environment in Augmented Realitybased Illusion System (ARIS). Finally, a complete novel upper limb rehabilitation system, Augmented Reality-based Illusion System (ARIS) is developed. The system incorporates some of the developments in RehaBio and real time VHA model to develop the illusion environment. By conducting the rehabilitation training with ARIS, user’s neural plasticity will be stimulated to reestablish the neural pathways and synapses that are able to control mobility. This is achieved via an illusion concept where an illusion scene is created in AR environment to remove the impaired real arm virtually and replace it with VHA model to be perceived as part of the user’s own body. The job of the VHA model in ARIS is when the real arm cannot perform the required task, it will take over the job of the real one and will let the user perceive the sense that the user is still able to perform the reaching movement by their own effort to the destination point. Integration with AR based therapeutic exercises and motivated immediate intrinsic and extrinsic feedback in ARIS leads to serve as a novel upper limb rehabilitation system in a clinical setting. The usability tests and verification process of the proposed systems are conducted and provided with very encouraging results. Furthermore, the developments have been demonstrated to the clinical experts in the rehabilitation field at Port Kembla Hospital. The feedback from the professionals is very positive for both the RehaBio and ARIS systems and they have been recommended to be used in the clinical setting for paralyzed patients.

Download Full Text PDF

, , , , , ,

Leave a comment

[ARTICLE] Effect of a mixed reality-based intervention on arm, hand, and finger function on chronic stroke – Full Text

Abstract

Background

Virtual and mixed reality systems have been suggested to promote motor recovery after stroke. Basing on the existing evidence on motor learning, we have developed a portable and low-cost mixed reality tabletop system that transforms a conventional table in a virtual environment for upper limb rehabilitation. The system allows intensive and customized training of a wide range of arm, hand, and finger movements and enables interaction with tangible objects, while providing audiovisual feedback of the participants’ performance in gamified tasks. This study evaluates the clinical effectiveness and the acceptance of an experimental intervention with the system in chronic stroke survivors.

Methods

Thirty individuals with stroke were included in a reversal (A-B-A) study. Phase A consisted of 30 sessions of conventional physical therapy. Phase B consisted of 30 training sessions with the experimental system. Both interventions involved flexion and extension of the elbow, wrist, and fingers, and grasping of different objects. Sessions were 45-min long and were administered three to five days a week. The body structures (Modified Ashworth Scale), functions (Motricity Index, Fugl-Meyer Assessment Scale), activities (Manual Function Test, Wolf Motor Function Test, Box and Blocks Test, Nine Hole Peg Test), and participation (Motor Activity Log) were assessed before and after each phase. Acceptance of the system was also assessed after phase B (System Usability Scale, Intrinsic Motivation Inventory).

Results

Significant improvement was detected after the intervention with the system in the activity, both in arm function measured by the Wolf Motor Function Test (p < 0.01) and finger dexterity measured by the Box and Blocks Test (p < 0.01) and the Nine Hole Peg Test (p < 0.01); and participation (p < 0.01), which was maintained to the end of the study. The experimental system was reported as highly usable, enjoyable, and motivating.

Conclusions

Our results support the clinical effectiveness of mixed reality interventions that satisfy the motor learning principles for upper limb rehabilitation in chronic stroke survivors. This characteristic, together with the low cost of the system, its portability, and its acceptance could promote the integration of these systems in the clinical practice as an alternative to more expensive systems, such as robotic instruments.

Continue —>  Effect of a mixed reality-based intervention on arm, hand, and finger function on chronic stroke | Journal of NeuroEngineering and Rehabilitation | Full Text

Fig. 3 Description of the exercises. The exercises covered a wide range of hand and arm movements, mostly focusing on the flexion and extension of the elbow and the wrist. a Exercise: to sweep the crumbs from the table. Movement: flexion-extension of the wrist without involving the fingers. b Exercise: to grate. Movement: Grasping and flexion-extension of the wrist. c Exercise: to knock on doors. Movement: flexion-extension of the wrist against gravity. d Exercise: to cook. Movement: grasping involving flexion-extension of the elbow and rotation of the shoulders. e Exercise: to squeeze a sponge. Movement: flexion-extension of the metacarpophalangeal-interphalangeal joint. f Exercise: to dial a number. Movement: tapping. g Exercise: to play piano. Movement: flexion-extension of the thumb, index, and middle finger. h Exercise: to buy items. Movement: pincer grasping with the thumb and index involving flexion-extension of the elbow and rotation of the shoulders

 

, , , , , ,

Leave a comment

[Survey] A Survey on Assistive Technology using Natural User Interface(NUI) computing to support and augment therapeutic rehabilitation – Full Text PDF

Abstract

Therapeutic rehabilitation is a specialty in medical field that deals with diagnosis, evaluation, treatment and management of people with all ages with physical or psychosomatic disabilities due to congenital disorder, accidents or aging problem. It deals with deduction and reduction in disabilities by providing improvement and restoration of movement and functional ability through regular and repetitive physical therapy exercises continued after discharge from the hospital.

However, the efficient treatment sessions are not guaranteed due to lack of therapists and facilities, patients were alone for over 60% of the day, patients engaged in ‘activity’ for only 13% of the day, undergoing some traditional therapies make patients to lose their interest and motivating patients to continue the exercises is lagging, which in turn makes longer time for recovery.

Thus, there is a need to find ways of cost effective, engaging and motivated training to support and improve recovery and rehabilitation. The focus is to use technology as a solution involving various computing techniques as a supplementary treatment to traditional rehabilitation and continued assessment of disabled patients.

Natural User Interface (NUI) is the emerging technique with the ability to interact with computers or smart devices using the human body. NUI computing is powered by human touch, gesture, voice, thoughts and senses.

This paper is a survey on assistive technology using emerging NUI computing techniques like touch computing, gesture computing, surface computing, brain computing and applications of virtual reality and augmented reality to support and augment therapeutic rehabilitation.

Download Full Text PDF

, , , , , , , , , , ,

Leave a comment

[ARTICLE] Choice of Human–Computer Interaction Mode in Stroke Rehabilitation

Abstract

Background and Objective. Advances in technology are providing new forms of human–computer interaction. The current study examined one form of human–computer interaction, augmented reality (AR), whereby subjects train in the real-world workspace with virtual objects projected by the computer. Motor performances were compared with those obtained while subjects used a traditional human–computer interaction, that is, a personal computer (PC) with a mouse.

Methods. Patients used goal-directed arm movements to play AR and PC versions of the Fruit Ninja video game. The 2 versions required the same arm movements to control the game but had different cognitive demands. With AR, the game was projected onto the desktop, where subjects viewed the game plus their arm movements simultaneously, in the same visual coordinate space. In the PC version, subjects used the same arm movements but viewed the game by looking up at a computer monitor.

Results. Among 18 patients with chronic hemiparesis after stroke, the AR game was associated with 21% higher game scores (P = .0001), 19% faster reaching times (P = .0001), and 15% less movement variability (P = .0068), as compared to the PC game. Correlations between game score and arm motor status were stronger with the AR version.

Conclusions. Motor performances during the AR game were superior to those during the PC game. This result is due in part to the greater cognitive demands imposed by the PC game, a feature problematic for some patients but clinically useful for others. Mode of human–computer interface influences rehabilitation therapy demands and can be individualized for patients.

via Choice of Human–Computer Interaction Mode in Stroke Rehabilitation.

, , , , , , , , , , ,

Leave a comment

[ARTICLE] Choice of Human–Computer Interaction Mode in Stroke Rehabilitation

Abstract

Background and Objective. Advances in technology are providing new forms of human–computer interaction. The current study examined one form of human–computer interaction, augmented reality (AR), whereby subjects train in the real-world workspace with virtual objects projected by the computer. Motor performances were compared with those obtained while subjects used a traditional human–computer interaction, that is, a personal computer (PC) with a mouse.

Methods. Patients used goal-directed arm movements to play AR and PC versions of the Fruit Ninja video game. The 2 versions required the same arm movements to control the game but had different cognitive demands. With AR, the game was projected onto the desktop, where subjects viewed the game plus their arm movements simultaneously, in the same visual coordinate space. In the PC version, subjects used the same arm movements but viewed the game by looking up at a computer monitor.

Results. Among 18 patients with chronic hemiparesis after stroke, the AR game was associated with 21% higher game scores (P = .0001), 19% faster reaching times (P = .0001), and 15% less movement variability (P = .0068), as compared to the PC game. Correlations between game score and arm motor status were stronger with the AR version.

Conclusions. Motor performances during the AR game were superior to those during the PC game. This result is due in part to the greater cognitive demands imposed by the PC game, a feature problematic for some patients but clinically useful for others. Mode of human–computer interface influences rehabilitation therapy demands and can be individualized for patients.

via Choice of Human–Computer Interaction Mode in Stroke Rehabilitation.

, , , , , , , ,

Leave a comment

[ARTICLE] Utility of Augmented Reality in Relation to Virtual Reality in Stroke Rehabilitation

Abstract

Introduction: Virtual Reality (VR) has been found useful for numerous rehabilitation applications, but has some intrinsic constraints such as the need for a visuospatial transformation when guiding movements. Augmented Reality (AR) is a new approach to human-computer interaction that enables patients to interact directly with virtual objects. The current study compared AR and VR in a stroke rehabilitation setting.

ΦωτόMETHODS: The Fruit Ninja game simulates a rehab setting by having subjects perform repeated goal-directed wrist/hand reaching tasks. Subjects held a cup-shaped color-marker in the paretic hand, then reached for a virtual fruit target that sliced in 2 when reached. This game was implemented in both AR and VR settings, with identical movement demands across the two. The target plus real-time visual feedback on hand movements were provided by a computer monitor in VR, and by a projection onto a tabletop in AR. After undergoing baseline assessments (arm motor Fugl-Meyer scale (FMA) and Box and Blocks (B&B)), 10 patients with hemiparetic stroke >6 mo prior and age >18 yr played three 1-min rounds each of the AR and VR games; 4 other subjects who were unable to hold the color-marker object were excluded from current analysis.

RESULTS: Of the 10 patients, age = 59±10 yr (mean±SD), FMA score = 57±11 (range 31-66), Hand/Wrist FMA subscore = 22±3 (range 15-24), and B&B score = 41±13 (range 16-58). When playing the exact same Fruit Ninja game, all 10 patients scored significantly (p<0.0001) higher in the AR setting (60±9 targets, range 48-78) as compared to the VR setting (48±8 targets, range 37-64 setting. Also, AR scores were stronger correlates of FM Hand/Wrist (rho=0.68, p<0.04) and B&B scores (rho=0.70, p<0.03) than were VR scores.

CONCLUSIONS: This study shows promising results with use of Augmented Reality in a patient-computer interface. Results also suggest advantages as compared to use of a Virtual Reality approach, possibly due to the fact that moving the hand requires a visuospatial transform in the VR setting but not in the AR setting. Compared to VR, AR scores were higher and correlated better with clinical scores, suggesting great potential for the use of Augmented Reality in a patient-computer interface during stroke rehabilitation.

via Abstract T MP43: Utility of Augmented Reality in Relation to Virtual Reality in Stroke Rehabilitation.

, , , , , , , , , , ,

Leave a comment

[ARTICLE] Physio@Home: Exploring visual guidance and feedback techniques for physiotherapy exercises – Full Text PDF

ABSTRACT

Physiotherapy patients exercising at home alone are at risk of re-injury since they do not have corrective guidance from a therapist. To explore solutions to this problem, we designed Physio@Home, a prototype that guides people through pre-recorded physiotherapy exercises using realtime visual guides and multi-camera views. Our design addresses several aspects of corrective guidance, including: plane and range of movement, joint positions and angles, and extent of movement. We evaluated our design, comparing how closely people could follow exercise movements under various feedback conditions. Participants were most accurate when using our visual guide and multi-views. We provide suggestions for exercise guidance systems drawn from qualitative findings on visual feedback complexity.

Full Text PDF

, , , ,

Leave a comment

[ARTICLE] REAL TIME BIOSIGNAL-DRIVEN ILLUSION SYSTEM FOR UPPER LIMB REHABILITATION

Abstract

This paper presents design and development of real time biosignal-driven illusion system: Augmented Reality based Illusion System (ARIS) for upper limb motor rehabilitation. ARIS is a hospital / home based self- motivated whole arm rehabilitation system that aims to improve and restore the lost upper limb functions due to Cerebrovascular Accident (CVA) or stroke.

Taking the advantage of human brain plasticity nature, the system incorporates with number of technologies to provide fast recovery by re-establishing the neural pathways and synapses that able to control the mobility. These technologies include Augmented Reality (AR) where illusion environment is developed, computer vision technology to track multiple colors in real time, EMG acquisition system to detect the user intention in real time and 3D modelling library to develop Virtual Arm (VA) model where human biomechanics are applied to mimic the movement of real arm. The system operates according to the user intention via surface electromyography (sEMG) threshold level. In the case of real arm cannot reach to the desired position, VA will take over the job of real arm to complete the exercise.

The effectiveness of the developed ARIS has evaluated via questionnaire, graphical and analytical measurements which provided with positive results.

via [Abstract] REAL TIME BIOSIGNAL-DRIVEN ILLUSION SYSTEM FOR UPPER LIMB REHABILITATION.

, , , , , , , , , ,

Leave a comment

[ARTICLE] An augmented reality system for upper-limb post-stroke motor rehabilitation: a feasibility study

…Implications for Rehabilitation

Gain of range of motion of flexion and abduction of the shoulder of post-stroke patients can be achieved through an augmented reality system containing exercises to promote the mental practice.

NeuroR system provides a mental practice method combined with visual feedback for motor rehabilitation of chronic stroke patients, giving the illusion of injured upper-limb (UL) movements while the affected UL is resting. Its application is feasible and safe.

This system can be used to improve UL rehabilitation, an additional treatment past the traditional period of the stroke patient hospitalization and rehabilitation…

via An augmented reality system for upper-limb post-stroke motor rehabilitation: a feasibility study, Disability and Rehabilitation: Assistive Technology, Informa Healthcare.

 

, , , , , , ,

Leave a comment

%d bloggers like this: