Posts Tagged augmented reality

[WEB SITE] Beyond haptics: blurring the line between your virtual avatar and your body

Main image credit: Teslasuit

In the space of a few short years, virtual reality has gone from being a technology of the future to part of the mainstream. Devices ranging from the humble Google Cardboard to the Oculus Rift have invaded our living rooms, and VR is set to transform everything from education to the sex industry.

But if VR is to achieve the mass appeal many are predicting then it needs to feel, as well as look, as real as possible, and not just like we’re passively watching a TV set strapped to our faces; the rest of our body needs to be as engaged as fully as our eyes.

Let’s get physical

Enter haptic technology, which allows us to literally feel what we’re experiencing in VR. You’ve likely come across haptic tech, sometimes referred to as just ‘haptics’, before, for example when you’ve played a video game and felt a rumble in the handset.

Now companies like Tesla Suit and Hardlight VR are bringing that experience to your whole body, with suits that can move and shake and vibrate in specific areas as you explore virtual worlds.

Design sketches for the Teslasuit, which enables the wearer to experience touch, heat and cold. Image credit: Teslasuit

But let’s slow down for a second. We have to because this haptic tech is far from becoming mainstream and, crucially, you can’t just put a haptic suit on someone and expect a VR experience to feel real.

That’s why there’s a lot of research going on into what’s known as ‘virtual embodiment’.

This is a complex and fairly new area of study, but it’s concerned with using technology, virtual representation, avatars, storytelling, haptics and all kinds of other subtle visual, auditory and sensory cues to make you feel like you’re inhabiting another body. Whether that’s an avatar of yourself, someone else or even something else.

Exploring the body/mind connection

Virtual embodiment might be a new area of study, but it’s built on research about the connection between our minds and our bodies that goes back more than a decade.

One example is what’s known as the ‘rubber hand illusion’. This was an experiment that essentially proved that, with the right stimuli, people took full ownership of a rubber hand as their own.

Fast-forward to the present day and similar studies have put the rubber hand illusion to the test in a VR setting.

In a 2010 study, researchers found that synchrony between touch, visual information and movement could induce a believable illusion that people actually had full ownership of a virtual arm.

Similar studies have looked at the efficacy of using avatars for rehabilitation and visual therapy, with research suggesting that, in most cases, our virtual bodies can feel as real as our physical bodies.

Defining virtual embodiment

To find out where the research is right now, we spoke to Dr Abraham Campbell, Chief Research Officer at MeetingRoom and Head of the VR Lab at University College Dublin.

“Virtual embodiment is a difficult thing to define as it can mean a lot to different people in different fields,” Campbell explained. He proposes that we look at virtual embodiment in three categories, all of which are a modified version of Tom Ziemke’s work on embodiment.

“Firstly, structural coupling is the most basic and classic definition of embodiment,” Campbell told us. “You’re connected to some form of structure. For example, a body. You move your limb in real life, and a virtual limb moves mimicking your actions…you are embodied within the VR world.”

Campbell offers the example of moving the HTC Vive controller in the real world, and that becoming a hand that’s moving in the virtual world.

Structural coupling – for example controlling virtual limbs with your own limbs – is the basic definition of embodiment within a virtual world. Image credit: Razer OSVR

Next up is historical embodiment, which is when the VR world you enter ‘remembers’ what’s happened in the past. Campbell uses the example of drawing on a white board, when what you’ve drawn stays there when you return in a day, a week or year from now.

“Finally, social embodiment is when you interact with real or artificial entities within the VR world,” Campbell says. “These interactions have to be behaviorally realistic, so you feel that your body is able to interact with them in the environment.”

And why is studying embodiment important? Campbell explains: “The more embodied the agent or human is within the environment, the more capable they are of interacting and sensing that world.”

Social interaction and education

Campbell’s main focus is on social collaboration in a recreational and educational setting.

“I’m examining the use of telepresence and VR in education, and exploring how I can remotely teach from Ireland to China using technology like Kinect [Microsoft’s motion-sensing input devices for its Xbox game consoles] to scan me in real time, while at the same time view the classroom in China using a set of projectors,” he said.

Bringing a teacher into a distant, virtual classroom will certainly be useful. But the next challenge is working on the intricacies of social interaction, such as facial expressions.

”Being able to convincingly communicate with others in a face-to-face way at a distance is one of the most exciting possibilities.”

Dr Gary McKeown

And although interacting with people may not sound like the most interesting use of virtual embodiment, it’s one that’s bound to get attention.

“It is also clearly an industry goal,” Dr Gary McKeown, Senior Lecturer at the School of Psychology at Queen’s University Belfast, tells us. “It is not a coincidence that the company with the most to gain from making the social interaction aspects of virtual reality function well – Facebook – is the one that bought Oculus.”

Remote instruction

Imagine being able to remotely control machinery, or just help out family fullyfrom thousands of miles away – it would change so much about work, commuting and social interaction.

This is one area that’s particularly interesting to Campbell is using embodiment research to aid telepresence or telerobotics, which is the use of virtual reality or augmented reality (AR) to do just that.

“I’m fascinated by Remote Expert, which is being pioneered by DAQRI,” he told us. “This allows an expert in a field to be remotely placed in augmented reality beside a non-expert to perform a complex task. DAQRI are looking at medical and industry fields to apply this technology, but you can imagine lots more applications.”

Campbell explained that one of the many uses for this kind of tech could be if an oil pipeline bursts, and the engineer who designed it is in another country. A local engineer could go out to fix the pipeline , with the designer advising them in real time using VR or AR and a stereo 360-degree camera.

A European Space Agency astronaut remotely guides a surface rover around a test site in California as part of NASA’s Surface Telerobotics program. Image credit: NASA

As we learned above, the tech enables the presence element of this. But where embodiment research comes in is making it more engaging, more realistic… ultimately more real, and with it the power to really offer help unhindered by technology.

Campbell explained: “The remote expert needs to be able to use hand gestures to demonstrate what the non-expert should do.

“The expert should be scanned in 3D in real time along with the remote world they are being placed into. This embodiment will allow them to truly be able to assist in whatever complex ask they’re asked to perform.”

The implications of this are massive, and could radically change a number of industries.

NASA already has a telerobotics research arm that’s looking at using this technology for space exploration, and it’s being introduced into other fields, from engineering to medicine.

Campbell believes this kind of telepresence will have a big impact on the medical industry as technology advances too.

“One solution I hope to explore in future is to use a full hologram projector pyramid,” he told us. “This approach has been suggested to me by medical professionals who want to meet patients remotely by using a full size projector pyramid [i.e. one that’s about two metres tall]. With this kind of tech, the doctor will be better able to diagnose a patient.”

Therapy and rehabilitation

Virtual embodiment doesn’t just have huge implications for exploring physical presence, but mental and emotional presence too.

In a 2016 study, researchers discovered that virtual avatars that look like our physical selves can help people feel a sense of embodiment and immersion that it’s believed could enable them to better work through mental health challenges, as well as real-world trauma.

VR software developer ProReal uses virtual environments containing avatars to play out scenarios that help people deal with a range of challenges, from bullying to PTSD and rehabilitation.

As the tech advances, it could provide a whole new area of therapy for those who aren’t getting the results they need from talking therapies or medication.

Avatars are useful for exploring different perspectives in complex therapeutic situations. Image credit: ProReal

But it’s not just more serious mental health challenges, like PTSD, that can be explored; avatars can be used to increase confidence or change our perception of ourselves. Campbell told us about the time he noticed that those with bigger avatars felt more powerful.

“One accidental discovery I had, when I looked at games in VR, was that when the avatar is on average one foot taller than the other characters, it makes the player feel more powerful than the computer-controlled characters,” he explained.

That observation mirrors a 2009 study in which researchers found that those given taller, bigger avatars behaved differently and more aggressively in interactions with others.

So aside from therapy and mental health use cases, it’s possible to imagine VR being used in corporate settings, to make people feel more confident before presenting to a boardroom.

The challenges of embodiment

If it’s easy for researchers to think of creative ways in which embodiment could have a positive impact on our lives, it’s not much of a leap to consider the negatives too.

Some tech commentators believe social isolation could be an issue as the use of VR headsets becomes more widespread and experiences become more immersive, a concern that’s likely to become more prevalent in gaming.

But many within the industry believe the focus on social isolation is just scaremongering.

“I haven’t witnessed people feeling isolation,” Campbell explained. “Even students who are interested in VR for pure escapism want to share it with others afterwards, and have become evangelists for VR in its ability to be an empathy machine, as with embodiment you can truly get a sense of seeing things from someone else’s perspective.”

Another important talking point is around dissociation or detachment from your own body after exploring virtual worlds. There’s been very little research in this area, but one study from 2006 found that VR can increase dissociative experiences, especially if you’ve been immersed in a virtual world for a long time.

Home – A VR Spacewalk, created by Rewind, lets you take a trip to the International Space Station. Image credit: Home A VR Spacewalk/Rewind

More than a decade on, and with better VR technology and content it’s no surprise that lots of anecdotal evidence points to a similar ‘post virtual reality sadness’, in which the real world doesn’t quite compare.

One potentially problematic side-effect Campbell thinks we do need to consider right away is addiction. But he explains that, unlike with traditional gaming addiction, VR can be designed differently.

“In VR, the user needs to replicate the real-world actions and thus shortcuts the traditional dopamine-hit reward cycle that people often become addicted to,” he told us.

So, for example, if you win a gaming level within VR you’ve likely put a lot of physical effort and exertion in, perhaps by killing the big bad boss at the end of the level. You’re likely to be physically tired. That’s the difference.

Of course you can still get addicted to that feeling – people get addicted to working out – but Campbell tells us: “It’s the responsibility of game designers to make sure that VR games reward a player for real effort and not make a game that’s hard at first to complete, but then actually gets easier.”

As tech and embodiment research advances, the potential of VR could know no bounds. Image credit: Pexels.com

We spoke to Sol Rogers, the Founder and CEO of creative agency and VR production studio Rewind, about some of these concerns.

“We’ve studied how humans interact in and with the real world for hundreds of years, and we’ll probably need the same amount of time to study how humans behave in VR and what the implications are,” he told us.

But he urged people to be excited about the prospect of what VR can do, not scared. “While we can only speculate about the impact VR will have, we need to progress with watchful caution rather than hysteria,” he added.

“Self-regulation from content creators is key, but a governing body also needs to take on some responsibility. Ultimately we need more research, and more time, to fully understand the implications.”

The recipe for greater embodiment

But embodiment is only convincing if everything else in the experience is up to scratch. We asked Rogers about how his team works with tech to create the most realistic experiences.

“Achieving a lifelike user experience in VR is now possible because of tremendous advancements in computer processing power, graphics, video and display technologies,” he told us.

“But the tech needs to stay out of the way; it needs to be entirely inconsequential to the experience, otherwise the spell is broken.”

”The tech needs to be entirely inconsequential to the experience, otherwise the spell is broken.”

Sol Rogers, CEO of Rewind

And he adds that the tech is only half of the equation; his job is to ensure the content is telling the best possible story, every step of the way. “Content is also key to creating presence. While the tech is no doubt important, no user is going to suspend disbelief if the experience is awful.”

From entertainment and social interaction to engineering and performing medical procedures, the more we understand, test and implement embodiment experiments, the more we can engineer experiences to feel real – and in turn be more effective.

With advances in research from the likes of Campbell and his team, along with advances in tech to make headsets slimmer, sensory feedback easier to implement and full-body holograms a reality, the sky is only virtually the limit.

This article is brought to you in association with Vodafone.

Source: Beyond haptics: blurring the line between your virtual avatar and your body | TechRadar

Advertisements

, , , , , ,

Leave a comment

[WHITE PAPER] Virtual and augmented reality based balance and gait training – Full Text PDF

The use of virtual and augmented reality for rehabilitation has become increasingly popular and has received much attention in scientific publications (over 1,000 papers). This white paper aims to summarize the scientific background and efficacy of using virtual and augmented reality for balance and gait training. For many patients with movement disorders, balance and gait training is an important aspect of their rehabilitation process and physical therapy treatment. Indications for such training include, among others, stroke, Parkinson’s disease, multiple sclerosis, cerebral palsy, vestibular disorders, neuromuscular diseases, low back pain, and various orthopedic complaints, such as total hip or knee replacement. Current clinical practice for balance training include exercises, such as standing on one leg, wobble board exercises and standing with eyes closed. Gait is often trained with a treadmill or using an obstacle course. Cognitive elements can be added by asking the patient to simultaneously perform a cognitive task, such as counting down by sevens. Although conventional physical therapy has proven to be effective in improving balance and gait,1,2 there are certain limitations that may compromise treatment effects. Motor learning research has revealed some important concepts to optimize rehabilitation: an external focus of attention, implicit learning, variable practice, training intensity, task specificity, and feedback on performance.3 Complying with these motor learning principles using conventional methods is quite challenging. For example, there are only a limited number of exercises, making it difficult to tailor training intensity and provide sufficient variation. Moreover, performance measures are not available and thus the patient usually receives little or no feedback. Also, increasing task specificity by simulating everyday tasks, such as walking on a crowded street, can be difficult and time consuming. Virtual and augmented reality could provide the tools needed to overcome these challenges in conventional therapy. The difference between virtual and augmented reality is that virtual reality offers a virtual world that is separate from the real world, while augmented reality offers virtual elements as an overlay to the real world (for example virtual stepping stones projected on the floor). In the first part of this paper we will explain the different motor learning principles, and how virtual and augmented reality based exercise could help to incorporate these principles into clinical practice. In the second part we will summarize the scientific evidence regarding the efficacy of virtual reality based balance and gait training for clinical rehabilitation.

Full Text PDF

, , , , ,

Leave a comment

[Editorial] E-Rehabilitation: New Reality or Virtual Need?

 

This is an era of digitalization, internet, wifi, use of mobile and smart phones, virtual world, applications and technology. On one hand these are contributing to cyber psychopathology, on the other hand these have a potential for management.

With the understanding of disability as a complex interaction between the effects of illness and contextual factors, both personal and environmental, the relevance of new avenues to deliver rehabilitative services is profound. A significant proportion of the population is underserved, with the National Mental Health Survey of India 2016- a survey which covered 34,802 individuals from 12 states of India- showing a mental morbidity of 10.6% in those over the age of 18 years, and 7.3% in those between the ages of 13 and 17, but with a treatment gap of 28–83% (and 86% for alcohol use disorders). In addition, “three out of four persons with a severe mental disorder experienced significant disability in work, social and family life” [1]. Given the extent of the need and the dearth of services, the report recommends the following, “Technology based applications for near-to-home-based care using smart-phone by health workers, evidence-based (electronic) clinical decision support systems for adopting minimum levels of care by doctors, creating systems for longitudinal follow-up of affected persons to ensure continued care through electronic databases and registers can greatly help in this direction. To facilitate this, convergence with other flagship schemes such as Digital India needs to be explored” [1]. Recent data has shown that smartphone user base in India has crossed 300 million users in 2016, making it the second largest smartphone market in the world [2]. The potential for service delivery via internet enabled devices seems likely only to rise over time, but what are the possibilities before us now, and equally important, what are the challenges to such approaches?

An exploration of the role of modern technology in rehabilitation in January, 2016, has highlighted the various possibilities in terms of social networking and peer support, telepsychiatry, E health services as well as smartphones and apps [3]. It’s interesting that estimates at the time alluded to smartphone users crossing the 200 million mark in 2016, a 100 million users less than later estimates! Looking ahead these are the ways new and emerging technologies could change the ways we approach and conceptualise recovery,

  1. (a)

    Information access: Access to information and more specifically, access to relevant and accurate information have to potential to allow caregivers and patients to recognise mental health issues early, and seek help. Some of this information will be from traditional media, such as radio and television, but a significant proportion of people are likely to glean this information from social media sites and communication apps—such as the almost ubiquitous Whatsapp—on which they also consume other services and obtain their daily news and information from. Search algorithms and the way they rank different sources of information are likely to play an important role in the way people form their opinions about the illnesses they suffer from and the way they seek help. There is a need for curated information on mental health, especially in the Indian context and in vernacular languages, that people can not only refer to themselves, but which they can direct their friends and family toward as reliable sources of information too. Health care professionals must be prepared to help their patients learn ‘eHealth literacy’ [4].

  2. (b)

    Automation: Work is something that most people with mental illness aspire to do, and this can enhance their quality of life significantly [5]. Automation and applications of artificial intelligence are poised to change the face of industry as well as our lifestyles. Some traditional jobs such as fabrication and driving are poised to radically change. This will mean that vocational rehabilitation programmes will have to keep pace with a changing environment, and look to integrating industry expertise in the designing of courses and course materials which remain relevant to patients. Government programmes such as the Skill India initiative have the potential to help evolve this flexibility in course design, and to skill or re-skill persons in their quest to obtain and sustain jobs.

    Workplace is being replaced by home based workstations, computers, laptops and notebooks. People accustomed to these run their office from anywhere and everywhere. There will be a need to redefine ‘work place’ as ‘where ever the laptop is’. Thus, in future, persons undergoing rehabilitation, can ‘work from home’, provided they have the facilities, and job to do. Staying and working from home for persons with mental health problems, will prevent them from ‘live’ socialising, using social skills, and giving respite to family caregivers. On the other hand, they would be under direct supervision of the family, reducing their concerns and anxieties.

  3. (c)

    Digital identities and digital payments: With the increasing digitisation of access to services, there is a growing need for education in digital literacy and security. Programmes which teach life skills will have to help their users familiarise themselves with the advantages of new technologies as well as the risks they bring. A number of records related to disability are likely to form parts of central databases, such as the Unique Disability ID [6], and the potential to offer a number of services through a single user interface to those with disability is significant. It would also ease the accessing of such benefits even when patients travel or move to other states, whether temporarily or permanently. The storage of health records in electronic formats, e-health records, would allow patients to exert control over access to their own records and enable transfers from one healthcare provider to another without delay or loss of information. An e-health record format which is shared among different providers and which allows different hospital information systems to effectively share information is an important need. There can be a possibility to maintain a central registry of persons receiving mental health rehabilitation services.

  4. (d)

    Wearables and digital phenotyping: The mobile devices and other wearable accessories we use have the potential to collect vast amounts of information about our health. Newer approaches look to collect information such as changes in the speed of our typing or motor movements, or the searches we repeat and use these to make estimates about the status of our cognitive and neurological health in real time–an approach called digital phenotyping. This could aid in monitoring persons suffering from dementia or mild cognitive deficits. It could also be used to explore trajectories of development in children and adolescents, and could help inform early intervention programmes. Over and above monitoring, the use of digital assistants could be used to guide and shape behaviour in real time, provide cognitive aids and reduce dependency as well as the burden on caregivers for some tasks.

  5. (e)

    Virtual Reality and Augmented reality: Virtual reality (VR) refers to an interactive immersive experience wherein a computer generated world which a user can interact with is simulated with either a screen or a heads-up display. Augmented reality systems allow perception of the environment around along with the simulated projection. It’s also used to refer to situations where mobile phones or wearables can be used to interact with the environment around to either generate a virtual experience or provide additional information.

    It’s been used as an application for interventions in phobias for some time. Recent gains in the technology have coincided with an expansion of uses to cognitive rehabilitation, social skills training and even craving management in alcohol use disorders [7]. The number of mental health professionals available to deliver these services is low compared to demand and unequally distributed. With the evolution of mobile systems that can deliver VR experiences, such as the Google Daydream platform, it may be possible to translate some of these packages into content that can be delivered across such platforms with fidelity. There is still some work to be done about how perception of such experiences can affect symptoms in those with mental illness, and even if the same visual illusions are perceived differently.

  6. (f)

    Social networks, communication apps and peer support: Social networks and social media increasingly influence information access and viewpoints. They can serve as accepting communities to which people can feel as if they belong. They can also carry risks, including the spread of myths and misconceptions. Peer support groups, much like other networks, are now easier to form and to find. Hence, the potential for persons with mental illness to be involved in advocacy movements and to influence public policy is unprecedented, if still underutilised. The ability to use social networks and the internet to market products and expand networks can help those who chose to be entrepreneurs have greater reach and exposure. The ability to use these networks effectively, and other marketing skills, would also become a skill set that requires mentoring in.

  7. (g)

    The use of learning networks: Virtual classrooms and virtual learning networks have the potential to raise standards of care delivery by spreading best care practices and knowledge. Initiatives like the ECHO network and the Virtual Knowledge Network, NIMHANS can help spread the expertise of institutes by mentoring professionals who are involved in care delivery. They can also serve as ways to allow different institutes to demonstrate their own best practices and innovative models of service delivery to their peers.

The future of psychiatric practice, including psychiatric rehabilitation, in relation to virtual reality, technology and gadgets is likely to change with advances in technology and their usage [8]. While the tools that are available are changing, they will still be guided by the principles that form the bedrock of good practice in rehabilitation. Patients and their families may be drawn to online resources for rehabilitation.

The current issue of the journal is rather healthy with seventeen articles. And there is a good global distribution as well, with descriptions of mental health and rehabilitation services in Vietnam, Nigeria, USA, UK, Canada, Malaysia, and Iran. These have also covered a wide range of themes, from recovery scales, models for community based rehabilitation and community participation, in patient services, first episode psychosis, helping mothers with intellectual disabilities, and infertility. In addition, a book review on a very useful book on challenges of care giving for mental illness, cover an interesting spectrum of articles.

Source: E-Rehabilitation: New Reality or Virtual Need? | SpringerLink

, , , , , , , , , , , ,

Leave a comment

[THESIS] AUGMENTED REALITY SYSTEM FOR REHABILITATION: NEW APPROACH BASED ON HUMAN INTERACTION AND BIOFEEDBACK – Full Text PDF

Abstract

Rehabilitation is the process of training for someone in order to recover or improve their lost functions caused by neurological deficits. The upper limb rehabilitation system provides relearning of motor skills that are lost due to any neurological injuries via motor rehabilitation training. The process of motor rehabilitation is a form of motor learning via practice or experience. It requires thorough understanding and examination of neural processes involved in producing movement and learning as well as the medical aspects that may affect the central nervous system (CNS) or peripheral nervous system (PNS) in order to develop an effective treatment system. Although there are numerous rehabilitation systems which have been proposed in literatures, a low cost upper limb rehabilitation system that maximizes the functional recovery by stimulating the neural plasticity is not widely available. This is due to lack of motivation during rehabilitation training, lack of real time biofeedback information with complete database, the requirement of one to one attention between physiotherapist and patient, the technique to stimulate human neural plasticity.

Therefore, the main objective of this thesis is to develop a novel low cost rehabilitation system that helps recovery not only from loss of physical functions, but also from loss of cognitive functions to fulfill the aforementioned gaps via multimodal technologies such as augmented reality (AR), computer vision and signal processing. In order to fulfill such ambitious objectives, the following contributions have been implemented.

Firstly, since improvements in physical functions are targeted, the Rehabilitation system with Biofeedback simulation (RehaBio) is developed. The system enhances user’s motivation via game based therapeutic exercises and biofeedback. For this, AR based therapeutic games are developed to provide eye-hand coordination with inspiration in motivation via immediate audio and visual feedback. All the exercises in RehaBio are developed in a safe training environment for paralyzed patients. In addition to that, realtime biofeedback simulation is developed and integrated to serve in two ways: (1) from the patient’s point of view, the biofeedback simulation motivates the user to execute the movements since it will animate the different muscles in different colors, and (2) from the therapist’s point of view, the muscle simulations and EMG threshold level can be evaluated as patient’s muscle performance throughout the rehabilitation process.

Secondly, a new technique that stimulates the human neural plasticity is proposed. This is a virtual human arm (VHA) model that driven by proposed continuous joint angle prediction in real time based on human biological signal, Electromyogram (EMG). The VHA model simulation aims to create the illusion environment in Augmented Realitybased Illusion System (ARIS).

Finally, a complete novel upper limb rehabilitation system, Augmented Reality-based Illusion System (ARIS) is developed. The system incorporates some of the developments in RehaBio and real time VHA model to develop the illusion environment. By conducting the rehabilitation training with ARIS, user’s neural plasticity will be stimulated to reestablish the neural pathways and synapses that are able to control mobility. This is achieved via an illusion concept where an illusion scene is created in AR environment to remove the impaired real arm virtually and replace it with VHA model to be perceived as part of the user’s own body. The job of the VHA model in ARIS is when the real arm cannot perform the required task, it will take over the job of the real one and will let the user perceive the sense that the user is still able to perform the reaching movement by their own effort to the destination point. Integration with AR based therapeutic exercises and motivated immediate intrinsic and extrinsic feedback in ARIS leads to serve as a novel upper limb rehabilitation system in a clinical setting.

The usability tests and verification process of the proposed systems are conducted and provided with very encouraging results. Furthermore, the developments have been demonstrated to the clinical experts in the rehabilitation field at Port Kembla Hospital. The feedback from the professionals is very positive for both the RehaBio and ARIS systems and they have been recommended to be used in the clinical setting for paralyzed patients.

Download Full Text PDF

, , , , , , , , ,

Leave a comment

[THESIS] AUGMENTED REALITY SYSTEM FOR REHABILITATION: NEW APPROACH BASED ON HUMAN INTERACTION AND BIOFEEDBACK – Full Text PDF

Abstract

Rehabilitation is the process of training for someone in order to recover or improve their lost functions caused by neurological deficits. The upper limb rehabilitation system provides relearning of motor skills that are lost due to any neurological injuries via motor rehabilitation training. The process of motor rehabilitation is a form of motor learning via practice or experience. It requires thorough understanding and examination of neural processes involved in producing movement and learning as well as the medical aspects that may affect the central nervous system (CNS) or peripheral nervous system (PNS) in order to develop an effective treatment system. Although there are numerous rehabilitation systems which have been proposed in literatures, a low cost upper limb rehabilitation system that maximizes the functional recovery by stimulating the neural plasticity is not widely available. This is due to lack of motivation during rehabilitation training, lack of real time biofeedback information with complete database, the requirement of one to one attention between physiotherapist and patient, the technique to stimulate human neural plasticity. Therefore, the main objective of this thesis is to develop a novel low cost rehabilitation system that helps recovery not only from loss of physical functions, but also from loss of cognitive functions to fulfill the aforementioned gaps via multimodal technologies such as augmented reality (AR), computer vision and signal processing. In order to fulfill such ambitious objectives, the following contributions have been implemented. Firstly, since improvements in physical functions are targeted, the Rehabilitation system with Biofeedback simulation (RehaBio) is developed. The system enhances user’s motivation via game based therapeutic exercises and biofeedback. For this, AR based therapeutic games are developed to provide eye-hand coordination with inspiration in motivation via immediate audio and visual feedback. All the exercises in RehaBio are developed in a safe training environment for paralyzed patients. In addition to that, realtime biofeedback simulation is developed and integrated to serve in two ways: (1) from the patient’s point of view, the biofeedback simulation motivates the user to execute the movements since it will animate the different muscles in different colors, and (2) from the therapist’s point of view, the muscle simulations and EMG threshold level can be evaluated as patient’s muscle performance throughout the rehabilitation process. Secondly, a new technique that stimulates the human neural plasticity is proposed. This is a virtual human arm (VHA) model that driven by proposed continuous joint angle prediction in real time based on human biological signal, Electromyogram (EMG). The VHA model simulation aims to create the illusion environment in Augmented Realitybased Illusion System (ARIS). Finally, a complete novel upper limb rehabilitation system, Augmented Reality-based Illusion System (ARIS) is developed. The system incorporates some of the developments in RehaBio and real time VHA model to develop the illusion environment. By conducting the rehabilitation training with ARIS, user’s neural plasticity will be stimulated to reestablish the neural pathways and synapses that are able to control mobility. This is achieved via an illusion concept where an illusion scene is created in AR environment to remove the impaired real arm virtually and replace it with VHA model to be perceived as part of the user’s own body. The job of the VHA model in ARIS is when the real arm cannot perform the required task, it will take over the job of the real one and will let the user perceive the sense that the user is still able to perform the reaching movement by their own effort to the destination point. Integration with AR based therapeutic exercises and motivated immediate intrinsic and extrinsic feedback in ARIS leads to serve as a novel upper limb rehabilitation system in a clinical setting. The usability tests and verification process of the proposed systems are conducted and provided with very encouraging results. Furthermore, the developments have been demonstrated to the clinical experts in the rehabilitation field at Port Kembla Hospital. The feedback from the professionals is very positive for both the RehaBio and ARIS systems and they have been recommended to be used in the clinical setting for paralyzed patients.

Download Full Text PDF

, , , , , ,

Leave a comment

[ARTICLE] Effect of a mixed reality-based intervention on arm, hand, and finger function on chronic stroke – Full Text

Abstract

Background

Virtual and mixed reality systems have been suggested to promote motor recovery after stroke. Basing on the existing evidence on motor learning, we have developed a portable and low-cost mixed reality tabletop system that transforms a conventional table in a virtual environment for upper limb rehabilitation. The system allows intensive and customized training of a wide range of arm, hand, and finger movements and enables interaction with tangible objects, while providing audiovisual feedback of the participants’ performance in gamified tasks. This study evaluates the clinical effectiveness and the acceptance of an experimental intervention with the system in chronic stroke survivors.

Methods

Thirty individuals with stroke were included in a reversal (A-B-A) study. Phase A consisted of 30 sessions of conventional physical therapy. Phase B consisted of 30 training sessions with the experimental system. Both interventions involved flexion and extension of the elbow, wrist, and fingers, and grasping of different objects. Sessions were 45-min long and were administered three to five days a week. The body structures (Modified Ashworth Scale), functions (Motricity Index, Fugl-Meyer Assessment Scale), activities (Manual Function Test, Wolf Motor Function Test, Box and Blocks Test, Nine Hole Peg Test), and participation (Motor Activity Log) were assessed before and after each phase. Acceptance of the system was also assessed after phase B (System Usability Scale, Intrinsic Motivation Inventory).

Results

Significant improvement was detected after the intervention with the system in the activity, both in arm function measured by the Wolf Motor Function Test (p < 0.01) and finger dexterity measured by the Box and Blocks Test (p < 0.01) and the Nine Hole Peg Test (p < 0.01); and participation (p < 0.01), which was maintained to the end of the study. The experimental system was reported as highly usable, enjoyable, and motivating.

Conclusions

Our results support the clinical effectiveness of mixed reality interventions that satisfy the motor learning principles for upper limb rehabilitation in chronic stroke survivors. This characteristic, together with the low cost of the system, its portability, and its acceptance could promote the integration of these systems in the clinical practice as an alternative to more expensive systems, such as robotic instruments.

Continue —>  Effect of a mixed reality-based intervention on arm, hand, and finger function on chronic stroke | Journal of NeuroEngineering and Rehabilitation | Full Text

Fig. 3 Description of the exercises. The exercises covered a wide range of hand and arm movements, mostly focusing on the flexion and extension of the elbow and the wrist. a Exercise: to sweep the crumbs from the table. Movement: flexion-extension of the wrist without involving the fingers. b Exercise: to grate. Movement: Grasping and flexion-extension of the wrist. c Exercise: to knock on doors. Movement: flexion-extension of the wrist against gravity. d Exercise: to cook. Movement: grasping involving flexion-extension of the elbow and rotation of the shoulders. e Exercise: to squeeze a sponge. Movement: flexion-extension of the metacarpophalangeal-interphalangeal joint. f Exercise: to dial a number. Movement: tapping. g Exercise: to play piano. Movement: flexion-extension of the thumb, index, and middle finger. h Exercise: to buy items. Movement: pincer grasping with the thumb and index involving flexion-extension of the elbow and rotation of the shoulders

 

, , , , , ,

Leave a comment

[Survey] A Survey on Assistive Technology using Natural User Interface(NUI) computing to support and augment therapeutic rehabilitation – Full Text PDF

Abstract

Therapeutic rehabilitation is a specialty in medical field that deals with diagnosis, evaluation, treatment and management of people with all ages with physical or psychosomatic disabilities due to congenital disorder, accidents or aging problem. It deals with deduction and reduction in disabilities by providing improvement and restoration of movement and functional ability through regular and repetitive physical therapy exercises continued after discharge from the hospital.

However, the efficient treatment sessions are not guaranteed due to lack of therapists and facilities, patients were alone for over 60% of the day, patients engaged in ‘activity’ for only 13% of the day, undergoing some traditional therapies make patients to lose their interest and motivating patients to continue the exercises is lagging, which in turn makes longer time for recovery.

Thus, there is a need to find ways of cost effective, engaging and motivated training to support and improve recovery and rehabilitation. The focus is to use technology as a solution involving various computing techniques as a supplementary treatment to traditional rehabilitation and continued assessment of disabled patients.

Natural User Interface (NUI) is the emerging technique with the ability to interact with computers or smart devices using the human body. NUI computing is powered by human touch, gesture, voice, thoughts and senses.

This paper is a survey on assistive technology using emerging NUI computing techniques like touch computing, gesture computing, surface computing, brain computing and applications of virtual reality and augmented reality to support and augment therapeutic rehabilitation.

Download Full Text PDF

, , , , , , , , , , ,

Leave a comment

[ARTICLE] Choice of Human–Computer Interaction Mode in Stroke Rehabilitation

Abstract

Background and Objective. Advances in technology are providing new forms of human–computer interaction. The current study examined one form of human–computer interaction, augmented reality (AR), whereby subjects train in the real-world workspace with virtual objects projected by the computer. Motor performances were compared with those obtained while subjects used a traditional human–computer interaction, that is, a personal computer (PC) with a mouse.

Methods. Patients used goal-directed arm movements to play AR and PC versions of the Fruit Ninja video game. The 2 versions required the same arm movements to control the game but had different cognitive demands. With AR, the game was projected onto the desktop, where subjects viewed the game plus their arm movements simultaneously, in the same visual coordinate space. In the PC version, subjects used the same arm movements but viewed the game by looking up at a computer monitor.

Results. Among 18 patients with chronic hemiparesis after stroke, the AR game was associated with 21% higher game scores (P = .0001), 19% faster reaching times (P = .0001), and 15% less movement variability (P = .0068), as compared to the PC game. Correlations between game score and arm motor status were stronger with the AR version.

Conclusions. Motor performances during the AR game were superior to those during the PC game. This result is due in part to the greater cognitive demands imposed by the PC game, a feature problematic for some patients but clinically useful for others. Mode of human–computer interface influences rehabilitation therapy demands and can be individualized for patients.

via Choice of Human–Computer Interaction Mode in Stroke Rehabilitation.

, , , , , , , , , , ,

Leave a comment

[ARTICLE] Choice of Human–Computer Interaction Mode in Stroke Rehabilitation

Abstract

Background and Objective. Advances in technology are providing new forms of human–computer interaction. The current study examined one form of human–computer interaction, augmented reality (AR), whereby subjects train in the real-world workspace with virtual objects projected by the computer. Motor performances were compared with those obtained while subjects used a traditional human–computer interaction, that is, a personal computer (PC) with a mouse.

Methods. Patients used goal-directed arm movements to play AR and PC versions of the Fruit Ninja video game. The 2 versions required the same arm movements to control the game but had different cognitive demands. With AR, the game was projected onto the desktop, where subjects viewed the game plus their arm movements simultaneously, in the same visual coordinate space. In the PC version, subjects used the same arm movements but viewed the game by looking up at a computer monitor.

Results. Among 18 patients with chronic hemiparesis after stroke, the AR game was associated with 21% higher game scores (P = .0001), 19% faster reaching times (P = .0001), and 15% less movement variability (P = .0068), as compared to the PC game. Correlations between game score and arm motor status were stronger with the AR version.

Conclusions. Motor performances during the AR game were superior to those during the PC game. This result is due in part to the greater cognitive demands imposed by the PC game, a feature problematic for some patients but clinically useful for others. Mode of human–computer interface influences rehabilitation therapy demands and can be individualized for patients.

via Choice of Human–Computer Interaction Mode in Stroke Rehabilitation.

, , , , , , , ,

Leave a comment

[ARTICLE] Utility of Augmented Reality in Relation to Virtual Reality in Stroke Rehabilitation

Abstract

Introduction: Virtual Reality (VR) has been found useful for numerous rehabilitation applications, but has some intrinsic constraints such as the need for a visuospatial transformation when guiding movements. Augmented Reality (AR) is a new approach to human-computer interaction that enables patients to interact directly with virtual objects. The current study compared AR and VR in a stroke rehabilitation setting.

ΦωτόMETHODS: The Fruit Ninja game simulates a rehab setting by having subjects perform repeated goal-directed wrist/hand reaching tasks. Subjects held a cup-shaped color-marker in the paretic hand, then reached for a virtual fruit target that sliced in 2 when reached. This game was implemented in both AR and VR settings, with identical movement demands across the two. The target plus real-time visual feedback on hand movements were provided by a computer monitor in VR, and by a projection onto a tabletop in AR. After undergoing baseline assessments (arm motor Fugl-Meyer scale (FMA) and Box and Blocks (B&B)), 10 patients with hemiparetic stroke >6 mo prior and age >18 yr played three 1-min rounds each of the AR and VR games; 4 other subjects who were unable to hold the color-marker object were excluded from current analysis.

RESULTS: Of the 10 patients, age = 59±10 yr (mean±SD), FMA score = 57±11 (range 31-66), Hand/Wrist FMA subscore = 22±3 (range 15-24), and B&B score = 41±13 (range 16-58). When playing the exact same Fruit Ninja game, all 10 patients scored significantly (p<0.0001) higher in the AR setting (60±9 targets, range 48-78) as compared to the VR setting (48±8 targets, range 37-64 setting. Also, AR scores were stronger correlates of FM Hand/Wrist (rho=0.68, p<0.04) and B&B scores (rho=0.70, p<0.03) than were VR scores.

CONCLUSIONS: This study shows promising results with use of Augmented Reality in a patient-computer interface. Results also suggest advantages as compared to use of a Virtual Reality approach, possibly due to the fact that moving the hand requires a visuospatial transform in the VR setting but not in the AR setting. Compared to VR, AR scores were higher and correlated better with clinical scores, suggesting great potential for the use of Augmented Reality in a patient-computer interface during stroke rehabilitation.

via Abstract T MP43: Utility of Augmented Reality in Relation to Virtual Reality in Stroke Rehabilitation.

, , , , , , , , , , ,

Leave a comment

%d bloggers like this: