[ARTICLE] Transfer of motor skill between virtual reality viewed using a head-mounted display and conventional screen environments – Full Text

Abstract

Background

Virtual reality viewed using a head-mounted display (HMD-VR) has the potential to be a useful tool for motor learning and rehabilitation. However, when developing tools for these purposes, it is important to design applications that will effectively transfer to the real world. Therefore, it is essential to understand whether motor skills transfer between HMD-VR and conventional screen-based environments and what factors predict transfer.

Methods

We randomized 70 healthy participants into two groups. Both groups trained on a well-established measure of motor skill acquisition, the Sequential Visual Isometric Pinch Task (SVIPT), either in HMD-VR or in a conventional environment (i.e., computer screen). We then tested whether the motor skills transferred from HMD-VR to the computer screen, and vice versa. After the completion of the experiment, participants responded to questions relating to their presence in their respective training environment, age, gender, video game use, and previous HMD-VR experience. Using multivariate and univariate linear regression, we then examined whether any personal factors from the questionnaires predicted individual differences in motor skill transfer between environments.

Results

Our results suggest that motor skill acquisition of this task occurs at the same rate in both HMD-VR and conventional screen environments. However, the motor skills acquired in HMD-VR did not transfer to the screen environment. While this decrease in motor skill performance when moving to the screen environment was not significantly predicted by self-reported factors, there were trends for correlations with presence and previous HMD-VR experience. Conversely, motor skills acquired in a conventional screen environment not only transferred but improved in HMD-VR, and this increase in motor skill performance could be predicted by self-reported factors of presence, gender, age and video game use.

Conclusions

These findings suggest that personal factors may predict who is likely to have better transfer of motor skill to and from HMD-VR. Future work should examine whether these and other predictors (i.e., additional personal factors such as immersive tendencies and task-specific factors such as fidelity or feedback) also apply to motor skill transfer from HMD-VR to more dynamic physical environments.

Background

The use of virtual reality (VR) in rehabilitation has been growing exponentially over recent years [1, 2]. Clinical applications of VR have been shown to be engaging and motivating [3, 4] with promising results suggesting VR interventions are comparable [5] or in some cases superior [6, 7] to conventional rehabilitation. However, while a number of studies have reported benefits of using VR for cognitive and motor rehabilitation, there are also reports on the limitations of using these devices for clinical applications [8, 9]. In particular, some studies have shown that VR interventions are not effective at improving motor performance in the real world due to a lack of motor skill transfer (i.e., the application of a motor skill in a novel task or environment [10]) [11, 12].

Concerns about motor skill transfer from virtual to real environments are even greater when specifically considering the use of VR viewed using a head-mounted display (HMD-VR). HMD-VR provides a more immersive experience compared to conventional environments (e.g., computer screens) and results in increased levels of presence (i.e., the illusion of actually being present in the virtual environment) and embodiment (i.e., the perceptual ownership of a virtual body in a virtual space) [13, 14] that modulate behavior [15] and impact performance on motor learning and rehabilitation applications (e.g., gait, balance, neurofeedback tasks) [16,17,18]. Additionally, motor learning in HMD-VR (e.g., upper extremity visuomotor adaptation) has been shown to rely on different learning processes compared to a conventional screen environment [19]. Given the differences in immersive experiences and learning processes between HMD-VR and conventional environments, it can be assumed that individuals may experience these environments as separate contexts. Studies have found the context of the training environment to affect the transfer of motor skills [20], where motor performance may decrease when testing occurs in an environment different from training [21]. However, only a small number of studies have specifically explored motor skill transfer of from an HMD-VR training environment to a more conventional environment (e.g., computer screen or real world) [22,23,24,25,26]. Among these studies, there are again conflicting results, with some studies finding successful motor skill transfer from HMD-VR to the real world [22, 23], and others not [24,25,26].

There is also large interindividual variability within the results, and this variability suggests there may be particular tasks or particular individuals that will be more successful in transferring HMD-VR motor skills to the real world. Understanding the task-related or personal factors that mediate learning and transfer from HMD-VR environments should be examined in order to understand what makes HMD-VR interventions effective. One advantage of HMD-VR over conventional screen environments is the ability to realistically simulate the real world which allows for greater task specificity [27]. Task-related factors such as fidelity (i.e., imitation of the real environment) and dimensionality (i.e., matching dimensions between virtual and real environments) between HMD-VR and the real world have been shown to influence lower extremity motor performance [28] and have been suggested to have an influence on transfer in both lower and upper extremity motor transfer [29, 30]. Individual differences in personal factors such as gender, age, video game experience, prior technical computer literacy, and computer efficacy seemed to influence transfer from HMD-VR to the real world in studies examining the transfer of spatial knowledge acquired in an HMD-VR environment [26, 31]. However, the individual differences on both task-related and personal factors have not been extensively examined in HMD-VR motor skill transfer. We begin to address this gap by examining whether individual personal factors facilitate better transfer from upper extremity motor skill acquisition in HMD-VR to a conventional screen environment.

In the current study, we examined: (1) whether transfer of upper extremity motor skills occurs between HMD-VR and conventional screen environments, and (2) what personal factors predict transfer between environments. Given the variability of motor skill learning and transfer in previous studies [22,23,24,25,26, 29], we hypothesized that individual motor performance would vary after transfer to a novel environment, and that this variability could be predicted by individual differences in variables such as presence in the training environment, prior experience with HMD-VR, or non-VR video games.

Methods and materials

Participants

Seventy-four healthy adults were recruited. Participants were randomized into two groups (Train-HMD-VR, Train-Screen). Three participants in the Train-Screen group were excluded from the analysis as a result of performing all trials in the Baseline training block incorrectly (see Analyses) and one participant in the Train-HMD-VR group was excluded from the analysis as a result of being an outlier, which was defined as being beyond three standard deviations from the group mean motor skill in at least one of the blocks. This resulted in a total of seventy participants (53 females/16 males/1 other, aged: M = 25.81, SD = 4.71) with thirty-five participants in each group included in the analysis. A statistical power analysis was performed for sample size estimation based on data from a pilot study of this work (N = 12) [32]. The effect size in this study was d = 0.38. With an alpha = 0.05 and power = 0.60, the projected sample size need with this effect size was approximately N = 35. Eligibility criteria included healthy, self-reported right-handed individuals and no previous experience with the motor skill task (see Experimental design). Written informed consent was obtained from all subjects. The experimental protocol was approved by the University of Southern California Institutional Review Board and performed in accordance with the 1964 Declaration of Helsinki.

Experimental design

Figure 1a provides an overview of the experimental design. The experiment consisted of training and testing blocks in which participants completed a modified version of the Sequential Visual Isometric Pinch Task (SVIPT) [33]. In this task, participants were instructed to apply varying degrees of isometric force between their thumb and index finger to a small pinch force sensor (Futek Pinch Sensor FSH01465; Futek IPM FSH03633; Fig. 1b) to move a cursor between numbered colored gates as quickly and accurately as possible (Fig. 1c). A small circle at the bottom of the screen changed from red to green to indicate the start of each trial. For each trial, no time limit was given and trial completion time was recorded. At the end of each trial, the small circle at the bottom of the screen changed from green to red and participants received auditory feedback (a pleasant “ding” if the cursor correctly entered all the gates or an unpleasant “buzz” if the cursor missed one or more of the gates). A two-second time interval was given between each trial.

Experimental paradigm. a Experimental design. b Pinch force between the thumb and index finger was applied to a small force transducer to move the cursor in the SVIPT. c Sequential Visual Isometric Pinch Task (SVIPT) display. Participants were asked to apply force to the force transducer, which translated into the movement of a small black cursor (shown at the home position in the white bar) moving horizontally to the right in the environment. The cursor moved left by reducing force. Instructions were to move the cursor between the gates, in order from 1 to 5, as quickly and accurately as possible, without over- or under-shooting any of the gates

[…]

Cobtinue —-> Transfer of motor skill between virtual reality viewed using a head-mounted display and conventional screen environments | Journal of NeuroEngineering and Rehabilitation | Full Text

[ARTICLE] What is the impact of user affect on motor learning in virtual environments after stroke? A scoping review – Full Text

Abstract

Purpose

The purported affective impact of virtual reality (VR) and active video gaming (AVG) systems is a key marketing strategy underlying their use in stroke rehabilitation, yet little is known as to how affective constructs are measured or linked to intervention outcomes. The purpose of this scoping review is to 1) explore how motivation, enjoyment, engagement, immersion and presence are measured or described in VR/AVG interventions for patients with stroke; 2) identify directional relationships between these constructs; and 3) evaluate their impact on motor learning outcomes.

Methods

A literature search was undertaken of VR/AVG interventional studies for adults post-stroke published in Medline, PEDro and CINAHL databases between 2007 and 2017. Following screening, reviewers used an iterative charting framework to extract data about construct measurement and description. A numerical and thematic analytical approach adhered to established scoping review guidelines.

Results

One hundred fifty-five studies were included in the review. Although the majority (89%; N = 138) of studies described at least one of the five constructs within their text, construct measurement took place in only 32% (N = 50) of studies. The most frequently described construct was motivation (79%, N = 123) while the most frequently measured construct was enjoyment (27%, N = 42). A summative content analysis of the 50 studies in which a construct was measured revealed that constructs were described either as a rationale for the use of VR/AVGs in rehabilitation (76%, N = 38) or as an explanation for intervention results (56%, N = 29). 38 (76%) of the studies proposed relational links between two or more constructs and/or between any construct and motor learning. No study used statistical analyses to examine these links.

Conclusions

Results indicate a clear discrepancy between the theoretical importance of affective constructs within VR/AVG interventions and actual construct measurement. Standardized terminology and outcome measures are required to better understand how enjoyment, engagement, motivation, immersion and presence contribute individually or in interaction to VR/AVG intervention effectiveness.

Introduction

An increasing evidence base supports the use of virtual reality (VR) and active video gaming (AVG) systems to promote motor learning in stroke rehabilitation [1, 2, 3, 4]. However, practical and logistical barriers to VR/AVG implementation in clinical sites have been well described [5, 6, 7]. To support their use, researchers and developers often emphasize the potential advantages of VR/AVG systems over conventional interventions, including that these technologies may enhance a patient’s affective experience in therapy for the purpose of facilitating recovery [8, 9, 10, 11]. Examining the role of affective factors for motor learning is an emerging area of emphasis in rehabilitation [2, 12, 13, 14, 15].

VR/AVG use may enhance patients’ motivation to participate in rehabilitation as well as their engagement in therapeutic tasks. Motivation encourages action toward a goal by eliciting and/or sustaining goal-directed behavior [16]. Motivation can be intrinsic (derived from personal curiosity, importance or relevance of the goal) or extrinsic (elicited via external reward) [17]. Engagement is a cognitive and affective quality or experience of a user during an activity [16]. Many characteristics of VR/AVG play can contribute to user motivation and engagement, such as novelty, salient audiovisual graphics, interactivity, feedback, socialization, optimal challenge [14], extrinsic rewards, intrinsic curiosity or desire to improve in the game, goal-oriented tasks, and meaningful play [18].

Motivation and engagement are hypothesized to support motor learning either indirectly, through increased practice dosage leading to increased repetitive practice, or directly, via enhanced dopaminergic mechanisms influencing motor learning processes [15, 16]. Yet evidence is required to support these claims. A logical first step is to understand how these constructs are being measured within VR/AVG intervention studies. Several studies have used practice dosage or intensity as an indicator of motivation or engagement [19, 20, 21]. To the authors’ knowledge, few have specifically evaluated the indirect mechanistic pathway by correlating measurement of patient motivation or engagement in VR/AVGs with practice dosage or intensity. While participants in VR/AVG studies report higher motivation as compared to conventional interventions [22, 23, 24], conclusions regarding the relationship between motivation and intervention outcomes are limited by lack of consistency and rigour in measurement, including the use of instruments with poor psychometric properties [22, 23].

The body of research exploring the direct effects of engagement or motivation on motor learning is still in its infancy. Lohse et al. [16] were the first to evaluate whether a more audiovisually enriched as compared to more sterile version of a novel AVG task contributed to skill acquisition and retention in typically developing young adults, finding that participants who played under the enriching condition had greater generalized learning and complex skill retention. Self-reported engagement (User Engagement Scale; UES) was higher in the enriched group, but the only difference in self-reported motivation was in the Effort subscale of the Intrinsic Motivation Inventory (IMI), where the enriched group reported less effort as compared to the sterile group. The authors did not find a significant correlation between engagement, motivation and retention scores. A follow-up study using electroencephalography did not replicate the finding that the more enriched practice condition enhanced learning, it did show that more engaged learners had increased information processing, as measured by reduced attentional reserve [25].

Enjoyment, defined as ‘the state or process of taking pleasure in something’ [26], has less frequently been the subject of study in motor learning research, but has become popular as a way of describing patient interaction with VR/AVGs. Enjoyment may be hypothesized to be a precursor to both motivation and engagement. Given that the prevailing marketing of VR/AVGs is that they are ‘fun’ and ‘enjoyable’ [1, 3, 14, 27], it is important to evaluate its measurement in the context of other constructs.

Motivation, engagement and enjoyment in VR/AVGs may be influenced by the additional constructs of immersion and presence. Immersion is defined as “the extent to which the VR system succeeds in delivering an environment which refocuses a user’s sensations from the real world to a virtual world” [13, 28]. Immersion is considered as an objective construct referring to how the computational properties of the technology can deliver an illusion of reality through hardware, software, viewing displays and tracking capabilities [29, 30]. A recent systematic review [13] could not conclusively state effect of immersion on user performance. Immersion is distinct from presence, defined as the “psychological product of technological immersion” [31]. Presence is influenced by many factors, including the characteristics of the user, the VR/AVG task, and the VR/AVG system [28]. While presence is thought to be related to enhanced motivation and performance [32], relationships between this and other constructs of interest require exploration. Table 1 outlines definitions of constructs of interest to this scoping review.

Table 1

Construct definitions

Construct	Definition	Reference
Motivation	Motivation encourages action toward a goal by eliciting and/or sustaining goal-directed behavior.	[16]
Engagement	Engagement is a cognitive and affective quality or experience of a user during an activity.	[16]
Enjoyment	The state or process of taking pleasure in something.	[26]
Immersion	The extent to which the VR system succeeds in delivering an environment which refocuses a user’s sensations from the real world to a virtual world.	[13, 28]
Presence	The psychological product of technological immersion.	[31]

The purpose of this scoping review is to explore the impact of these affective constructs on motor learning after stroke. This greater understanding will enhance the clinical rationale for VR/AVG use and inform directions for subsequent research. Specifically, our objectives were to:

1. 1.

   Describe how VR/AVG studies measure or report client enjoyment, motivation, engagement, immersion and presence.
2. 2.

   Evaluate the extent to which motivation, enjoyment, engagement, immersion, and presence impact motor learning.
3. 3.

   Propose directional relationships between enjoyment, motivation, engagement, immersion, presence and motor learning.

[…]

 

Continue —> What is the impact of user affect on motor learning in virtual environments after stroke? A scoping review | Journal of NeuroEngineering and Rehabilitation | Full Text

Fig. 2Proposed relationships between the five constructs and motor learning