[ARTICLE] USEQ: A Short Questionnaire for Satisfaction Evaluation of Virtual Rehabilitation Systems – Full Text HTML


New emerging technologies have proven their efficacy in aiding people in their rehabilitation. The tests that are usually used to evaluate usability (in general) or user satisfaction (in particular) of this technology are not specifically focused on virtual rehabilitation and patients. The objective of this contribution is to present and evaluate the USEQ (User Satisfaction Evaluation Questionnaire). The USEQ is a questionnaire that is designed to properly evaluate the satisfaction of the user (which constitutes part of usability) in virtual rehabilitation systems. Forty patients with balance disorders completed the USEQ after their first session with ABAR (Active Balance Rehabilitation), which is a virtual rehabilitation system that is designed for the rehabilitation of balance disorders. Internal consistency analysis and exploratory factor analysis were carried out to identify the factor structure of the USEQ. The six items of USEQ were significantly associated with each other, and the Cronbach alpha coefficient for the questionnaire was 0.716. In an analysis of the principal components, a one-factor solution was considered to be appropriate. The findings of the study suggest that the USEQ is a reliable questionnaire with adequate internal consistency. With regard to patient perception, the patients found the USEQ to be an easy-to-understand questionnaire with a convenient number of questions.

1. Introduction

1.1. Usability

Usability is an important quality attribute of a user’s experience when interacting with a system or tool, and it is also an important attribute in helping users to achieve the suggested goals [1]. With regard to HCI (Human–Computer Interface) and usability, Bevan states in [2] that standards related to usability can be categorized as being primarily concerned with the use of the product (effectiveness, efficiency, and satisfaction in a specific context of use).
The categorization of Bevan is coherent with the ISO 9241-11 standard [3,4,5], which describes a widely accepted definition of usability. This standard indicates the rules that are needed in terms of ergonomics, hardware, software, and environments in order to obtain good usability for a product or system. Section 8.1 describes the term usability as “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use”.

1.2. Usability in Virtual Rehabilitation

One of the promising and emerging fields within rehabilitation therapies for different pathologies is virtual rehabilitation (VRh) [6,7,8,9]. VRh systems are designed to assist clinical specialists and patients in the rehabilitation process [10]. The use of ground-breaking technologies together with the emergence of entertaining and playful virtual environments (VE) have demonstrated promising results in the rehabilitation process [11,12,13,14], improving the adherence to treatments [12]. However, these systems should be tested regarding important aspects such as usability.
Currently, there are different questionnaires that are designed to evaluate usability in general-purpose systems. The best-known usability questionnaire is the system usability scale (SUS) [15,16], which measures the feeling of usability of the users when using computer systems. It is composed of 10 questions with a five-point Likert attitude scale (from strongly disagree to strongly agree). This questionnaire has been used in different domains such as: security software [17], mobile phones [18,19], PDA [20], Social Network sites [21,22], wiki sites [23], serious games [24], or robotics [25]. Even though the SUS questionnaire is not specifically designed for VRh systems, it has also been used for rehabilitation purposes due to the lack of questionnaires that focus on VRh systems. Meldrum et al. [26] tested balance in patients with vestibular and other neurological diseases using VRh and quantified the usability of the Nintendo Wii Fit Plus®. Duvinage et al. [27] assessed the usability of a P300 system (using Brain–Computer interfaces) for lower-limb rehabilitation purposes. One considerable advantage of the SUS questionnaire is the reasonable number of questions that are to be answered at the end of the first session. However, the concepts of this questionnaire are too generic (computers, PDAs, Websites, etc.). The main drawback of the SUS questionnaire is that it does not include questions to obtain responses about specific items related to Virtual Rehabilitation.
Another well-known usability questionnaire is VRUSE [28]. Fitzgerald et al. [29] assessed the usability of the E-Yoga system using VRUSE, with the goal of improving postural control and biomechanical alignment of the subjects in a rehabilitation process. The VRUSE evaluates a wide range of concepts: functionality, user input, system output (display), user guidance and help, consistency, flexibility, simulation fidelity, error correction/handling and robustness, sense of immersion/presence, and overall system usability. The main drawback of this test is the large number of questions that the patients are required to answer [28]: the complete questionnaire has 100 questions. This drawback is especially important if the patients involved in a rehabilitation process have neurological and/or cognitive disorders. Other simplified usability questionnaires for VRh with reasonable outcomes are described in [30,31,32,33], but the drawback of these questionnaires is that the internal consistency has not yet been validated.
Kizony et al. [34] published the Short Feedback Questionnaire (SFQ), which is a questionnaire that is related to Witmer and Singer’s Presence Questionnaire [35]. It is composed of eight questions with a five-point Likert attitude scale, and it has been used in virtual reality environments [36,37,38]. The SFQ questionnaire evaluates the user’s sense of presence, perceived difficulty of the task, and any discomfort that users may have felt during the experience. This questionnaire does not focus on VRh systems.
To our knowledge, there are no validated questionnaires for testing usability or satisfaction of virtual rehabilitation systems. A questionnaire for this purpose must have a reasonable number of questions and internal consistency reliability.
Following the definitions of usability in [2,3,4,5], usability can be divided into three components: efficiency, effectiveness, and satisfaction. Focusing on VRh, efficiency and effectiveness can usually be measured through a clinical trial. With a classical clinical trial, we can compare an experimental group (using a VRh system) with a control group (following a traditional rehabilitation program) by evaluating efficacy and comparing the recovery level of the two groups. With regard to effectiveness, we can measure, for instance, the number of sessions that each group needs to reach a certain level. However, the third component of usability, satisfaction, cannot be evaluated in the same way as efficiency and effectiveness: a reliable and consistent questionnaire (with an adequate number of questions) is necessary to measure the satisfaction of the users.
The aim of the present study is to introduce the USEQ, a user satisfaction questionnaire that is specifically designed to evaluate satisfaction with virtual rehabilitation systems, and to validate their reliability by analyzing their internal consistency.

2. USEQ: The User Satisfaction Evaluation Questionnaire

2.1. SEQ: The Suitability Evaluation Questionnaire

In [39], the SEQ was introduced as a 14-question questionnaire that is designed to test items such as satisfaction, acceptance, and security of use in virtual rehabilitation systems. The SEQ was designed by a multidisciplinary team of clinical and technical experts. Factors such as the length of the questionnaire, the type of questions to be asked and what to ask were taken into account in the design of the questionnaire. For the length of the questionnaire, the clinical experts that collaborated in the design of the SEQ estimated that a maximum of 15 questions would be an acceptable length for patients.
For the type of questions, the designers of the SEQ considered 13 questions with a five-point Likert Scale, plus an open-ended question offering patients the possibility to add comments if necessary. The SEQ has a five-point Likert Scale questions (instead of other options such as seven-point Likert Scale questions) because the authors considered five options of answers to be good enough, and, also, it is coherent with the main usability questionnaires that are currently being used: SUS [15], VRUSE [28], and SFQ [34] also use five-point Likert Scale questions.
For what to ask about, the designers of the SEQ composed the questions taking into account the usability questionnaires available and their own experience, both in the technical and in the clinical field.
A previous study evaluating the suitability of virtual rehabilitation for the elderly was carried out using the SEQ [40]. The SEQ was used to evaluate the ABAR (Active Balance Rehabilitation) system, the VRh system that is used in this study. The study presented in [40] allowed the evaluation of the perceived length and difficulty of the SEQ. In [40], the patients completed the questionnaire without any problems. None of the patients considered the questionnaire to be too long. The main drawback of SEQ is that it is composed of different dimensions; therefore, it is not possible to evaluate their internal consistency.

Source: Sensors | Free Full-Text | USEQ: A Short Questionnaire for Satisfaction Evaluation of Virtual Rehabilitation Systems | HTML

Figure 1. Patient interacting with the ABAR system.



, , , , , , , , ,

  1. Leave a comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: