Posts Tagged Outcome measure

[ARTICLE] Outcome measures in post-stroke arm rehabilitation trials: do existing measures capture outcomes that are important to stroke survivors, carers, and clinicians? – Full Text

We sought to (1) identify the outcome measures currently used across stroke arm rehabilitation randomized trials, (2) identify and compare outcomes important to stroke survivors, carers and clinicians and (3) describe where existing research outcome measures capture outcomes that matter the most to stroke survivors, carers and clinicians and where there may be discrepancies.

First, we systematically identified and extracted data on outcome measures used in trials within a Cochrane overview of arm rehabilitation interventions. Second, we conducted 16 focus groups with stroke survivors, carers and clinicians using nominal group technique, supplemented with eight semi-structured interviews, to identify these stakeholders’ most important outcomes following post-stroke arm impairment. Finally, we described the constructs of each outcome measure and indicated where stakeholders’ important outcomes were captured by each measure.

We extracted 144 outcome measures from 243 post-stroke arm rehabilitation trials. The Fugl-Meyer Assessment Upper Extremity section (used in 79/243 trials; 33%), Action Research Arm Test (56/243; 23%), and modified Ashworth Scale (53/243; 22%) were most frequently used. Stroke survivors (n = 43), carers (n = 10) and clinicians (n = 58) identified 66 unique, important outcomes related to arm impairment following stroke. Between one and three outcomes considered important by the stakeholders were captured by the three most commonly used assessments in research.

Post-stroke arm rehabilitation research would benefit from a reduction in the number of outcome measures currently used, and better alignment between what is measured and what is important to stroke survivors, carers and clinicians.

Up to 77% of stroke survivors experience upper limb (arm) impairment,1 which affects function2 and reduces health-related quality of life.3 Rehabilitation strategies, including those for the arm after stroke, should be based on research evidence. However, only moderate-quality evidence supports the use of interventions to rehabilitate the arm in current clinical practice.4 There is a demand from stroke survivors, carers, clinicians and researchers for research into interventions to improve arm function after stroke.5,6

Efficacy of interventions should be demonstrated using measures that accurately and consistently capture change following treatment.7 Researchers currently use a wide range of measures to assess the efficacy of arm interventions after stroke within randomized controlled trials; recent work has identified at least 48 arm-related measures,8 indicating heterogeneity in what is in current use, as well as the wide range of possible targets for arm interventions including specific impairments, spasticity, pain or task-specific function. The measures in current use are highly varied in their focus and methods, impacting on researchers’ ability to compare and aggregate data from different studies to examine overall efficacy. Consensus on appropriate measures would enhance our ability to detect efficacy of interventions through pooled analysis.9 It has been acknowledged that selection of measures for use in trials should capture domains of importance to patients, carers and clinicians, consider the psychometric properties of measures, and feasibility for use in clinical and research settings.10,11

There is a need for consensus on measure use in post-stroke arm rehabilitation trials.8 The Core Outcome Measures in Effectiveness Trials (COMET) initiative10,12 provides guidance on development of consensus recommendations, highlighting the importance of targeting outcomes that are important and relevant to patients and clinicians.

Considering the views of stroke survivors, clinicians and researchers, the National Institute for Neurological Disorders and Stroke-Common Data Element13 recommends items for inclusion as part of standardized data collection across all stroke trials, and the International Consortium for Health Outcomes Measurement recommends measures for standardized data collection in stroke clinical practice.14 Other recommendations exist for general stroke outcomes and reflect physicians’ opinions on important outcomes according to the World Health Organization International Classification of Functioning, Disability and Health framework.15

The Stroke Recovery and Rehabilitation Round Table, consisting of researchers and clinicians, has generated consensus recommendations for core data collection across sensorimotor stroke rehabilitation trials, including a recommendation to use the Action Research Arm Test for measurement of arm activity limitation across trials.16 In addition, work has been completed to describe the psychometric properties of 53 available arm measures17 in order to inform selection. However, due to the wide-ranging impact of stroke on people’s lives,18 arm-specific measures are unlikely to capture all important outcomes.

To date, there is no clear consensus recommendation for the selection of measures in post-stroke arm rehabilitation randomized trials. Furthermore, there is a lack of information about which outcomes are most meaningful to stroke survivors, carers and clinicians. With a view to inform recommendations for selecting measures in future trials, we sought to investigate (1) existing measures used in post-stroke arm rehabilitation research studies, (2) outcomes important to stroke survivors, their carers and practising clinicians and (3) where important outcomes are captured by existing measures.[…]


Continue —-> Outcome measures in post-stroke arm rehabilitation trials: do existing measures capture outcomes that are important to stroke survivors, carers, and clinicians? – Julie Duncan Millar, Frederike van Wijck, Alex Pollock, Myzoon Ali, 2019

, , , , , , , ,

Leave a comment

[ARTICLE] A composite robotic-based measure of upper limb proprioception – Full Text



Proprioception is the sense of the position and movement of our limbs, and is vital for executing coordinated movements. Proprioceptive disorders are common following stroke, but clinical tests for measuring impairments in proprioception are simple ordinal scales that are unreliable and relatively crude. We developed and validated specific kinematic parameters to quantify proprioception and compared two common metrics, Euclidean and Mahalanobis distances, to combine these parameters into an overall summary score of proprioception.


We used the KINARM robotic exoskeleton to assess proprioception of the upper limb in subjects with stroke (N = 285. Mean days post-stroke = 12 ± 15). Two aspects of proprioception (position sense and kinesthetic sense) were tested using two mirror-matching tasks without vision. The tasks produced 12 parameters to quantify position sense and eight to quantify kinesthesia. The Euclidean and Mahalanobis distances of the z-scores for these parameters were computed each for position sense, kinesthetic sense, and overall proprioceptive function (average score of position and kinesthetic sense).


A high proportion of stroke subjects were impaired on position matching (57%), kinesthetic matching (65%), and overall proprioception (62%). Robotic tasks were significantly correlated with clinical measures of upper extremity proprioception, motor impairment, and overall functional independence. Composite scores derived from the Euclidean distance and Mahalanobis distance showed strong content validity as they were highly correlated (r = 0.97–0.99).


We have outlined a composite measure of upper extremity proprioception to provide a single continuous outcome measure of proprioceptive function for use in clinical trials of rehabilitation. Multiple aspects of proprioception including sense of position, direction, speed, and amplitude of movement were incorporated into this measure. Despite similarities in the scores obtained with these two distance metrics, the Mahalanobis distance was preferred.


Stroke is heterogeneous, affecting sensory, motor, and cognitive functions that are required for daily activities. While there are well validated tools to assess motor and speech functions (eg. Fugl-Meyer Assessment (FMA) [1], the National Institute of Health Stroke Scale (NIHSS) [2], Chedoke-McMaster Stroke Assessment Impairment Inventory (CMSA) [3]) the use of high quality, validated assessment tools for measuring sensory function post-stroke (proprioception in particular) is limited [4], and there is still a lack of a gold standard assessment. While the FMA and NIHSS have sensory components to the assessment, they are seldom used as a sole measure of sensory impairment in research studies focused on sensation as they are based on relatively coarse scales. Yet, sensory and proprioceptive impairments have a significant negative impact on functional recovery following stroke [56789]. Individuals with sensory and motor impairments, compared to those with just motor impairments, have longer lengths of hospitalization and fewer discharges home [101112]. Furthermore, it has recently been shown that motor and proprioceptive impairments can occur independently after stroke [13].

Some commonly used clinical assessments of proprioception post-stroke include: 1) simple passive limb movement detection test [14] in which an examiner moves a subject’s limb segment with their eyes closed, and subjects are asked to say which direction the limb was moved; 2) the Revised Nottingham Sensory Assessment [1516] in which the subject is asked to mirror match the movement of a passively moved limb by a therapist; and 3) the Thumb Localizing Test [17] which involves passive movement of a subject’s arm and hand to a random position overhead, and is followed by subjects reaching to grasp their thumb with the opposite (less affected) hand. These assessments are scored crudely as normal, slightly impaired, or absent, and lack the sensitivity to detect smaller changes in proprioceptive function in part due to poor inter- and intrarater reliability [1819]. Therefore, establishing an objective and reproducible method to assess proprioceptive impairments post-stroke is vital to evaluating the efficacy of different treatments.

Other more advanced methods to assess proprioception have been developed [20212223], with many using robotic technology to measure the kinematics of an individual’s movements. Assessment devices can now measure position sense and kinesthetic impairments after stroke using arm contralateral matching [13242526], in which a subject’s affected arm is passively moved by the robot to a position, and the subject mirror-matches the movement/position with their less affected limb. Another paradigm involves passive movement of a subject’s limb to a specified position, returning the limb to the starting position, and then having subjects actively move the same arm to this remembered position [2126]. This method has an advantage in that it does not require interhemispheric transfer of information, but has limited value in assessing people with concurrent motor deficits, or in assessing kinematic aspects of proprioception, such movement speed and amplitude perception. Further, results can be confounded by problems with spatial working memory. Threshold for detection of passive movement paradigms have also been used to assess proprioception [2728]. This paradigm eliminates confounds due to motor impairment and interhemispheric transfer of information but again, little information about the kinematics of movement perception (e.g. speed or direction) are gained from this task, and it typically takes much longer to complete than position/movement matching. Lastly, Carey et al. [20] have developed and validated a wrist position sense test, where a subject’s wrist is moved to a position (wrist flexion or extension) and without vision of the wrist the subject has to use their other arm to move a cursor to the direction the wrist is pointing. This method minimizes confounds due to interhemispheric information transfer and motor deficits, but again does not provide information about kinesthetic impairments.

Many of these assessments are reliable, reproducible, objective, and provide quantitative measures of proprioceptive function in the upper limbs. Dukelow et al. [1324], used a KINARM robot (BKIN Technologies, Kingston, ON), and detailed a contralateral position-matching task for the upper extremities that can measure various aspects of an individual’s position sense including: absolute error, variability in matching positions, systematic shifts in perceived workspace, and perceived contraction or expansion of the workspace. Similarly, Semrau et al. [25] recently detailed a kinesthetic matching task using the KINARM robot that can measure an individual’s ability to mirror-match the speed, direction, and amplitude of a robotically moved limb [825]. These tasks are reliable [24], and provide numerous parameters that describe an individual’s position or kinesthetic sense impairments and can be used to guide a rehabilitation program tailored to the individual. Furthermore, these studies have shown a strong relationship between proprioceptive impairments and functional independence post-stroke, yet proprioceptive impairments are often not addressed in day-to-day therapy. Reliable and quantitative assessment tools are therefore critical for testing the efficacy of rehabilitation treatments, as in clinical rehabilitation trials.

While multiple kinematic parameters can provide a level of exactness around the nature of an individual’s proprioceptive impairments and are helpful for rehabilitation planning, a summary measure is needed for clinical therapeutic trials in rehabilitation. Thus, a single continuous metric of upper limb proprioceptive function that combines all parameters from the position and kinesthetic matching robotic tasks was developed using two common measures of distance, Euclidean distance (EDist) and Mahalanobis distance (MDist) [29]. The EDist was chosen as it is an easily interpretable calculation and considers each parameter independently. It is the square root of the sum of squared distances between data points (i.e. the straight-line distance between two points in three-dimensional space). The MDist is the next measure we used to compare with the EDist. It was chosen because the calculation accounts for correlations between parameters (by using the inverse of the variance-covariance matrix of the data set of interest), therefore preventing the overweighting of correlated parameters in the calculation. It is the distance between a point and the center of a distribution, measured along the major axes of variation (i.e. the standard deviation of an object in more than one dimension) [3031].. Because the kinematic parameters derived from the robotic tasks may demonstrate some degree of correlation with one another [13], the MDist can account for this auto-correlation. Theoretically, it should perform better at identifying stroke subjects who perform abnormally on the tasks and those who have atypical patterns of behavior relative to controls. The MDist is generally preferred over the EDist for multivariable data since it can cope with different structures of data [31].

MDist (or variants of it) has recently been used in other studies when examining reaching movements after stroke [32].. Our primary aim was to examine differences and similarities between two summary scores (EDist and MDist) in their ability to differentiate proprioceptive impairment in individuals with stroke from controls in a large patient sample. We hypothesized that using a composite proprioception score calculated from the Mahalanobis distance would more accurately identify impaired proprioception in individuals with stroke compared to a proprioception score calculated from the Euclidean distance.[…]


Continue —>  A composite robotic-based measure of upper limb proprioception | Journal of NeuroEngineering and Rehabilitation | Full Text


Fig. 1a KINARM robotic exoskeleton (BKIN Technologies, Kingston, ON, Canda). Subjects are seated in the wheelchair base with arms supported by the arm troughs. b Top-down view of the position matching task. The stroke affected arm was positioned by the robot (black targets, green lines) and subjects were required to mirror-match the target positions with their opposite hand (open targets, blue lines). Nine targets were matched to six times each for a total of 54 trials, presented in pseudorandom order. c Top-down view of an exemplar subject performing one trial of the kinesthetic matching task. The stroke affected arm was moved by the robot between two targets (green lines) and subjects were required to mirror match the speed, direction, and amplitude of movement as soon as they felt the robot move their arm (blue lines). The speed versus time profile represents the temporal aspects of the task, by measuring the response latency (time to initiation of the active arm movement) and peak speed ratio (difference between peak speeds of the passive (green) and active (blue) hands)

, , , , , , , , , ,

Leave a comment

[ARTICLE] Accelerometry-enabled measurement of walking performance with a robotic exoskeleton: a pilot study | Journal of NeuroEngineering and Rehabilitation – Full Text HTML

Published: 31 March 2016



Clinical scores for evaluating walking skills with lower limb exoskeletons are often based on a single variable, such as distance walked or speed, even in cases where a host of features are measured. We investigated how to combine multiple features such that the resulting score has high discriminatory power, in particular with few patients. A new score is introduced that allows quantifying the walking ability of patients with spinal cord injury when using a powered exoskeleton.


Four spinal cord injury patients were trained to walk over ground with the ReWalk™ exoskeleton. Body accelerations during use of the device were recorded by a wearable accelerometer and 4 features to evaluate walking skills were computed. The new score is the Gaussian naïve Bayes surprise, which evaluates patients relative to the features’ distribution measured in 7 expert users of the ReWalk™. We compared our score based on all the features with a standard outcome measure, which is based on number of steps only.


All 4 patients improved over the course of training, as their scores trended towards the expert users’ scores. The combined score (Gaussian naïve surprise) was considerably more discriminative than the one using only walked distance (steps). At the end of training, 3 out of 4 patients were significantly different from the experts, according to the combined score (p < .001, Wilcoxon Signed-Rank Test). In contrast, all but one patient were scored as experts when number of steps was the only feature.


Integrating multiple features could provide a more robust metric to measure patients’ skills while they learn to walk with a robotic exoskeleton. Testing this approach with other features and more subjects remains as future work.


Clinical scores of walking ability are crucial in many areas of physical rehabilitation to assess the efficacy of a therapeutic intervention or an assistive device, as well as to discriminate the ability between different patients [1, 2]. One domain of interest is evaluating functional ambulation in individuals who suffered a spinal cord injury (SCI). Even though many outcome measures target the SCI population [3, 4], currently there exist no specific measures targeting the ability of a patient to use a lower limb robotic exoskeleton to walk overground and achieve functional ambulation.

Lower limb exoskeletons are bilateral powered orthoses designed to provide assistance for sit-to-stand and for walking and, in some cases, to assist lower extremity function in individuals with incomplete or complete SCI [58]. Currently, several exoskeletons are transitioning from purely research and rehabilitation devices to personal mobility systems that individuals with SCI could use to walk inside their home and in their communities [9, 10]. A paradigmatic case is the ReWalk™, which has been approved by the Food and Drug Administration to be sold to individuals with SCI as a take-home personal mobility device.

Quantitative clinical assessment of exoskeletons is fundamental to evaluate their safety and effectiveness when used by individuals with disabilities. Specifically, individuals with complete SCI, who aim at taking an exoskeleton home as a personal mobility device, require an intensive training protocol to become independent users. Such training is typically delivered in a clinical setting and therefore clinicians need a robust metric to evaluate if a patient has reached a level of ability and expertise to independently use the device at home and in the community. Obtaining a robust index of the patients’ walking skills with an exoskeleton could also be used to inform health insurance companies about the actual improvements in functional mobility for potential reimbursement. This point is crucial as the cost of these devices is extremely high and therefore any support funding has to be justified.

The primary clinical outcome measures currently used to assess functional ambulation with exoskeletons are the 6-Minute-Walk-Test (6MWT) and the Ten-Meter-Walk-Test (10mWT) [11, 12]. These two tests measure, respectively, the distance walked in six minutes and the time to walk over a distance of 10 m, while walking at a constant speed. Despite being validated in spinal cord injury populations [13], it is questionable whether these measures are sufficient to fully evaluate a patient skill and the device efficiency. Indeed, other studies have measured additional features to characterize walking skills with robotic exoskeletons.

Specifically, amongst the features quantified there are: the kinematics of the hip, knee and ankle joints in patients trained to use the ReWalk™ [14], as recorded via a motion capture system; the exertion level based on the heart rate normalized to the walking speed (i.e. physiological cost index) [15] and the oxygen uptake [16, 17]. Other metrics used include the variation in vertical and lateral amplitude of the head motion [18], ground reaction forces analysis [19] and the ability to maintain eye contact to assess cognitive effort [20]. Even when multiple features were measured, each study reports the values of each feature individually to characterize functional ambulation with exoskeletons. Therefore it is unclear how each feature contributes to the overall expertise of a subject. Furthermore, some of the captured features require complex and expensive lab equipment, commonly seen only in large hospitals and university settings.

In the current study, we propose to combine multiple features of walking performance by estimating their probability distribution over a set of expert users who have been previously trained extensively to use the exoskeleton. New participants are then scored based on how well their features fit the experts’ probability distribution. Building on this principle, we define a new score to quantify walking ability with exoskeletons: the Gaussian Naïve Bayes surprise. The term surprise is derived from information theory and represents the amount of unexpected information provided by an event [21]. We apply our score to quantify the walking skills of four individuals with complete SCI, as they are trained to use the ReWalk™ exoskeleton. Four features are computed from the trunk accelerations, which are recorded using a commercial wearable accelerometer while subjects perform a 6MWT with the exoskeleton. We estimate the parameters of the features probability distribution from seven expert subjects (1 SCI and 6 able-bodied) that received extensive prior training with the device, and compute the Gaussian naïve Bayes surprise of the four SCI participants with respect to the experts. The score based on all four features is compared with one based only on number of steps (an equivalent of distance walked), in terms of the separation between experts and patients that is yielded by the two indices.


The ReWalk™ exoskeleton

The ReWalk™ (ReWalk Robotics Inc., Marlborough, MA, USA) is a motorized lower limb exoskeleton suit designed to provide legged mobility to paraplegic patients who suffered a spinal cord injury from level T4 to L5. The device has two actuated degrees of freedom – one at the hip and one at the knee on each side – and has a passive spring-assisted dorsiflexion joint at the ankle. Figure 1a shows a schematic of the device.

Fig. 1 ReWalk™ exoskeleton and measured trunk angles. a Schematic of the ReWalk™ exoskeleton suit. The tri-axial wearable accelerometer attached on the right flank of the robot recorded the body accelerations while the subject walked with the device. Features to score walking quality were computed from the accelerations (see text). b The trunk angles in the frontal (x-y) and lateral (y-z) plane during walking (φ, frontal, blue line; α, lateral, green line). c The power spectral density plots of the trunk angles: the x-value of each maximum corresponds to the frequency of the oscillations in the plane. The step frequency corresponds to the maximum in the x-y plane

Continue —> Accelerometry-enabled measurement of walking performance with a robotic exoskeleton: a pilot study | Journal of NeuroEngineering and Rehabilitation | Full Text

, , , , , , , ,

Leave a comment

%d bloggers like this: