Robot-aided assessment of lower extremity functions: a review
Maggioni et al. Journal of NeuroEngineering and Rehabilitation
Robot-aided assessment of lower extremity functions: a review
Serena Maggioni 0 1
Alejandro Melendez-Calderon 0
Edwin van Asseldonk
Verena Klamroth-Marganska 1
Lars Lünenburger 0
Robert Riener 1
Herman van der Kooij
0 Hocoma AG , Volketswil , Switzerland
1 Sensory-Motor Systems (SMS) Lab, Institute of Robotics and Intelligent Systems (IRIS), Department of Health Sciences and Technology (D-HEST), ETH Zürich , Zürich , Switzerland
The assessment of sensorimotor functions is extremely important to understand the health status of a patient and its change over time. Assessments are necessary to plan and adjust the therapy in order to maximize the chances of individual recovery. Nowadays, however, assessments are seldom used in clinical practice due to administrative constraints or to inadequate validity, reliability and responsiveness. In clinical trials, more sensitive and reliable measurement scales could unmask changes in physiological variables that would not be visible with existing clinical scores. In the last decades robotic devices have become available for neurorehabilitation training in clinical centers. Besides training, robotic devices can overcome some of the limitations in traditional clinical assessments by providing more objective, sensitive, reliable and time-efficient measurements. However, it is necessary to understand the clinical needs to be able to develop novel robot-aided assessment methods that can be integrated in clinical practice. This paper aims at providing researchers and developers in the field of robotic neurorehabilitation with a comprehensive review of assessment methods for the lower extremities. Among the ICF domains, we included those related to lower extremities sensorimotor functions and walking; for each chapter we present and discuss existing assessments used in routine clinical practice and contrast those to state-of-the-art instrumented and robot-aided technologies. Based on the shortcomings of current assessments, on the identified clinical needs and on the opportunities offered by robotic devices, we propose future directions for research in rehabilitation robotics. The review and recommendations provided in this paper aim to guide the design of the next generation of robot-aided functional assessments, their validation and their translation to clinical practice.
Assessment; ICF; Robotic rehabilitation; Walking; Muscle force; Range of motion; Proprioception; Synergies; Joint impedance; Gait; Reliability; Validity; Responsiveness; Exoskeleton; Translational research
Standardized sensorimotor assessments after
neurological disorders have the potential to aid the
understanding of recovery and to support the design of
effective therapeutic interventions, with the ultimate
goal of maximizing the patient’s chances of
rehabilitation. Despite the general consensus on this statement
among clinicians, neuroscientists and rehabilitation
engineers, sensorimotor assessments are not routinely
performed in the clinical practice [
]. Duncan et al.
identified four high-level determinants that impact
routine assessments in practice: i) Knowledge, Education,
and Perceived Value in Outcome Measurement (i.e.
information on validity and reliability); ii) Support/Priority
for Outcome Measure Use (i.e. organizational and
management factors); iii) Practical Considerations (e.g. time,
cost); iv) Patient Considerations (e.g. usefulness of the
assessment to the patient’s treatment). The limited use
of assessments in clinical practice reduces the chances
to obtain feedback on the therapeutic intervention and
consequently decreases the efficiency of therapy
planning and adjustment [
1, 3, 4
]. Objective proofs are
needed to justify healthcare expenses and reimbursement
from insurances [
]. In research, the lack of sensitive
and reliable outcome measures can hamper the results of
clinical trials aimed at determining the efficacy of new
treatments, if changes due to the intervention under study
fail to be detected [
]. Thus, valid, reliable and sensitive
assessments are useful in areas that encompass
therapeutic, research and financial domains (Fig. 1).
The last decades have seen an increasing use of robotic
devices for neurorehabilitation training in clinical centers
]. Besides training, translational researchers in
neurorehabilitation have proposed the use of robotic devices
to overcome some of the limitations in traditional clinical
assessments. Robotic devices represent an alternative
method to provide more objective, sensitive, reliable and
time-efficient assessments in clinical practice [
6, 9, 10
Sensors are embedded or can be easily added in robotic
devices in order to provide quantitative measures of
variables such as, for example, joint angles. Instrumented
devices enable the recording of new variables (e.g.,
smoothness) that were not accessible before. Standardized
assessment protocols and repeatable conditions can be
achieved with the use of robotic devices. Patient’s
motivation, which is a factor that can influence the assessment
], can be promoted by using virtual reality
applications to provide constant engagement, along with
standardized instructions. Moreover, assessments can be
integrated into the training session without requiring
additional setup and measurement time. Training variables
(e.g., duration, number of repetitions) can also be used to
provide an indication of the patient’s performance and
allow comparison between sessions.
However, the frequent criticism from clinicians
towards these engineering solutions is that the outcome
measures provided by robotic devices are too abstract,
do not translate to function and lack ecological validity.
Moreover, robotic devices often require a long setup
time and a certain degree of technical knowledge to be
]. In a typical setting, the therapist has
between 30 min and 1 h to deliver the therapeutic
intervention. If the assessment protocol takes too much time
to be performed, the solution may not be adopted. In
some cases, the increase in sensitivity and reliability is
discarded in favor of an existing subjective, yet
timeefficient, assessment that can be applied in any clinical
setting. These may be some of the reasons why
robotbased assessments have not yet been integrated in
clinical practice at a large scale. Therefore, future
developments in rehabilitation robotics should enable the
clinician to choose among a set of assessment tools
according to the specific needs of the patient. We
encourage engineers to develop assessment technologies that
are not limited by practical constraints and
administrative burdens. We believe that the barriers that prevent
the translation of robotic assessments to clinical use
must be understood so that they can be overtaken.
Hence, to guide the development of future robotic-based
assessment tools, it is fundamental that we understand
the needs of the key players and adjust our motivation
to develop new technological solutions.
This paper is targeted to researchers and technical
developers in the field of robotic neurorehabilitation. The goal is
to provide a comprehensive review of the state-of-art
robot-assisted methods, with focus on the lower limb, and
identify gaps in which robotic technologies can solve
current issues in the assessment of sensorimotor functions.
We present and discuss existing assessment methods for
lower limb functions used in routine clinical practice and
contrast those to state-of-the-art instrumented and robotic
technologies. We also provide guidelines and
recommendations for the development and validation of new
sensorand robotic-based assessment methods, taking into account
the clinical needs. The review and recommendations
provided in this paper aim to guide the design of the next
generation of robotic devices.
Walking recovery is among the most desired goals of
patients after a neurological injury [
]. In order to
maximize the recovery of the walking function, an
optimal therapeutic plan should be defined and adjusted
according to the patient’s progress. However, the lack of
quantitative and sensitive assessments of lower limb
functions that can be used during every day clinical
practice limits the possibility to record the patient’s
progress over time. For this reason, the scope of this review
is constituted by measures and assessment methods that
target body functions of the lower limbs, with a
particular focus on those related to walking. We decided to
exclude assessments of functions that, although needed for
walking, are influenced by body systems other than
lower limbs (e.g. balance). The methods and papers
mentioned in this review were selected from an
electronic search in PubMed and Google Scholar.
Concerning the robotic measures, for each section we searched
for the particular topic (e.g. “range of motion”) and the
word “robotic” OR “robot”. Only papers relating to the
lower extremities were considered. We looked also at
the literature relevant to the robotic gait trainers and
exoskeletons. The recent review from Zhang et al. 
provided a good list of references on ankle devices. We
also performed a manual search among the references
considered relevant that we found in the selected
articles. We aimed at a comprehensive, but not necessarily
systematic or exhaustive review.
Assessments of sensorimotor functions can be discussed
in the framework of a comprehensive classification for
describing health and health-related states developed in 2001
by the World Health Organization. The International
Classification of Functioning, Disability and Health (ICF) forms
a conceptual basis for the definition, measurement and
policy formulations for health and disability [
]. The main
aim of the ICF is to provide decision-makers in heath
related sectors with a planning and policy tool. Moreover,
relevant data can be collected in a consistent and
internationally comparable manner. In the ICF, limitations of
function and disability are not considered to be
etiologyspecific but rather are seen as reflecting common
manifestations of underlying health conditions [
]. In the same
way, the assessments discussed in this review are not
disease-specific but are applicable to different kind of
populations. The ICF is a useful framework to conceive new
robot-based assessment tools and to categorize existing
ones. The ICF describes health and health-related states by
means of three categories: functioning at the level of body
or body part (Body functions and structures), the whole
person (Activity), and the whole person in a social context
]. The functions addressed by this review
are listed together with their ICF classification in Table 1.
Assessments validation – psychometric properties
In this section, we will present some of the most relevant
statistical analyses that are commonly used to evaluate the
psychometric properties of an assessment tool.
Throughout the paper, we will refer at these definitions to describe
the adequacy of the clinical and robotic-based assessment
methods. One of the main challenges for the acceptance
of new robot-based assessments in clinical practice is their
validation. The lack of information on the validity and
reliability of an assessment has been identified as one of the
barriers to its use [
Reliability must be tested first when developing a new
assessment method. An instrument cannot be valid if the
values it provides from repeated measurements are not
]. The most common methods to assess the
reliability of an instrument in medicine and sport are the
Intra-class Correlation Coefficient (ICC) and the Standard
Error of Measurement (SEM). The ICC targets the relative
reliability (the degree to which individuals maintain their
position in a sample over repeated measurements); the
SEM measures absolute reliability (the degree to which
repeated measurements vary for individuals) [
two methods are, therefore, complementary. ICC values
Control of voluntary
b735, b7500, b7650
Muscle tone functions,
Stretch motor reflex,
The sections of the current review in the framework of the ICF. The ICF lists a broad range of health-related components under the categories of Body function (b), Body
structures (s), Activities and Participation (d), Environmental factors (e). In each category it is possible to find a complete list of health-related components divided in
are strongly influenced by the heterogeneity of the
subjects (i.e. a high ICC can be obtained even if large
differences between trials are present, provided that
between-subjects variability is high) [
]. Results of
reliability test measured with ICC in a particular population
cannot be extended to a study including a different
population. The SEM quantifies the precision of individual
scores within the subjects [
], but its direct calculation
involves the determination of the standard deviation (SD)
of a large number of scores from an individual. In practice
this is not possible; therefore, the SEM is estimated
(Table 2). SEM is independent of the population from
which it was determined and it is not affected by
betweensubjects variability as is the ICC [
]. Absolute reliability
can be also evaluated using the Bland-Altman plots [
Here, for each subject, the mean of two measurements is
plotted against their difference.
The presence of systematic bias is confirmed when the
mean of the differences between the two tests is
significantly different from zero. The limits of agreement (LOA)
are another measure of absolute reliability: they indicate
the range where, for a new individual from the studied
population, the difference between any two tests will lie
with a 95 % probability [
]. When the test is used to
detect changes between sessions within the same individual,
these changes can be considered significant only if they
fall outside the LOA. Therefore, the broader the LOA, the
larger the minimal detectable change (MDC) would be for
a given sample size in an experiment.
Validity assessment is usually more complex because
generally the “true” value of a measure is not known
with absolute certainty. The general approach for
validating robot-based assessments so far consisted in
applying correlation between instrumented measures and
clinical scores in order to find which parameters
measured by robots are able to reconstruct established
clinical tests (concurrent validity). However, tying the
validation of an instrumented method to a score that is
subjective and ordinal-based could be questionable.
When a gold standard is already established (e.g.
isokinetic dynamometer for muscle strength measurement),
concurrent validity can be tested against it. Without
such standards, validity is tested indirectly as the ability
of a tool to measure the underlying theoretical construct
(construct validity) [
Responsiveness is the ability of a test to accurately detect
change when it has occurred [
]. Reliability highly
influences responsiveness because real changes can be masked
by the measurement error if the reliability of the test is
poor. Measures characterized by a limited number of
Responsiveness Ability to accurately detect changes. Internal responsiveness: Internal responsiveness: Cohen’s effect size: observed change in
ability of a measure to change over a particular specified time score divided by the SD of baseline score. Standardized response
frame. External responsiveness: extent to which changes in a mean (SRM): observed change score divided by SD of change
measure over a specified time frame relate to corresponding score in the group.
cMhinanimgaelsDineteacgtaobldle sCthanandgaerd( M[2D1C7)]: minimal amount of change that Ebxatseerdnaolnreasnpoenxsteivrennaelscsr:itReOrioCncu[2r1v7e]s:MseDnCsit¼ivitySEvMsspec1if:i9c6ity pffi2ffi
is not likely to be due to random variation in measurement [
] MCID: anchor-based (compare a change score with external
Minimal clinically important difference (MCID): smallest amount measure of clinically relevant change) or distribution-based
of change in an outcome that might be considered important methods (based on statistical characteristics of the sample) [
by the patient or clinician [
]. Floor and ceiling effects: percentage of the number of scores
Floor and ceiling effects: the extent to which scores cluster at clustered at bottom/top.
the bottom or top, respectively, of the scale range.
categories have intrinsically low responsiveness because
large changes in status usually are required to the patient in
order to change category. Ceiling and floor effects limit
responsiveness at the extremes of the score range, since
further improvement or deterioration cannot be monitored.
The minimal clinically important difference (MCID) is a
concept useful to consider the patient’s perspective when
dealing with assessments. It involves both a minimal
amount of patient reported change and changes important
enough to modify patient management [
Overview of clinical assessments and robotic measures of lower limb functions
The following sections provide an overview of
assessments methods for different outcome measures. For
each outcome measure, its definition and relevance,
the ways it is measured in clinic and in research
settings are presented. For each of the available
instrumented and robotic measures, the advantages over
the current clinical assessments as well as points for
improvement are also discussed. An overview of the
validity, reliability and responsiveness of the clinical
assessments discussed in this paper can be found in
Table 3. Table 4 provides a list of psychometric
properties of available robot-aided assessments. However,
the limited amount of studies on validation of the
proposed robotic measures prevented the
completeness of the table.
Range of motion
Definition of the measure
Range of Motion (ROM) can be defined as the range,
measured in degrees, through which a joint can be
moved around one of its axes. Active ROM (aROM) is
performed by the voluntary movement of the patient,
while the assessment of passive ROM (pROM) implies
that the therapist (or a robotic devices) rotates the
patient’s joint distal segment with respect to the proximal
] while the patient tries to relax. A
minimum level of joint ROM is required to perform activities
of daily life in a safe and energy-efficient way [
For example, reduced knee ROM in the sagittal plane
prevents an adequate foot clearance and leads to
compensatory mechanisms . After a neurological injury
it is common to observe a decreased ROM and a
pathological behavior at the extremes of the ROM. To
quantify this pathological behavior the “end feel” is sensed,
which is defined as the resistance of the joint in response
to a gentle overpressure applied at the end of the ROM
]. A decreased ROM and pathological end feel can be
due to weakness, spasticity, pain, tendon and muscle
contractures or ectopic bone formation [
Clinical assessment and open issues
The most common instrument used in clinical practice
for measuring joint ROM is the universal goniometer. The
therapist must place the axis of the instrument over the
axis of movement of the joint, aligning the stationary arm
with the proximal segment and the moveable arm with
the distal segment. pROM is assessed to determine the
mobility of a joint regardless of the voluntary ability of the
patient and it is usually slightly greater than aROM and
much greater in case of muscle weakness. aROM values
can be diminished when the movement is performed
against gravity, especially in weak patients. When
assessing the end-feel, the therapist manually determines the
type of this resistance (e.g. “hard”, “soft”, “firm” etc.),
which is indicative of different pathologies or conditions
that can affect the normal ROM of a joint [
Moderate to substantial intra-rater reliability and
validity for ROM measurements can be achieved by means of
the universal goniometer (Table 3), but inter-rater
reliability is generally lower and highly dependent on the
therapist’s experience [
]. The inter-rater reliability of
pROM and of end-feel measurements is particularly
critical because it depends on the torque exerted by the
therapist on the patient’s joint [
]. Therefore, it is highly
recommended that the assessment is performed by the
same therapist following a rigid standardized
measurement protocol [
]. Additional sources of errors in the
measurements are the incorrect identification of the joint
axis, the improper alignment of the goniometer arms with
the body segments (also due to the movement of the joint)
and the parallax error when reading the scale [
Moreover, the measures can be affected by compensatory
motions occurring at other joints.
State of the art in rehabilitation robotics
Measures of ROM are obtained through angular position
sensors, for which different technologies are available.
Within the existing robotic devices available for clinical
use, isokinetic dynamometers (see section Muscle force)
embed ROM measurement procedures [
gait robots for treadmill walking (e.g. Lokomat ,
], ALEX [
], ARTHuR [
]) and exoskeletons
for overground walking (e.g. Vanderbilt [
], ReWalk [
], Ekso [
], H2 [
], Vlexo [
embed potentiometers or encoders in the robotic joints
to measure joint angles. Nevertheless, the only method
for pROM assessment in a gait trainer available for
clinical use is implemented in the Lokomat: the procedure
requires the therapist to move the limbs of the patient
strapped in the device [
]. For research purposes,
several attempts to obtain instrumented measurements of
the ankle joint have been made, often embedding ROM
and stiffness evaluation (see section Joint impedance) in
the same device. For example, potentiometers were used
ρ = 0.89
ρ = − 0.95
vs WISCI II:
ρ = 0.795
in two ankle robots to train and assess active and/or
passive plantar- and dorsiflexion ROM in stroke patients
] and in a device able to assess ankle rotations in
the 3 planes [
]. Another robotic ankle trainer, the
Anklebot, embeds encoders to estimate the ankle
dorsiplantarflexion and inversion-eversion angles [
End-feel assessment, at the best of our knowledge, has
not been realized yet in a lower limb device. Nevertheless,
attempts to develop an instrumented end-feel assessment
were made for the shoulder joint [
]. The authors
used a force sensor to measure the applied force and a
motion tracking system to assess the joint displacement.
The rationale behind this approach is that the end-feel
can be interpreted as the displacement induced by a force
applied at the end of the joint ROM. It is, therefore, a
measure of stiffness and as such it can be quantified by
applying a known force and measuring the joint
displacement at the end of the ROM . However, research in
this field is still at an early stage and no information on
validity and reliability of the measurements are available.
Future developments in rehabilitation robotics
Rehabilitation and assistive robots usually make use of
angular position sensors in their hardware for control
purposes and it would, therefore, be natural to conceive
robot-aided joint ROM assessments. The development
of new technologies in rehabilitation robotics can
address many of the issues of current clinical measures of
joint motion. aROM measures can be improved by using
robots that are able to compensate for gravity while the
subject performs active movements, making the
assessment independent of the body orientation with respect
to gravity. Transparency of robots must be ensured by
means of backdrivable actuators or particular control
strategies (e.g. admittance control [
]). The mechanical limits of a robotic joint
should be designed in order to allow a subject to reach
the whole ROM. Otherwise, the measures will saturate
to this limit, leading to an underestimation of the
patient’s ROM [
]. The stabilization of the patient’s joints
other than the joint of interest and the reduction of
compensatory movements can be provided by
mechanical fixation to the robotic device. Nevertheless,
compensatory movements can be very difficult to detect,
especially when they occur within the same joint under
test; in this case they can only be identified from the
careful eye of the examiner [
]. During the
measurement of pROM and end-feel, robots can impose a
standardized movement in terms of torque and/or speed
]. This would improve the reliability of the test
making it independent of the operator. Moreover,
predefined sequences of movements can be programmed
using robotic devices in order to have a standardized
Exoskeletons for overground walking could
potentially be used for measurements in static and dynamic
conditions provided that gravity, friction and inertia
are adequately compensated (see section Walking
function/Gait pattern). For example, a versatile
passive exoskeletal research platform (Vlexo) developed
to study human-robot interactions was designed to
have robotic joint ROM higher than the human ROM
]. Each degree of freedom could be blocked to
avoid compensatory movement. Thanks to the high
adaptability and instrumentation possibilities, it would
potentially become a good tool for measuring
simultaneously the ROM of hip (abd-adduction, int-ext
rotation, flex-extension) and knee in static and dynamic
End-feel assessment procedures can be implemented
with a similar approach as for the shoulder joint [
using for example motorized exoskeleton devices [
or ankle robots [
46, 48, 49
] equipped with angular
position and force sensor.
Concerning the measurement technology, the most
used angular sensors in robotics are potentiometers,
due to their robustness, accuracy and low price.
However, since they must be aligned with the joint’s axis
of rotation, the measures could potentially suffer from
misalignments when the anatomical joint does not
have a single axis of rotation or when the setup is
not properly done. To overcome this issue, other
sensor technologies that do not require the identification
of the joint axis can be used. Flexible goniometers
based on strain gauge technology are available on the
market (e.g. Biometrics Ltd. – uniaxial or biaxial,
]). The end blocks are fixed to the segments that
form the joint and the angle of flexion-extension and
abduction-adduction can be recorded, provided that
the device is attached in a suitable plane. They have
very good performances both in static and dynamic
], but they are at present not
sufficiently robust for daily clinical usage. In wearable
applications, strain sensors [
] and optic fibers [
have been used due to their low encumbrance and
low weight, but at the moment their performance is
not adequate for accurate measurements. Among the
wearable sensor technologies, Inertial Measurement
Units (IMUs) are promising instruments, given the
good performances shown so far, especially in knee
dynamic ROM measurements [
]. However, they
require calibration and signal processing algorithms that
perform sensor fusion and compensate for possible
inaccuracies due to electromagnetic interferences.
Further studies are recommended to define the
hardware configuration, the sensor technology and the
measurement protocol that maximize the validity and
reliability of the aROM, pROM and end-feel assessment
in a clinical context, with the temporal and economic
limitations that this implies. Wearable technologies
could give an insight of the ROM that the patient is able
to display in a real-life situation.
Definition of the measure
Muscle strength is defined as the amount of force
generated by muscle contraction [
]. Muscle weakness, or the
inability to generate normal levels of force, has clinically
been recognized as one of the limiting factors in the motor
rehabilitation of patients following stroke [
] and it is one
of the major clinical manifestation in hereditary
neuromuscular disorders and injuries of the spinal cord [
amount of preserved voluntary muscle contraction has
been proven to be highly correlated with walking ability in
incomplete SCI [
] and stroke [
]. In the elderly
population, lower limb muscle weakness has been associated with
an increased risk of falls [
]. In the lower limbs, muscle
weakness can be ascribed to disuse atrophy and to the
disruption in descending neural pathways leading to
inadequate recruitment of motoneuron pools [
Assessing muscle strength is important to determine the severity
of the injury, to plan the therapy and to monitor the effects
of rehabilitation treatments .
Clinical assessment and open issues
In clinical practice, muscle strength is typically assessed
using manual muscle testing (MMT) (e.g. Medical
Research Council scale [
]). MMT grades strength
according to the ability of a muscle to act against gravity
or against a resistance applied by an examiner (0: no
muscle contraction, 5: holds test position against
maximal resistance) [
]. However, the accuracy and
sensitivity of MMT is low and the same grade in MMT
corresponds to a large range of absolute strength values
]. It was reported by [
] that Beasley found that a
variation of less than 25 % in muscle strength for the knee
extensor cannot be detected by MMT [
]. MMT is strongly
influenced by the experience of the examiner, who must
avoid compensatory movements by the subject and ensure
a standard positioning. MMT suffers from ceiling effects,
because the maximum score (5.0) is assigned before a
normal level of muscle strength is truly reached [
was found not adequate as a screening tool and insufficient
in tracking the progress of a patient undergoing therapy
]. Subtle increases in muscle strength are only
detectable with instrumented methods.
Quantitative measures of muscle strength can be
performed during isometric, isoinertial or isokinetic
contraction. In an isometric test the subject is asked to perform a
maximum voluntary contraction (MVC) against a fixed
resistance and the maximum value of the force/torque is
retained. In clinical practice, this test is mostly performed
with a hand-held dynamometer (HHD) or myometer. The
HHD is a portable force sensing device that can be placed
between the hand of the examiner and the body segment
to test, similar to how an examiner would perform a
]. The examiner must be able to apply a
resistance equal or greater than the patient’s force. Like the
MMT, the myometry is, therefore, depending on the
amount of resistant force the practitioner is able to apply
to the segment of interest and on his ability to stabilize
proximal joints [
]. Nevertheless, with respect to
MMT, myometry has higher sensitivity and it is less prone
to ceiling effects . Reliability and validity of HHD
measures can be further increased by fixating the device with
a belt [
], so that the resistance applied against the
movement is not dependent on the examiner’s force. Load
cells mounted on supportive frames can also be used for
this purpose . Isoinertial tests consist in lifting a
constant load throughout the joint ROM and the outcome is
the maximum load that can be lifted once (1-RM) [
Isoinertial tests are usually executed using sport devices,
like the leg extension machine, modified in order to
record the joint angle [
]. During an isokinetic contraction
the joint angular velocity is kept constant by a machine,
the isokinetic dynamometer. The subject is asked to
forcefully contract the muscles during the whole ROM while
the peak torque is calculated. This test can only be
performed with a robotic device and it will be discussed in
the next section. Isokinetic tests could be useful to
unmask speed-dependent strength impairments [
Although the isokinetic dynamometer is considered the gold
standard for muscle strength measurements, price,
encumbrance and setup time limit its use in a clinical
setting. Therefore, it was proposed to use preferably
isometric or isoinertial tests in clinical practice due to
their reduced cost and easiness of use [
]. The three
test modalities have indeed similar good construct validity
(relation with physical function) and substantial test-retest
reliability  and high correlations have been found
between isometric and isokinetic torque measures, although
isometric tests lead normally to higher values of muscle
]. It is important that users are aware that
these three conditions provide different estimates of
muscle strength. Nevertheless, it was demonstrated that
using the HHD according to standard procedures and
fixation, excellent inter and intra-tester reliability and a good
correlation with the isokinetic dynamometer can be
73, 79, 81, 88
]. Therefore, given the cost and
long measurement time (around 25 min) required by the
isokinetic dynamometer, it was suggested to favor the use
of HHD in clinical practice [
State of the art in rehabilitation robotics
The most known device for muscle strength measures is
the isokinetic dynamometer. This machine allows the
measurement of joint torques in controlled conditions:
isometric at selected joint angles or isokinetic at selected
angular velocities [
]. A servo-controlled lever arm
provides resistance to the subject’s joint when it reaches a
defined angular velocity (≥0 deg/s). Different mechanical
configurations allow testing of hip flexion-extension and
ab-adduction, knee flexion-extension, ankle
plantardorsiflexion and eversion-inversion. The patient’s trunk and
the segments proximal to the joint tested must be stabilized
with straps and the axis of the dynamometer must be
carefully aligned with the axis of the joint to test to avoid
measurement inaccuracy . In isokinetic tests the subject is
asked to push “as hard and as fast as possible” while the
device provides resistance to the movement of the limb so
that it cannot accelerate beyond the machine’s preset
angular speed [
]. A continuous passive motion (CPM) has
been proposed for severely impaired subjects, where the
robot moves the limb and the dynamometer lever arm at a
preset velocity while recording forces applied to the lever
]. Reliability and validity of the isokinetic
dynamometer are substantial but the high cost and the long setup
time limit its use in everyday clinical practice.
In rehabilitation robotics, muscle strength has been
measured integrating force sensors into the structure of
exoskeletal devices for quantifying physical human-robot
interaction and estimating the force exerted by the
patient. Directly measuring the interaction force at the
attachment points requires a load cell, placed at the
connection between the cuff/orthosis and the
exoskeleton link, such as in a modified version of the Lokomat
]. Otherwise, the estimation of interaction torques
can be achieved through a force sensor in series with the
actuators, like in the Lokomat  and in the ALEX
], or through linear potentiometers for measuring the
length of the springs used in the actuators of the LOPES
]. The torques produced at each joint are calculated
online from the joint position and the linear force
values. The Lokomat, in particular, allows the execution
of hip and knee isometric strength tests in the sagittal
plane: the patient is positioned with 30° hip flexion and
45° knee flexion and asked to flex or extend the joints
against the resistance provided by the orthosis. A
moderate to substantial inter- and intra-rater reliability of
this method was found with patients with and without
neuro-muscular disorders [
The ankle joint is usually measured separately from
the hip and knee joints with dedicated devices used in a
sitting position [
]. An ankle robot constituted by a
footplate fixed through a six-axis force sensor to a
servomotor shaft that controls its angular position and speed
was used for measuring isometric muscle strength: the
subject’s ankle was locked at the 0° ankle dorsiflexion,
and maximal voluntary contraction was taken [
Isometric torques of the ankle joint in different
kinematic configurations were obtained from a device able
to measure ankle torques around the three articular axes
(plantar-/dorsiflexion, int-/external rotation and
pronation/supination). The 6-DOF structure allows linear and
angular displacement of the ankle with respect to the
shank. Each DOF is blockable in different configurations
and torques and angles can be measured [
Future developments in rehabilitation robotics
Despite the poor psychometric properties of the MMT,
methods alternative to this test that can be easily
integrated in a clinical setting are lacking. Robotic devices can
address many of the problematics previously identified.
The responsiveness of muscle strength tests is important
for detecting small changes during the progression of
rehabilitation. Therapy goals can be set based on the
minimum force required for performing activities of daily
living, like walking or sit-to-stand [
]. It is important
that a test is able to detect changes at least equal to the
MCID. However, MCID of muscle strength changes in
patients with neurological disorders have not yet been
established. Ceiling effects must be avoided in order to have a
measurement scale that can be used also with mildly
affected patients. Robotic devices have the potential to
provide more sensitive assessments thanks to the sensors
embedded in their structure. Standard and repeatable
testing conditions can be achieved by implementing a system
for fixating the patient to the device and preventing
undesired movements and by programming a standard
sequence of movements that should avoid fatigue effects
. Moreover, assessment procedures can be integrated
in a therapy session performed with a rehabilitation device
without requiring additional setup time.
The isokinetic dynamometer is a first attempt to
provide a state-of-the-art robotic assessment method [
A large body of research on this device have unraveled
the possible shortcomings and studied different
applications and measurement protocols. In particular, factors
such as gravity compensation, damping of the system,
human-machine interface and alignment of the human
and robot axes of rotation have been considered in many
85, 89, 99
]. This knowledge can be applied
to the development of future robot-aided muscle
strength assessments, despite the fact that the
differences in hardware prevent the complete reproducibility
of the results. Testing subjects with severe weakness
requires particular attention because subtle levels of
muscle strengths can be masked by the use of device
that is too heavy for the patient or the use of a position
that does not eliminate the effect of gravity [
the motivation of the patient plays an important role
] and it would be worthy to investigate how this
human factor affects the outcome measures and
consequently to standardize the protocol and the instructions.
Definition of the measure
Proprioception can be defined as the ability of an
individual to determine joint and body movement (kinaesthesia)
as well as position (statesthesia) of the body, or body
segments, in space [
]. It is based on sensory signals
provided to the brain from muscle, joint, and skin
receptors , with muscle spindles playing the major role
]. Proprioceptive feedback has been demonstrated to
be a key component of motor control and functional joint
]. A diminished proprioceptive acuity at the
ankle joint is associated with a lower unipedal stance time,
which is a measure relevant for evaluating frontal plane
postural control [
]. Loss of proprioception has been
reported both in neurological (e.g. stroke, SCI, peripheral
neuropathy) and in orthopedic patients (e.g. knee
osteoarthritis) and it has been associated with an increased risk
of falls in the elderly [
Clinical assessment and open issues
Assessment of lower limb proprioception in clinical
practice is based mainly on two rather simple tests: the
movement detection at the big toe and the Romberg
]. In the first the examiner moves the patient’s
toe upward or downward and asks to detect the
direction and the amplitude of the movement. In the
Romberg test, the subject is asked to close his eyes while
standing with his feet close together. A non-specific
proprioceptive deficit would usually result in the loss of
balance. While useful as a quick method to detect the
presence of proprioceptive abnormalities, these tests are
not sensitive enough to detect mild impairments or to
track changes over time. Moreover, the test at the big
toe depends strongly on the pressure applied by the
examiner and the amplitude of the movement imposed
]. Furthermore, only the distal segments of the
upper and lower limb are tested and no assessments of
the proximal joints are performed. A more specific test,
even if less used in clinical practice and mainly in upper
limb examination, is the joint position reproduction or
matching (JPR) [
]. In this test the patient is
blindfolded and the examiner moves his/her limb to a target
position. The patient is then asked to match this
position either with the contralateral limb or with the same
limb after it has been brought back to the starting
position. This test is normally performed without any
instrument and the visually observed mismatch in position is
retained as a rough measure of proprioceptive precision
]. Goniometers can also be used to measure the
joint angle before and after the matching but their
reliability and measurement error have been shown to vary
widely . Items related to proprioception are included
also in the sensory-related section of the Fugl-Meyer
Score for stroke patients. Here small alterations in the
position of hip, knee, ankle and great toe are evaluated
]. However, the stimulus provided by the examiner is
inherently subjective and sensitivity is limited to 3 levels
(absent, impaired or normal proprioception).
State of the art in rehabilitation robotics
Instrumented tests for proprioception in lower limbs
have been developed using motorized devices or
isokinetic dynamometers. An overview of these experimental
devices and methods can be found in [
The classic JPR test discussed above can be easily
instrumented. A machine moves the subject’s limb to the
target position. The subject is then asked to match this
position, either by actively moving the limb or by
pressing a button when the limb passively moved by the
machine reaches the target position. However, it has to be
taken into account that active and passive motion of the
limbs are not equal in terms of sensory feedback [
JPR methods are not suitable for people with cognitive
impairments since they are highly dependent on
]. Moreover, they have been found to have slight
to moderate reliability [
]. A JPR test for assessing hip
and knee joint proprioception has been implemented in
the robotic gait orthosis Lokomat and tested in healthy
subjects and 23 incomplete SCI subjects [
subject’s leg was positioned at a target hip and knee angle
and then moved away to a distractor position. The
subject was then asked to place the limb at the remembered
target position using a joystick to control the robot. The
absolute error between target and remembered position
was retained as outcome measure. The test-retest
reliability in SCI was found to be fair at hip joint and
substantial at the knee joint but the Bland-Altman plots
showed broad LOA that indicate a low sensitivity in SCI
individuals. Heteroscedasticity was also reported.
Nevertheless, the score correlated well with the clinical
assessment of proprioception and a significant difference
between SCI patients and healthy subjects was found.
A second approach for measuring proprioception is
the threshold to detection of passive motion
(TTDPM). In this test the body segment under test is
moved by a machine in a predefined direction. The
subject is asked to press a button as soon as he/she
detects a movement. Movements are presented at
different velocities since the proprioceptive threshold
decreases with increasing speed [
]. A motorized
apparatus for testing hip, knee, ankle and toe
detection threshold was developed by Refshauge et al. and
the influence of speed and joint position on the test
outcomes was studied [
]. A modified
isokinetic dynamometer and a chair with motorized arms
have been used for assessing passive flexion/extension
and varus/valgus movements of the knee in healthy
subjects and osteoarthritic patients (OA) [
From the initial posture, the servomotor rotated the
knee at a constant low velocity of below or equal to
1°/s). The threshold position of detection of the
movement was retained, with smaller threshold values
indicating greater proprioceptive acuity. Reliability
was found to be excellent both within and between
raters, both for OA and healthy subjects. In both
studies the subjects wore headphones and an eye
mask. The TTDPM was tested also using the
Lokomat : hip and knee separately were moved
according to a randomized order of speeds (0.5–4°/s),
directions and catch trials (no movement). Angle and
reaction time were used to calculate a movement
detection score. The score presented substantial reliability
and a high correlation with a clinical score of
proprioception, showing better sensitivity (it is possible to measure
reaction times ≥ 50 ms) and no ceiling effects. Faster speeds
were able to elicit a response in severely impaired subjects
that could not detect movements at 0.5 °/s. The TTDPM
test leads generally to more precise and less variable
measures of proprioception acuity than the JPR test.
Interestingly, the two tests have shown no concurrent validity [
Future developments in rehabilitation robotics
These studies demonstrate that instrumented and robotic
assessments of proprioception are feasible and present
several advantages over clinical assessments of
proprioception. Measures of proprioception in clinical practice
are rather coarse and lack granularity. Standardization is
nearly absent and the outcome of clinical tests is often a
Lower limb robotic devices provide the possibility to
maintain a high consistency in the protocol (speed,
points of contact, timing) between trials. The
responsiveness of the robot-based measure was demonstrated
also by the ability to detect a wide range of angle errors
in subjects that were judged unimpaired by the clinical
]. Moreover, the influence of motor
impairment on the control of lower limbs can be
eliminated because the leg can be passively moved by the
robot. Lastly, robotic devices can provide useful
information on joints that are not normally addressed in
clinical practice, where the most common examination
involves only the big toe . It is likely that specific
information on other joints might provide an insight on
different components of sensory function useful to track
changes in recovery after injury [
]. On the other side,
the straps of exoskeletal devices may provide additional
cutaneous feedback to the subject, thus influencing the
]. When designing a new robotic
device or protocol for proprioception assessment it is
important to consider that the test methods (JPR or
TTDPM) do not provide the same information [
Different versions of the protocol exist also within the
same test and again their choice can highly influence the
]. The speed of a TTDPM test highly
influences the outcome measures [
] and must be
accurately controlled by the robotic device. Active and
passive movements are likely to activate different
proprioceptive mechanisms .
Robot-based assessments of proprioception require
longer time of administration with respect to clinical
assessments, but they are able to provide reliable and
sensitive information on proprioceptive acuity that allows a
more detailed examination useful for diagnosis or
accurate tracking of the recovery of the patient.
Abnormal joint torque coupling and synergies
Definition of the measure
Due to cortical damage, stroke survivors and cerebral
palsy (CP) children can lose the ability to move their
joints independently, which result in abnormal coupled,
pathophysiological movement patterns, also called
synergies. The loss of independent control of joint moments
is caused by involuntary co-activation of muscles over
multiple joints [
] defined two often occurring
pathophysiological synergies in the lower extremities:
1. Extension synergy consisting of internal rotation,
adduction and extension of the hip, extension of the
knee, and plantar flexion and inversion of the ankle
2. Flexion synergy consisting of external rotation,
abduction, and flexion of the hip, flexion of the knee
and dorsal flexion and eversion of the ankle
Clinical assessment and open issues
Loss of independent joint control limits the performance
on activities of daily living. Therefore, in both clinical
and in research settings abnormal joint torque coupling
is often being assessed and this is mostly done using the
Fugl-Meyer Assessment of Physical Performance [
This scale has been shown to be a reliable, sensitive and
valid method for the assessment of motor impairment
after stroke [
]. However, it can be argued that
for the quantification of abnormal joint torque coupling
this scale lacks sensitivity due to the use of a 3-point
scale (0 = cannot perform,1 = performs partially, 2 =
performs fully) for the assessment of each component of
State of the art in rehabilitation robotics
Robotic and robot-related measures could possibly
provide more accurate information. Over the last
decade several studies have investigated abnormal joint
torque coupling using robotic and robot-related
68, 92, 120–126
]. The majority of these studies
quantified the synergies in static situations during
isometric contractions and used a similar approach.
Subjects were strapped into a (robotic) device (most
often the Lokomat) that constrains every movement
of the concerned leg and the pelvis. The device was
equipped with force sensors to measure all the interaction
forces/torques that the subject exerts with this leg on the
device, for instance the cuffs of the Lokomat were
instrumented with 6-DOF load cells [
92, 120, 121
produced isometric torques in a particular direction
(primary), while torques in all other the directions (secondary)
were also measured. Abnormal torque coupling was
quantified as the difference in secondary torque production
between healthy individuals and stroke survivors. Studies
differed in the amount of joints and planes that were
investigated and the position in which the coupling was
assessed. Thelen et al. [
] assessed the coupling while
subjects were positioned in an adjustable chair with ankle
fixed to six degree- of-freedom load cell, whereas others
assessed the coupling while subject where standing in the
toe-off and/or mid-swing position with the test leg
Thelen et al. showed that individuals with cerebral
palsy produced a knee extension moment during hip
extension and vice versa whereas healthy subjects
produced a knee flexion moment during hip extension and
a hip flexion moment during knee extension.
Quantification of abnormal joint couplings using a (robotic) device
has provided evidence for different couplings. Neckel
et al. [
] found that stroke survivors only showed an
abnormal coupling between hip abduction and flexion
and had similar couplings as found in healthy subjects
for the other degrees of freedom. Cruz and Dhaher [
observed that stroke survivor coupled knee extension
with hip adduction. Tan et al. [
] found strong
coupling between ankle frontal plane torque and hip sagittal
plane torques and vice versa that were not present in the
healthy control subjects (ankle plantar flexion with hip
adduction, ankle eversion with hip extension and ankle
inversion with hip flexion). Recently, Sanchez et al. [
also found evidence for the earlier found coupling
between hip extension and adduction, and ankle plantar
flexion and hip adduction. So, evidence starts to
accumulate that stroke survivors have abnormal coupling
between hip adduction, hip extension and plantar flexion.
To our knowledge only one study has attempted to
identify abnormal joint torque coupling during walking
]. In this study participants were moved along a
predetermined locomotor trajectory using the Lokomat
while interaction and ground reaction forces were
measured. However, the difficulty with this setup is that it is
hard to disentangle the torques required for walking and
maintaining balance and those resulting from the
abnormal joint torque coupling. Therefore, although assessed
in a quasi-dynamic situation, the results may not be
generalizable to voluntary walking.
The reliability (test-retest, inter-rater, intra-rater) has
not yet been assessed for these abnormal couplings, nor
has its responsiveness been determined. The criterion
validity has not explicitly been investigated, however
Cruz and colleagues [
] demonstrated using step wise
regression that the coupling between knee extension and
hip adduction was the best predictor of gait speed
amongst other strength and coupling variables. None of
the aforementioned studies did correlate their coupling
measures with a clinical scale like the Fugl-Meyer to
assess the construct validity.
Future developments in rehabilitation robotics
To summarize, robotic measures may be able to quantify
abnormal joint torque coupling more precisely compared
to clinical measures such as the Fugl-Meyer Assessment
of Physical Performance. However, the reliability,
responsiveness and validity of these measures need to be further
investigated. Additionally, robotic assessment is still
performed under static or quasi-dynamic conditions, which
might not quantify well how these couplings limit walking.
For assessing abnormal couplings in the upper
extremities, the assessments have moved from a static approach
] to a dynamic approach where the couplings are
assessed during reaching movements using robotic devices
]. We foresee that a similar shift will happen for the
lower extremities. Integration of the principle used in the
robotic assessment under static conditions in robotic gait
trainers could provide the tools to assess abnormal joint
torque coupling during walking.
Definition of the measure
In the clinical field, the term joint stiffness has been used
to express the sensation of difficulty in moving a joint
]. While this term is commonly used in the clinical
practice, the notion of stiffness used in this context does
not match the definition of stiffness in classical
mechanics. To describe all the mechanisms that contribute to
the resistance of motion, the term impedance is usually
preferred. In motor control literature, the term
mechanical impedance is defined as the dynamic operator that
specifies the force an object generates in response to an
imposed motion [
]. The latter definition includes all
motion-dependent effects, i.e. those terms that specify
the force generated by changes in position (e.g. stiffness,
non-elastic forces), in velocity (e.g. viscosity, damping)
and in acceleration (e.g. inertia) [
]. In biomechanics,
the term joint impedance relates the motion of the joint
and the torque acting about it [
]. Joint impedance is
usually estimated by applying a torque or force
perturbation and measuring the resulting change in position or
applying a position perturbation and measuring the
resulting change in torque of force.
Joint impedance is mainly determined by three sources:
i) the passive biomechanical properties of the muscles,
tendons and tissue around the joint and limb inertia –
passive components; ii) the resistance produced by the
muscles in response to reflexes [
] – reflexive
components; and iii) the resistance produced by the muscle
fibers due to non-reflexive, neural-driven contractions –
intrinsic components . Since the reflexive and
intrinsic component are both related to muscle activation, their
sum is commonly referred to as active component1.
In neurological populations, an abnormal increase in
joint impedance can result from spasticity, rigidity or
]. The intrinsic and reflexive components
have also been shown to be affected in neurological
Joint impedance varies with muscle contraction [
joint position [
], rotation amplitude [
the duration of the applied perturbation, since after
approximately 30 ms cross-bridges break [
] and the
contribution of cross bridge stiffness to the overall joint
impedance will diminish. Joint position affects joint
impedance measurements because the intrinsic component
increases towards the extreme joint angles as the
ligaments get more stretched. Additionally the different
muscles vary their active contribution to the joint
impedance depending on their length (and therefore on the
corresponding joint configuration), due to the particular
shape of the length-tension curve of the muscle [
The reflex activity is also known to be speed dependent
] and only contributes above a threshold [
Finally, the task instruction given to the subject will also
shape the joint impedance [
]. Most common task
instructions are ‘relax’,resist the perturbation’, or ‘keep the
Clinical assessment and open issues
The Modified Ashworth Scale [
] is the most widely
used clinical assessment to quantify an abnormal
increase in joint impedance due to excessive muscle tone.
The MAS consists of moving the limb of the patient
through its range of motion and rating the resistance on
a 6-point scale. The MAS is widely accepted, even
though the validity and reliability of the measure are
] since especially inter-rater reliability
was slight to fair. Moreover, the MAS may also lack
sensitivity. The MAS assess joint impedance only in passive
conditions, where the subject is asked to relax, which
might not be indicative for how spasticity influences
dynamic movements. Another test to assess the increased
resistance to movement in a more quantitative way is the
pendulum test, first described by Wartenberg [
test quantifies movements of the lower leg following its
drop from a horizontal position by deriving the angle of
first reversal, the maximal angular velocity or number of
oscillations. The pendulum test has shown good
convergent validity, reliability and sensitivity [
limitations of this test are that it is done in relaxed
conditions – which is difficult to achieve - and can only be used
for the knee. Additionally, measuring equipment
(electrogoniometers, inertial sensors) are needed to record the leg
motion and to extract the variables.
While measurement of joint impedance in not
commonly performed on the everyday clinical practice, it has
implications in understanding a potential cause of
impairment. For instance, Mirbagheri et al. [
] was able
to isolate abnormal active contributions in spinal cord
injury patients based on measurement of joint
impedance of the ankle. Such measurements can also point
out to different pathologies such as spasticity, rigidity or
State of the art in rehabilitation robotics
As mentioned earlier, joint impedance is dependent on
joint position, muscle contraction levels, and amplitude,
velocity and duration of the perturbation. Therefore, the
use of robotic devices is advantageous because these
factors can be precisely controlled at the same time
relevant signals are been recorded. Several instrumented
and robotic measures have been developed to asses
either the reflexive and/or intrinsic components of joint
45, 49, 136, 138, 155–162
]. We will not
review all devices and methods. In particular for the ankle
joint many devices have been developed, which have
recently been reviewed . To assess passive joint
impedance, the joint of the participant is moved by a robotic
manipulator or manually over a certain angle often
measured using a potentiometer while the resisting force is
measured using force sensors integrated in the (robotic)
device. For accessing the passive joint impedance it is
important that no muscle activity is present. Therefore,
the participant is asked to (try to) relax and the angular
velocity is kept low to avoid the excitation of reflex
contractions. In the push and pull test, the joint is moved
with small increments and kept static for approximately
5 s in every position. The net moment (after removing
gravity) provided by an external device to keep the
segment in equilibrium is retained for each incremental
]. Both isokinetic dynamometers and
custom made joint actuators have been used as assessment
devices. With a manually operated device the passive
ankle impedance could be estimated reliably in healthy
subjects (ICC values between 0.71 and 0.85,[
]) and in
CP children (ICC = 0.82, [
]). In the study of Chesworth
et al. [
] a custom made torque motor system was
used to assess passive joint impedance of the ankle with
a comparable reliability (ICC: 0.77–0.94).
The contribution of active components (i.e. intrinsic
and reflexive) to joint impedance have also been
investigated using similar experimental setups. In a typical
setup, the subject is either asked to actively resist an
angular displacement or to (try to) exert a constant force.
At some point, either an angular position perturbation is
applied while the resisting force is measured or a force
perturbation is applied and the resulting angle is
measured. Impedance measured under this condition
contains the three components: passive, intrinsic and
reflexive. To be able to distinguish between these
components, different strategies have been used. For
example, in the study of McHugh et al. [
] the passive
component is subtracted from the total impedance to
determine the active component. Also more complex
methods exist, which are based on system identification
techniques. In the method of Mirbagheri et al. [
system identification method is applied to distinguish
between intrinsic and reflexive components. In this
method, pseudo-random continuous rotations of the
ankle are applied, and the ankle torque and EMG of
involved muscle groups are recorded. The model consists
of an intrinsic component and a unidirectional delayed
velocity feedback pathway representing the reflexive
component. Input to the model is ankle rotation, and
the model parameters are optimized to minimize the
error between the predicted and recorded torque. The
EMG is used to determine the latency of the reflex
component. In healthy subjects a good intra-rater (r > 0.8)
reliability was found [
]. De Vlugt et al. [
similar techniques but instead of continuous rotations,
they applied ramp-and-hold ankle dorsiflexion rotations
with different speed profiles. They employed a nonlinear
neuromuscular model that is more complex than the
one used by Mirbagheri to predict the recorded ankle
torque. Results showed that stroke survivors could be
distinguished from control subjects by tissue stiffness and
viscosity and to a lesser extent by reflexive torque from
the soleus muscle. These parameters were also sensitive to
discriminate different patients, who were clinically graded
by the MAS. In a subsequent study [
] these researchers
adapted their model and protocol slightly by applying both
ankle plantar and dorsiflexion rotations. The estimated
model parameters could discriminate between patients
with CP and control subjects. Soleus background activity
was sensitive to MAS spasticity severity, but reflex activity
was not. Preliminary data indicated that reflex activity was
reduced after spasticity treatment. The between-trial (ICC:
0.76–0.99) and between-day repeatability (ICC: 0.64–0.95)
was moderate to substantial for tissue stiffness and
background activity, but not for reflex parameters.
A shortcoming of most of the studies on joint
impedance is that the assessment is done for static or passive
tasks where the participants are in a supine, prone-lying
or sitting position. The ankle impedance has also been
determined in more natural active conditions, such as stance
] using very fast dorsi- and plantar-flexion
rotations with a motorized footplate and non-parametric
Aforementioned studies and approaches all made use
of dedicated assessment setups. However also robotic
gait trainers can be used to derive measures of joint
impedance. For instance, the Lokomat has a built-in
function to assess overall joint impedance of the hip and
knee joints by passively moving the limbs at different
speed profiles and recording the resulting joint torque.
Using this technique, a moderate correlation between
joint impedance and MAS scores could be seen [
Koopman et al. [
] used the LOPES robotic gait
trainer and multi input multi output system
identification techniques to assess joint impedance of the hip and
knee. Healthy subjects were assessed while in the toe-off
or heel strike position and were asked to resist the
movement of the device or apply no force at all. Results
showed that the effect of biarticular muscles on the
inter-joint impedance could not be ignored.
Future developments in rehabilitation robotics
Although research on the accuracy and reliability of
robotic devices to assess joint impedance is not
available for all developed devices and methods, it can be
argued that the use of integrated sensors and robotic
actuators will show better psychometric properties
compared to the MAS score. Another advantage of
robotic measures is that they can help to develop
methods to estimate the active and passive or
intrinsic from reflexive components, while the MAS only
measures the resistance of motion but not the
underlying cause. The pendulum test could be implemented
in combination with transparent devices that do not
hinder the natural oscillation of the shank (e.g. soft
exoskeletons). However, the reviewed robotic
assessments are still performed under non-functional and
static or passive conditions. Therefore, further
development is necessary to be able to assess joint
impedance during a dynamic task such as walking. A
method to estimate joint impedance during gait is to
use musculoskeletal models and using optimization
techniques to estimate muscle forces that are related
in the model to muscle impedance [
alternative method is to apply time-varying system
identification algorithms to estimate the changing impedance
of the human knee over the gait cycle [
ensemble-based correlation technique averages over
repetitions instead as over the time cycle [
Averaging over repetitions and over time within a short
data segment within repetition can also be combined
] with the advantage that less repetitions are
needed. A testing platform consisting of a knee
perturbator has been built in order to deliver velocity
perturbations during walking and record reaction
torques, with the aim of determining the knee
impedance using system identification techniques [
ensemble-based correlation technique has also been
applied to estimate the modulation of the ankle
impedance from the end of the stance phase to heel
contact with MIT’s AnkleBot [
comparing the estimated knee impedance of the
ensemblebased correlation method with the model-based
] shows order of magnitude differences
in the estimated knee impedance. Hence, more
research is needed to reliably estimate the impedance of
multiple joints during gait.
Walking function/Gait pattern
Definition of the measure
Walking can be defined as a repetitious sequence of limb
motions that move the body forward while
simultaneously maintaining stance stability [
]. Gait refers to
the manner or style of walking [
]. Gait is composed
by a cyclic series of motion patterns performed by the
hip, knee and ankle. The gait cycle can be divided in phases,
the main ones being swing and stance [
174, 176, 177
Walking can be described according to different
domains: i) the capacity of performing activities related
to walking (e.g. walking without assistance, sit-to-stand);
ii) the spatio-temporal characteristics (e.g. speed, step
length, cadence, stance/swing ratio); iii) the “quality” of
gait pattern, which concerns the ability to coordinate
lower-limb segments and joints (e.g., simultaneous
coordination of hip and knee angles) [
Clinical assessment and open issues
In clinical practice, walking is mainly assessed by
examining the spatio-temporal characteristics and the
capacity of performing walking-related tasks. Like the other
assessments discussed in this paper, measuring walking
is also influenced by time constraints. Therefore,
measures that are relatively easy and fast to administer are
Among these, the capacity of performing functional
walking activities is commonly assessed using
ordinalbased clinical scores. These tests have a low administrative
burden and they can be useful to grossly categorize the
patients according to their walking capacity, but they are
not sensitive enough to detect small improvements in
]. The Walking Index for Spinal Cord
Injury (WISCI II), for example, assigns a score between 0
and 20 based on the amount of assistance required for
walking (e.g., walking with one/two crutches). Consistent
floor and ceiling effects and a low responsiveness were
]. Moreover, the different levels are unevenly
spaced, meaning that a change of 1 point in the score has
a different relevance depending on the position along the
]. Several other activity-based tests were
developed (e.g., Functional Ambulation Category (FAC),
Dynamic Gait Index (DGI)) to assess walking function but,
although very useful for gaining information on the overall
walking process, they are unable to provide any detailed
information on the way it is realized.
Time-based tests are often performed, since they provide
quantitative measures and have shown substantial
interand intra-rater reliability [
]. For example, in the
10-mWalking-Test (10MWT) a stopwatch is used to measure
the time required to walk 10 m [
]. Thus, the test
provides a measure of short-duration walking speed and it has
substantial correlation with other time-based walking tests
and with other walking-related functions like muscle
strength of the lower limb (Table 3) [
]. However, the
information obtained with this test is limited to gait speed,
which, although normally used as a surrogate measure for
gait quality [
], is not able to provide information on
complex alterations of walking (e.g., compensatory
]. 10MWT and other time-based walking tests
(e.g., Time-Up-and-Go, 6-min-Walking-Test) present floor
and ceiling effects since non-ambulatory subjects score 0
and mildly impaired patients could walk longer distances at
the same speed [
]. Other spatio-temporal parameters
can be obtained using more sophisticated instruments like
] and pressure mats [
]. Heel strike
events can be detected from an IMU placed at the lower
]. A more detailed step segmentation is possible if
the IMUs are placed directly on the feet [
Parameters such as step duration, step length and swing/stance
time ratio can provide important additional information on
gait impairments and on the progresses during recovery.
For example, there is evidence that step variability (i.e.
variability in stride time, stride length and gait speed) is altered
in patients with neurodegenerative diseases . In stroke
patients, asymmetry in right and left step time and
altered stance/swing time ratio were reported using IMUs
]. IMU-based systems are not yet widely integrated
in clinical practice, even if new systems are now
commercialized (e.g. McRoberts DynaPort [
], GaitUp [
It is important to evaluate the patient’s gait pattern to
understand whether the person is using compensatory
strategies. These strategies might indeed not be visible
in the spatio-temporal gait characteristics, which can be
similar to physiological ones even in presence of an
aberrant muscle activity [
]. This is especially important
in longitudinal studies which aim at demonstrating
whether improvements in walking speed are attained
either by using compensatory strategies or by restoration
of the pre-morbid gait patterns [
]. By using
measurements able to capture the quality of the gait pattern it is
possible to discriminate between the two different
recovery strategies - compensation or restoration of
physiological gait. However, at present the quality of the gait
pattern can only be accurately assessed using a motion
tracking system and force plates. This instrumented gait
analysis provides an accurate measure of joint angles,
moments and powers but requires a costly equipment
and a long administration time.
A major issue related to walking assessment is that
non-ambulatory subjects are often assigned the lowest
score in every test (e.g. 0 m/s in the 10MWT, 0 score in
the WISCI II), irrespective of their residual lower limb
functions. These subjects are therefore excluded from it
because most of the scales’ floor effect. It would be
possible to assess non-ambulatory subjects indirectly by
measuring other variables that correlate with walking
ability, like muscle strength or balance. However, these
tests are performed usually while sitting or lying, in
contexts very dissimilar to walking.
State of the art in rehabilitation robotics
Driven gait robots for treadmill walking and
exoskeletons for overground assistance can be used to record
joint kinematics while walking in order to obtain
information on the quality of the gait pattern. Robotic
exoskeletons equipped with angular position sensors have
been utilized to record joint kinematics during treadmill
or overground walking [
36, 42, 199–201
The Lokomat and the LOPES have been used to
measure hip and knee angles in various studies, where the
reduced impedance of the joints allowed the subjects to
impose their own gait pattern. The joints’ kinematics
was evaluated mainly by comparing it with a reference
angular trajectory: e.g. timing error within a tunnel
around the desired spatial path [
] or spatial tracking
]. A method to assess retraining in stroke
patients based on the areal difference between a healthy
reference and the patient’s trajectory during the swing
phase was implemented in the ALEX gait trainer [
However, at present, robotic gait trainers might not be
the most suitable devices for performing an assessment
equivalent to camera-based gait analysis, due to the
influence that their mechanical constraints have on the
gait pattern. Wearable and lightweight devices that do
not hinder human movements are required for this
purpose. Particularly suitable for this condition would be
the soft lower limb exoskeletons (“exosuits”) that have
been recently developed to improve human-robot
interaction and to allow a more natural walking pattern [
]. In an active soft orthotic ankle device two
IMUs placed on the shank and on the foot are used to
compute the ankle joint angle [
]. Alternatively, strain
sensors embedded in the suit spanning over a joint are
]. Although this is a promising approach for
measurements in dynamic conditions, the sensing
accuracy is at present not high enough for accurate
measurements, due to relative movements between the suit and
the skin of the subject. Moreover, sensor calibration is
required every time a user wears the suit.
The robotic assistance required for walking has been
proposed as an alternative method for assessing the
walking function. For example, adaptive algorithms
automatically adjust the support provided by the device
based on the patient’s ability to follow a predefined
trajectory or to perform a specific task (e.g. foot clearance)
200, 201, 206, 207
]. The algorithms update a control
parameter K (usually the impedance of the joints and
the unloading of the body weight) at each walking step s
based on a forgetting factor γ < 1 and on the weighted
error g ⋅ e calculated in the previous step:
K sþ1 ¼ γ⋅K s þ g⋅es
After a certain number of steps, the parameter K
converges to a value that can be retained as a measure of
the subject’s impairment [
]. For example, an overall
score can be obtained summing the torques required at
each joint averaged during the last 10 steps [
Future developments in rehabilitation robotics
Robotic gait trainers and exoskeletons for overground
assistance can be easily instrumented to provide
kinematic and kinetic data that can be used to derive metrics
useful for assessing the gait pattern and the walking
function. Since these devices enable non-ambulatory
patients to walk in a safe and functional manner, they
allow the assessment of these category of subjects,
limiting unwanted floor effects of the tests. Although these
systems are expensive, they are already used in many
clinical centers worldwide for providing gait training. In
these contexts, subjects can be tested during gait
training, requiring no or little additional time. Repeatable
assessment procedures can be programmed in order to
standardize the testing conditions (e.g., speed, unloading
of body weight). Accurate measurements of the gait
pattern can be obtained if the effects of the device dynamics
(i.e. weight, inertia) are minimized. Moreover, the
exoskeletons should have enough degrees of freedom to
avoid constraining physiological walking movements.
The compliant fixation of the patient’s leg to the orthosis
could lead to measurement inaccuracy and errors [
therefore standardized procedures need to be established
in order to make the patient’s setup in the device as
independent as possible of the operator. When the
transparency of the device is guaranteed by hardware design
(e.g. soft exoskeletons) or by software compensation
], the robot can be used for measuring joint
kinematics or spatio-temporal gait parameters. When this
condition is not met or when the subject is too impaired to be
able to walk without the support of the device, other
assessment methods must be used. It would be misleading,
in fact, to measure standard gait parameters in a robotic
gait trainer that affects the patient’s walking pattern. To
address this problem, new outcome measures can be
proposed. For example, the amount of support (i.e. joint
impedance or unloading of the body weight) required to
achieve a functional walking pattern can be used as an
indicator of the subject’s impairment. Further studies in this
direction are needed to establish the concurrent validity of
this outcome measure with existing clinical scores. It can
be hypothesized that a correlation with clinical scores that
address the amount of support required for walking (e.g.
WISCI II, FAC) exists. Moreover, if the algorithm adapts
the support of the device to the particular needs of the
single gait phases, it would be also possible to identify
specific impairments localized within the gait cycle .
However, the results of this method depend on the
performance metric used. If a measure of the deviation from
a reference trajectory is applied, the resulting support will
depend also on the similarity between the prescribed
trajectory and the patient’s individual gait pattern. A dead
band around the reference trajectory, as in [
partially address this problem.
Discussion and conclusion
In this review we have discussed how novel robot-aided
functional assessments can address the current issues
related to the evaluation of the health-related status of a
patient in clinical practice. Although essential for
maximizing the individual therapy outcomes, the use of
assessment methods in routine practice is at present
insufficient. Among the reasons that contribute to this dearth,
poor quality of the existing assessment scores and high
administrative burden have been identified [
]. In the
different sections of this review we have highlighted
additional issues in current clinical assessments specific for
different lower limb functions. We have explained how
robotic devices for rehabilitation have the potential to solve
these issues by providing high quality assessments (i.e.
objective, reliable and valid) and by integrating the
assessment procedure in a training program. Based on the
existing shortcomings and on the possibilities offered by
robotic technologies, we have proposed solutions and
recommendations for the development of novel robot-aided
assessment tools. The quality of the assessment methods
must be determined by studying their psychometric
properties, as discussed in the section Assessments validation.
We believe that the increasing use of robots for
rehabilitation is not only beneficial for the therapy outcome, but
also represents a huge opportunity for improving the
assessment quality and increasing their frequency of
administration. Indeed, robotic devices can be equipped with
sensors for recording data useful for developing
quantitative and objective assessment metrics. Secondly, robots
can potentially assure the standardized execution of the
assessment procedure, which is essential for reducing the
measurement error and increasing the reliability. Moreover,
robot-based assessments can reach higher inter-rater and
intra-rater reliability if the robotic device is designed to limit
fixation errors and to reduce inter-operator differences.
Cuffs positioning, misalignments and different tightening of
the fixation to the patient’s limbs may have a huge impact
on the reliability of the assessment outcomes. User-friendly
and ergonomic robotic device, along with a rigorous
training of the operators may contribute to solve this problem.
A known issue of the current assessments used in clinical
practice is their administrative burden (mainly time-wise)
that limits the frequency at which they are administered.
Assessments executed with rehabilitation robotic devices
can be performed during the therapy session, measuring
relevant parameters directly during the training, while the
patient is using the device. Robotic assessments are able
not only to complement existing clinical measurements,
but also to enlarge the measurable range of an impairment:
because of the quantifiable assistance that robots can
provide, robotic assessments can be administered even if the
patient is not able to perform the movement without
]. Moreover, measurements that have been only
subjectively addressed before (e.g., proprioception) can now be
targeted by instrumented tests. New variables that were not
readily accessible before (e.g., smoothness, joint coupling)
become available. Further research on neurophysiological
mechanisms must be encouraged to determine how these
variables relate to sensorimotor functions and whether they
can provide information on recovery [
]. The increased
sensitivity and the reduced measurement error of the
robot-based assessments can be of utmost importance
when the outcome measures are used in a clinical trial
aimed at demonstrating the efficacy of a new therapy. Often
little can be concluded because the effects of the therapy
under study are masked by high inter-subject variability or
they are not captured by conventional clinical assessments
]. Assessments able to distinguish the contribution of
restoration of physiological patterns and the effect of
compensatory mechanisms to the recovery will help to orient
future therapeutic approaches [
]. Not least, more sensitive
measurements could also contribute to increasing the
motivation of the patients, when even a slight improvement
can be documented.
Before starting a research study aiming at developing a
new assessment method, researchers must consider several
issues. First of all, an inter-disciplinary approach involving
research institutes, clinical facilities and medical device
manufacturers is encouraged in all the phases of
development: researchers must take into account the clinical
relevance of the proposed measure (is the information
provided by the measure useful for adjusting the therapy?),
the interpretability of the outcome parameter (what is its
physiological meaning?), the feasibility of the method for
both its use in clinical practice (is it safe? Are its
administrative and respondent burdens reasonable?) and the
manufacturability and large-scale implementation. If these steps
are missing, the risk is that the assessment method will
never be routinely used in the clinical practice. On the
other side, it is also important that researchers go “beyond”
the limits of established clinical tests by developing new
and independent standards based on robotic
measurements. The final aim of robotic assessments, indeed, should
not be to reproduce existing clinical scales that, even if
widely accepted in the clinical practice, are not comparable
by their nature to instrumented and robot-based
]. Lastly, when developing assessment metrics of a
same variable for different lower extremities devices,
researchers should try to make the results independent of the
platform on which they are obtained. In this way the same
metric could be implemented in different devices and
results from several studies could be compared. Even if the
dynamics of the device will most likely influence some of
the assessment metrics, comparative measures can be used
(e.g. normalizing patient’s data against healthy normative
data recorded in the same device).
A crucial requirement for the acceptance of a new
assessment method in the rehabilitation community is its
clinical validation: reliability must be assessed with an
adequate sample size and validity should be established
either by comparing the score with a gold-standard - if it
exists - (concurrent validity) or by studying the
relationship of the new assessment score with the underlying
constructs of interest (construct validity). Guidelines for the
validation of new robot-based assessments should be
developed to help the researchers to define adequate clinical
validation studies and to use the correct statistical tools.
Moreover, it is necessary to develop indications for
interpreting the different scoring systems: clinicians must be
able to identify whether a change in score is clinically
significant or it is due to measurement error [
We think that robotic assessments represent a
challenging “green field” where researchers have the possibility –
and the urgency – to develop methods that will have a
strong impact on rehabilitation outcomes. Better
assessments of lower extremities functions will allow the
clinicians to prescribe therapeutic and rehabilitation plans that
optimize the individual recovery while minimizing
unnecessary effort and costs [
]. We believe, therefore, that
research for developing valid, reliable and responsive
assessment methods is strongly needed for clinical practice,
for studies on new therapies and, overall, for improving the
rehabilitation outcome and decreasing the time of recovery.
1The terminology used in different publications is not
always consistent. In the work of Kearny and colleagues [
the intrinsic component includes the passive properties of
the joint and the properties of the active muscle fibers.
10MWT, 10-m-walking-test; 1-RM, one repetition maximum; 6MWT,
6-minwalking-test; AB, able-bodied; CI, confidence interval; CP, cerebral palsy; CPM,
continuous passive motion; DF, dorsiflexion; DGI, dynamic gait index; DOF,
degree of freedom; EMG, electromyography; FAC, functional ambulation
category; HHD, hand-held dynamometer; ICC, intra-class correlation coefficient;
ICF, international classification of function, disability and health; JPR, joint
position reproduction; LOA, limits of agreement; MAS, modified Ashworth scale;
MCID, minimum clinically important difference; MDC, minimum detectable
change; MMT, manual muscle test; MVC, maximum voluntary contraction; OA,
osteoarthritis; PF, plantarflexion; ROM, range of motion; SD, standard deviation;
SEM, standard error of measurement; TTDPM, time to detection of passive
motion; TUG, timed-up-and-go; WISCI II, walking index for spinal cord injury
This work was conceived among the actions of “State of the Art
RobotSupported assessments (STARS)” in the frame of the COST Action TD1006
“European Network on Robotics for NeuroRehabilitation”. We would like to
thank Rick Bosveld for the initial contribution to the conception of the
SM at the time of writing of this work was employed within the Industrial
Academic Initial Training Network “Moving Beyond”
(www.movingbeyond.eu) that received funding from the People Programme (Marie Curie
Actions) of the European Union’s Seventh Framework Programme
(FP7/20072013) under REA grant agreement n° 316639.
Availability of data and supporting materials
SM wrote the manuscript. AM-C contributed substantially to the discussion
and the section on Joint Impedance and to drafting of the introduction and
conclusion sections. EVA and HVDK contributed substantially to the section
on Joint Impedance and wrote the section on Abnormal Joint Torque Coupling
and Synergies. All the authors participated in the conception of the scope
and structure of the paper, edited, reviewed and approved the manuscript.
At the time of writing, SM, AM-C and LL are employed by Hocoma AG, a
manufacturer of robotic devices for rehabilitation.
Consent for publication
Ethics approval and consent to participate
1Sensory-Motor Systems (SMS) Lab, Institute of Robotics and Intelligent
Systems (IRIS), Department of Health Sciences and Technology (D-HEST), ETH
Zürich, Zürich, Switzerland. 2Hocoma AG, Volketswil, Switzerland. 3Spinal
Cord Injury Center, Balgrist University Hospital, University Zürich, Zürich,
Switzerland. 4Department of Physical Medicine and Rehabilitation,
Northwestern University, Chicago, IL, USA. 5Laboratory of Biomechanical
Engineering, MIRA Institute for Biomedical Technology and Technical
Medicine, University of Twente, Enschede, The Netherlands. 6Department of
Biomechanical Engineering, Delft University of Technology, Delft, The
1. Duncan EA , Murray J. The barriers and facilitators to routine outcome measurement by allied health professionals in practice: a systematic review . BMC Health Serv Res . 2012 ; 12 : 96 .
2. Jette DU , Halbert J , Iverson C , Miceli E , Shah P . Use of standardized outcome measures in physical therapist practice: perceptions and applications . Phys Ther . 2009 ; 89 : 125 - 35 .
3. Copeland J . Outcome measures: why physiotherapists must use them . Phys Ther Rev . 2009 ; 14 : 367 - 8 .
4. Van Hedel HJA. Improvement in function after spinal cord injury : the blackbox entitled rehabilitation . Swiss Med Wkly . 2012 ; 142 : w13673 .
5. Steeves JD , Lammertse D , Curt A , Fawcett JW , Tuszynski MH , Ditunno JF , Ellaway PH , Fehlings MG , Guest J , Kleitman N , Bartlett P , Blight A , Dietz V , Dobkin B , Grossman R , Short D , Nakamura M , Coleman W , Gaviria M , Privat A. Guidelines for the conduct of clinical trials for spinal cord injury (SCI) as developed by the ICCP panel: clinical trial outcome measures . Spinal Cord . 2007 ; 45 : 206 - 21 .
6. Lambercy O , Maggioni S , Lünenburger L , Gassert R , Bolliger M. Robotic and wearable sensor technologies for measurements/clinical assessments . In: Neurorehabilitation Technology 2nd edition. Edited by Dietz V , Reinkensmeyer DJ . Springer International; 2016 .
7. Mehrholz J , Elsner B , Werner C , Kugler J , Pohl M . Electromechanical-assisted training for walking after stroke (Review) . Cochrane Libr . 2013 ; 44 ( 10 ): e127 .
8. Diaz I , Gil JJ , Sanchez E. Lower-Limb Robotic Rehabilitation : Literature Review and Challenges . J Robot . 2011 .
9. Keller U , Schölch S , Albisser U , Rudhe C , Curt A , Riener R , Klamroth-Marganska V . Robot-assisted Arm assessments in spinal cord injured patients: a consideration of concept study . PLoS One . 2015 ; 10 : e0126948 .
10. Bolliger M , Banz R , Dietz V , Lünenburger L . Standardized voluntary force measurement in a lower extremity rehabilitation robot . J Neuroeng Rehabil . 2008 ; 5 : 23 .
11. Tiffreau V , Ledoux I , Eymard B , Thévenon A , Hogrel J-Y. Isokinetic muscle testing for weak patients suffering from neuromuscular disorders: a reliability study . Neuromuscul Disord . 2007 ; 17 : 524 - 31 .
12. Ditunno PL , Patrick M , Stineman M , Ditunno JF . Who wants to walk? Preferences for recovery after SCI: a longitudinal and cross-sectional study . Spinal Cord . 2008 ; 46 : 500 - 6 .
13. Bohannon RW , Horton MG , Wikholm JB . Importance of four variables of walking to patients with stroke . Int J Rehabil Res . 1991 ; 14 : 246 - 50 .
14. Zhang M , Davies TC , Zhang Y , Xie S , Eng P . Reviewing effectiveness of ankle assessment techniques for use in robot-assisted therapy . J Rehabil Res Dev . 2014 ; 51 : 517 - 34 .
15. World Health Organization. Towards a Common Language for Functioning, Disability and Health ICF . 2002 .
16. Raghavendra P , Bornman J , Granlund M , Björck-Akesson E. The world health Organization's international classification of functioning, disability and health: implications for clinical and research practice in the field of augmentative and alternative communication . Augment Altern Commun . 2007 ; 23 : 349 - 61 .
17. Atkinson G , Nevill A . Statistical methods for assessing measurement error (reliability) in variables relevant to sports medicine . Sport Med . 1998 ; 26 : 217 - 38 .
18. Weir JP . Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM . J Strength Cond Res . 2005 ; 19 : 231 - 40 .
19. Altman DG , Bland JM . Measurement in medicine : the analysis of method comparison studies . Stat . 1983 ; 32 ( July 1981 ): 307 - 17 .
20. Andresen EM . Criteria for assessing the tools of disability outcomes research . Arch Phys Med Rehabil . 2000 ; 81 : S15 - 20 .
21. Roach KE . Measurement of health outcomes : reliability, validity and responsiveness . J Prosthetics Orthot . 2006 ; 18 : 1 - 5 .
22. Cook CE . Clinimetrics corner: the minimal clinically important change score (MCID): a necessary pretense . J Man Manip Ther . 2008 ; 16 : E82 - 3 .
23. Clarkson H . Joint motion and function assessment: a research-based practical guide . Philadelphia: Lippincott Williams and Wilkins; 2000 .
24. Rowe PJ , Myles CM , Walker C , Nutton R . Knee joint kinematics in gait and other functional activities measured using flexible electrogoniometry: how much knee motion is sufficient for normal daily life? Gait Posture . 2000 ; 12 : 143 - 55 .
25. Charbonnier C , Chagué S , Schmid J , Kolo FC , Bernardoni M , Christofilopoulos P. Analysis of hip range of motion in everyday life: a pilot study . Hip Int . 2015 ; 25 : 82 - 90 .
26. Anderson FC , Goldberg SR , Pandy MG , Delp SL . Contributions of muscle forces and toe-off kinematics to peak knee flexion during the swing phase of normal gait : an induced position analysis . J Biomech . 2004 ; 37 : 731 - 7 .
27. Ribbers G . Brain injury: long term outcome after traumatic brain injury . In: Stone J , Blouin M , editors. International Encyclopedia of Rehabilitation . 2010 .
28. Pohl M , Mehrholz J , Rockstroh G , Rückriem S , Koch R . Contractures and involuntary muscle overactivity in severe brain injury . Brain Inj . 2007 ; 21 : 421 - 32 .
29. Brosseau L , Balmer S , Tousignant M , O'Sullivan JP , Goudreault C , Goudreault M , Gringras S . Intra- and intertester reliability and criterion validity of the parallelogram and universal goniometers for measuring maximum active knee flexion and extension of patients with knee restrictions . Arch Phys Med Rehabil . 2001 ; 82 : 396 - 402 .
30. Gajdosik RL , Bohannon RW. Clinical measurement of range of motion. Review of goniometry emphasizing reliability and validity . Phys Ther . 1987 ; 67 : 1867 - 72 .
31. Piriyaprasarth P , Morris ME . Psychometric properties of measurement tools for quantifying knee joint position and movement: a systematic review . Knee . 2007 ; 14 : 2 - 8 .
32. van Trijffel E , van de Pol RJ , Oostendorp RB , Lucas C . Inter-rater reliability for measurement of passive physiological movements in lower extremity joints is generally low: a systematic review . J Physiol Lond . 2010 ; 56 : 223 - 35 .
33. Dijkstra PU , de Bont LG , Van der Weele LT BG . Joint mobility measurements: reliability of a standardized method . Cranio J Craniomandib Pract . 1994 ; 12 : 52 - 7 .
34. Jung I-G , Yu I-Y , Kim S-Y , Lee D-K , Oh J-S . Reliability of ankle dorsiflexion passive range of motion measurements obtained using a hand-held goniometer and Biodex dynamometer in stroke patients . J Phys Ther Sci . 2015 ; 27 : 1899 - 901 .
35. Bok S-K , Lee TH , Lee SS . The effects of changes of ankle strength and range of motion according to aging on balance . Ann Rehabil Med . 2013 ; 37 : 10 - 6 .
36. Banala SK , Kim SH , Agrawal SK , Scholz JP . Robot assisted gait training with Active Leg Exoskeleton (ALEX) . IEEE Trans Neural Syst Rehabil Eng . 2009 ; 17 : 2 - 8 .
37. Emken JL , Wynne JH , Harkema SJ , Reinkensmeyer DJ . Robotic device for manipulating human stepping . IEEE Trans Robot . 2006 ; 22 : 185 - 9 .
38. Farris RJ , Quintero H , Goldfarb M. Preliminary evaluation of a powered lower limb orthosis to aid walking in paraplegic individuals . IEEE Trans Neural Syst Rehabil Eng . 2011 ; 19 : 652 - 9 .
39. del-Ama AJ , Gil-Agudo A , Pons JL , Moreno JC . Hybrid FES-robot cooperative control of ambulatory gait rehabilitation exoskeleton . J Neuroeng Rehabil . 2014 ; 11 : 27 .
40. Esquenazi A , Talaty M , Packel A , Saulino M. The ReWalk powered exoskeleton to restore ambulatory function to individuals with thoracic-level motor-complete spinal cord injury . Am J Phys Med Rehabil . 2012 ; 91 : 911 - 21 .
41. Strausser KA , Swift TA , Zoss AB , Kazerooni H , Bennett BC . Mobile Exoskeleton for Spinal Cord Injury: Development and Testing . Arlington: ASME 2011 Dynamic Systems and Control Conference; 2011 .
42. Bortole M , Venkatakrishnan A , Zhu F , Moreno JC , Francisco GE , Pons JL , Contreras-Vidal JL . The H2 robotic exoskeleton for gait rehabilitation after stroke: early findings from a clinical study . J Neuroeng Rehabil . 2015 ; 12 : 54 .
43. Bartenbach V , Wyss D , Seuret D , Riener R. A Lower Limb Exoskeleton Research Platform to Investigate Human-Robot Interaction . Singapore: IEEE International Conference on Rehabilitation Robotics; 2015 .
44. Riener R , Lünenburger L , Maier IC , Colombo G , Dietz V . Locomotor training in subjects with sensori-motor deficits : an overview of the robotic gait orthosis lokomat . J Healthc Eng . 2010 ; 1 : 197 - 216 .
45. Peng Q , Park H-S , Shah P , Wilson N , Ren Y , Wu Y-N , Liu J , Gaebler-Spira DJ , Zhang L-Q . Quantitative evaluations of ankle spasticity and stiffness in neurological disorders using manual spasticity evaluator . J Rehabil Res Dev . 2011 ; 48 : 473 - 81 .
46. Waldman G , Yang C-Y , Ren Y , Liu L , Guo X , Harvey RL , Roth EJ , Zhang L-Q. Effects of robot-guided passive stretching and active movement training of ankle and mobility impairments in stroke . NeuroRehabilitation . 2013 ; 32 : 625 - 34 .
47. Zhang L-Q , Chung SG , Bai Z , Xu D , Rey EMT V , Rogers MW , Johnson ME , Roth EJ . Intelligent stretching of ankle joints with contracture/spasticity . IEEE Trans Neural Syst Rehabil Eng . 2002 ; 10 : 149 - 57 .
48. Giacomozzi C , Cesinaro S , Basile F , De Angelis G , Giansanti D , Maccioni G , Masci E , Panella a , Paolizzi M , Torre M , Valentini P , Macellari V . Measurement device for ankle joint kinematic and dynamic characterisation . Med Biol Eng Comput . 2003 ; 41 : 486 - 93 .
49. Roy A , Krebs HI , Williams DJ , Bever CT , Forrester LW , Macko RM , Hogan N. Robot-aided neurorehabilitation: a novel robot for ankle rehabilitation . IEEE Trans Robot . 2009 ; 25 : 569 - 82 .
50. Borsa PA , Sauers EL , Herling DE , Manzour WF . In vivo quantification of capsular End- point in the nonimpaired glenohumeral joint using an instrumented measurement system . J Orthop Sports Phys Ther . 2001 ; 31 : 419 - 31 .
51. McQuade K , Price R , Liu N , Ciol MA . Objective assessment of joint stiffness: a clinically oriented hardware and software device with an application to the shoulder joint . J Nov Physiother . 2012 ; 2 : 7 .
52. Jarrassé N , Proietti T , Crocher V , Robertson J , Sahbani A , Morel G , Roby-Brami A . Robotic exoskeletons: a perspective for the rehabilitation of Arm coordination in stroke patients . Front Hum Neurosci . 2014 ; 8 (December): 1 - 13 .
53. van Dijk W, van der Kooij H , Koopman B , van Asseldonk EHF . Improving the transparency of a rehabilitation robot by exploiting the cyclic behaviour of walking . IEEE Int Conf Rehabil Robot . 2013 ; 2013 : 6650393 .
54. Weiselfish-Giammatteo S , Giammatteo T. Integrative Manual Therapy for Biomechanics: Application of Muscle Energy and “Beyond” Technique : Treatment of the Spine, Ribs, and Extremities . North Atlantic Books; 2003 . [Integrated Manual Therapy Series].
55. Veneman JF , Kruidhof R , Hekman EEG , Ekkelenkamp R , Asseldonk EHF V. Design and evaluation of the LOPES exoskeleton robot for interactive gait rehabilitation . IEEE Trans Neural Syst Rehabil Eng . 2007 ; 15 : 379 - 86 .
56. Colombo G , Wirz M , Dietz V . Driven gait orthosis for improvement of locomotor training in paraplegic patients . Spinal Cord . 2001 ; 39 : 252 - 5 .
57. Rowe P , Myles C , Hillmann S , Hazlewood M. Validation of flexible electrogoniometry as a measure of joint kinematics . Physiotherapy . 2001 ; 87 : 479 - 88 .
58. Tesio L , Monzani M , Gatti R , Franchignoni F. Flexible electrogoniometers: kinesiological advantages with respect to potentiometric goniometers . Clin Biomech (Bristol, Avon) . 1995 ; 10 : 2 - 4 .
59. Bronner S , Agraharasamakulam S , Ojofeitimi S. Reliability and validity of electrogoniometry measurement of lower extremity movement . J Med Eng Technol . 2010 ; 34 : 232 - 42 .
60. Piriyaprasarth P , Morris ME , Winter A , Bialocerkowski AE . The reliability of knee joint position testing using electrogoniometry . BMC Musculoskelet Disord . 2008 ; 9 : 6 .
61. Menguc Y , Park YL , Martinez-Villalpando E , Aubin P , Zisook M , Stirling L , Wood RJ , Walsh CJ . Soft wearable motion sensing suit for lower limb biomechanics measurements . Proc - IEEE Int Conf Robot Autom . 2013 ; 5309 - 16 .
62. Donno M , Palange E , Nicola F Di, Member S , Bucci G , Ciancetta F . A New Flexible Optical Fiber Goniometer for Dynamic Angular Measurements : Application to Human Joint Movement Monitoring . IEEE Trans. Instrum. Meas . 2008 ; 57 : 1614 - 20 .
63. Favre J , Jolles BM , Aissaoui R , Aminian K. Ambulatory measurement of 3D knee joint angle . J Biomech . 2008 ; 41 : 1029 - 35 .
64. Luinge HJ , Veltink PH . Inclination measurement of human movement using a 3-D accelerometer with autocalibration . IEEE Trans Neural Syst Rehabil Eng . 2004 ; 12 : 112 - 21 .
65. Dejnabadi H , Jolles BM , Aminian K. A new approach to accurate measurement of uniaxial joint angles based on a combination of accelerometers and gyroscopes . IEEE Trans Biomed Eng . 2005 ; 52 : 1478 - 84 .
66. Djurić-Jovičić MD , Jovičić NS , Popović DB , Djordjević AR . Nonlinear optimization for drift removal in estimation of gait kinematics based on accelerometers . J Biomech . 2012 ; 45 : 2849 - 54 .
67. MeSH - Muscle Strength . [http://www.ncbi.nlm.nih.gov/mesh/68053580]. Accessed 26 July 2016 .
68. Neckel N , Pelliccio M , Nichols D , Hidler J . Quantification of functional weakness and abnormal synergy patterns in the lower limb of individuals with chronic stroke . J Neuroeng Rehabil . 2006 ; 3 : 17 .
69. Wirz M , van Hedel HJ , Rupp R , Curt A , Dietz V . Muscle force and gait performance: relationships after spinal cord injury . Arch Phys Med Rehabil . 2006 ; 87 : 1218 - 22 .
70. Kim C , Eng J. The relationship of lower-extremity muscle torque to locomotor performance in people with stroke . Phys Ther . 2003 ; 83 : 49 - 57 .
71. Cho KH , Bok SK , Kim YJ , Hwang SL . Effect of lower limb strength on falls and balance of the elderly . Ann Rehabil Med . 2012 ; 36 : 386 - 93 .
72. Newham DJ , Hsiao SF . Knee muscle isometric strength, voluntary activation and antagonist co-contraction in the first six months after stroke . Disabil Rehabil . 2001 ; 23 : 379 .
73. Noreau L , Vachon J . Comparison of three methods to assess muscular strength in individuals with spinal cord injury . Spinal Cord . 1998 ; 36 : 716 - 23 .
74. Paternostro-Sluga T , Grim-Stieger M , Posch M , Schuhfried O , Vacariu G , Mittermaier C , Bittner C , Fialka-Moser V . Reliability and validity of the Medical Research Council (MRC) scale and a modified scale for testing muscle strength in patients with radial palsy . J Rehabil Med . 2008 ; 40 : 665 - 71 .
75. Beasley W. Quantitative muscle testing: principles and applications to research and clinical services . Arch Phys Med Rehabil . 1961 ; 42 : 398 - 425 .
76. Herbison GJ , Isaac Z , Cohen ME , Ditunno JF . Strength post-spinal cord injury: myometer vs manual muscle test . Spinal Cord . 1996 ; 34 : 543 - 8 .
77. Bohannon RW . Manual muscle testing: does it meet the standards of an adequate screening test? Clin Rehabil . 2005 ; 19 : 662 - 7 .
78. Merlini L , Mazzone ES , Solari A , Morandi L . Reliability of hand-held dynamometry in spinal muscular atrophy . Muscle Nerve . 2002 ; 26 : 64 - 70 .
79. Stark T , Walker B , Phillips JK , Fejer R , Beck R . Hand-held dynamometry correlation with the gold standard isokinetic dynamometry: a systematic review . PM R . 2011 ; 3 : 472 - 9 .
80. Marmon AR , Pozzi F , Alnahdi AH , Zeni J. The validity of plantarflexor strength measures obtained through hand-held dynamometry measurements of force . Int J Sports Phys Ther . 2013 ; 8 : 820 - 7 .
81. Kim WK , Kim DK , Seo KM , Kang SH . Reliability and validity of isometric knee extensor strength test with hand-held dynamometer depending on its fixation: a pilot study . Ann Rehabil Med . 2014 ; 38 : 84 - 93 .
82. Meldrum D , Cahalane E , Keogan F , Hardiman O . Maximum voluntary isometric contraction: investigation of reliability and learning effect . Amyotroph Lateral Scler Other Motor Neuron Disord . 2003 ; 4 : 36 - 44 .
83. Colombo R , Mazzini L , Mora G , Parenzan R , Creola G , Pirali I , Minuco G . Measurement of isometric muscle strength: a reproducibility study of maximal voluntary contraction in normal subjects and amyotrophic lateral sclerosis patients . Med Eng Phys . 2000 ; 22 : 167 - 74 .
84. Lauermann SP , Lienhard K , Item-Glatthorn JF , Casartelli NC , Maffiuletti N. Assessment of quadriceps muscle weakness in patients after total knee arthroplasty and total hip arthroplasty: methodological issues . J Electromyogr Kinesiol . 2014 ; 24 : 285 - 91 .
85. Lienhard K , Lauermann SP , Schneider D , Item-Glatthorn JF , Casartelli NC , Maffiuletti N. Validity and reliability of isometric, isokinetic and isoinertial modalities for the assessment of quadriceps muscle strength in patients with total knee arthroplasty . J Electromyogr Kinesiol . 2013 ; 23 : 1283 - 8 .
86. Lum PS , Patten C , Kothari D , Yap R. Effects of velocity on maximal torque production in poststroke hemiparesis . Muscle Nerve . 2004 ; 30 : 732 - 42 .
87. Knapik JJ , Wright JE , Mawdsley RH , Braun J . Isometric, isotonic, and isokinetic torque variations in four muscle groups through a range of joint motion . Phys Ther . 1983 ; 63 : 938 - 47 .
88. Whiteley R , Jacobsen P , Prior S , Skazalski C , Otten R , Johnson A . Correlation of isokinetic and novel hand-held dynamometry measures of knee flexion and extension strength testing . J Sci Med Sport . 2012 ; 15 : 444 - 50 .
89. Van Campen A , De Groote F , Jonkers I , De Schutter J. An extended dynamometer setup to improve the accuracy of knee joint moment assessment . IEEE Trans Biomed Eng . 2013 ; 60 : 1202 - 8 .
90. Clark DJ , Condliffe EG , Patten C . Reliability of concentric and eccentric torque during isokinetic knee extension in post-stroke hemiparesis . Clin Biomech (Bristol, Avon) . 2006 ; 21 : 395 - 404 .
91. Hidler J . Robotic-assessment of walking in individuals with gait disorders . Conf Proc Annu Int Conf IEEE Eng Med Biol Soc IEEE Eng Med Biol Soc Annu Conf . 2004 ; 7 : 4829 - 31 .
92. Neckel ND , Blonien N , Nichols D , Hidler J . Abnormal joint torque patterns exhibited by chronic stroke subjects while walking with a prescribed physiological gait pattern . J Neuroeng Rehabil . 2008 ; 5 : 19 .
93. Veneman JF , Kruidhof R , Hekman EEG , Ekkelenkamp R , Van Asseldonk EHF , Van Der Kooij H . Design and evaluation of the LOPES exoskeleton robot for interactive gait rehabilitation . Neural Syst Rehabil Eng IEEE Trans . 2007 ; 15 : 379 - 86 .
94. Forrester LW , Roy A , Goodman RN , Rietschel J , Barton JE , Krebs HI , Macko RF. Clinical application of a modular ankle robot for stroke rehabilitation . NeuroRehabilitation . 2013 ; 33 : 85 - 97 .
95. Bohannon RW . Knee extension strength and body weight determine sit-to-stand independence after stroke . Physiother Theory Pract . 2007 ; 23 : 291 - 7 .
96. Bohannon RW . Relevance of Muscle Strength to Gait Performance in Patients with Neurologic Disability . Neurorehabil Neural Repair . 1989 .
97. Nadler SF , DePrince ML , Hauesien N , Malanga G , Stitik TP , Price E. Portable dynamometer anchoring station for measuring strength of the hip extensors and abductors . Arch Phys Med Rehabil . 2000 ; 81 : 1072 - 6 .
98. Drouin JM , Valovich-mcLeod TC , Shultz SJ , Gansneder BM , Perrin DH . Reliability and validity of the Biodex system 3 pro isokinetic dynamometer velocity, torque and position measurements . Eur J Appl Physiol . 2004 ; 91 : 22 - 9 .
99. Rothstein JM , Lamb RL , Mayhew TP. Clinical uses of isokinetic measurements . Critical issues Phys Ther . 1987 ; 67 : 1840 - 4 .
100. Han J , Waddington G , Adams R , Anson J , Liu Y . Assessing proprioception: A critical review of methods . J Sport Heal Sci . 2015 .
101. Suetterlin KJ , Sayer AA . Proprioception: where are we now? A commentary on clinical assessment, changes across the life course, functional implications and future interventions . Age Ageing . 2013 ; 43 : 1 - 6 .
102. Goble DJ . Proprioceptive acuity assessment via joint position matching: from basic science to general practice . Phys Ther . 2010 ; 90 : 1176 - 84 .
103. Proske U , Gandevia SC . The proprioceptive senses: their roles in signaling body shape, body position and movement, and muscle force . Physiol Rev . 2012 ; 92 : 1651 - 97 .
104. Cammarata ML , Dhaher YY . Proprioceptive acuity in the frontal and sagittal planes of the knee: a preliminary study . Eur J Appl Physiol . 2011 ; 111 : 1313 - 20 .
105. Allet L , Kim H , Ashton-Miller J , De Mott T , Richardson JK . Frontal plane hip and ankle sensorimotor function, not age, predicts unipedal stance time . Muscle Nerve . 2012 ; 45 : 578 - 85 .
106. Domingo A , Lam T. Reliability and validity of using the Lokomat to assess lower limb joint position sense in people with incomplete spinal cord injury . J Neuroeng Rehabil . 2014 ; 11 : 167 .
107. Elangovan N , Herrmann A , Konczak J . Assessing proprioceptive function: evaluating joint position matching methods against psychophysical thresholds . Phys Ther . 2014 ; 94 : 553 - 61 .
108. Clark NC , Röijezon U , Treleaven J . Proprioception in musculoskeletal rehabilitation. Part 2: clinical assessment and intervention . Man Ther . 2015 ; 20 : 378 - 87 .
109. Sanford J , Moreland J , Swanson LR , Stratford PW , Gowland C . Reliability of the Fugl-Meyer assessment for testing motor performance in patients following stroke . Phys Ther . 1993 ; 73 : 447 - 54 .
110. Hillier S , Immink M , Thewlis D : Assessing Proprioception: A Systematic Review of Possibilities . Neurorehabil Neural Repair 2015 ; 29 : 933 - 49 .
111. Refshauge KM , Taylor JL , Mccloskey DI , Gianoutsos M , Mathews P , Fitzpatrick RC . Movement detection at the human big toe . J Physiol . 1998 ; 513 : 307 - 14 .
112. Refshauge KM , Chan R , Taylor JL , Mccloskey DI . Detection of movements imposed on human hip, knee, ankle and toe joints . J Physiol . 1995 ; 488 : 231 - 41 .
113. Hurkmans EJ , van der Esch M , Ostelo RWJG , Knol D , Dekker J , Steultjens MPM . Reproducibility of the measurement of knee joint proprioception in patients with osteoarthritis of the knee . Arthritis Rheum . 2007 ; 57 : 1398 - 403 .
114. Chisholm AE , Domingo A , Jeyasurya J , Lam T. Quantification of lower extremity kinesthesia deficits using a robotic exoskeleton in people with a spinal cord injury . Neurorehabil Neural Repair . 2015 ; 30 : 199 - 208 .
115. Brunnstrom S. Movement therapy in hemiplegia-a neurophysiological approach . New York: Harper & Row Publishers, Inc.; 1970 .
116. Fugl-Meyer AR , Jääskö L , Leyman I , Olsson S , Steglind S. The post-stroke hemiplegic patient. 1. a method for evaluation of physical performance . Scand J Rehabil Med . 1975 ; 7 : 13 - 31 .
117. Duncan PW , Propst M , Nelson SG . Reliability of the fugl-meyer assessment of sensorimotor recovery following cerebrovascular accident . Phys Ther . 1983 ; 63 : 1606 - 10 .
118. Hsueh I-P , Hsu M-J , Sheu C-F , Lee S , Hsieh C-L , Lin J-H . Psychometric comparisons of 2 versions of the fugl-meyer motor scale and 2 versions of the stroke rehabilitation assessment of movement . Neurorehabil Neural Repair . 2008 ; 22 : 737 - 44 .
119. Dettmann MA , Linder MT , Sepic SB . Relationships among walking performance, postural stability, and functional assessments of the hemiplegic patient . Am J Phys Med . 1987 ; 66 : 77 - 90 .
120. Tan AQ , Dhaher YY . Evaluation of lower limb cross planar kinetic connectivity signatures post-stroke . J Biomech . 2014 ; 47 : 1 - 8 .
121. Cruz TH , Dhaher Y. Evidence of abnormal lower-limb torque coupling after stroke: an isometric study . Stroke . 2008 ; 39 : 139 - 47 .
122. Krishnan C , Dhaher Y. Corticospinal responses of quadriceps are abnormally coupled with hip adductors in chronic stroke survivors . Exp Neurol . 2012 ; 233 : 400 - 7 .
123. Thelen DD , Riewald S , Asakawa DS , Sanger TD , Delp SL . Abnormal coupling of knee and hip moments during maximal exertions in persons with cerebral palsy . Muscle Nerve . 2003 ; 27 : 486 - 93 .
124. Hidler JM , Carroll M , Federovich EH . Strength and coordination in the paretic Leg of individuals following acute stroke . J Biomech . 2007 ; 15 : 526 - 34 .
125. Sanchez N , Dewald JPA . Constraints imposed by the lower extremity extensor synergy in chronic hemiparetic stroke: Preliminary findings . Conf Proc Annu Int Conf IEEE Eng Med Biol Soc IEEE Eng Med Biol Soc Conf . 2014 ; 2014 : 5804 - 7 .
126. Lunardini F , Casellato C , D'Avella A , Sanger T , Pedrocchi A . Robustness and reliability of synergy-based myocontrol of a multiple degree of freedom robotic Arm . IEEE Trans Neural Syst Rehabil Eng . 2015 .
127. Sanchez N , Acosta A , Stienen A , Dewald J : A Multiple Degree of Freedom Lower Extremity Isometric Device to Simultaneously Quantify Hip, Knee and Ankle Torques . IEEE Trans Neural Syst Rehabil Eng 2015 ; 23 : 765 - 75 .
128. Cruz TH , Lewek MD , Dhaher YY . Biomechanical impairments and gait adaptations post-stroke: multi-factorial associations . J Biomech . 2009 ; 42 : 1673 - 7 .
129. Dewald JP , Beer R . Abnormal joint torque patterns in the paretic upper limb of subjects with hemiparesis . Muscle Nerve . 2001 ; 24 ( 2 ): 273 - 83 .
130. Ellis MD , Sukal T , DeMott T , Dewald JPA . Augmenting clinical evaluation of hemiparetic arm movement with a laboratory-based quantitative measurement of kinematics as a function of limb loading . Neurorehabil Neural Repair . 2008 ; 22 : 321 - 9 .
131. Latash M , Zatsiorsky VM . Biomechanics and Motor Control: Defining Central Concepts . Academic Press; 2015 .
132. Hogan N. Mechanical impedance of single- and multi-articular systems . In: Winters JM , Woo S -Y, editors. Multiple Muscle Systems . New York: Springer; 1990 . p. 149 - 64 .
133. Latash ML , Zatsiorsky VM . Joint stiffness: Myth or reality? Hum Mov Sci . 1993 ; 12 ( 6 ): 653 - 92 .
134. Ludvig D , Perreault EJ . Estimation of joint impedance using short data segments . Conf Proc Annu Int Conf IEEE Eng Med Biol Soc IEEE Eng Med Biol Soc Conf . 2011 ; 2011 : 4120 - 3 .
135. Kearney RE , Hunter IW . System identification of human joint dynamics . Crit Rev Biomed Eng . 1990 ; 18 : 55 - 87 .
136. Mirbagheri MM , Barbeau H , Kearney RE . Intrinsic and reflex contributions to human ankle stiffness: variation with activation level and position . Exp Brain Res . 2000 ; 135 : 423 .
137. Sinkjaer T , Magnussen I . Passive, intrinsic and reflex-mediated stiffness in the ankle extensors of hemiparetic patients . Brain . 1994 ; 117 : 355 - 63 .
138. Chung SG , Van Rey E , Bai Z , Roth EJ , Zhang L-Q . Biomechanic changes in passive properties of hemiplegic ankles with spastic hypertonia . Arch Phys Med Rehabil . 2004 ; 85 : 1638 - 46 .
139. Mirbagheri MM , Barbeau H , Ladouceur M , Kearney RE . Intrinsic and reflex stiffness in normal and spastic, spinal cord injured subjects . Exp Brain Res . 2001 ; 141 : 446 .
140. Hunter I , Kearney R . Dynamics of human ankle stiffness: variation with mean ankle torque . J Biomech . 1982 ; 15 : 747 - 52 .
141. Gottlieb GL , Agarwal GC . Response to sudden torques about ankle in man: myotatic reflex . J Neurophysiol . 1979 ; 42 ( 1 Pt 1 ): 91 - 106 .
142. Weiss P , Kearney R , Hunter I. Position dependence of ankle joint dynamics-II. Active mechanics . J Biomech . 1986 ; 19 : 737 - 51 .
143. Weiss P , Kearney R , Hunter I. Position dependence of ankle joint dynamics-I. Passive mechanics . J Biomech . 1986 ; 19 : 727 - 35 .
144. Kearney R , Hunter I. Dynamics of human ankle stiffness: variation with displacement amplitude . J Biomech . 1982 ; 15 : 753 - 6 .
145. de Vlugt E , van Eesbeek S , Baines P , Hilte J , Meskers CGM , de Groot JH. Short range stiffness elastic limit depends on joint velocity . J Biomech . 2011 ; 44 : 2106 - 12 .
146. Zhang LQ , Nuber G , Butler J , Bowen M , Rymer WZ . In vivo human knee joint dynamic properties as functions of muscle contraction and joint position . J Biomech . 1998 ; 31 : 71 - 6 .
147. Kearney RE , Hunter IW . Nonlinear identification of stretch reflex dynamics . Ann Biomed Eng. 1988 ; 16 : 79 - 94 .
148. Powers RK , Marder-Meyer J , Rymer WZ . Quantitative relations between hypertonia and stretch reflex threshold in spastic hemiparesis . Ann Neurol . 1988 ; 23 : 115 - 24 .
149. Mugge W , Abbink DA , van der Helm FCT. Reduced power method: how to evoke low-bandwidth behaviour while estimating full-bandwidth dynamics . 2007 . p. 575 - 81 .
150. Bohannon RW , Smith MB . Interrater reliability of a modified ashworth scale of muscle spasticity . Phys Ther . 1987 ; 67 : 206 - 7 .
151. Blackburn M , van Vliet P , Mockett SP . Reliability of measurements obtained with the modified Ashworth scale in the lower extremities of people with stroke . Phys Ther . 2002 ; 82 : 25 - 34 .
152. Wartenberg R . Pendulousness of the legs as a diagnostic test . Neurology . 1951 ; 1 : 18 .
153. Bohannon RW , Harrison S , Kinsella-Shaw J . Reliability and validity of pendulum test measures of spasticity obtained with the Polhemus tracking system from patients with chronic stroke . J Neuroeng Rehabil . 2009 ; 6 : 30 .
154. Fowler EG , Nwigwe AI , Ho TW . Sensitivity of the pendulum test for assessing spasticity in persons with cerebral palsy . Dev Med Child Neurol. 2000 ; 42 : 182 - 9 .
155. Kearney E , Weiss L , Morier R . System identification of human ankle dynamics: Intersubject variability and intrasubject reliability . Clin Biomech . 1990 ; 5 : 205 - 17 .
156. Wilken J , Rao S , Estin M , Saltzman CL , Yack HJ . A new device for assessing ankle dorsiflexion motion: reliability and validity . J Orthop Sports Phys Ther . 2011 ; 41 : 274 - 80 .
157. Lünenburger L , Colombo G , Riener R , Dietz V. Clinical assessments performed during robotic rehabilitation by the gait training robot Lokomat . In: Proceedings of the 2005 IEEE 9th International Conference on Rehabilitation Robotics . 2005 . p. 345 - 8 .
158. Chesworth BM , Vandervoort BM . Reliability of a torque motor system for measurement of passive ankle joint stiffness in control subjects . Physiother Canada . 1988 ; 40 : 300 - 3 .
159. Franzoi AC , Castro C , Cardone C. Isokinetic assessment of spasticity in subjects with traumatic spinal cord injury (ASIA A) . Spinal Cord . 1999 ; 37 : 416 - 20 .
160. McHugh MP , Hogan DE . Effect of knee flexion angle on active joint stiffness . Acta Physiol Scand . 2004 ; 180 : 249 - 54 .
161. Blackburn JT , Padua DA , Riemann BL , Guskiewicz KM . The relationships between active extensibility, and passive and active stiffness of the knee flexors . J Electromyogr Kinesiol . 2004 ; 14 : 683 - 91 .
162. de Vlugt E , de Groot JH , Schenkeveld KE , Arendzen JH , van der Helm FCT , Meskers CGM . The relation between neuromechanical parameters and Ashworth score in stroke patients . J Neuroeng Rehabil . 2010 ; 7 : 35 .
163. Androwis GJ , Michael PA , Strongwater A , Foulds RA . Estimation of intrinsic joint impedance using quasi-static passive and dynamic methods in individuals with and without Cerebral Palsy . In: IEEE Engineering in Medicine and Biology Society. Annual Conference . 2014 . p. 4403 - 6 .
164. Mirbagheri MM , Kearney RE , Barbeau H . Quantitative, objective measurement of ankle dynamic stiffness: Intrasubject reliability and intersubject variability . In: Annual International Conference of the IEEE Engineering in Medicine and Biology - Proceedings . 1996 . p. 585 - 6 .
165. Sloot LH , van der Krogt MM , de Gooijer-van de Groep KL , van Eesbeek S, de Groot J , Buizer AI , Meskers C , Becher JG , de Vlugt E , Harlaar J. The validity and reliability of modelled neural and tissue properties of the ankle muscles in children with cerebral palsy . Gait Posture . 2015 ; 42 : 7 - 15 .
166. Loram I , Lakie M , Maganaris C. Reply from Ian D. Loram , Constantinos N. Maganaris and Martin Lakie . J Physiol . 2005 ; 569 : 706 .
167. Vlutters M , Boonstra TA , Schouten AC , Van der Kooij H. Direct measurement of the intrinsic ankle stiffness during standing . J Biomech . 2015 ; 48 : 1258 - 63 .
168. Koopman B , van Asseldonk EHF , Van Der Kooij H. In vivo measurement of human knee and hip dynamics using MIMO system identification . Buenos Aires: Proceedings of 32nd Annual Internation conference of the IEEE EMBS; 2011 . p. 3426 - 9 .
169. Pfeifer S , Vallery H , Hardegger M , Riener R , Perreault EJ . Model-based estimation of knee stiffness . IEEE Trans Biomed Eng . 2012 ; 59 : 2604 - 12 .
170. Ludvig D , Pfeifer S , Hu X , Perreault EJ . Time-varying system identification for understanding the control of human knee impedance . In: IFAC Proceedings Volumes (IFAC-PapersOnline) . 2012 . p. 1306 - 10 .
171. Ludvig D , Starret Visser T , Giesbrecht H , Kearney R . Identification of timevarying intrinsic and reflex joint stiffness . IEEE Trans Biomed Eng . 2011 ; 58 : 1 .
172. Ludvig D , Perreault EJ . System identification of physiological systems using short data segments . IEEE Trans Biomed Eng . 2012 ; 59 : 3541 - 9 .
173. Tucker MR , Moser A , Lambercy O , Sulzer J , Gassert R . Design of a wearable perturbator for human knee impedance estimation during gait . IEEE Int Conf Rehabil Robot 2013 . 2013 ; 2013 : 6650372 .
174. Perry J. Gait Analysis . Normal and Pathological Function . Thorofare: SLACK Incorporated; 1992 .
175. Keane-Miller , O 'Toole MT . Miller-Keane Encyclopedia & Dictionary of Medicine, Nursing & Allied Health. 7th ed . 2005 .
176. Winter DA . Biomechanics and Motor Control of Human Gait: Normal, Elderly and Pathological. 2nd edition . Waterloo Biomechanics; 1991 .
177. Baker R : Measuring Walking: A Handbook of Clinical Gait Analysis . 1st edition . London: Mac Keith Press; 2013 .
178. Awai L , Curt A . Intralimb coordination as a sensitive indicator of motorcontrol impairment after spinal cord injury . Front Hum Neurosci . 2014 ; 8 (March): 148 .
179. Hedel HJA V, Wirz M , Curt A . Improving walking assessment in subjects with an incomplete spinal cord injury : responsiveness . 2006 . p. 352 - 6 .
180. van Hedel HJ , Wirz M , Dietz V . Standardized assessment of walking capacity after spinal cord injury: the European network approach . Neurol Res . 2008 ; 30 : 61 - 73 .
181. Van Hedel HJ , Wirz M , Dietz V . Assessing Walking Ability in Subjects With Spinal Cord Injury : Validity and Reliability of 3 Walking Tests . Arch Phys Med Rehabil . 2005 ; 86 : 190 - 6 .
182. Jackson AB , Carnel CT , Ditunno JF , Read MS , Boninger ML , Schmeler MR , Apt OTRL , Williams SR , Donovan WH . Outcome measures for gait and ambulation in the spinal cord injury population . J Spinal Cord Med . 2008 ; 31 : 487 - 99 .
183. Schmid A , Duncan PW , Studenski S , Lai SM , Richards L , Perera S , Wu SS . Improvements in speed-based gait classifications are meaningful . Stroke . 2007 ; 38 : 2096 - 100 .
184. Awai L , Bolliger M , Ferguson AR , Courtine G , Curt A . Influence of spinal cord integrity on gait control in human spinal cord injury . Neurorehabil Neural Repair . 2015 .
185. Jasiewicz JM , Allum JHJ , Middleton JW , Barriskill A , Condie P , Purcell B , Li RCT . Gait event detection using linear accelerometers or angular velocity transducers in able-bodied and spinal-cord injured individuals . Gait Posture . 2006 ; 24 : 502 - 9 .
186. Rueterbories J , Spaich EG , Larsen B , Andersen OK . Methods for gait event detection and analysis in ambulatory systems . Med Eng Phys . 2010 ; 32 : 545 - 52 .
187. Aminian K , Najafi B , Büla C , Leyvraz P-F , Robert P . Spatio-temporal parameters of gait measured by an ambulatory system using miniature gyroscopes . J Biomech . 2002 ; 35 : 689 - 99 .
188. Bilney B , Morris M , Webster K. Concurrent related validity of the GAITRite walkway system for quantification of the spatial and temporal parameters of gait . Gait Posture . 2003 ; 17 : 68 - 74 .
189. Micó-Amigo ME , Kingma I , Ainsworth E , Walgaard S , Niessen M , van Lummel RC , van Dieën JH. A novel accelerometry-based algorithm for the detection of step durations over short episodes of gait in healthy elderly . J Neuroeng Rehabil . 2016 ; 13 : 38 .
190. Boutaayamou M , Schwartz C , Stamatakis J , Denoël V , Maquet D , Forthomme B , Croisier JL , Macq B , Verly JG , Garraux G , Brüls O . Development and validation of an accelerometer-based method for quantifying gait events . Med Eng Phys . 2015 ; 37 : 226 - 32 .
191. Mariani B , Rouhani H , Crevoisier X , Aminian K. Quantitative estimation of foot-flat and stance phase of gait using foot-worn inertial sensors . Gait Posture . 2013 ; 37 : 229 - 34 .
192. Hausdorff JM . Gait variability: methods, modeling and meaning . J Neuroeng Rehabil . 2005 ; 9 : 1 - 9 .
193. Allen JL , Kautz S , Neptune RR . Step length asymmetry is representative of compensatory mechanisms used in post-stroke hemiparetic walking . Gait Posture . 2011 ; 33 : 538 - 43 .
194. Chen G , Patten C , Kothari DH , Zajac FE . Gait differences between individuals with post-stroke hemiparesis and non-disabled controls at matched speeds . Gait Posture . 2005 ; 22 : 51 - 6 .
195. Olney SJ , Richards C . Hemiparetic gait following stroke . Part I : Characteristics. Gait Posture . 1996 ; 4 : 136 - 48 .
196. McRoberts . [http://www.mcroberts. nl/]. Accessed 26 July 2016 .
197. GaitUp. [http://www.gaitup.com/]. Accessed 26 July 2016 .
198. Awai L , Curt A . Intralimb coordination as a sensitive indicator of motor-control impairment after spinal cord injury . Front Hum Neurosci . 2014 ; 8 (March): 1 - 8 .
199. Hidler J , Wisman W , Neckel N. Kinematic trajectories while walking within the Lokomat robotic gait-orthosis . Clin Biomech (Bristol, Avon) . 2008 ; 23 : 1251 - 9 .
200. Koopman B , van Asseldonk EHF , van der Kooij H. Selective control of gait subtasks in robotic gait training: foot clearance support in stroke survivors with a powered exoskeleton . J Neuroeng Rehabil . 2013 ; 10 : 3 .
201. Emken JL , Harkema SJ , Beres-jones JA , Ferreira CK , Reinkensmeyer DJ . Feasibility of manual teach-and-replay and continuous impedance shaping for robotic locomotor training following spinal cord injury . IEEE Trans Biomed Eng . 2008 ; 55 : 322 - 34 .
202. Duschau-Wicke A , von Zitzewitz J , Caprez A , Luenenburger L , Riener R . Path control: a method for patient-cooperative robot-aided gait rehabilitation . IEEE Trans neural Syst Rehabil Eng . 2010 ; 18 : 38 - 48 .
203. Krishnan C , Ranganathan R , Dhaher YY , Rymer WZ . A pilot study on the feasibility of robot-aided Leg motor training to facilitate active participation . PLoS One . 2013 ; 8 : e77370 .
204. Park YL , Chen BR , Young D , Stirling L , Wood RJ , Goldfield E , Nagpal R . Bioinspired active soft orthotic device for ankle foot pathologies . IEEE Int Conf Intell Robot Syst . 2011 ; 4488 - 95 .
205. Bartenbach V , Schmidt K , Naef M , Wyss D , Riener R . Concept of a Soft Exosuit for the Support of Leg Function in Rehabilitation . Singapore: IEEE International Conference on Rehabilitation Robotics; 2015 .
206. Maggioni S , Lunenburger L , Riener R , Melendez-Calderon A . Robot-aided assessment of walking function based on an adaptive algorithm . In: Rehabilitation Robotics (ICORR) , 2015 IEEE International Conference on. 2015 . p. 804 - 9 .
207. Duschau-Wicke A , Felsenstein S , Riener R . Adaptive body weight support controls human activity during robot-aided gait training . Kyoto: 2009 IEEE 11th International Conference on Rehabilitation Robotics; 2009 . p. 413 - 8 .
208. Wolbrecht ET , Chan V , Reinkensmeyer DJ , Bobrow JE . Optimizing compliant, model-based robotic assistance to promote neurorehabilitation . IEEE Trans Neural Syst Rehabil Eng . 2008 ; 16 : 286 - 97 .
209. Neckel ND , Blonien N , Nichols D , Hidler J . Abnormal joint torque patterns exhibited by chronic stroke subjects while walking with a prescribed physiological gait pattern . J Neuroeng Rehabil . 2008 ; 13 : 1 - 13 .
210. Niu X , Varoqui D , Kindig M , Mirbagheri MM . Prediction of gait recovery in spinal cord injured individuals trained with robotic gait orthosis . J Neuroeng Rehabil . 2014 ; 11 : 1 - 9 .
211. ICF Browser. [http://apps.who.int/classifications/icfbrowser/].
212. Hallgren KA . Computing inter-rater reliability for observational data: an overview and tutorial . Tutor Quant Methods Psychol . 2012 ; 8 : 23 - 34 .
213. Shrout PE , Fleiss JL . Intraclass correlations : uses in assessing rater reliability . Psychon Bull . 1979 ; 86 : 420 - 8 .
214. Shrout PE . Measurement reliability and agreement in psychiatry . Stat Methods Med Res . 1998 ; 7 : 301 - 17 .
215. McHugh ML . Interrater reliability: the kappa statistic . Biochemia Medica . 2012 ; 276 - 282 .
216. Sivan M , Connor RJO , Makower S , Hons BA , Phys D , Levesley M , Bhakta B . Systematic review of outcome measures used in the evaluation of robotassisted upper limb exercise in stroke . J Rehabil Med . 2011 ; 43 : 181 - 9 .
217. Husted JA , Cook RJ , Farewell VT , Gladman DD . Methods for assessing responsiveness : a critical review and recommendations . J. Clin. Epidemiol . 2000 ; 53 : 459 - 68 .
218. Wright A , Hannon J , Hegedus EJ , Kavchak AE . Clinimetrics corner : a closer look at the minimal clinically important difference (MCID) . J Man Manip Ther . 2012 ; 20 : 160 - 6 .
219. Gogia PP , Braatz JH , Rose SJ , Norton BJ . Reliability and validity of goniometric measurements at the knee . Phys Ther . 1987 ; 67 : 192 - 5 .
220. Currier LL , Froehlich PJ , Carow SD , McAndrew RK , Cliborne AV , Boyles RE , Mansfield LT , Wainner RS . Development of a clinical prediction rule to identify patients with knee pain and clinical evidence of knee osteoarthritis Who demonstrate a favorable short-term response to Hip mobilization . Phys Ther . 2007 ; 87 : 1106 - 19 .
221. Poulsen E , Christensen HW , Penny JØ , Overgaard S , Vach W , Hartvigsen J . Reproducibility of range of motion and muscle strength measurements in patients with hip osteoarthritis - an inter-rater study . BMC Musculoskelet Disord . 2012 ; 13 : 242 .
222. Elveru RA , Rothstein JM , Lamb RL . Goniometric reliability in a clinical setting . Phys Ther . 1988 ; 68 : 672 .
223. Watkins MA , Riddle DL , Lamb RL , Personius WJ . Reliability of goniometric measurements and visual estimates of ankle joint active range of motion obtained in a clinical setting . Phys Ther . 1991 ; 71 : 90 - 6 .
224. Wakefield CB , Halls A , Difilippo N , Cottrell GT . Reliability of goniometric and trigonometric techniques for measuring Hip-extension range of motion using the modified thomas test . J Athl Train . 2015 ; 50 : 150219105224004 .
225. Hayes KW , Petersen CM . Reliability of assessing end-feel and pain and resistance sequence in subjects with painful shoulders and knees . J Orthop Sports Phys Ther . 2001 ; 31 : 432 - 45 .
226. Fan E , Ciesla ND , Truong AD , Bhoopathi V , Zeger SL , Needham DM . Interrater reliability of manual muscle strength testing in ICU survivors and simulated patients . Intensive Care Med . 2010 ; 36 : 1038 - 43 .
227. Escolar DM , Henricson EK , Mayhew J , Florence J , Leshner R , Patel KM , Clemens PR. Clinical evaluator reliability for quantitative and manual muscle testing measures of strength in children . Muscle Nerve . 2001 ; 24 : 787 - 93 .
228. Jain M , Smith M , Cintas H , Koziol D , Wesley R , Harris-Love M , Lovell D , Rider LG , Hicks J : Intra-Rater and Inter-Rater Reliability of the 10- Point Manual Muscle Test (MMT) of Strength in Children with Juvenile Idiopathic Inflammatory Myopathies (JIIM) . Phys Occup Ther Pediatr 2006 ; 26 : 1541 - 3144 .
229. Schache MB , McClelland JA , Webster KE . Reliability of measuring hip abductor strength following total knee arthroplasty using a hand-held dynamometer . Disabil Rehabil . 2015 ; 38 : 597 .
230. www.rehabmeasures.org. [http://www.rehabmeasures.org]. Accessed 26 July 2016 .
231. Lam T , Noonan VK , Eng JJ . SCIRE research team: a systematic review of functional ambulation outcome measures in spinal cord injury . Spinal Cord . 2008 ; 46 : 246 - 54 .
232. Zhang M , Davies TC , Nandakumar A , Quan S. A novel assessment technique for measuring ankle orientation and stiffness . J Biomech . 2015 ; 48 : 3527 - 9 .
233. Chesworth BM , Vandervoort AA . Age and passive ankle stiffness in healthy women . Phys Ther . 1989 ; 69 : 217 - 24 .