Vision and force/torque integration for realtime estimation of fast-moving object under intermittent contacts
Bae et al. Robomech J
Vision and force/torque integration for realtime estimation of fast-moving object under intermittent contacts
Hyunki Bae 0
Soo Jeon 0
Jan P. Huissoon 0
0 Department of Mechanical and Mechatronics Engineering, University of Waterloo, 200 University Ave W , Waterloo, ON N2L 3G1 , Canada
This paper considers the fast and accurate estimation of motion variables of a rigid body object whose movement occurs from intermittent contacts with coordinating manipulators in nonprehensile manipulation tasks. The estimator operates under multiple sensory data including visual, joint torque, joint position and/or tactile measurements which are combined at the lower level to compensate for the latency and the slow sampling of the visual data. The estimator is real-time in the sense that it provides the motion data of the target object at the same fast sample rate of the servo controller without delay. The basic formulation is the multi-rate Kalman filter with the contact force vector as its process input, and the visual observation as its measurement output signal which is the down-sampled and delayed version of the configuration of the target object. Experimental tests are conducted for the case of planar object manipulation as well as the non-centroidal rotation under gravity using a robotic hand, and their results are presented to demonstrate the validity of the proposed estimation scheme.
Coordinated manipulation; Kalman filter; Sensors and sensing systems
Background
Coordinating multiple manipulators for handling an
object has potential to substantially enhance dexterity
and versatility, the level of which is not possible by
traditional stand-alone manipulators [1–3]. Although we
expect that this will bring us numerous benefits, there
are a number of technical issues to be resolved before we
demonstrate the practical use of coordinated
manipulation with human-like dexterity. From a hardware
perspective, there has been much progress in recent years
in new designs of complex manipulation systems such
as anthropomorphic robotic hands that are capable of
highly coordinated manipulation tasks with a large
number of degrees of freedom [4–7]. On the other hand,
there is a consensus (see, for example, A Roadmap for US
Robotics [1]) that we are still far behind in some key
technology areas such as perception, robust high fidelity
sensing and planning and control to achieve highly dexterous
object manipulation capabilities. As for the sensing in
particular, it is recognized that processing sensory data
with multiple modalities (e.g. vision, joint angle, tactile
and force) will play a key role in synthesizing
dexterous manipulation skills [1], which is clearly the case for
human dexterity as we have ample evidence from
neuroscience [8, 9].
This paper considers the situation where we want to
keep track of the movement of the object in realtime,
while the object is being manipulated by intermittent
contact forces from the coordinating manipulators. Such
scenarios occur when we attempt to manipulate a rigid
body object through, so called nonprehensile
manipulation [10] which refers to a class of dexterous
manipulation tasks that make an explicit use of the dynamic forces
(e.g. inertial, gravitational, or centrifugal) to allow extra
motion freedoms to the object. Typical examples of
nonprehensile manipulation include pushing, throwing,
caging, flipping, twirling, etc. In conventional approaches to
coordinated (or dexterous) manipulation [11], the object
is being held stably through force-closure condition
during manipulation. The force closure condition ensures
that the object and manipulating bodies are kinematically
related throughout the manipulation process, and thus
we can easily keep track of the movement of the object
using the associated kinematic relations. On the
contrary, in the nonprehensile situations, the movement of
the target object cannot be inferred from the kinematic
relation because the object frequently loses its contact
with some or all of its manipulating bodies, making the
contact force applied in an intermittent way. Therefore,
an additional sensor such as a vision camera is often
necessary to provide the information of object motions
in nonprehensile cases. The visual data, however, may
not be enough to provide all the information in a timely
manner. The vision may be blocked from a certain
direction (occlusion) [12]. Also, the movement data obtained
from visual images are often slow and delayed (latency)
[13, 14], thereby restricting the speed and the accuracy of
achievable object manipulation tasks, especially when the
object is moving relatively fast, as is often the case with
nonprehensile tasks mentioned above.
The main idea of this paper is to design a real-time
rigid body estimator by augmenting the visual data with
other sensory inputs available from manipulating
bodies including the joint force/torque sensor or the tactile
sensor mounted at the surface of the end-effect (...truncated)