Position/Force Hybrid Control System for High Precision Aligning of Small Gripper to Ring Object

International Journal of Automation and Computing, Aug 2013

A position/force hybrid control system based on impedance control scheme is designed to align a small gripper to a special ring object. The vision information provided by microscope vision system is used as the feedback to indicate the position relationship between the gripper and the ring object. Multiple image features of the gripper and the ring object are extracted to estimate the relative positions between them. The end-effector of the gripper is tracked using the extracted features to keep the gripper moving in the field of view. The force information from the force sensor serves as the feedback to ensure that the contact force between the gripper and the ring object is limited in a small safe range. Experimental results verify the effectiveness of the proposed control strategy.

A PDF file should load here. If you do not see its contents the file may be temporarily unavailable at the journal website or you do not have a PDF plug-in installed and enabled in your browser.

Alternatively, you can download the file locally and open with any standalone PDF reader:

https://link.springer.com/content/pdf/10.1007%2Fs11633-013-0732-y.pdf

Position/Force Hybrid Control System for High Precision Aligning of Small Gripper to Ring Object

Juan Zhang 0 Zheng-Tao Zhang 0 0 Research Center of Precision Sensing and Control, Institute of Automation, Chinese Academy of Sciences , Beijing 100190, China A position/force hybrid control system based on impedance control scheme is designed to align a small gripper to a special ring object. The vision information provided by microscope vision system is used as the feedback to indicate the position relationship between the gripper and the ring object. Multiple image features of the gripper and the ring object are extracted to estimate the relative positions between them. The end-efiector of the gripper is tracked using the extracted features to keep the gripper moving in the fleld of view. The force information from the force sensor serves as the feedback to ensure that the contact force between the gripper and the ring object is limited in a small safe range. Experimental results verify the efiectiveness of the proposed control strategy. Introduction Micro-assembly technology is used widely in microelectro-mechanical systems (MEMS) and micro-optoelectro-mechanical systems (MOEMS) industry. It refers to the process of handling and assembling micro-sized components into micro-devices. These components may have small size of several m to several mm. To fabricate them, micro-assembly techniques, such as micro-gripper design, control system design, and modeling of adhesive force, are all face challenges[15]. Heterogeneous micro-parts produced from various fabrication processes are used for producing complex 3D microstructures in some micro-assembly tasks[69]. The microassembly workstation which comprises a micromanipulator, high precision positioning stages, a set of visual systems and micro-force sensors is commonly used[1013]. Microassembly has high requirements in accuracy and exibility. As the microscope vision has the feature of small view fleld, it is not easy to obtain micro-parts precise position. Furthermore, visual instability exists in the micro-assembly process, and the micro-positioning accuracy is hampered by the volume of sensors[14]. Traditional movement control strategy based on position is not sucient for the microassembly tasks. In addition, as the frangibility of the microparts, it is easy to be damaged in the assembly process. Position-based method does not take interaction force into account which has a great inuence on the measurement at the micro-scale. Proper force control can decrease the high requirement of the position detection accuracy and increase the security in the micro-assembly process. In the literatures, many studies have been done on using force control for improving the micro-assembly results. Despite the diversity of the approaches, force control is mainly used in two ways. First, the force is introduced to guide the assembly task. Rabenorosoa et al.[15; 16] introduced force to guide the micro-part to insert into the rail. The micro-part is gripped by two sensorized flngers. The two sensorized flnManuscript received July 13, 2012; revised March 1, 2013 This work was supported by National Natural Science Foundation of China (No. 61105036 and 61227804). gers can detect the contact side of the micro-gripper and the rail, and can estimate the contact force between them. The contact force control can correct the trajectory of the micropart when the contact appears. However, the manipulation eciency is always low. And the complex micro-gripper with force feedback is designed to ensure the security of the micro-parts[1719]. The manipulation force is controlled in real-time for gripping the micro-parts. Choi et al.[20] designed a variable stifiness joint (VSJ) for a robot manipulator, as well as a control scheme to control the stifiness and the position of the VSJ. However, the design of the gripper is always complex. To align a small gripper and a special ring object conveniently and safely, without complex gripper design, an effective control strategy based on microscope vision and contact force is proposed in the paper. The microscope vision system is weakly calibrated. The distance of the gripper and ring object is calculated in real-time based on feature extraction from the image captured by microscope vision system. The end-efiector of the small gripper is tracked in real-time to keep the small gripper moving in the small fleld of view. The force of the ring object is detected in real-time. Once the gripper moves into the near region, the force of the ring object detected by the force sensor is introduced to control the small gripper movement. The force of the ring object is in a safe range in the alignment process. The rest of this paper is organized as follows. In Section 2, the microscope vision system is introduced, the distance calculation method and the feature extraction algorithm are presented. The force sensor and the movement control strategy are described in Section 3. In Section 4, experiments are provided to verify the efiectiveness of the control strategy. The paper is concluded in Section 5. Micro-assembly system and distance computation The task of the vision measurement system is to extract features of the image of the small gripper and the ring object, and to compute the distance between them. System conflguration The visual system is shown in Fig. 1. The System consists of a charge-coupled device (CCD) camera, a microscopic lens and a light source. The image captured by the camera is sent to the host computer via local area network (LAN) and processed. The force sensor is connected to the ring object via a connector, and its signals are sent to the computer in real-time. The small gripper is flxed to 3 degree-of-freedoms (DOFs) motor stage. The motor is controlled by the host computer through serial port. Fig. 1 The sketch of visual system The object is a hollow thin cylinder ring with four circular holes spread evenly. Its shape is given in Fig. 2 (a). It is a hollow cylinder with an outer diameter of 5.36 mm, 0.156 mm thickness and 3 mm height. There are four circular holes with diameter of 2.1 mm arranged on the cylinder surface evenly. The small gripper is shown in Fig. 2 (b). The end-efiector is part of a cylinder surface, which is used to absorb the ring object via negative pressure. Three selected feature points are denoted as O, A and B. The feature line is denoted as L, as shown in Fig. 3 (a). The point O is the center point of the ellipse C1. The line L is the center line of the small gripper. The point A is the right intersection of the line L and the ellipse C2. The point B is the right intersection of the line L0 and the ellipse C1. The line L0 is parallel with the line L, and the point O is on it. Ellipse C1 is the fltting ellipse of the outer edge of the ring object. Ellipse C2 is the fltting ellipse of the arc edge of the end-efiector of the small gripper. A frame is established at the point O, whose X-axis is from the point O to the point B and Y -axis is vertical to the line L0, as shown in Fig. 3. The image distance of the gripper and the ring object in the Y -axis direction is calculated as where k and b are the parameters of the line L, (Ox, Oy) is the image coordinate of the point O, Dyi is the image distance of the point O to the line L, i.e., the image distance of the small gripper to the ring object in the Y -axis direction. The image distance of the small gripper and the ring object in the X-axis direction is calculated as Dxi = (Ax Bx)2 + (Ay By)2 Dy2i where (Ax, Ay) and (Bx, By) are the image coordinates of points A and B, Dxi is the image distance of the small gripper to the ring object in the X-axis direction. The image distances Dxi and Dyi need to be converted to the Cartesian space distances to control the motor movement. The calculation formula is shown in (3). (Dx = fiDxi Dy = fiDyi Distance computation The position relationship between the small gripper and the ring object in the image is shown in Fig. 3 (a). The pose of the ring object relative to the microscope vision system is adjusted well in advance[21]. The ring object in the image is a thin circle. The end-efiector of the small gripper in the image is part of a circle. The position relationship after alignment is shown in Fig. 3 (b). where Dx is the Cartesian space distance in the X-axis direction, Dy is the Cartesian space distance in the Y -axis direction, fi is the Cartesian space length of one image pixel, and fi is calibrated ofi-line[21]. Feature extraction The background image with the ring object is recorded before the end-efiector moves into the fleld of view (FOV). Then, the images are captured and processed with the endefiector moving into the FOV. The process of the feature extraction and the distance calculation is divided into four steps, as shown in Fig. 4. Firstly, the image area of interest (ROI) Ra is set manually by the position of the ring object in the image, the ring object outer arc edge C1 is extracted from the image ROI Ra. The point O is computed, as shown in Fig 5 (a). Secondly, the image ROI Rb is calculated by background difierence of the current image and the background image. The small gripper upper edgeline L1 and lower edge line L2 are extracted from Rb. The lines and ellipses are extracted by improved random sample consensus (RANSAC) algorithm[21]. The central line L is determined using the lines L1 and L2, as shown in Fig. 5 (b). Thirdly, L0 is determined from line L and the point O. The point A is computed from the line L and arc edge C2, the point B is calculated from line L0 and arc edge C1. The distances Dx and Dy are calculated, as shown in Fig. 5 (c). Fourth, the area of the end-efiector of the small gripper is computed, which is denoted as a rectangle PQMN. The image ROI Rb is updated by the position of PQMN, as shown in Fig. 5 (d). The last three steps are repeated in the small gripper movement control process. Fig. 4 The ow chart of feature extraction algorithm Fig. 5 The sketch of the feature extraction algorithm 3 Impedance control system As error exists in the posture adjustment of the ring object, the ring object is not parallel with the small gripper in the horizontal direction sometimes, as shown in Fig. 6. Fig. 6 The sketch of the position relationship of the small gripper and ring object in the horizontal direction In the situation as shown in Fig. 6 (a), the calculated distance can indicate the actual distance between the gripper and the ring object well. In the situation in Fig. 6 (b), the calculated distance cannot correctly reect the actual distance between them. The gripper will collide to the ring object in the alignment procedure with the guidance of the microscope vision system. It will damage the gripper and the ring object. To avoid the damage due to collision, the contact force is introduced into the control system as the feedback to control the motor movement. The vision-based position/force hybrid control system is designed by using the impedance control scheme, as shown in Fig. 7. Fig. 7 Block diagram of control system In Fig. 7, Ex and Ey are the expected positions of the small gripper in the X-axis and Y -axis direction. kf is the conversion coecient from force to position. Fx and Fy are the forces acting on the ring object. ex and ey are the calculated distance errors, and they are calculated in (4). ex = Ex Dx kf Fx ey = Ey Dy kf Fy: The choice of the coecient kf is important in the control system. If kf is set to a small value, the small gripper will have a strong stifiness. If kf is set to a big value, it will have an obvious compliance. The calculation of the coecient kf is given in (5). kf = where et is a threshold distance that the contact will probably happen. If the distance is larger than the threshold, the contact is considered not happened. c is a constant value calculated using (6). c = where F0 is the allowable maximum contact force of the small gripper acting on the ring object in the alignment process. The controller for the gripper0s movement adopts proportional-integral (PI) controller with variable structure, as given in (7). When the relative distance of the gripper and the ring object is larger, the gripper moves a flxed length in a control circle in order to control the small gripper approach to the ring object quickly. When the relative distance is smaller, the gripper is controlled by the PI algorithm to align with high precision. >><KP [e(k) e(k 1)] + KI e(k); eT 2 6 je(k)j 6 eT 1 (7) where k is the control cycle index, u(k) is the control value before the k-th step movement, Ls is the step length in the far distance range, eT 1 is the distance threshold, eT 2 is the allowable given error, e(k) is the calculated distance error before the k-th control cycle, KP is the proportional coecient, and KI is the integral coecient. The movement control in the X-axis and Y -axis are executed simultaneously. Once ex and ey are both in the given range, i.e., jexj < eT 2 and jeyj < eT 2, the movement control is stopped. Experiments and results The experimental platform is shown in Fig. 8. The image resolution of the CCD camera (GC2450) was 2448 2050 in pixel, its maximum frame rate was 15 frames per second (FPS). The magniflcation factor of the microscope (Navitar) was from 1.05 to 6.75. The full scale of the force sensor (Nano43) was 18 N. The working range of the 3-DOFs stage was 100 mm in each axis, and the maximum movement speed was 400 mm/s. error range of ex and ey were set to [10 m, 10 m]. The threshold eT 1 was set to 500 m, and the step length Ls was set to 200 m. Fig. 9 The images in the approaching process Fig. 8 Experiment platform In the experiments, the PI parameters KP x, KIx, KP y and KIy were set to 0.5, 0.15, 0.4 and 0.07 according to Ziegler-Nichols method[22]. The expected values Ex and Ey were set to 50 m and 0 m, respectively. The allowable Fig. 10 Force in the experiment 1 As the statistical analysis of the force is obtained through the force sensor, we have found that the noise exist in the force. As the Kalman fllter can work very well to fllter the noise, the force was flltered by Kalman fllter. The force sensor used in the experiments was Nano43, which is well decoupled in X and Y directions. Therefore, the noises in X and Y directions can be separately dealt with. In experiments, the forces from the force sensor in an interval about 14 s were collected at a stationary state. The noise variance was calculated based on the collected data. The noise variance in the X-axis was 9 104, and in the Y -axis was 5 104. As the allowable force of the ring object, the contact force F0 after flltering was set to 1 N. The threshold et was set to 150 m. Therefore, the constant c was 150 m/N. It is calculated by (6). The sample time of the force was 10 ms. The control time was set to 300 ms to ensure that the stepper motor could flnish the given motion and the vision measurement could be surely flnished. The entire image processing time was about 80 ms. The introduction of the force is very helpful to avoid collision. A series of experiments were done to verify the efiectiveness of the control strategy. Two sets of experimental results were given. Experiment 1 was conducted in the normal situation when the contact did not happen between the small gripper and the ring object. Experiment 2 was conducted in the situation when the contact happened. Noncontact approaching The force in Experiment 1 is shown in Fig 10. Fig 10 (a) is the force in the X-axis direction, and Fig 10 (b) is the force in the Y -axis direction. It can be seen that the force after Kalman flltering varied in a small range. The contact of the small gripper and the ring object did not happen. The distance error in the process is shown in Fig. 11. The distance error in the X-axis was decreased to 6.3 m at last, and in Y -axis was decreased to 4.7 m. They were in the given range. It can be seen that the error decreased to the given range efiectively. Fig. 11 Distance error in the Experiment 1 A series of experiments of the small gripper aligning to the ring object were conducted well with the guidance of the microscope vision and the force information. The control system used was described in Section 3. The images of the alignment process are shown in Fig. 9. Fig. 9 (a) is the image before alignment, Fig. 9 (f) is the image after alignment, and Figs. 9 (b){9 (e) are the images in the alignment process. It can be seen that the end-efiector of the gripper was well tracked, and the end-efiector was kept in the fleld of view. Fig. 12 Force in the Experiment 2 Contact approaching The force in the Experiment 2 is shown in Fig. 12. Fig. 12 (a) is the force in the X-axis direction, Fig. 12 (b) is the force in the Y -axis direction. It can be seen that the force after Kalman flltering had an obvious variation at the time of 5.7 s. It is the time the small gripper contacted the ring object. Accordingly, the contact happened at the 19th step movement. The movement was not stopped until ex and ey were in the given range. Fig. 13 is the distance error in the alignment process. The error in the Y -axis was decreased to 39.8 m at last, and in the X-axis was decreased to 0:4 m. The maximal contact force in the X-axis was less than 0.1 N, and in the Y -axis was less than 0.4 N. They were in the safe force range. The security of the small gripper and the ring object in the alignment process was ensured. the given range. Table 2 lists the results of the contact situation. The maximal contact force and the distance errors at last in the alignment process are listed. It can be seen that the max force in every experiment was less than 1 N, and the last errors were out of the given range because of the force. The security in the alignment process was ensured. As seen above, the control system fulfllls the system requirement efiectively. The experimental results verifled the efiectiveness of the proposed control system. In the noncontact situation, the controller controls the small gripper aligning to the ring object efiectively. In the contact situation, the contact force of the small gripper and the ring object is controlled in a safe range. The security of the small gripper and the ring object in the alignment process are ensured. Experimental results in noncontact situation in X-axis (m) in Y -axis (m) Distance error in the Experiment 2 Ten times of experimental results are given in Table 1. It is in the noncontact situation. The flnal distance error in the X-axis and Y -axis are listed. Both of them were in Under the guidance of microscope vision and force information, postion/force hybrid control system is proposed to Experimental results in contact situation control the small gripper aligning to the ring object. The control system mainly has two contributions. First, the microscope vision is weakly calibrated. Only the scale from the image to Cartesian space is calibrated ofi-line. The relative position of the two objects is calculated in real-time by several image geometry features. Second, to avoid collision of the ring object and the gripper, the force of the ring object is introduced into the control system. The force is converted into the distance error to control the gripper movement. The force of the ring object is in a given safe range in the alignment process. The control system does not have high requirement about the gripper structure for alignment safely. Furthermore, the control strategy is easy to be implemented, and it can be used in the other microassembly processes. The efiectiveness of the proposed control system was verifled in the experiments. In the contact situation, the absolute value of force is less than 1 N. In the noncontact situation, the absolute value of the flnal distance error is less than 10 m. The entire image processing time is less than 80 ms. The gripper is controlled to align to the ring object fast and safely. In the future work, we will focus on the assembly of the ring object with other micro-parts of small size and complex structure in the three-dimensional Cartesian space. The goal is to assemble a complex micro-object for special purpose. The research of complex geometry structure micro-parts assembly is valuable for the development of the EMES and micro-optical-electro-mechanical systems and micro-electro-mechanical systems. De Xu received the B. Sc. and M. Sc. degrees in control science and engineering from Shandong University of Technology, China in 1985 and 1990, respectively, and the Ph.D. degree in control science and engineering from Zhejiang University, China in 2001. Since 2001, he has been with the Institute of Automation, Chinese Academy of Sciences (IACAS), China. He is currently a professor with the Research Center of Precision Sensing and Control, IACAS. His research interests include robotics and automation, control of robots, visual control and intelligent control. E-mail: Zheng-Tao Zhang received the B. Sc. degree from China University of Petroleum, China in 2004, and M. Sc. degree in control science and engineering from Beijing Institute of Technology, China in 2007, He is an assistant researcher in Institute of Automation, Chinese Academy of Sciences, China. His research interests include visual measurement, micro-assembly, and automation. E-mail: assembly. E-mail: (Corresponding author) Juan Zhang received her B. Eng. degree in biomedical engineering, and the M. Eng. degree in detection technology and automatic equipment, from Zhengzhou University, China in 2007 and 2010, respectively. She is currently a Ph. D. candidate in Institute of Automation, Chinese Academy of Sciences (CAS), China. Her research interests include visual measurement and control, and microWen-Sheng Zhang received the Ph. D. degree in pattern recognition and intelligent system from the Institute of Automation, Chinese Academy of Sciences, China in 2000. He joined the Institute of Software, China in 2001. He is currently a professor of machine learning and data mining, Institute of Automation, CAS. His research interests include intelligent information processing, pattern recognition, artiflcial intelligence and computer human interaction. E-mail:


This is a preview of a remote PDF: https://link.springer.com/content/pdf/10.1007%2Fs11633-013-0732-y.pdf

Juan Zhang, De Xu, Zheng-Tao Zhang, Wen-Sheng Zhang. Position/Force Hybrid Control System for High Precision Aligning of Small Gripper to Ring Object, International Journal of Automation and Computing, 2013, 360-367, DOI: 10.1007/s11633-013-0732-y