Vision-assisted micromanipulation using closed-loop actuation of multiple microrobots

Rahman et al. Robot. Biomim. Vision-assisted micromanipulation using closed-loop actuation of multiple microrobots M. Arifur Rahman 0 Noboru Takahashi 1 Kawai F. Siliga 0 Nigel K. Ng 0 Zhidong Wang 1 Aaron T. Ohta 0 0 Department of Electrical Engineering, University of Hawaii at Manoa , 2540 Dole Street, Holmes Hall 483, Honolulu, HI 96822 , USA 1 Department of Advanced Robotics, Chiba Institute of Technology , 2-17-1 Tsudanuma, Narashino, Chiba 275-0016 , Japan Accurate control and precise positioning of opto-thermocapillary flow-addressed bubble microrobots are necessary for micromanipulation. In addition, micromanipulation using the simultaneous actuation of multiple microrobots requires a robust control system to enable independent motion. This paper demonstrates a hybrid closed-loop visionassisted control system capable of actuating multiple microrobots simultaneously and positioning them at precise locations relative to micro-objects under manipulation. A vision-assisted grasp-planning application was developed and used to calculate the necessary trajectories of the microrobots to form cages around micro-objects. The location of the microrobots and the micro-objects was detected at the caging locations using a particle-tracking application that used image feedback for precise positioning. The closed-loop image feedback information enabled the position update of the microrobots, allowing them to precisely follow the trajectory and caging locations calculated by the grasp-planning application. Four microrobots were assigned to cage a star-shaped micro-object using the closedloop control system. Once caged, the micro-object was transported to a location within the workspace and uncaged, demonstrating the micromanipulation task. This microrobotic system is well suited for the micromanipulation of single cells. Microrobot; Opto-thermocapillary flow; Micromanipulation; Closed-loop control; Grasp planning; Path planning; Caging Background Microrobots in a liquid medium are an efficient tool for bio-micromanipulation. They have broad applications in biological engineering [ 1–4 ], biomedical engineering [ 5, 6 ], and tissue engineering [ 7, 8 ]. Microrobots, which are untethered submillimeter actuators, have been utilized for the transportation of micro-objects including living cells [ 5, 7, 9 ], with micron to submicron resolution [ 5, 7, 9–11 ]. Microrobot-assisted bio-micromanipulation is capable of cell separation and sorting [ 10, 12, 13 ], cell trapping, isolation, and transport [ 12, 14 ], cellladen hydrogel assembly [ 7 ], and cell lysis and molecular delivery [ 15 ]. Microrobots can be actuated using various actuation methods including magnetic actuation [ 8, 9, 16–20 ], bacterial propulsion [ 3, 4 ], biomimetic propulsion [ 21, 22 ], and opto-thermal actuation [ 7, 12, 13, 15 ]. However, the degree of control required and achieved by the various methods of actuation depends upon the actuation force and the number of microrobots being addressed independently. One significant challenge to controlling many microrobots independently is the use of a global actuation force, as employed by magnetic microrobots [ 9, 23 ] and bio-inspired magnetic swimming microrobots [ 6, 24 ]. Despite the challenges, the independent actuation of multiple magnetic microrobots has been demonstrated and was made possible by fabricating the microrobots with different dimensions to obtain different magnetic signatures [ 25, 26 ]. Bacteria-propelled microrobots based on the motility of the bacteria have a limited degree of controllability [27], but the use of electrical signals [ 28 ], UV light [ 4 ], or chemical energy [ 3, 27 ] in conjunction with bacteria propulsion helps to achieve higher controllability, and has been used to actuate multiple microrobots [3]. Bacteria-inspired microrobots actuated by magnetic [ 29 ], acoustic [ 30 ], or a combination of these forces [ 31 ] is capable of parallel control, but complex motion, such as actuation along multiple trajectories, is more difficult [ 32 ]. One type of microrobot that allows the parallel actuation of many microrobots independently is the opto-thermocapillary flow-addressed bubble (OFB) microrobot [ 12 ]. This microrobot consists of a gas bubble in a liquid medium and is actuated by opto-thermal forces [ 33 ]. Parallel actuation of many OFB microrobots allows higherthroughput micromanipulation [ 34 ] and cooperative manipulation [ 35 ]. However, the simultaneous actuation of many microrobots is beyond the capacity of a human operator using a manual control interface, such as a mouse [ 35 ] or touch screen [ 36 ]. An automated, closedloop control system is required to control many microrobots at the same time. In previous work, a small number of OFB microrobots performed microassembly [ 35–37 ], single-cell assembly [ 12 ], cell-laden hydrogel assembly [ 7 ], and singlecell poration [ 15 ]. These applications require microscale accuracy under optical microscopy. As an example, cellladen hydrogel structures need to be positioned in contact for cell culturing. A closed-loop automated control system benefits the above micromanipulation tasks by enabling increased throughput, cooperative microassembly, and the automated planning and execution of assembly tasks. Individual and independent control of many OFB microrobots in parallel requires automated and sophisticated control, including features such as grasp planning for caging, path planning for obstacle avoidance, and moving along the shortest path to a destination. Moreover, when multiple OFB microrobots are simultaneously actuated, precise positioning and actuation, as well as knowledge of the payload location, will enhance the accuracy and resolution of the micromanipulation. In this work, a hybrid closed-loop control system for an OFB microrobot system was developed. The hybrid system uses an open-loop computer-generated holographic control system (developed in LabVIEW) to generate the optical patterns necessary to actuate multiple OFB microrobots simultaneously. The closed-loop part of the hybrid system includes an image-processing algorithm (developed in MATLAB) that provides image feedback control; this allows the actuation of multiple OFB microrobots and the knowledge of the locations of the objects under manipulation. The control system also includes a grasp-planning algorithm (developed in MATLAB) that determines the shortest path and suitable grasping point of micro-objects. Finally, the closed-loop automatic actuation of four microrobots was demonstrated; the OFB microrobots cooperatively caged a star-shaped SU-8 microstructure and transported it to a desired location within the workspace. The hybrid control system achieved higher accuracy compared to open-loop actuation. Methods A 1064-nm Nd:YAG, single-mode (TEM00) linearly polarized laser (Laser Quantum, Ventus 1064, 1.5  + W) with a 2-mm beam diameter was expanded using a 3x    beam expander (Fig.  1a). The expanded beam was incident at approximately 10 degrees on a spatial light modulator (SLM, Hamamatsu, LCOS-SLM X1046807) with an active area of 16  mm by 12  mm. The collimated laser beam was modulated by the SLM, which displayed an 800 pixel by 600-pixel computer-generated hologram (CGH) created using a modified version of the Red Tweezers program [ 38 ]. The modulated wavefront containing the user-defined pattern was 4-f imaged by placing lens L1 at its focal length from the SLM, and lens L2 at its focal length away from the Fourier plane. A 0.42-N.A. 10X objective lens (Mitutoyo) was placed at the focal length of lens L2, focusing the optical pattern on the light-absorbing coating of the substrate. The 1-mmthick glass substrate is coated with 100  nm of titanium and formed a fluidic chamber when bonded to a 1-mmthick glass slide with a spacer layer consisting of 160-μm double-sided polyimide tape. The liquid medium used in the fluidic chamber was silicone oil (Fisher Scientific, S159-500). A 1600 pixel by 1200 pixel camera (Point Grey Flea3) was used for image capture. One pixel in the image corresponds to 1.02 μm on the workspace. The spatial light modulator creates the optical pattern desired by the user (Fig.  1b). Each optical spot that is focused on the light-absorbing layer of the substrate produces a localized hot spot, vaporizing a small volume of the liquid, and thus generating an OFB microrobot [ 28 ]. The thermocapillary flow generated by the temperature gradient and resulting surface tension gradient at the gas–liquid interface of the bubble pulls the OFB microrobot toward the center of the localized hot spot [ 28 ]. Vision‑assisted closed‑loop control system The hybrid closed-loop control system developed in this work has two different parts: the microrobot actuation block developed in LabVIEW, and the image-processing and grasp-planning block developed in MATLAB. The data transfer between the LabVIEW and the MATLAB control blocks was accomplished using the MathScript module of LabVIEW. Figure  2 shows the block diagram of the complete control system. The actuation of microrobots was accomplished by the “microrobot actuation block.” An open-loop computer-generated holographic (CGH) control system was developed in LabVIEW [ 33 ]. An optical spot is represented by a circular spot on the LabVIEW user interface and is maneuvered using manual or automatic control. The user enters the target location of each microrobot, and the MathScript module calculates the navigation parameters, such as initial location, destination, frame rate, and frame size, and passes them to the control application. The control application in LabVIEW then sends the data to the OpenGL Shader hologram engine, which calculates the corresponding hologram using the direct superposition algorithm. The holograms corresponding to the optical pattern are then displayed on the SLM. The feedback block was developed in MATLAB (Fig.  2). The major function of the feedback loop is to process the camera image of the workspace and output data on the OFB microrobots and the micro-objects under manipulation. The captured image was analyzed by the Hough transform algorithm, providing the microrobot’s locations and sizes, and the micro-object’s locations and sizes. The Hough transform detected the shapes in the image, which were then matched to prestored object data. The image data were converted to grayscale and then to a binary image using Otsu thresholding [ 39 ] to make the image suitable for the Hough transform. Detection of unwanted objects, such as the white circular spot at the center of Microrobot 1 in Fig.  4a, was reduced by defining the size ranges of the objects to detect. A grasp-planning algorithm was also developed in MATLAB to plan the course, speed, and other microassembly-related parameters for efficient manipulation. Once the microrobot data and microobject data are available in the MATLAB workspace, the grasp-planning module utilizes user-desired preplanned conditions to calculate the final location of each microrobot for accurate grasping. Once the final destination of each microrobot is calculated, it is saved in a.mat file for subsequent use by the microrobot actuation block. During initiation of the actuation, the destination data from the .mat file are loaded into the MathScript module of LabVIEW. Results The hybrid control system described above was used to perform closed-loop actuation of a single microrobot, grasp planning of multiple microrobots for a micromanipulation task, closed-loop actuation of multiple microrobots, and open-loop micromanipulation of a star-shaped SU-8 microstructure. Closed‑loop actuation of a single microrobot One of the benefits of the closed-loop control of OFB microrobots is the ability to accurately update the position of the microrobot using data from the image-processing algorithm. Here, we have demonstrated actuation of one microrobot along a preset zigzag path using the hybrid control system (Fig. 3). The microrobot was actuated from its initial location to 5 waypoints (waypoints 2–6 in Fig.  3) using automated open-loop control sequences. Each point-to-point actuation (1–2, 2–3, 3–4, 4–5, and 5–6; see Fig. 3a) consisted of a series of 30 smaller actuation segments, at a rate of 2 Hz. The actuation velocity between the waypoints varied. First, the microrobot was actuated from position 1–2 using open-loop control, a distance of 1048  μm (Fig.  3a, b). At location 2 (Fig.  3b), the feedback block captured an image of the workspace, detected the physical bubble location within the workspace, and compared it with the intended destination set in the LabVIEW user interface. The microrobot was then moved to a new location (2′ in Fig.  3c) to correct for the positioning error. Similarly, the microrobot position was determined by the feedback block at each waypoint (3–6 in Fig. 3d, f, h, j), compared with the preset destination in LabVIEW actuation block, and then moved to minimize the difference between the preset destination and the actual position (3′, 4′, 5′, 6′ in Fig. 3e, g, i, k). The position error calculated by the feedback block at 2′, 3′, 4′, 5′, 6′ in Fig. 3 was 5.25, 14.1, 5.1, 16.5, and 2.8  μm, respectively. The position error was then reduced to approximately 1  µm after the microrobot locations were updated using the hybrid closed-loop control system. Grasp planning Path planning refers to determining a collision-free path for a moving object among obstacles [ 18 ]. In this work, a grasp-planning algorithm determines the geometry and location of a microrobot and a micro-object payload. The output of the grasp-planning algorithm is sequences of microrobot locations that form a trajectory from its initial position to its goal position, which is in reference to the payload. There were no obstacles present in the workspace when caging the payload, but the algorithm is capable of determining collision-free path about obstacles. (Additional file  1: Figure S4). The grasp-planning algorithm was used to create a cage of four OFB microrobots around a star-shaped SU-8 microstructure, and transport the object to another location. The star-shaped microstructure consisted of four arms, each 59  μm in length, and a hollow circular center with an inner diameter of 62 μm. The width of the wall around the circular center and the width of the arms were approximately 66  μm. The thickness of the SU-8 was 50  μm, and the structure had an approximate mass of 2.35 μg. Initially, four OFB microrobots were generated at random locations around the micro-object (Fig.  4a) by momentarily increasing the optical power in each spot using the actuation block (Fig.  2). The feedback block then detects the location and size of the microrobots and the structure. The grasp-planning algorithm uses the location of the structure within the workspace to calculate the caging positions (1′, 2′, 3′, and 4′ in Fig. 4a) at user-defined equidistant locations around the microobject. The grasp-planning algorithm allows the adjustment of the caging locations based on visual feedback and the shape of the object. In this experiment, the caging formation was rotated clockwise (Fig. 4a–c) to allow a better grasp of the micro-object. The final caging configuration (Fig.  4d) puts the microrobots in positions that will allow them to grasp in between the arms of the micro-object when the caging formation is contracted. Hybrid closed‑loop actuation of multiple microrobots The caging formation (1′, 2′, 3′, and 4′ in Fig. 4d) calculated by the grasp-planning algorithm of the feedback block sets the destination location for the individual microrobots. These positions were saved in a.mat file and subsequently loaded into the MathScript module of the LabVIEW actuation block. However, the caging locations require a transformation to match the coordinate system of the actuation block. Once the transformed final destinations of the microrobots are serially loaded into the MathScript module from the.mat file, the destination of each microrobot is mapped to the corresponding locations on the LabVIEW user interface. In this experiment, the microrobots numbered 1, 2, 3, and 4 were assigned the caging locations 1′, 2′, 3′, and 4′ (Fig. 4d) as their destinations in the actuation block. Figure 5 shows the open-loop actuation of four microrobots from their initial positions to the caging locations. Microrobots 1, 2, 3, and 4 were simultaneously actuated at velocities of 19, 29.6, 44.1, and 31.83  μm/s, respectively, using the actuation block (Fig.  5a–c). Here, the simultaneous actuation of multiple microrobots with different speeds demonstrates a capability of the microrobot control system: parallel, uncoupled movement of microrobots along trajectories that vary in direction and distance traveled during the same actuation time. The microrobot actuation took 15  s (Additional file 2: Video S1). Figure  5d shows the path of each microrobot from its initial position to the caging location. Here, the simultaneous actuation of multiple microrobots with different speeds demonstrates a capability of the microrobot control system: parallel, uncoupled movement of microrobots along trajectories that vary in direction and distance traveled during the same actuation time. Figure  6 shows the OFB microrobots at the caging positions. The image-processing algorithm determined the locations of the microrobots and compared them to the caging locations calculated by the grasp-planning algorithm. Figure 6a shows the locations of Microrobots 1, 2, 3, and 4 as determined by the image-processing algorithm (white dotted circles), and the desired caging locations set by the grasp-planning algorithm (red circles). The feedback block calculates the error between the actual location of the microrobot and the desired caging position, and calculates the new microrobot destination to minimize the error. Microrobots 1, 2, 3, and 4 in Fig. 6a were 27, 24.7, 14.28, and 21.5 μm away from their desired caging positions, respectively. This information is passed to the MathScript node of the actuation block, which moves the microrobots to the new destinations (Fig. 6b). The microrobots were actuated to their new positions at speeds of 5.4, 4.94, 2.86, and 4.3 μm/s for Microrobots 1, 2, 3, and 4, respectively (Fig. 6b). After updating the location using open-loop actuation, the physical location of the microrobot within the workspace was determined by the image-processing algorithm of the feedback loop, as shown in Fig.  6c. The red circles in Fig.  6c are the caging position set by the graspplanning algorithm at the beginning of the actuation. In Fig.  6c, the Microrobots at 1′, 2′, 3′, and 4′ were 8, 12, 5, and 19  μm away from their desired locations, corresponding to a reduction in the position error of approximately 50%. Micromanipulation The hybrid closed-loop vision-assisted control system allowed an accurate placement of a caging formation, as described above. Figure  7a shows the open-loop actuation of a microrobot matrix approaching a micro-object to grasp it for manipulation. The matrix of microrobots was manually controlled by user input to the actuation block. The object was grasped by contracting the microrobot formation at an average speed of 7.6  μm/s (Fig.  7b and Additional file 3: Video S2). After grasping, the microrobot formation attempted to transport the micro-object in the positive x-direction. However, the micro-object was stuck to the floor of the fluid chamber, resulting in the dislocation of the microrobots from their actuation patterns (Fig.  7c). This phenomenon is more obvious for Microrobots 2 and 3, marked with red arrows in Fig. 7c, as they were moved in the positive x-direction and left the micro-object behind. To free the micro-object from the surface, the formation of the microrobots was rotated to create a torque on the object while maintaining a firm grasp (Fig.  7d). The twisting of the object helped to overcome its stiction, and the micro-object could then be transported. The microobject was transported along various trajectories (Fig. 7e) and at various speeds up to approximately 90  μm/s (Additional file 4: Video S3). A graph of the planned trajectory and completed trajectory during the micro-object manipulation is included in Additional file  1: Figure S5. Upon completion of the micromanipulation, the microobject was released by expanding the microrobot formation (Fig. 7f ). Discussion The hybrid closed-loop vision-based control of OFB microrobots with open-loop actuation has leveraged the functionality of two different platforms to perform the micromanipulation tasks. This integration of MATLAB and LabVIEW utilizes the hardware support and rapid configuration of LabVIEW and the advantages of MATLAB’s processing of complex image data. Aside from providing more precise manipulation, closed-loop position updates allow the use of standard macroscale robotic functions such as grasp planning, collision avoidance, and detection, grasping, and payload detection and delivery. Also, the hybrid control allows the detection of failed actuation of individual microrobots (Additional file 1: Figure. S6). Moreover, the OFB microrobot system, which is capable of the independent actuation of many microrobots, utilizes vision-based automatic actuation for the simultaneous participation of multiple microrobots in micromanipulation, as it is difficult for a human operator to control many microrobots. In this work, the OFB microrobot system enabled a team of microrobots to transport a large object, which is not possible with a single microrobot. This was quantified in previous work, as it was observed one or two microrobots could produce limited rotational movement of an SU-8 microstructure, but no translational movement. Three or four microrobots were necessary to translate the micro-object [ 40 ]. The closed-loop position updates increased the accuracy of the caging locations by 50% compared to one iteration of open-loop actuation. The results suggest that multiple iterations of closed-loop position updating may reduce the error further, but this needs more investigation. The causes of the position error during open-loop actuation can be broadly divided into two categories: system error and mechanical error. The system error is due to the spatial resolution of the SLM and any misalignment of the optical elements. The mechanical error is caused by the misalignment of the image coordinates compared to the LabVIEW user interface coordinates. This misalignment exists due to the mechanical adjustment of the camera position when attempting to match to the LabVIEW coordinates. The software-defined coordinates in LabVIEW were considered the ideal coordinates, and the camera was adjusted by hand such that a single pixel on both the MATLAB image-processing module and the LabVIEW actuation module had the same dimensions. Despite the careful adjustment, an error of approximately 6–8 pixels was present; this was quantified by taking multiple measurements of a stationary micro-object. The closed-loop control system helps with the system error, but is unable to correct for the mechanical error. The average error calculated during open-loop actuation (Fig.  5) was 21.87  μm per microrobot. The closed-loop position update reduced the average error to 11  μm per microrobot. The positioning tolerance of the micromanipulation varies with the size of the objects under manipulation and the type of manipulation. For example, stable caging requires the microrobots to be placed at a distance smaller than the payload size from each other [ 41, 42 ]. The caging of the 300-μm-diameter star-shaped microobject using 130-μm-diameter microrobots required the microrobots to be placed less than 300  μm from each other for stable trapping and at least 130  µm (one body length) apart to avoid merging of the bubble microrobots. Thus, the calculated microrobot separation of 280  μm with  ±  10  μm position tolerance satisfies these conditions. The hybrid closed-loop control system was able to reduce the average position error from 21.9 to 11 μm, satisfying the position tolerance for this micromanipulation task. The image acquisition, object detection, and path planning in MATLAB take approximately 1.6  s to compute while running on a PC with an AMD Phenom II × 6 3.31GHz processor and 16 GB RAM. The object locations and path planning data calculated by the feedback block (Fig. 2) are saved in a.mat file. A MathScript node included in the actuation block in LabVIEW reads the data from the.mat file and sends it to the sequence generator for actuating the microrobots along the planned path. The feedback block runs in MATLAB and the actuation block runs in LabVIEW, so the operator needs to manually enter the.mat file location in the MathScript node and click the run button in the actuation block. This operation takes approximately 5 s, so there is a total delay of approximately 6.5 s between the image acquisition and the microrobot actuation. The image feedback of the proposed hybrid control system can detect more than four particles at once; it can detect as many as distinct objects that can fit in the camera field of view. The control algorithm is also not limited to four particles; it can set destinations and waypoints for a number of objects detected in the workspace. The open-loop control part of the hybrid system has been demonstrated to control an array of 50 OFB microrobots [ 40 ], and this system can control at least the same amount of microrobots at once. In this work, the vision-based closed-loop position update was executed at the waypoints for single microrobot actuation and at the caging locations for multiple microrobot actuations, instead of iteratively after each camera video frame. The closed-loop position update was not implemented after each frame since the position errors per frame were usually less than one micron. Moreover, the processing time for the image analysis of the high-resolution image (1600 px by 1200) at each frame would increase the overhead on the overall computation process and time. Conclusions A hybrid closed-loop vision-assisted control system was developed in MATLAB and LabVIEW to control multiple OFB microrobots automatically. The control system was used to demonstrate open-loop actuation of a single microrobot and simultaneous actuation of multiple microrobots along with closed-loop position updates. A grasp-planning algorithm was also developed in MATLAB and utilized to calculate the precise locations for grasping a micro-object using a team of four microrobots. The position of each microrobot and the micro-object under manipulation was detected by the closed-loop feedback module to minimize the error between the physical location calculated by the imageprocessing algorithm and the intended destination. This closed-loop actuation allows automatic and simultaneous actuation of multiple microrobots for micromanipulation with precise positioning, beyond the capacity of a human operator. However, this hybrid control scheme requires certain human operators to switch between the applications (MATLAB and LabVIEW). In the future, this system can be upgraded to use image processing based on the seamless integration of LabVIEW and MATLAB, allowing minimum user interaction. Additional files Additional file 1. Supplementary information. Additional file 2: Video S1. Automatic actuation of four OFB microrobots into the caging formation. Additional file 3: Video S2. Grasping the micro-object. Additional file 4: Video S3. Micro-object manipulation. Authors’ contributions ATO and MAR conceived and designed the experiments; MAR performed the experiments; MAR, NT, and ATO analyzed the data; NT, MAR, KFS, NKN, and ZW developed and modified the control application; MAR and ATO wrote the paper. All authors read and approved the final manuscript. Acknowledgements ATO acknowledges the Japan Society for the Promotion of Science (JSPS) for an overseas researcher fellowship. Competing interests The authors declare that they have no competing interests Funding This project was supported in part by Grant Number 1R01EB016458 from the National Institute of Biomedical Imaging and Bioengineering of the National Institutes of Health (NIH). These contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH. The funding sponsors had no role in the design of the study, in the collection, analyses, or interpretation of data, in the writing of the manuscript, and in the decision to publish the results. Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. 1. Rubenstein M , Cornejo A , Nagpal R . Programmable self-assembly in a thousand-robot swarm . Science . 2014 ; 345 : 795 - 9 . 2. Wang X , Chen S , Sun D . Robot-aided micromanipulation of biological cells with integrated optical tweezers and microfluidic chip . In: Sun Y , Liu X , editors. Micro- and nanomanipulation tools . 1st ed. Weinheim: Wiley; 2015 . p. 393 - 415 . 3. Kim D , Liu A , Diller E , Sitti M. Chemotactic steering of bacteria propelled microbeads . Biomed Microdev . 2012 ; 14 : 1009 - 17 . 4. Steager EB , Sakar MS , Kim DH , Kumar V , Pappas GJ , Kim MJ . Electrokinetic and optical control of bacterial microrobots . J Micromech Microeng . 2011 ; 21 : 035001 . 5. Sitti M , Ceylan H , Hu W , Giltinan J , Turan M , Yim S , Diller E . Biomedical applications of untethered mobile milli/microrobots . Proc IEEE . 2015 ; 103 : 205 - 24 . 6. Peyer KE , Zhang L , Nelson BJ . Bio-inspired magnetic swimming microrobots for biomedical applications . Nanoscale . 2013 ; 5 : 1259 - 72 . 7. Hu W , Fan Q , Ohta AT . Assembly of cell-laden microgels by an optically controlled bubble manipulator . In: International Conference on Miniaturized Systems for Chemistry and Life Sciences, Freiburg , Germany, October, 2013 . 8. Knowlton S , Li D , Ersoy F , Cho YK , Tasoglu S. Building blocks for bottomup neural tissue engineering: tools for in vitro assembly and interrogation of neural circuits . In: Zhang LG , Kaplan DL , editors. Neural Engineering . 1st ed. Berlin: Springer International Publishing; 2016 . p. 123 - 44 . 9. Steager EB , Selman Sakar M , Magee C , Kennedy M , Cowley A , Kumar V . Automated biomanipulation of single cells using magnetic microrobots . Int J Rob Res . 2013 ; 32 : 346 - 59 . 10. Chen CY , Chen CF , Yi Y , Chen LJ , Wu LF , Song T . Construction of a microrobot system using magnetotactic bacteria for the separation of Staphylococcus aureus . Biomed Microdev . 2014 ; 16 : 761 - 70 . 11. Cappelleri DJ , Fu Z , Fatovic M . Caging for 2D and 3D micromanipulation . J Micro-Nano Mech . 2012 ; 7 : 115 - 29 . 12. Hu W , Fan Q , Ohta AT . An opto-thermocapillary cell micromanipulator . Lab Chip . 2013 ; 13 : 2285 - 91 . 13. Cheng J , Rahman MA , Ohta AT . Optical manipulation of cells . In: Lee W, Tseng P , Di Carlo D, editors. Microtechnology for Cell Manipulation and Sorting . 1st ed. Berlin: Springer International Publishing; 2017 . p. 93 - 128 . 14. Vikram Singh A , Sitti M. Targeted drug delivery and imaging using mobile milli/microrobots: a promising future towards theranostic pharmaceutical design . Curr Pharm Des . 2016 ; 22 : 1418 - 28 . 15. Fan Q , Hu W , Ohta AT . Efficient single-cell poration by microsecond laser pulses . Lab Chip . 2015 ; 15 : 581 - 8 . 16. Salehizadeh M , Diller E . Two-agent formation control of magnetic microrobots in two dimensions . J Micro-Bio Robot . 2017 ; 1 : 1 - 11 . 17. Diller E , Giltinan J , Lum GZ , Ye Z , Sitti M . Six-degree-of-freedom magnetic actuation for wireless microrobotics . Int J Robot Res . 2016 ; 35 : 114 - 28 . 18. Chowdhury S , Johnson BV , Jing W , Cappelleri DJ . Designing local magnetic fields and path planning for independent actuation of multiple mobile microrobots . J Micro-Bio Robot . 2017 ; 12 : 21 - 31 . 19. Pawashe C , Floyd S , Sitti M. Multiple magnetic microrobot control using electrostatic anchoring . Appl Phys Lett . 2009 ; 94 : 164108 . 20. Pelrine R , Wong-Foy A , McCoy B , Holeman D , Mahoney R , Myers G , Herson J , Low T. Diamagnetically levitated robots: an approach to massively parallel robotic systems with unusual motion properties . In: IEEE International Conference on Robot Automation, St. Paul, USA, May 2012 . 21. Cheang UK , Kim MJ . Fabrication and control of simple low Reynolds number microswimmers . Appl Phys Lett . 2016 ; 109 : 034101 . 22. Ding Y , Qiu F , Casadevall i Solvas X , Chiu FWY , Nelson BJ , deMello A. Microfluidic-based droplet and cell manipulations using artificial bacterial flagella . Micromachines . 2016 ; 7 : 25 . 23. Pawashe C , Floyd S , Diller E , Sitti M . Two-dimensional autonomous microparticle manipulation strategies for magnetic microrobots in fluidic environments . IEEE Trans Robot . 2012 ; 28 : 467 - 77 . 24. Martel S , Mohammadi M , Felfoul O , Zhao L , Pouponneau P . Flagellated magnetotactic bacteria as controlled MRI-trackable propulsion and steering systems for medical nanorobots operating in the human microvasculature . Int J Robot Res . 2009 ; 28 : 571 - 82 . 25. Diller E , Giltinan J , Sitti M. Independent control of multiple magnetic microrobots in three dimensions . Int J Robot Res . 2013 ; 32 : 614 - 31 . 26. Frutiger DR , Vollmers K , Kratochvil BE , Nelson BJ . Small, fast, and under control: wireless resonant magnetic micro-agents . Int J Robot Res . 2010 ; 29 : 613 - 36 . 27. Behkam B , Sitti M. Bacterial flagella-based propulsion and on/off motion control of microscale objects . Appl Phys Lett . 2007 ; 90 : 023902 . 28. Kim PSS , Becker A , Ou Y , Julius AA , Kim MJ . Imparting magnetic dipole heterogeneity to internalized iron oxide nanoparticles for microorganism swarm control . J Nanoparticle Res . 2015 ; 17 : 1 - 15 . 29. Ghosh A , Fischer P . Controlled propulsion of artificial magnetic nanostructured propellers . Nano Lett . 2009 ; 9 : 2243 - 5 . 30. Kaynak M , Ozcelik A , Nourhani A , Lammert PE , Crespi VH , Huang TJ . Acoustic actuation of bioinspired microswimmers . Lab Chip . 2017 ; 17 : 395 - 400 . 31. Li J , Li T , Xu T , Kiristi M , Liu W , Wu Z , Wang J . Magnetoacoustic hybrid nanomotor . Nano Lett . 2015 ; 15 : 4814 - 21 . 32. Chowdhury S , Jing W , Cappelleri DJ . Controlling multiple microrobots: recent progress and future challenges . J Micro-Bio Robot . 2015 ; 10 : 1 - 11 . 33. Rahman MA , Cheng J , Fan Q , Ohta AT . Automated actuation of multiple bubble microrobots using computer-generated holograms . In: Proceedings of SPIE Defense + Commercial Sensing , Baltimore, MD , April, 2015 . 34. Rahman MA , Cheng J , Ohta AT . Parallel actuation and independent addressing of many bubble microrobots . In: 11th Annual IEEE International Conference on Nano/Micro Engineered and Molecular Systems , Matsushima, Japan, April 2016 . 35. Rahman MA , Cheng J , Ohta AT . Automated cooperative micro-assembly using multiple bubble microrobots . Paper presented at the 9th IEEE International Conference on Nano/Molecular Medicine and Engineering , Honolulu, HI, 15 - 18 November 2015 . 36. Hu W , Fan Q , Ohta AT . Interactive actuation of multiple optothermocapillary flow-addressed bubble microrobots . Robot Biomim . 2014 ; 1 : 14 . 37. Takahashi N , Rahman MA , Cheng J , Wang Z , Ohta AT . Automated microobject caging using bubble microrobots . In: 11th Annual IEEE International Conference on Nano/Micro Engineered and Molecular Systems , Matsushima, Japan, 17 - 20 April 2016 . 38. Bowman RW , Gibson GM , Linnenberger A , Phillips DB , Grieve JA , Carberry DM , Padgett MJ . “ Red Tweezers”: fast, customisable hologram generation for optical tweezers . Comput Phys Commun . 2014 ; 185 : 268 - 73 . 39. Otsu N. A threshold selection method from gray-level histograms . IEEE Trans Syst Man Cybern . 1979 ; 9 : 62 - 6 . 40. Rahman MA , Cheng J , Wang Z , Ohta AT . Cooperative micromanipulation using the independent actuation of fifty microrobots in parallel . Sci Rep . 2017 ; 7 : 3278 . 41. Cappelleri DJ , Fu Z. Towards flexible, automated microassembly with caging micromanipulation . In: IEEE International Conference on Robot Automation , Karlsruhe, Germany, 6 -10 May 2013 . 42. Takahashi N , Rahman MA , Cheng J , Wang Z , Ohta AT . Automated microobject caging using bubble microrobots . In: 11th Annual IEEE International Conference on Nano/Micro Engineered and Molecular Systems , Matsushima, Japan, April 2016 .