Vision-assisted micromanipulation using closed-loop actuation of multiple microrobots
Rahman et al. Robot. Biomim.
Vision-assisted micromanipulation using closed-loop actuation of multiple microrobots
M. Arifur Rahman 0
Noboru Takahashi 1
Kawai F. Siliga 0
Nigel K. Ng 0
Zhidong Wang 1
Aaron T. Ohta 0
0 Department of Electrical Engineering, University of Hawaii at Manoa , 2540 Dole Street, Holmes Hall 483, Honolulu, HI 96822 , USA
1 Department of Advanced Robotics, Chiba Institute of Technology , 2-17-1 Tsudanuma, Narashino, Chiba 275-0016 , Japan
Accurate control and precise positioning of opto-thermocapillary flow-addressed bubble microrobots are necessary for micromanipulation. In addition, micromanipulation using the simultaneous actuation of multiple microrobots requires a robust control system to enable independent motion. This paper demonstrates a hybrid closed-loop visionassisted control system capable of actuating multiple microrobots simultaneously and positioning them at precise locations relative to micro-objects under manipulation. A vision-assisted grasp-planning application was developed and used to calculate the necessary trajectories of the microrobots to form cages around micro-objects. The location of the microrobots and the micro-objects was detected at the caging locations using a particle-tracking application that used image feedback for precise positioning. The closed-loop image feedback information enabled the position update of the microrobots, allowing them to precisely follow the trajectory and caging locations calculated by the grasp-planning application. Four microrobots were assigned to cage a star-shaped micro-object using the closedloop control system. Once caged, the micro-object was transported to a location within the workspace and uncaged, demonstrating the micromanipulation task. This microrobotic system is well suited for the micromanipulation of single cells.
Microrobot; Opto-thermocapillary flow; Micromanipulation; Closed-loop control; Grasp planning; Path planning; Caging
Background
Microrobots in a liquid medium are an efficient tool for
bio-micromanipulation. They have broad applications
in biological engineering [
1–4
], biomedical engineering
[
5, 6
], and tissue engineering [
7, 8
]. Microrobots, which
are untethered submillimeter actuators, have been
utilized for the transportation of micro-objects including
living cells [
5, 7, 9
], with micron to submicron
resolution [
5, 7, 9–11
]. Microrobot-assisted
bio-micromanipulation is capable of cell separation and sorting [
10, 12,
13
], cell trapping, isolation, and transport [
12, 14
],
cellladen hydrogel assembly [
7
], and cell lysis and
molecular delivery [
15
]. Microrobots can be actuated using
various actuation methods including magnetic
actuation [
8, 9, 16–20
], bacterial propulsion [
3, 4
], biomimetic
propulsion [
21, 22
], and opto-thermal actuation [
7, 12,
13, 15
]. However, the degree of control required and
achieved by the various methods of actuation depends
upon the actuation force and the number of microrobots
being addressed independently.
One significant challenge to controlling many
microrobots independently is the use of a global actuation
force, as employed by magnetic microrobots [
9, 23
] and
bio-inspired magnetic swimming microrobots [
6, 24
].
Despite the challenges, the independent actuation of
multiple magnetic microrobots has been demonstrated
and was made possible by fabricating the microrobots
with different dimensions to obtain different magnetic
signatures [
25, 26
]. Bacteria-propelled microrobots based
on the motility of the bacteria have a limited degree of
controllability [27], but the use of electrical signals [
28
],
UV light [
4
], or chemical energy [
3, 27
] in conjunction
with bacteria propulsion helps to achieve higher
controllability, and has been used to actuate multiple
microrobots [3]. Bacteria-inspired microrobots actuated by
magnetic [
29
], acoustic [
30
], or a combination of these
forces [
31
] is capable of parallel control, but complex
motion, such as actuation along multiple trajectories, is
more difficult [
32
].
One type of microrobot that allows the parallel
actuation of many microrobots independently is the
opto-thermocapillary flow-addressed bubble (OFB) microrobot
[
12
]. This microrobot consists of a gas bubble in a liquid
medium and is actuated by opto-thermal forces [
33
].
Parallel actuation of many OFB microrobots allows
higherthroughput micromanipulation [
34
] and cooperative
manipulation [
35
]. However, the simultaneous actuation
of many microrobots is beyond the capacity of a human
operator using a manual control interface, such as a
mouse [
35
] or touch screen [
36
]. An automated,
closedloop control system is required to control many
microrobots at the same time.
In previous work, a small number of OFB microrobots
performed microassembly [
35–37
], single-cell
assembly [
12
], cell-laden hydrogel assembly [
7
], and
singlecell poration [
15
]. These applications require microscale
accuracy under optical microscopy. As an example,
cellladen hydrogel structures need to be positioned in
contact for cell culturing. A closed-loop automated control
system benefits the above micromanipulation tasks by
enabling increased throughput, cooperative
microassembly, and the automated planning and execution of
assembly tasks.
Individual and independent control of many OFB
microrobots in parallel requires automated and
sophisticated control, including features such as grasp planning
for caging, path planning for obstacle avoidance, and
moving along the shortest path to a destination.
Moreover, when multiple OFB microrobots are simultaneously
actuated, precise positioning and actuation, as well as
knowledge of the payload location, will enhance the
accuracy and resolution of the micromanipulation.
In this work, a hybrid closed-loop control system for an
OFB microrobot system was developed. The hybrid
system uses an open-loop computer-generated holographic
control system (developed in LabVIEW) to generate
the optical patterns necessary to actuate multiple OFB
microrobots simultaneously. The closed-loop part of the
hybrid system includes an image-processing algorithm
(developed in MATLAB) that provides image feedback
control; this allows the actuation of multiple OFB
microrobots and the knowledge of the locations of the objects
under manipulation. The control system also includes
a grasp-planning algorithm (developed in MATLAB)
that determines the shortest path and suitable grasping
point of micro-objects. Finally, the closed-loop
automatic actuation of four microrobots was demonstrated;
the OFB microrobots cooperatively caged a star-shaped
SU-8 microstructure and transported it to a desired
location within the workspace. The hybrid control
system achieved higher accuracy compared to open-loop
actuation.
Methods
A 1064-nm Nd:YAG, single-mode (TEM00) linearly
polarized laser (Laser Quantum, Ventus 1064, 1.5 + W)
with a 2-mm beam diameter was expanded using a
3x beam expander (Fig. 1a). The expanded beam was
incident at approximately 10 degrees on a spatial light
modulator (SLM, Hamamatsu, LCOS-SLM
X1046807) with an active area of 16 mm by 12 mm. The
collimated laser beam was modulated by the SLM, which
displayed an 800 pixel by 600-pixel computer-generated
hologram (CGH) created using a modified version of the
Red Tweezers program [
38
]. The modulated wavefront
containing the user-defined pattern was 4-f imaged by
placing lens L1 at its focal length from the SLM, and
lens L2 at its focal length away from the Fourier plane.
A 0.42-N.A. 10X objective lens (Mitutoyo) was placed at
the focal length of lens L2, focusing the optical pattern on
the light-absorbing coating of the substrate. The
1-mmthick glass substrate is coated with 100 nm of titanium
and formed a fluidic chamber when bonded to a
1-mmthick glass slide with a spacer layer consisting of 160-μm
double-sided polyimide tape. The liquid medium used
in the fluidic chamber was silicone oil (Fisher Scientific,
S159-500). A 1600 pixel by 1200 pixel camera (Point Grey
Flea3) was used for image capture. One pixel in the image
corresponds to 1.02 μm on the workspace.
The spatial light modulator creates the optical pattern
desired by the user (Fig. 1b). Each optical spot that is
focused on the light-absorbing layer of the substrate
produces a localized hot spot, vaporizing a small volume of
the liquid, and thus generating an OFB microrobot [
28
].
The thermocapillary flow generated by the temperature
gradient and resulting surface tension gradient at the
gas–liquid interface of the bubble pulls the OFB
microrobot toward the center of the localized hot spot [
28
].
Vision‑assisted closed‑loop control system
The hybrid closed-loop control system developed in this
work has two different parts: the microrobot actuation
block developed in LabVIEW, and the image-processing
and grasp-planning block developed in MATLAB. The
data transfer between the LabVIEW and the MATLAB
control blocks was accomplished using the MathScript
module of LabVIEW. Figure 2 shows the block diagram
of the complete control system. The actuation of
microrobots was accomplished by the “microrobot actuation
block.” An open-loop computer-generated holographic
(CGH) control system was developed in LabVIEW [
33
].
An optical spot is represented by a circular spot on the
LabVIEW user interface and is maneuvered using manual
or automatic control. The user enters the target location
of each microrobot, and the MathScript module
calculates the navigation parameters, such as initial
location, destination, frame rate, and frame size, and passes
them to the control application. The control application
in LabVIEW then sends the data to the OpenGL Shader
hologram engine, which calculates the corresponding
hologram using the direct superposition algorithm. The
holograms corresponding to the optical pattern are then
displayed on the SLM.
The feedback block was developed in MATLAB
(Fig. 2). The major function of the feedback loop is to
process the camera image of the workspace and output
data on the OFB microrobots and the micro-objects
under manipulation. The captured image was
analyzed by the Hough transform algorithm, providing the
microrobot’s locations and sizes, and the micro-object’s
locations and sizes. The Hough transform detected the
shapes in the image, which were then matched to
prestored object data. The image data were converted to
grayscale and then to a binary image using Otsu
thresholding [
39
] to make the image suitable for the Hough
transform. Detection of unwanted objects, such as the
white circular spot at the center of Microrobot 1 in
Fig. 4a, was reduced by defining the size ranges of the
objects to detect. A grasp-planning algorithm was also
developed in MATLAB to plan the course, speed, and
other microassembly-related parameters for efficient
manipulation. Once the microrobot data and
microobject data are available in the MATLAB workspace, the
grasp-planning module utilizes user-desired preplanned
conditions to calculate the final location of each
microrobot for accurate grasping. Once the final destination of
each microrobot is calculated, it is saved in a.mat file for
subsequent use by the microrobot actuation block.
During initiation of the actuation, the destination data from
the .mat file are loaded into the MathScript module of
LabVIEW.
Results
The hybrid control system described above was used to
perform closed-loop actuation of a single microrobot,
grasp planning of multiple microrobots for a
micromanipulation task, closed-loop actuation of multiple
microrobots, and open-loop micromanipulation of a
star-shaped SU-8 microstructure.
Closed‑loop actuation of a single microrobot
One of the benefits of the closed-loop control of OFB
microrobots is the ability to accurately update the
position of the microrobot using data from the
image-processing algorithm. Here, we have demonstrated actuation
of one microrobot along a preset zigzag path using the
hybrid control system (Fig. 3). The microrobot was
actuated from its initial location to 5 waypoints (waypoints
2–6 in Fig. 3) using automated open-loop control
sequences. Each point-to-point actuation (1–2, 2–3,
3–4, 4–5, and 5–6; see Fig. 3a) consisted of a series of 30
smaller actuation segments, at a rate of 2 Hz. The
actuation velocity between the waypoints varied.
First, the microrobot was actuated from position 1–2
using open-loop control, a distance of 1048 μm (Fig. 3a,
b). At location 2 (Fig. 3b), the feedback block captured
an image of the workspace, detected the physical bubble
location within the workspace, and compared it with the
intended destination set in the LabVIEW user interface.
The microrobot was then moved to a new location (2′
in Fig. 3c) to correct for the positioning error. Similarly,
the microrobot position was determined by the feedback
block at each waypoint (3–6 in Fig. 3d, f, h, j), compared
with the preset destination in LabVIEW actuation block,
and then moved to minimize the difference between the
preset destination and the actual position (3′, 4′, 5′, 6′ in
Fig. 3e, g, i, k). The position error calculated by the
feedback block at 2′, 3′, 4′, 5′, 6′ in Fig. 3 was 5.25, 14.1, 5.1,
16.5, and 2.8 μm, respectively. The position error was
then reduced to approximately 1 µm after the
microrobot locations were updated using the hybrid closed-loop
control system.
Grasp planning
Path planning refers to determining a collision-free path
for a moving object among obstacles [
18
]. In this work,
a grasp-planning algorithm determines the geometry
and location of a microrobot and a micro-object payload.
The output of the grasp-planning algorithm is sequences
of microrobot locations that form a trajectory from its
initial position to its goal position, which is in reference
to the payload. There were no obstacles present in the
workspace when caging the payload, but the algorithm is
capable of determining collision-free path about
obstacles. (Additional file 1: Figure S4). The grasp-planning
algorithm was used to create a cage of four OFB
microrobots around a star-shaped SU-8 microstructure, and
transport the object to another location. The star-shaped
microstructure consisted of four arms, each 59 μm in
length, and a hollow circular center with an inner
diameter of 62 μm. The width of the wall around the circular
center and the width of the arms were approximately
66 μm. The thickness of the SU-8 was 50 μm, and the
structure had an approximate mass of 2.35 μg.
Initially, four OFB microrobots were generated at
random locations around the micro-object (Fig. 4a) by
momentarily increasing the optical power in each spot
using the actuation block (Fig. 2). The feedback block
then detects the location and size of the microrobots
and the structure. The grasp-planning algorithm uses
the location of the structure within the workspace to
calculate the caging positions (1′, 2′, 3′, and 4′ in Fig. 4a)
at user-defined equidistant locations around the
microobject. The grasp-planning algorithm allows the
adjustment of the caging locations based on visual feedback
and the shape of the object. In this experiment, the
caging formation was rotated clockwise (Fig. 4a–c) to allow
a better grasp of the micro-object. The final caging
configuration (Fig. 4d) puts the microrobots in positions
that will allow them to grasp in between the arms of the
micro-object when the caging formation is contracted.
Hybrid closed‑loop actuation of multiple microrobots
The caging formation (1′, 2′, 3′, and 4′ in Fig. 4d)
calculated by the grasp-planning algorithm of the feedback
block sets the destination location for the individual
microrobots. These positions were saved in a.mat file and
subsequently loaded into the MathScript module of the
LabVIEW actuation block. However, the caging locations
require a transformation to match the coordinate system
of the actuation block. Once the transformed final
destinations of the microrobots are serially loaded into the
MathScript module from the.mat file, the destination of
each microrobot is mapped to the corresponding
locations on the LabVIEW user interface. In this experiment,
the microrobots numbered 1, 2, 3, and 4 were assigned
the caging locations 1′, 2′, 3′, and 4′ (Fig. 4d) as their
destinations in the actuation block.
Figure 5 shows the open-loop actuation of four
microrobots from their initial positions to the caging
locations. Microrobots 1, 2, 3, and 4 were simultaneously
actuated at velocities of 19, 29.6, 44.1, and 31.83 μm/s,
respectively, using the actuation block (Fig. 5a–c). Here,
the simultaneous actuation of multiple microrobots with
different speeds demonstrates a capability of the
microrobot control system: parallel, uncoupled movement of
microrobots along trajectories that vary in direction and
distance traveled during the same actuation time. The
microrobot actuation took 15 s (Additional file 2: Video
S1). Figure 5d shows the path of each microrobot from
its initial position to the caging location. Here, the
simultaneous actuation of multiple microrobots with different
speeds demonstrates a capability of the microrobot
control system: parallel, uncoupled movement of
microrobots along trajectories that vary in direction and distance
traveled during the same actuation time.
Figure 6 shows the OFB microrobots at the caging
positions. The image-processing algorithm determined
the locations of the microrobots and compared them to
the caging locations calculated by the grasp-planning
algorithm. Figure 6a shows the locations of Microrobots
1, 2, 3, and 4 as determined by the image-processing
algorithm (white dotted circles), and the desired caging
locations set by the grasp-planning algorithm (red
circles). The feedback block calculates the error between the
actual location of the microrobot and the desired caging
position, and calculates the new microrobot destination
to minimize the error. Microrobots 1, 2, 3, and 4 in Fig. 6a
were 27, 24.7, 14.28, and 21.5 μm away from their desired
caging positions, respectively. This information is passed
to the MathScript node of the actuation block, which
moves the microrobots to the new destinations (Fig. 6b).
The microrobots were actuated to their new positions at
speeds of 5.4, 4.94, 2.86, and 4.3 μm/s for Microrobots 1,
2, 3, and 4, respectively (Fig. 6b).
After updating the location using open-loop actuation,
the physical location of the microrobot within the
workspace was determined by the image-processing algorithm
of the feedback loop, as shown in Fig. 6c. The red
circles in Fig. 6c are the caging position set by the
graspplanning algorithm at the beginning of the actuation. In
Fig. 6c, the Microrobots at 1′, 2′, 3′, and 4′ were 8, 12,
5, and 19 μm away from their desired locations,
corresponding to a reduction in the position error of
approximately 50%.
Micromanipulation
The hybrid closed-loop vision-assisted control system
allowed an accurate placement of a caging formation, as
described above. Figure 7a shows the open-loop
actuation of a microrobot matrix approaching a micro-object
to grasp it for manipulation. The matrix of microrobots
was manually controlled by user input to the
actuation block. The object was grasped by contracting the
microrobot formation at an average speed of 7.6 μm/s
(Fig. 7b and Additional file 3: Video S2). After grasping,
the microrobot formation attempted to transport the
micro-object in the positive x-direction. However, the
micro-object was stuck to the floor of the fluid chamber,
resulting in the dislocation of the microrobots from their
actuation patterns (Fig. 7c). This phenomenon is more
obvious for Microrobots 2 and 3, marked with red arrows
in Fig. 7c, as they were moved in the positive x-direction
and left the micro-object behind.
To free the micro-object from the surface, the
formation of the microrobots was rotated to create a torque on
the object while maintaining a firm grasp (Fig. 7d). The
twisting of the object helped to overcome its stiction, and
the micro-object could then be transported. The
microobject was transported along various trajectories (Fig. 7e)
and at various speeds up to approximately 90 μm/s
(Additional file 4: Video S3). A graph of the planned
trajectory and completed trajectory during the micro-object
manipulation is included in Additional file 1: Figure S5.
Upon completion of the micromanipulation, the
microobject was released by expanding the microrobot
formation (Fig. 7f ).
Discussion
The hybrid closed-loop vision-based control of OFB
microrobots with open-loop actuation has leveraged
the functionality of two different platforms to perform
the micromanipulation tasks. This integration of
MATLAB and LabVIEW utilizes the hardware support and
rapid configuration of LabVIEW and the advantages of
MATLAB’s processing of complex image data. Aside
from providing more precise manipulation, closed-loop
position updates allow the use of standard macroscale
robotic functions such as grasp planning, collision
avoidance, and detection, grasping, and payload detection and
delivery. Also, the hybrid control allows the detection of
failed actuation of individual microrobots (Additional
file 1: Figure. S6). Moreover, the OFB microrobot system,
which is capable of the independent actuation of many
microrobots, utilizes vision-based automatic actuation
for the simultaneous participation of multiple
microrobots in micromanipulation, as it is difficult for a human
operator to control many microrobots. In this work, the
OFB microrobot system enabled a team of microrobots
to transport a large object, which is not possible with a
single microrobot. This was quantified in previous work,
as it was observed one or two microrobots could produce
limited rotational movement of an SU-8 microstructure,
but no translational movement. Three or four
microrobots were necessary to translate the micro-object [
40
].
The closed-loop position updates increased the
accuracy of the caging locations by 50% compared to one
iteration of open-loop actuation. The results suggest that
multiple iterations of closed-loop position updating may
reduce the error further, but this needs more
investigation. The causes of the position error during open-loop
actuation can be broadly divided into two categories:
system error and mechanical error. The system error is due
to the spatial resolution of the SLM and any misalignment
of the optical elements. The mechanical error is caused
by the misalignment of the image coordinates compared
to the LabVIEW user interface coordinates. This
misalignment exists due to the mechanical adjustment of
the camera position when attempting to match to the
LabVIEW coordinates. The software-defined coordinates
in LabVIEW were considered the ideal coordinates, and
the camera was adjusted by hand such that a single pixel
on both the MATLAB image-processing module and the
LabVIEW actuation module had the same dimensions.
Despite the careful adjustment, an error of approximately
6–8 pixels was present; this was quantified by taking
multiple measurements of a stationary micro-object.
The closed-loop control system helps with the system
error, but is unable to correct for the mechanical error.
The average error calculated during open-loop actuation
(Fig. 5) was 21.87 μm per microrobot. The closed-loop
position update reduced the average error to 11 μm per
microrobot.
The positioning tolerance of the micromanipulation
varies with the size of the objects under manipulation
and the type of manipulation. For example, stable
caging requires the microrobots to be placed at a distance
smaller than the payload size from each other [
41, 42
].
The caging of the 300-μm-diameter star-shaped
microobject using 130-μm-diameter microrobots required the
microrobots to be placed less than 300 μm from each
other for stable trapping and at least 130 µm (one body
length) apart to avoid merging of the bubble
microrobots. Thus, the calculated microrobot separation of
280 μm with ± 10 μm position tolerance satisfies these
conditions. The hybrid closed-loop control system was
able to reduce the average position error from 21.9 to
11 μm, satisfying the position tolerance for this
micromanipulation task.
The image acquisition, object detection, and path
planning in MATLAB take approximately 1.6 s to compute
while running on a PC with an AMD Phenom II × 6
3.31GHz processor and 16 GB RAM. The object locations and
path planning data calculated by the feedback block (Fig. 2)
are saved in a.mat file. A MathScript node included in the
actuation block in LabVIEW reads the data from the.mat
file and sends it to the sequence generator for actuating the
microrobots along the planned path. The feedback block
runs in MATLAB and the actuation block runs in
LabVIEW, so the operator needs to manually enter the.mat file
location in the MathScript node and click the run button
in the actuation block. This operation takes approximately
5 s, so there is a total delay of approximately 6.5 s between
the image acquisition and the microrobot actuation.
The image feedback of the proposed hybrid control
system can detect more than four particles at once; it can
detect as many as distinct objects that can fit in the
camera field of view. The control algorithm is also not limited
to four particles; it can set destinations and waypoints
for a number of objects detected in the workspace. The
open-loop control part of the hybrid system has been
demonstrated to control an array of 50 OFB
microrobots [
40
], and this system can control at least the same
amount of microrobots at once.
In this work, the vision-based closed-loop position
update was executed at the waypoints for single
microrobot actuation and at the caging locations for multiple
microrobot actuations, instead of iteratively after each
camera video frame. The closed-loop position update
was not implemented after each frame since the
position errors per frame were usually less than one micron.
Moreover, the processing time for the image analysis
of the high-resolution image (1600 px by 1200) at each
frame would increase the overhead on the overall
computation process and time.
Conclusions
A hybrid closed-loop vision-assisted control system
was developed in MATLAB and LabVIEW to control
multiple OFB microrobots automatically. The control
system was used to demonstrate open-loop actuation
of a single microrobot and simultaneous actuation of
multiple microrobots along with closed-loop position
updates. A grasp-planning algorithm was also
developed in MATLAB and utilized to calculate the precise
locations for grasping a micro-object using a team of
four microrobots. The position of each microrobot and
the micro-object under manipulation was detected by
the closed-loop feedback module to minimize the error
between the physical location calculated by the
imageprocessing algorithm and the intended destination. This
closed-loop actuation allows automatic and simultaneous
actuation of multiple microrobots for micromanipulation
with precise positioning, beyond the capacity of a human
operator. However, this hybrid control scheme requires
certain human operators to switch between the
applications (MATLAB and LabVIEW). In the future, this
system can be upgraded to use image processing based on
the seamless integration of LabVIEW and MATLAB,
allowing minimum user interaction.
Additional files
Additional file 1. Supplementary information.
Additional file 2: Video S1. Automatic actuation of four OFB
microrobots into the caging formation.
Additional file 3: Video S2. Grasping the micro-object.
Additional file 4: Video S3. Micro-object manipulation.
Authors’ contributions
ATO and MAR conceived and designed the experiments; MAR performed the
experiments; MAR, NT, and ATO analyzed the data; NT, MAR, KFS, NKN, and
ZW developed and modified the control application; MAR and ATO wrote the
paper. All authors read and approved the final manuscript.
Acknowledgements
ATO acknowledges the Japan Society for the Promotion of Science (JSPS) for
an overseas researcher fellowship.
Competing interests
The authors declare that they have no competing interests
Funding
This project was supported in part by Grant Number 1R01EB016458 from the
National Institute of Biomedical Imaging and Bioengineering of the National
Institutes of Health (NIH). These contents are solely the responsibility of the
authors and do not necessarily represent the official views of the NIH. The
funding sponsors had no role in the design of the study, in the collection,
analyses, or interpretation of data, in the writing of the manuscript, and in the
decision to publish the results.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in
published maps and institutional affiliations.
1. Rubenstein M , Cornejo A , Nagpal R . Programmable self-assembly in a thousand-robot swarm . Science . 2014 ; 345 : 795 - 9 .
2. Wang X , Chen S , Sun D . Robot-aided micromanipulation of biological cells with integrated optical tweezers and microfluidic chip . In: Sun Y , Liu X , editors. Micro- and nanomanipulation tools . 1st ed. Weinheim: Wiley; 2015 . p. 393 - 415 .
3. Kim D , Liu A , Diller E , Sitti M. Chemotactic steering of bacteria propelled microbeads . Biomed Microdev . 2012 ; 14 : 1009 - 17 .
4. Steager EB , Sakar MS , Kim DH , Kumar V , Pappas GJ , Kim MJ . Electrokinetic and optical control of bacterial microrobots . J Micromech Microeng . 2011 ; 21 : 035001 .
5. Sitti M , Ceylan H , Hu W , Giltinan J , Turan M , Yim S , Diller E . Biomedical applications of untethered mobile milli/microrobots . Proc IEEE . 2015 ; 103 : 205 - 24 .
6. Peyer KE , Zhang L , Nelson BJ . Bio-inspired magnetic swimming microrobots for biomedical applications . Nanoscale . 2013 ; 5 : 1259 - 72 .
7. Hu W , Fan Q , Ohta AT . Assembly of cell-laden microgels by an optically controlled bubble manipulator . In: International Conference on Miniaturized Systems for Chemistry and Life Sciences, Freiburg , Germany, October, 2013 .
8. Knowlton S , Li D , Ersoy F , Cho YK , Tasoglu S. Building blocks for bottomup neural tissue engineering: tools for in vitro assembly and interrogation of neural circuits . In: Zhang LG , Kaplan DL , editors. Neural Engineering . 1st ed. Berlin: Springer International Publishing; 2016 . p. 123 - 44 .
9. Steager EB , Selman Sakar M , Magee C , Kennedy M , Cowley A , Kumar V . Automated biomanipulation of single cells using magnetic microrobots . Int J Rob Res . 2013 ; 32 : 346 - 59 .
10. Chen CY , Chen CF , Yi Y , Chen LJ , Wu LF , Song T . Construction of a microrobot system using magnetotactic bacteria for the separation of Staphylococcus aureus . Biomed Microdev . 2014 ; 16 : 761 - 70 .
11. Cappelleri DJ , Fu Z , Fatovic M . Caging for 2D and 3D micromanipulation . J Micro-Nano Mech . 2012 ; 7 : 115 - 29 .
12. Hu W , Fan Q , Ohta AT . An opto-thermocapillary cell micromanipulator . Lab Chip . 2013 ; 13 : 2285 - 91 .
13. Cheng J , Rahman MA , Ohta AT . Optical manipulation of cells . In: Lee W, Tseng P , Di Carlo D, editors. Microtechnology for Cell Manipulation and Sorting . 1st ed. Berlin: Springer International Publishing; 2017 . p. 93 - 128 .
14. Vikram Singh A , Sitti M. Targeted drug delivery and imaging using mobile milli/microrobots: a promising future towards theranostic pharmaceutical design . Curr Pharm Des . 2016 ; 22 : 1418 - 28 .
15. Fan Q , Hu W , Ohta AT . Efficient single-cell poration by microsecond laser pulses . Lab Chip . 2015 ; 15 : 581 - 8 .
16. Salehizadeh M , Diller E . Two-agent formation control of magnetic microrobots in two dimensions . J Micro-Bio Robot . 2017 ; 1 : 1 - 11 .
17. Diller E , Giltinan J , Lum GZ , Ye Z , Sitti M . Six-degree-of-freedom magnetic actuation for wireless microrobotics . Int J Robot Res . 2016 ; 35 : 114 - 28 .
18. Chowdhury S , Johnson BV , Jing W , Cappelleri DJ . Designing local magnetic fields and path planning for independent actuation of multiple mobile microrobots . J Micro-Bio Robot . 2017 ; 12 : 21 - 31 .
19. Pawashe C , Floyd S , Sitti M. Multiple magnetic microrobot control using electrostatic anchoring . Appl Phys Lett . 2009 ; 94 : 164108 .
20. Pelrine R , Wong-Foy A , McCoy B , Holeman D , Mahoney R , Myers G , Herson J , Low T. Diamagnetically levitated robots: an approach to massively parallel robotic systems with unusual motion properties . In: IEEE International Conference on Robot Automation, St. Paul, USA, May 2012 .
21. Cheang UK , Kim MJ . Fabrication and control of simple low Reynolds number microswimmers . Appl Phys Lett . 2016 ; 109 : 034101 .
22. Ding Y , Qiu F , Casadevall i Solvas X , Chiu FWY , Nelson BJ , deMello A. Microfluidic-based droplet and cell manipulations using artificial bacterial flagella . Micromachines . 2016 ; 7 : 25 .
23. Pawashe C , Floyd S , Diller E , Sitti M . Two-dimensional autonomous microparticle manipulation strategies for magnetic microrobots in fluidic environments . IEEE Trans Robot . 2012 ; 28 : 467 - 77 .
24. Martel S , Mohammadi M , Felfoul O , Zhao L , Pouponneau P . Flagellated magnetotactic bacteria as controlled MRI-trackable propulsion and steering systems for medical nanorobots operating in the human microvasculature . Int J Robot Res . 2009 ; 28 : 571 - 82 .
25. Diller E , Giltinan J , Sitti M. Independent control of multiple magnetic microrobots in three dimensions . Int J Robot Res . 2013 ; 32 : 614 - 31 .
26. Frutiger DR , Vollmers K , Kratochvil BE , Nelson BJ . Small, fast, and under control: wireless resonant magnetic micro-agents . Int J Robot Res . 2010 ; 29 : 613 - 36 .
27. Behkam B , Sitti M. Bacterial flagella-based propulsion and on/off motion control of microscale objects . Appl Phys Lett . 2007 ; 90 : 023902 .
28. Kim PSS , Becker A , Ou Y , Julius AA , Kim MJ . Imparting magnetic dipole heterogeneity to internalized iron oxide nanoparticles for microorganism swarm control . J Nanoparticle Res . 2015 ; 17 : 1 - 15 .
29. Ghosh A , Fischer P . Controlled propulsion of artificial magnetic nanostructured propellers . Nano Lett . 2009 ; 9 : 2243 - 5 .
30. Kaynak M , Ozcelik A , Nourhani A , Lammert PE , Crespi VH , Huang TJ . Acoustic actuation of bioinspired microswimmers . Lab Chip . 2017 ; 17 : 395 - 400 .
31. Li J , Li T , Xu T , Kiristi M , Liu W , Wu Z , Wang J . Magnetoacoustic hybrid nanomotor . Nano Lett . 2015 ; 15 : 4814 - 21 .
32. Chowdhury S , Jing W , Cappelleri DJ . Controlling multiple microrobots: recent progress and future challenges . J Micro-Bio Robot . 2015 ; 10 : 1 - 11 .
33. Rahman MA , Cheng J , Fan Q , Ohta AT . Automated actuation of multiple bubble microrobots using computer-generated holograms . In: Proceedings of SPIE Defense + Commercial Sensing , Baltimore, MD , April, 2015 .
34. Rahman MA , Cheng J , Ohta AT . Parallel actuation and independent addressing of many bubble microrobots . In: 11th Annual IEEE International Conference on Nano/Micro Engineered and Molecular Systems , Matsushima, Japan, April 2016 .
35. Rahman MA , Cheng J , Ohta AT . Automated cooperative micro-assembly using multiple bubble microrobots . Paper presented at the 9th IEEE International Conference on Nano/Molecular Medicine and Engineering , Honolulu, HI, 15 - 18 November 2015 .
36. Hu W , Fan Q , Ohta AT . Interactive actuation of multiple optothermocapillary flow-addressed bubble microrobots . Robot Biomim . 2014 ; 1 : 14 .
37. Takahashi N , Rahman MA , Cheng J , Wang Z , Ohta AT . Automated microobject caging using bubble microrobots . In: 11th Annual IEEE International Conference on Nano/Micro Engineered and Molecular Systems , Matsushima, Japan, 17 - 20 April 2016 .
38. Bowman RW , Gibson GM , Linnenberger A , Phillips DB , Grieve JA , Carberry DM , Padgett MJ . “ Red Tweezers”: fast, customisable hologram generation for optical tweezers . Comput Phys Commun . 2014 ; 185 : 268 - 73 .
39. Otsu N. A threshold selection method from gray-level histograms . IEEE Trans Syst Man Cybern . 1979 ; 9 : 62 - 6 .
40. Rahman MA , Cheng J , Wang Z , Ohta AT . Cooperative micromanipulation using the independent actuation of fifty microrobots in parallel . Sci Rep . 2017 ; 7 : 3278 .
41. Cappelleri DJ , Fu Z. Towards flexible, automated microassembly with caging micromanipulation . In: IEEE International Conference on Robot Automation , Karlsruhe, Germany, 6 -10 May 2013 .
42. Takahashi N , Rahman MA , Cheng J , Wang Z , Ohta AT . Automated microobject caging using bubble microrobots . In: 11th Annual IEEE International Conference on Nano/Micro Engineered and Molecular Systems , Matsushima, Japan, April 2016 .