DETERMINATION OF THE UAV POSITION BY AUTOMATIC PROCESSING OF THERMAL IMAGES

International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XXXIX-B6, 2012 XXII ISPRS Congress, 25 August – 01 September 2012, Melbourne, Australia DETERMINATION OF THE UAV POSITION BY AUTOMATIC PROCESSING OF THERMAL IMAGES Wilfried Hartmann, Sebastian Tilch, Henri Eisenbeiss, Konrad Schindler ETH Zurich (Swiss Federal Institute of Technology), Institute of Geodesy and Photogrammetry Wolfgang-Pauli-Str. 15, 8093 Zurich, Switzerland (wilfried.hartmann, sebastian.tilch, henri.eisenbeiss, konrad.schindler)@geod.baug.ethz.ch KEY WORDS: Thermal, UAV, Camera, Calibration, Bundle, Photogrammetry, GPS/INS ABSTRACT: If images acquired from Unmanned Aerial Vehicles (UAVs) need to be accurately geo-referenced, the method of choice is classical aerotriangulation, since on-board sensors are usually not accurate enough for direct geo-referencing. For several different applications it has recently been proposed to mount thermal cameras on UAVs. Compared to optical images, thermal ones pose a number of challenges, in particular low resolution and weak local contrast. In this work we investigate the automatic orientation of thermal image blocks acquired from a UAV, using artificial ground control points. To that end we adapt the photogrammetric processing pipeline to thermal imagery. The pipeline achieves accuracies of about ± 1 cm in planimetry and ± 3 cm in height for the object points, respectively ± 10 cm or better for the camera positions, compared to ± 100 cm or worse for direct geo-referencing using on-board single-frequency GPS. 1 INTRODUCTION 1.2 Chapter 2 is dedicated to the data acquisition. In the first two sections, the employed UAV system (2.1) and the geometric calibration of the thermal camera (2.2) is described. The flight planning and the distribution of the ground control points (GCPs) is shown in section (2.3). The recording itself is briefly described in (2.4). The processing workflow is presented in chapter 3, including synchronization issues (3.1), automatic image measurements of the GCPs (3.2), and bundle triangulation (3.3). In chapter 4 the experimental results are presented, and the influence of different matching strategies (4.1) and number of GCPs (4.2) is evaluated. In (4.3) the use of GNSS and IMU measurements is discussed. Finally, the paper ends with conclusions and outlook in (5). The application of micro Unmanned Aerial Vehicles (UAVs) in geomatics is increasing since they facilitate the rapid and flexible acquisition of areas and objects at a medium scale. According to (van Blyenburgh, 2011), a micro UAV is defined as a small unmanned aircraft with a maximum payload of 5 kg and a flight altitude up to 250 m. Mostly, micro UAVs are used in mining, agriculture, urban and architectural mapping, as well as archaeology (Eisenbeiss, 2009). In agriculture, UAVs can be used e.g. to detect fawns before the harvest like in (Israel, 2011), or to measure the nitrogen status of sunflowers, as shown in (Agüera et al., 2011). 3D applications include different forms of topographic mapping. In (Neitzel and Klonowski, 2011) different methods for the generation of dense 3D point clouds are compared. An example application, which is presented in (Neitzel and Klonowski, 2011) is the mapping of a landfill. The mapping of landslides near road embankments is presented in (Carvajal et al., 2011). In contrast to manned aerial vehicles, micro UAVs are equipped only with low-cost GNSS receivers and IMU sensors due to payload limitations. The accuracy of the sensors is not sufficient to use their measurements for direct geo-referencing, which is why UAV imagery is normally oriented through aero-triangulation. The present paper investigates whether thermal images in combination with artificial ground control points (GCPs) may be used for direct geo-referencing. Thermal images have lower resolution than RGB images and at the same time more blur and more distortion, and usually only few, if any, crisp local features. Figure 1 shows an example image. 1.1 Paper structure Figure 1: Example for a thermal image acquired by a thermal camera FLIR Tau 640. Objectives 2 The goal of the project is to implement and evaluate a purely image-based approach for automatic geo-referencing of thermal images, i.e. the determination of the camera positions and orientations. The images are acquired by a thermal camera which is mounted on a UAV. DATA ACQUISITION The UAV system used for data acquisition as well as the thermal camera are introduced in this section. Furthermore, we discuss flight preparation, including the flight planning and the choice of artificial GCPs. 111 International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XXXIX-B6, 2012 XXII ISPRS Congress, 25 August – 01 September 2012, Melbourne, Australia 2.1 UAV system structure with a glass plate and an overlaid steel plate. Cross shaped openings in the steel plate serve as target features. In (Lagüela et al., 2011) a wooden board is equipped with a symmetrical pattern of lamps. In (Simmler, 2009) the indoor calibration field is a plate with embedded lamps that can be used as calibration targets. Because of their temperature gradient, the application of lamps in (Lagüela et al., 2011) and (Simmler, 2009) is not optimal: the lamps’ temperature decreases gradually with increasing distance to its centre and leads to fuzzy boundaries. This problem does not apply to the approaches of (Buyuksalih and Pétrie, 1999) and (Luhmann et al., 2011). In both approaches, the target appear sharp in the thermal images, with well-defined boundaries. Among the two, the calibration field in (Luhmann et al., 2011) does not need a separate power supply, while the one of (Buyuksalih and Pétrie, 1999) does. The passive calibration field is more flexible for both indoor and outdoor applications. Moreover, it is easy to build. Based on the review we have chosen to use the calibration field of (Luhmann et al., 2011) in this project. Video recording UAV control Figure 2: UAV system. 2.2.2 Calibration results In our experiment, an uncooled thermal camera FLIR Tau 640 with a relatively high geometric resolution of 640 × 512 pixel has been used, as shown in (Fig. 4). The vanadium oxide microbolometer detector is sensitive to wavelengths between 7.5 µm and 13.5 µm. The employed UAV system (Falcon 8 from Ascending Technologies), the remote control and the two required notebooks are illustrated in (Fig. 2). The Falcon 8 performs vertical take off and landing (VTOL). During the flight it can be controlled by either a human operator who uses the remote control, or by control software which runs on the first computer. A second computer is required to record the video signal from the thermal camera, which is down-linked during the flight. 2.2 Geometric camera calibration 2.2.1 Calibration method Before the image acquisition, the interior orientation of the therma (...truncated)