Least squares image matching: A comparison of the performance of robust estimators

ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume II-1, 2014 ISPRS Technical Commission I Symposium, 17 – 20 November 2014, Denver, Colorado, USA LEAST SQUARES IMAGE MATCHING: A COMPARISON OF THE PERFORMANCE OF ROBUST ESTIMATORS Zeyu Li a,*, Jinling Wang b a School of Civil and Environmental Engineering, University of New South Wales, Sydney, 2052,Australia – b School of Civil and Environmental Engineering, University of New South Wales, Sydney, 2052,Australia – Commission VI, WG VI/4 KEY WORDS: Least Squares Image Matching, Robust Estimators, Accuracy, Comparison, Systematic errors, Outliers ABSTRACT: Least squares image matching (LSM) has been extensively applied and researched for high matching accuracy. However, it still suffers from some problems. Firstly, it needs the appropriate estimate of initial value. However, in practical applications, initial values may contain some biases from the inaccurate positions of keypoints. Such biases, if high enough, may lead to a divergent solution. If all the matching biases have exactly the same magnitude and direction, then they can be regarded as systematic errors. Secondly, malfunction of an imaging sensor may happen, which generates dead or stuck pixels on the image. This can be referred as outliers statistically. Because least squares estimation is well known for its inability to resist outliers, all these mentioned deviations from the model determined by LSM cause a matching failure. To solve these problems, with simulation data and real data, a series of experiments considering systematic errors and outliers are designed, and a variety of robust estimation methods including RANSACbased method, M estimator, S estimator and MM estimator is applied and compared in LSM. In addition, an evaluation criterion directly related to the ground truth is proposed for performance comparison of these robust estimators. It is found that robust estimators show the robustness for these deviations compared with LSM. Among these the robust estimators, M and MM estimator have the best performances. 1. INTRODUCTION Image matching is an active research field in digital photogrammetry and computer vision. The main task in image matching is to find the corresponding pixel on two images of the same physical region. Usually the two images are referred as the reference image and the querying image respectively. In general, this is one fundamental step of various vision-based applications such as camera calibration, panorama generation, object recognition, structure from motion (SFM), 3D map generation and vision-based navigation. Different matching methods such as keypoint matching and area-based matching are proposed by many researchers. Among them, least squares image matching (LSM) is still attractive for its definite mathematical description of the two patches and high accuracy. Moreover, quality control in the surveying domain can be implemented on LSM. LSM’s creation can be traced back to the 1980s when Förstner (1982) firstly put forward the basic idea for LSM. Now the most widely used LSM utilises the intensity as the observation, and combines affine transformation model and linear radiometric model. Since it is a nonlinear model, Taylor linearization transfers the nonlinear model into the linear one to solve the unknowns through iterations. At the same time, because the number of observations exceeds that of unknowns, the solution can be obtained in a least squares sense. Finally, an accurate matching is acquired by the solution. Thus, LSM in essence is one process of nonlinear regression. * LSM has been widely researched and explored to enhance its adaptability and performance. For example, Bethmann and Luhmann (2011) employed the new projective transformation model in the functional model to improve the adaptability. In terms of the stochastic model, most researchers assumed that the measurements involved had equal precision and statistically independent, thus the weight matrix was a diagonal matrix. However, Wu et al. (2007) adapted the stochastic model by Blue estimator and the result showed that the new stochastic model improved matching accuracy by 0.2-0.4 pixel. Acting as a matching refinement method, LSM has been extensively applied in different types of photogrammetric software. For example, Zhang et al. (2011) put forward a threestep scheme for keypoint matching on low-attitude images acquired by remotely piloted aerial vehicles. Pyramid-based LSM acted as the last step for refinement to improve the precision of keypoint matching. Ultimately a 3D city model was generated. Debella-Gilo and Kääb (2012) explored LSM’s application on surface displacement and deformation of mass movements, and showed that LSM could match the images and computed longitudinal strain rates, transverse strain rates and shear strain rates reliably and accurately. However, LSM is subject to the divergence problem, which means that the solution from LSM will not promise a reliable result under certain circumstances. According to Bethmann and Luhmann (2011), the five aspects will affect the solution of LSM, which are the texture, the reference windows, geometric Corresponding author This contribution has been peer-reviewed. The double-blind peer-review was conducted on the basis of the full paper. doi:10.5194/isprsannals-II-1-37-2014 37 ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume II-1, 2014 ISPRS Technical Commission I Symposium, 17 – 20 November 2014, Denver, Colorado, USA distortions the quality of initial values, and the functional model. This paper focuses on robust estimation to eliminate the influence of derivations from the LSM model. Two sources are presented. Firstly, the initial values of the unknowns are important since LSM is based on Taylor linearization. Keypoint matching is one method to obtain the initial values, but mismatches may happen. Therefore if the initial values of the unknowns are not close enough to the correct solution, the error caused by linearization will be large, resulting in a matching failure in the end. However, according to Baine and Rattan (2012), compared with their corresponding points in the reference image, if all the corresponding points in the querying image have a constant bias with the same magnitude and direction, the biases can be classified as systematic errors. Thus, this paper simulates this special case for LSM applications. Secondly, another type of deviation—Salt & Pepper noise—is considered in this paper. It models the malfunction of the imaging sensor, and simulates other common conditions such as poor illumination, signal transmission and high temperature. Since LSM already defines the model for the data, which means that the “normal” data pattern has been determined. If one set of data is “faraway” from the main part of the data, it can be identified as deviations. Generally, if the deviation is (...truncated)