A ROBUST METHOD FOR FUNDAMENTAL MATRIX ESTIMATION WITH RADIAL DISTORTION

The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-3, 2018 ISPRS TC III Mid-term Symposium “Developments, Technologies and Applications in Remote Sensing”, 7–10 May, Beijing, China A ROBUST METHOD FOR FUNDAMENTAL MATRIX ESTIMATION WITH RADIAL DISTORTION Jun-Shi XUE 1,*, Xiang-Ning CHEN2, Hui YI 1 1 Department of Postgraduate, Aerospace Engineering University, Beijing, China - (, ) 2 Institute of Aerospace Information, Aerospace Engineering University, Beijing, China – . Commission III, ICWG III/I KEY WORDS: Machine Vision; 3D Reconstruction; Robust Method; Fundamental Matrix; Epipolar Geometry; SVD. ABSTRACT: Fundamental Matrix Estimation is of vital importance in many vision applications and is a core part of 3D reconstruction pipeline. Radial distortion makes the problem to be numerically challenging. We propose a novel robust method for radial fundamental matrix estimation. Firstly, two-sided radial fundamental matrix is deduced to describe epipolar geometry relationship between two distorted images. Secondly, we use singular value decomposition to solve the final nonlinear minimization solutions and to get the outliers removed by multiplying a weighted matrix to the coefficient matrix. In every iterative step, the criterion which is the distance between feature point and corresponding epipolar line is used to determine the inliers and the weighted matrix is update according to it. The iterative process has a fast convergence rate, and the estimation result of radial fundamental matrix remains stable even at the condition of many outliers. Experimental results prove that the proposed method is of high accuracy and robust for estimating the radial fundamental matrix. The estimation result of radial fundamental matrix could be served as the initialization for structure from motion. 1. INTRODUCTION Fundamental matrix describes the epipolar geometry relationship between two images in the same scene. It is independent of scene structure, and only depends on the camera internal parameters and motion parameters. Fundamental matrix estimation is a basic and key issue in computer vision. It plays an important role in many vision applications such as SLAM, motion segmentation, structure from motion, image stitching and dense stereo matching. Moreover, it’s one of the core parts of 3D reconstruction pipeline. Given its vital importance, many methods were proposed in the past decades. (Longust H, 1984) first proposed to apply epipolar geometry constraints to scene reconstruction. The five point relative pose solver with known camera internal parameters (Stewénius H, 2006) and the six point relative pose solver with unknown focal length (Stewenius H, 2005), the well-known 7-point and normalized 8-point algorithm (ArmanguéX, 2003) were the frequently used linear estimation algorithm. These methods had high computational efficiency regardless of false matches and outliers, but poor accuracy and stability if taking these factors into account. Robust methods, such as RANSAC, LMedS, MLESAC, MAPSAC, could deal with false matches and outliers based on multiple sampling estimation parameter model, which resulted in heavy computational burden and low efficiency. However, most of the above-mentioned method are based on the assumption that the camera follows pin-hole model, while with the popularity of unmanned aerial vehicle camera, cellphone cameras and GoPro Hero, the importance of radial distortion models increases, especially in 3D reconstruction and SLAM. A non-minimal method based on 15 correspondences for fundamental matrix estimation with radial distortion was first proposed in (Barreto J, 2005) A number of minimal problems for fundamental matrix estimation with radial distortion have been studied in (Kukelova Z, 2007a, 2007b), where practical solutions were given in some cases. Fast and robust algorithms for two minimal problems for simultaneous computation of fundamental matrix and two different radial distortion were given from 12 point correspondences based on a generalized eigenvalue formulation (Byrod M, 2008). A numerically stable and efficient solution for the calibrateduncalibrated image registration problem with radial distortion was presented in (José H, 2012). Based on the plumb-line assumption, constraints on the radial distortion center from epipolar geometry were derived (Henrique B., 2013). More recently, a fast and stable polynomial solver based on Gröbner basis method was derived (Jiang F, 2014), which enables simultaneous auto-calibration of focal length and radial distortion. In the meantime, the detail of using numerical Gröbner basis computations techniques was given. A more efficient and stable solution using 10 image correspondences was proposed by using the Sturm sequences method, which can be used in real-time applications (Kukelova Z, 2015). Moreover, a new formulation in which distortion center can be absorbed into the radial fundamental matrix was presented by (Brito J, 2013). These solutions make great progress in numerical stability and efficiency. However, the fast, accurate and robust solutions for the fundamental matrix with radial distortion estimation need to be further studied. In this paper, we propose a new robust method for estimating the fundamental matrix with radial distortion based on 14 image correspondences. Unlike traditional robust estimation method, * Corresponding author This contribution has been peer-reviewed. https://doi.org/10.5194/isprs-archives-XLII-3-2029-2018 | © Authors 2018. CC BY 4.0 License. 2029 The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-3, 2018 ISPRS TC III Mid-term Symposium “Developments, Technologies and Applications in Remote Sensing”, 7–10 May, Beijing, China our method integrate the outlier removal procedure into the normalized 14-point algorithm, so as to estimate radial fundamental matrix robustly and fast. The main content of the paper is as follows: In section 2, the formulation of radial fundamental matrix is introduced and in section 3, the robust method for radial fundamental matrix estimation is discussed in detail. Finally, in section4, experiments on synthetic data and real images are implemented to prove the accuracy and robust of the proposed method.  xdi ydi 1 xdi 2  ydi 2  ,  xdi = the extended corresponding points. Using Kronecker products, equation (4) can be written as: ( xdi , ydi ,1, xdi 2  ydi 2 )  ( xdi , ydi ,1, xdi2  ydi2 ) f  0 (5)  2.1 Problem Formulation   f  ( F11 , Where 2. MAIN BODY ydi 1 xdi2  ydi2  , F44 )T is called vectorization of F . From each correspondence, we obtain a different row vector Ai :  T and mui   xui , yui ,1 are T Assuming that mui  xui , yui ,1  the point correspondences on the two images I , I taken by different cameras in the same scene. According to epipolar (...truncated)