Congruence analysis of point clouds from unstable stereo image sequences

The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XL-5, 2014 ISPRS Technical Commission V Symposium, 23 – 25 June 2014, Riva del Garda, Italy CONGRUENCE ANALYSIS OF POINT CLOUDS FROM UNSTABLE STEREO IMAGE SEQUENCES Christian Jepping, Folkmar Bethmann, Thomas Luhmann Jade University of Applied Sciences, Institute for Applied Photogrammetry and Geoinformatics, Oldenburg, Germany ; ; Commission V, WG V/1 KEY WORDS: stereo image matching, 3D point tracking, congruence analysis ABSTRACT: This paper deals with the correction of exterior orientation parameters of stereo image sequences over deformed free-form surfaces without control points. Such imaging situation can occur, for example, during photogrammetric car crash test recordings where onboard high-speed stereo cameras are used to measure 3D surfaces. As a result of such measurements 3D point clouds of deformed surfaces are generated for a complete stereo sequence. The first objective of this research focusses on the development and investigation of methods for the detection of corresponding spatial and temporal tie points within the stereo image sequences (by stereo image matching and 3D point tracking) that are robust enough for a reliable handling of occlusions and other disturbances that may occur. The second objective of this research is the analysis of object deformations in order to detect stable areas (congruence analysis). For this purpose a RANSAC-based method for congruence analysis has been developed. This process is based on the sequential transformation of randomly selected point groups from one epoch to another by using a 3D similarity transformation. The paper gives a detailed description of the congruence analysis. The approach has been tested successfully on synthetic and real image data. 1. INTRODUCTION 1.1 Application This research is allocated to a special problem in car safety testing. Real crash tests and numerical crash simulations are used to investigate the behaviour of a car during heavy impacts of mechanical forces in order to evaluate the potential of injuries for involved persons (e.g. driver, pedestrians) and to develop efficient approaches to maximize safety. A number of different sensors and systems are used to measure different effects during the crash test. Photogrammetric high-speed camera systems are used to record the geometric behaviour of different object, for instance the 3D trajectories of dummy bodies or the surface of dynamically deformed car parts. The results of photogrammetric measurements are subsequently used for the verification and validation of finite-element simulations (FE) which are developed for every car type. These numerical models are successively improved by data from real experiments (Raguse et al., 2004). For this verification process it is required to provide data fusion of photogrammetric results with data from other sensors. Hence 3D data has to be transformed into the global coordinate system of the car XF, YF, ZF, see Figure 1. If calibrated stereo or multicamera systems are used that observe the car from an external viewpoint, the transformation can be provided by given control points that are targeted on stable parts of the car body. In contrast, using on-board camera systems that are observing only partially stable object areas, this approach cannot be applied. In addition, due to the massive forces during crash it cannot be assumed that the interior orientation of on-board cameras stays constant, not even for high-stability camera bodies. Figure 1. Possible camera- and object-movements, and object deformations between time t0 and time tn This contribution has been peer-reviewed. doi:10.5194/isprsarchives-XL-5-301-2014 301 The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XL-5, 2014 ISPRS Technical Commission V Symposium, 23 – 25 June 2014, Riva del Garda, Italy With respect to photogrammetric aspects and under consideration of the above described problem of data fusion, the following technical problems can be identified (Figure 1):     The relative orientation of cameras (in system XM, YM, ZM) will change from epoch to epoch (i.e. with each frame). The observed object area will be deformed; reference points with pre-defined coordinates XF, YF, ZF are not valid anymore; hence, calculation of new exterior camera orientation with respect to the coordinate system XF, YF, ZF is not possible any longer. The spatial transformation of the system XM, YM, ZM with respect to the car coordinate system XF, YF, ZF will change from epoch to epoch. The parameters of interior orientation can be changed. This paper will discuss some approaches for the solution of the above mentioned issues. The main part of the research deals with a RANSAC-based method for the detection of stable object areas (congruence analysis, see section 3) which allows for the successive re-orientation of camera system XM, YM, ZM relative to the object surface. Congruence analysis requires appropriate data which is described in the following section. 2. DATA Within the following sections the generation of the input data for congruence analysis is described. Section 2.1 presents the approach for stereo matching and 3D point tracking. Section 2.2 explains the necessity for re-calculation of the relative orientation of the cameras for each time step and gives a schematic description of the information within the 3D trajectories of the object points which are subsequently used for the congruence analysis. 2.1 Stereo matching and 3D point tracking For congruence analysis (section 3) the spatial trajectories of a high number of 3D points are required which lie on the observed object surface. For this purpose the software package PISA (Photogrammetric Image Sequence Analysis) is used that has been developed at IAPG. PISA allows for the extraction of a dense 3D point cloud starting in epoch 0 for a well-defined object area with sufficient texture. Image-based point matching is based on extended least-squares matching (Bethmann et al., 2009). In addition, each point can be tracked into the next epoch by robust tracking to give 3D point trajectories through the complete sequence. The tracking algorithm uses stereomatching local window-based matching techniques (normalized cross-correlation (NCC) and least-squares matching (LSM)) for the tracking of image points. Since in crash-test environments, and especially for on-board scenarios, a lot of disturbances (e.g. occlusions, dust, changing light reflection on the object surface and so on) may occur, the tracking algorithm should be able to handle these disturbances as robust as possible. Therefore, an algorithmic solution has been developed which focusses on integrating all available (stereo-) information as extensive as possible. Figure 2 illustrates the procedure of 3D point tracking exemplarily for the tracking of one point. (...truncated)