Photometric stereo for strong specular highlights

Computational Visual Media, Feb 2018

Photometric stereo is a fundamental technique in computer vision known to produce 3D shape with high accuracy. It uses several input images of a static scene taken from one and the same camera position but under varying illumination. The vast majority of studies in this 3D reconstruction method assume orthographic projection for the camera model. In addition, they mainly use the Lambertian reflectance model as the way that light scatters at surfaces. Thus, providing reliable photometric stereo results from real world objects still remains a challenging task. We address 3D reconstruction by use of a more realistic set of assumptions, combining for the first time the complete Blinn–Phong reflectance model and perspective projection. Furthermore, we compare two different methods of incorporating the perspective projection into our model. Experiments are performed on both synthetic and real world images; the latter do not benefit from laboratory conditions. The results show the high potential of our method even for complex real world applications such as medical endoscopy images which may include many specular highlights.

A PDF file should load here. If you do not see its contents the file may be temporarily unavailable at the journal website or you do not have a PDF plug-in installed and enabled in your browser.

Alternatively, you can download the file locally and open with any standalone PDF reader:

https://link.springer.com/content/pdf/10.1007%2Fs41095-017-0101-9.pdf

Photometric stereo for strong specular highlights

Photometric stereo for strong specular highlights Maryam Khanian Ali Sharifi Boroujerdi Michael Breuß Photometric stereo is a fundamental technique in computer vision known to produce 3D shape with high accuracy. It uses several input images of a static scene taken from one and the same camera position but under varying illumination. The vast majority of studies in this 3D reconstruction method assume orthographic projection for the camera model. In addition, they mainly use the Lambertian reflectance model as the way that light scatters at surfaces. Thus, providing reliable photometric stereo results from real world objects still remains a challenging task. We address 3D reconstruction by use of a more realistic set of assumptions, combining for the first time the complete Blinn-Phong reflectance model and perspective projection. Furthermore, we compare two different methods of incorporating the perspective projection into our model. Experiments are performed on both synthetic and real world images; the latter do not benefit from laboratory conditions. The results show the high potential of our method even for complex real world applications such as medical endoscopy images which may include many specular highlights. photometric stereo (PS); complete Blinn- Phong model; perspective projection; diffuse reflection; specular reflection - 1.1 Introduction Background Reconstruction of three dimensional (3D) information from 2D images is a classic problem in computer vision. Many approaches exist, as documented by a rich literature and a number of 1 Chair of Applied Mathematics, Brandenburg University of Technology, Cottbus 03046, Germany. E-mail: M. Khanian, ( ); A. S. Boroujerdi, ; M. Breuß, michael.breuss@ b-tu.de. Manuscript received: 2017-09-04; accepted: 2017-12-01 excellent monographs, among which are Refs. [ 1–3 ]. The survey in Ref. [ 4 ] concerns 3D reconstruction methods that are more oriented towards the computer graphics community. Following Ref. [ 3 ], one may distinguish approaches based on the point spread function as in depth from focus or defocus [ 5 ], triangulation-based methods such as stereo vision [ 6 ] or structure from motion [ 7 ], and intensity-based or photometric methods such as shape from shading and photometric stereo [ 1 ]. Generally speaking, specific approaches may be distinguished by the type of image data, the number of acquired input images, and whether the camera or objects in the scene are moving or not. For example, techniques based on specular flow [ 8–10 ] rely on relative motion between a specular object and its environment. Focusing on photometric approaches, as explained by Woodham [ 11 ] and Ihrke et al. [ 4 ], these typically employ a static view point and variations in illumination to obtain the 3D structure. While shape from shading is a photometric technique classically making use of just one input image [ 1 ], photometric stereo (PS) allows reconstruction of a depth map of a static scene from several input images taken from a fixed view point under different illumination conditions. Woodham pioneered PS in 1978 [ 11 ], and further developments were due to Horn et al. [ 12 ]. Woodham derived the underlying image irradiance equation as a relation between the image intensity and the reflectance map. It has been shown that for a Lambertian surface, orientation can be uniquely determined from the resulting appearance variations provided by at least three input images illuminated by single known, non-coplanar light sources [ 13 ]. As it is for instance also recognized in Ref. [ 4 ], most of the later approaches have followed Woodham’s idea and kept two simplifying assumptions. Of particular importance, the first one supposes that the surface reflects the light according to Lambert’s law [ 14 ]. This simple reflectance model can still be a reasonable assumption on certain types of materials, when the scene is composed of matte surfaces, but fails for shiny objects concentrating light distributions. Such surfaces can readily be seen in real world situations. It is quite well proved that a light source illuminating a rough surface, reflects a significant part of the light as described by a non-Lambertian reflectance model [ 15–17 ]. In such models, the intensity of reflected light depends not only on the light direction but also on the viewing angle, and the light is reflected in a mirrorlike way accompanied by a specular lobe. The second assumption in classic PS models is that scene points are projected orthographically during the photographic process. This is a reasonable assumption if objects are far away from the camera, but not if they are close in which the perspective effects grow to be important. The importance of using the perspective projection in such a situation has been demonstrated in the computer vision literature; in the context of photometric methods, let us refer for instance to the work Ref. [ 18 ] where (...truncated)


This is a preview of a remote PDF: https://link.springer.com/content/pdf/10.1007%2Fs41095-017-0101-9.pdf

Maryam Khanian, Ali Sharifi Boroujerdi, Michael Breuß. Photometric stereo for strong specular highlights, Computational Visual Media, 2018, pp. 83-102, Volume 4, Issue 1, DOI: 10.1007/s41095-017-0101-9