Photometric stereo for strong specular highlights
Photometric stereo for strong specular highlights
Maryam Khanian
Ali Sharifi Boroujerdi
Michael Breuß
Photometric stereo is a fundamental technique in computer vision known to produce 3D shape with high accuracy. It uses several input images of a static scene taken from one and the same camera position but under varying illumination. The vast majority of studies in this 3D reconstruction method assume orthographic projection for the camera model. In addition, they mainly use the Lambertian reflectance model as the way that light scatters at surfaces. Thus, providing reliable photometric stereo results from real world objects still remains a challenging task. We address 3D reconstruction by use of a more realistic set of assumptions, combining for the first time the complete Blinn-Phong reflectance model and perspective projection. Furthermore, we compare two different methods of incorporating the perspective projection into our model. Experiments are performed on both synthetic and real world images; the latter do not benefit from laboratory conditions. The results show the high potential of our method even for complex real world applications such as medical endoscopy images which may include many specular highlights.
photometric stereo (PS); complete Blinn- Phong model; perspective projection; diffuse reflection; specular reflection
-
1.1
Introduction
Background
Reconstruction of three dimensional (3D)
information from 2D images is a classic problem
in computer vision. Many approaches exist, as
documented by a rich literature and a number of
1 Chair of Applied Mathematics, Brandenburg University
of Technology, Cottbus 03046, Germany. E-mail: M.
Khanian, ( ); A. S.
Boroujerdi, ; M. Breuß, michael.breuss@
b-tu.de.
Manuscript received: 2017-09-04; accepted: 2017-12-01
excellent monographs, among which are Refs. [
1–3
].
The survey in Ref. [
4
] concerns 3D reconstruction
methods that are more oriented towards the
computer graphics community. Following Ref. [
3
],
one may distinguish approaches based on the point
spread function as in depth from focus or defocus [
5
],
triangulation-based methods such as stereo vision [
6
]
or structure from motion [
7
], and intensity-based or
photometric methods such as shape from shading
and photometric stereo [
1
]. Generally speaking,
specific approaches may be distinguished by the
type of image data, the number of acquired input
images, and whether the camera or objects in the
scene are moving or not. For example, techniques
based on specular flow [
8–10
] rely on relative motion
between a specular object and its environment.
Focusing on photometric approaches, as explained
by Woodham [
11
] and Ihrke et al. [
4
], these
typically employ a static view point and variations in
illumination to obtain the 3D structure. While shape
from shading is a photometric technique classically
making use of just one input image [
1
], photometric
stereo (PS) allows reconstruction of a depth map
of a static scene from several input images taken
from a fixed view point under different illumination
conditions. Woodham pioneered PS in 1978 [
11
], and
further developments were due to Horn et al. [
12
].
Woodham derived the underlying image irradiance
equation as a relation between the image intensity
and the reflectance map. It has been shown that for
a Lambertian surface, orientation can be uniquely
determined from the resulting appearance variations
provided by at least three input images illuminated
by single known, non-coplanar light sources [
13
].
As it is for instance also recognized in Ref. [
4
], most
of the later approaches have followed Woodham’s
idea and kept two simplifying assumptions. Of
particular importance, the first one supposes that
the surface reflects the light according to Lambert’s
law [
14
]. This simple reflectance model can still
be a reasonable assumption on certain types of
materials, when the scene is composed of matte
surfaces, but fails for shiny objects concentrating
light distributions. Such surfaces can readily be
seen in real world situations. It is quite well proved
that a light source illuminating a rough surface,
reflects a significant part of the light as described
by a non-Lambertian reflectance model [
15–17
]. In
such models, the intensity of reflected light depends
not only on the light direction but also on the
viewing angle, and the light is reflected in a
mirrorlike way accompanied by a specular lobe. The
second assumption in classic PS models is that
scene points are projected orthographically during
the photographic process. This is a reasonable
assumption if objects are far away from the camera,
but not if they are close in which the perspective
effects grow to be important. The importance of
using the perspective projection in such a situation
has been demonstrated in the computer vision
literature; in the context of photometric methods,
let us refer for instance to the work Ref. [
18
] where
(...truncated)