Project Description

This is a project on photometric stereo reconstruction. An object is observed by a fixed camera under different illumination. So we have a dense set of images to start with. The challenge is to infer a 2.5D surface description of the object (that is, a depth model), despite that the captured data are severely contaminated by shadows, highlights, transparency and that the light calibration is inaccurate. The steps of the project are:

  • Initial normal estimation
  • Normal refinement by MRF using graph cuts
  • Depth reconstruction by shape-from-shapelets

    Inputs

    The inputs of photometric stereo reconstruction are a set of images with same camera setting but different illumination angles. A set of input is shown below:

    Outputs

    The output of photometric stereo reconstruction is a 2.5D surface, a image with depth information. In our project, we get following outputs with different input datasets.

    Input example Output Output with texture rendered

    The viewworld model can be download here

    Notes

    There are some experience we learned from this project.

  • The denominator image can be selected either in the raw images or normalized images. However, if you select it from normalized images, you get a smoother result.

  • In the estimation procedure, the SVD cannot ganrantee the z of normal is positive, which is mandatory in the real case. Therefor you have to flip the normal to get a positive z.

  • After estimation, there are quite some noise points in the image, which is caused by the dark points. Because the pixel value, which is going to be denominated, is close to zero and easily to generate great error. You have to design a filter to filt them out.

  • When plotting the normals, carefully design the program to convert normal vectors to slant-tilt tuples.

    People

  • Bin XU,
    HK University of Science and Technology, Fok Ying Tung Graduate School

  • Haiyan YANG,
    HK University of Science and Technology, Dept. ECE

    Useful Links

  • COMP 5421 course website
  • COMP 5421 project4 website