[Numpy-discussion] Help with interpolating missing values from a 3D scanner

[Scott Sinclair]

> 2009/1/16 Robert Kern <robert.kern at gmail.com>:
> On Thu, Jan 15, 2009 at 16:55, David Bolme <bolme1234 at comcast.net> wrote:
>>
>> I am working on a face recognition using 3D data from a special 3D
>> imaging system.  For those interested the data comes from the FRGC
>> 2004 dataset.  The problem I am having is that for some pixels the
>> scanner fails to capture depth information.  The result is that the
>> image has missing values.  There are small regions on the face such as
>> eyebrows and eyes that are missing the depth information.  I would
>> like to fill in these region by interpolating from nearby pixels but I
>> am not sure of the best way to do that.
>
> Another approach (that you would have to code yourself) is to take a
> Gaussian smoothing kernel of an appropriate size, center it over each
> missing pixel, then average the known pixels under the kernel using
> the kernel as a weighting factor. Place that average value into the
> missing pixel. This is actually fairly similar to the Rbf method
> above, but will probably be more efficient since you know that the
> points are all gridded.

You might try using Rbf with a window of known pixels centred on your
missing pixels. You'll automatically get a smoothing kernel that
weights nearer known pixel values more heavily, the behaviour of the
kernel depends on the basis function you choose (so it's similar to
the Gaussian smoothing idea). The reason for using a window is
efficiency, Rbf will be grossly inefficient if you feed it all of the
known pixels in your image as known values. Using a window will gain
efficiency without significantly changing your result because very
distant known pixel values contribute little to the result anyway.

The iteration of image inpainting also sounds like a useful extension.

Cheers,
Scott