Dust and scratch removal (in post processing)

well, I constructed it in order to go wrong… :sunglasses:

More specifically, I worked in another life over a decade on human and animal vision, and precise optical flow is very important part of the computations going on in our brain (in-painting as well). So I kind of knew where to search…

Here are the three base frames of the first example:

First, the base frame before the frame in question:

Now the frame actually analyzed:

and finally, the frame just after the frame analyzed:

Notice something? As usual in hand-held cinematography, especially when only recording with 18 fps, things tend to become blurry occationally. Furthermore, the lettering consists of repeating patterns (sort of). Both things can throw motion-estimation algorithms (optical flow) off the track. And indeed, this is what happened here.

The second example’s original frame is this one:

Comparing with the failure example above, one notices that daVinci’s plugin (and in fact, you are right, I used daVinci to create the examples) did a pretty good job in detecting the outline of the “dust” - only it did fail in the in-filling part. I suspect the main reason is the difficulty to do proper motion-estimation on very detailed structures - the little antenna structure the dust blob obscures is no help here. The new approaches typically utlize “intelligent” segmentation techniques to do this better, albeit at the cost of increased processing times…

If you look closely at the first example given above, there is a tiny scratch close to the cockpit of the plane. There are quite a few different types of scratches and dust and other things which show up in old footage. For fun, have a look here for some other examples of one might encounter when scanning old film…

1 Like