Video inpainting methods are often required in industries such as
movie and TV production. For instance, when unwanted objects like
pedestrians or cars appear in the scene, or when crew members that are
required for the shot need to be removed in post-processing. However,
there are no robust video inpainting methods that can handle arbitrary
motions of objects and camera, let alone be applicable to high
resolution videos used in the industry. Additionally, in the
interaction of our research group with the movie industry, we learned
that the removal of unwanted objects from videos is still performed by
artists in a frame-by-frame basis.
To address the general video inpainting problem, I take advantage of
the redundancy found in video sequences to restore the scene behind
the unwanted objects: Objects are removed by reusing other views of
them that are available in other video frames.
We proposed two automatic video inpainting methods that follow this
principle: One for removing of objects that occlude other dynamic
objects from videos taken with static cameras, and one for removing
objects that occlude other static objects from videos taken with
dynamic cameras.
The inpaint result is produced by enforcing local consistency
constraints using a global optimizer (graph cuts).
In the experimental evaluation, we found that high quality inpaintings
can be produced whenever there is enough redundancy in the video, even
for high resolution videos.
Our hope is that, as automatic methods for video inpainting become
faster and more reliable, this type of editing tasks can be made less
time consuming for the users, and finally become widely available to
the general public.