Motion Inpainting on Dynamic Backgrounds with Large Object Removal


Timothy K. Shih, Joseph C. Tsai, Shwu-Huey Yen and Kanoksak Wattanachote



Abstract—Video inpainting technologies use special and temporal properties to recover lost portion of continuous area. Example applications include object removal for special effects, defect recovering in aged movies, and the recent 3D inpainting techniques for 3D videos. One of the difficult problems not yet completely solved is to remove objects or to recover lost defects in dynamic backgrounds such as fire, smoke, or waterfall. The difficult issue is to maintain a visual continuity, by using motion gestures. This article demonstrates a motion inpainting mechanism, to recover the lost of areas/objects in dynamic background, with visual pleasant results as well as quantitative evaluations. The mechanism needs to search for video clips from a set of irregular shaped frames, called patch streams. If the duration of a found patch stream is shorter than a target background video, the patch stream will be extended by our algorithm inspired by seam carving. Finally, the inpainted video is evaluated with a newly proposed motion coherence metric.


Index Terms—Motion Analysis, Motion Inpainting, Object Removal, Video Inpainting


——————————   <<>>   ——————————


(a) S1: original video

(b) S1: inpainting result with waterfall added

(c) S2: original video

(d) S2: inpainting result with bear removed

(e) S3: original video

(f) S3: inpainting result with additional smoke on the right

(g) S4: original video

(h) S4: inpainting result with a bigger eruption

(i) S5: original video

(j) S5: inpainting result to remove the person

(k) S6: original video

(l) S6: inpainting result with a larger waterfall

(m) S7: original video

(n) S7: inpainting result with a tree branch removed

(o) S8: original video

(p) S8: inpainting result with additional waterfall

(q) S9: original video

(r) S9: inpainting result with additional waterfall

(s) S10: original video

(t) S10: inpainting result with more fire on the left

(u) S11: original video

(v) S11: inpainting result with additional smoke

(w) S12: original video

(x) S12: inpainting result with additional smoke

(y) S13: original video

(z) S13: inpainting result with one more window on fire

(a) S14: original video

(b) S14: inpainting result with a bigger eruption