State-of-the-Art Computer Vision Technologies

Content-aware image resizing (also called content-aware image retargeting, content-aware image scaling) is to enlarge or reduce the size of an image with minimum distortion of salient or important areas of the image. This is a very useful image processing feature for professional photography, online publishing and handheld-device display.

The technology is called “Seam Carving“, i.e., based on the evaluation of high importance pixels (people, buildings, etc.) and other pixels, the algorithm(graph cut) detects seams (continuous lines of pixels joining opposite edges of the image) which have the lowest importance, and deletes those pixels or add interpolated pixels along the lowest importance seam. The image operations are done in gradient domain. The following video gives more details of the technology.

Later on, the authors extend the image retargeting work to videos, with another SIGGRAPH paper (M. Rubinstein, A. Shamir and S. Avidan, Improved Seam Carving for Video Retargeting, SIGGRAPH 2008) and the following demo video.

Will video retargeting go to iPhone or Adobe’s After Effects? From the computation point of view, it is unlikely this application will go to iPhone in the near future, due to iPhone’s low processing power, but it is possible that video retargeting will appear in the next release of Adobe’s After Effects.