Smarter image resizing

A bunch of us around the office were talking about the cool content aware image resizing algorithm that we saw on Doug’s blog.

The idea behind Shai Avidan and Ariel Shamir‘s work is to find information-neutral “seams” in the image and fold or expand the image along those constantly shifting fault lines. Unlike traditional methods for resizing images, “important” areas don’t become smaller at the same rate as low information areas. As a result, things can move around a bit in the resized images. Take a look at the video above to see this in action, or read the paper for information about how it works.

A seam is a connected path of low energy pixels in an image. On the left is the original image with one horizontal and one vertical seam. In the middle the energy function used in this example is shown (the magnitude of the gradient), along with the vertical and horizontal path maps used to calculate the seams. By automatically carving out seams to reduce image size, and inserting seams to extend it, we achieve content-aware resizing. The example on the top right shows our result of extending in one dimension and reducing in the other, compared to standard scaling on the bottom right.

Personally, I think this is another step in our cultural evolution with respect to imagery. First we believed every photograph represented an actual event that was faithfully transcribed. Photography was an optical-mechanical process of transcription, according to its earliest practitioners (and detractors). Then mid-20th century we realized imagery could be manipulated in order to entertain, mislead, or manipulate us; but we still more or less believed that images were inherently truthful. After another half-century we’re still coming around to the fact that images are surfaces that we project our thoughts and feeling onto and, as a result, must be treated rather skeptically.

Image manipulation techniques such as this — which change image content in a way that moves around visual elements automatically while attempting to retain the information within a scene — may finally highlight some latent connections between our mind’s images and those that are recorded. We’re constantly evaluating the content of scenes and unconsciously throwing out most of the “uninteresting” information, transforming the world’s “truthiness” along the way. Recorded images have no more inherent truth than people’s faulty memories do.

This entry was posted in Color and Vision, Fodder for Techno-weenies, Life Lessons, Photography. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>