Image Engineering: Dietmar Wueller has a great company that makes test equipment, targets, and software. I’ve had the good fortune of working a booth in a trade show next to them (more than once). I trust their expertise completely.

WyoFOTO, LLC: My mother and step-father are photographers and have a little web site of their own.

Trek 2.1 Road Bike: Every month brings a new event to distract me from posting here. In June, it’s my new bike.

Twitter: With millions of users, Twitter doesn’t really need me to get the message out. I think it’s a nice way to follow a small number of people who post links to photography, design, and software engineering articles/blogs/etc. that might interest me. Others use it differently — sometimes I do, too.

Flickr: They don’t need my advertising either.

The Wire: The final three episodes of “The Wire” should arrive in my mailbox today (via Netflix). If you’ve ever been hooked on the show, you’ll know another reason why I’m not posting more.

As usual, I learned something new and talked to a lot of people about imaging and MATLAB. Last year, the new-to-me things concerned image quality for mobile devices and image forensics. This year’s big surprises were 3-D imaging, which apparently is going mainstream. (According to Andrew Woods, the chair of the Stereoscopic Displays and Applications conference, there are already two million televisions in people’s homes which are capable of displaying 3D video.) Every year that I attend, I seem to get a different view of the conference; so I’m cautious to read too much into any given year. But somehow, 3D felt different this time.

The exhibitors behind us were demonstrating their more advanced 3D displays and video capture systems, and the person across from us was selling 3D notecards and prints. So my coworker and I thought we’d get in on the action during the waning hours of the exhibit. We decided to try our hand at making an anaglyph, one of the easier kinds of 3D images to make (along with stereograms, which are a bit lo-fi for MATLAB, after all).

The recipe calls for combining two images of the same scene from slightly different perspectives using a simple function that combines the red channel from the left image and the green and blue channels from the right image. And of course, you need the funny glasses. First, the images that we made with my camera phone:

Left image

Right image

Using some MATLAB code that we found on MATLAB Central, you get the following image:

The anaglyph

(Unfortunately, I can’t help you with the funny glasses. Go get yer own.)

I am told the results are “okay for a cha-cha picture.” I don’t have binocular vision, so what do I know?

Have fun. (And don’t be like the guy who said that the 1.57% of the population with monocular vision aren’t really worth worrying about.)

(Thanks to Steve on Image Processing for the link.)

Before I give you some scanty details about Hooman Mohseni’s work into single-photon detectors, I’ll do what he did in his presentation and discuss rod cells, the eye’s own single-photon detectors.

The rod cells in your eye are almost perfect single-photon detectors. When a photon of light in the visible range strikes a rod cell, it causes a chemical reaction with rhodopsin, a photo pigment. The rhodopsin molecule changes shape, closing an aperture in the rod cell wall. This partially interrupts the flow of charged sodium ions into the rod cell, and an electrical charge builds up. The charge causes an impulse that is transmitted to the rest of the visual system, possibly causing the sensation of seeing light. There are several layers of rhodopsin in the cell, allowing a greater impulse when more photons strike it. (You can read more details about rhodopsin and its paramour retinal if you want.)

Each rod cell can detect a single photon event, and rods are photon number resolving, meaning that the visual system can use the impulse to distinguish whether one photon, two photons or twenty photons struck the rod cell. And the false count rate is very low, approximately one per 100 seconds. And all of this happens at low electricity levels. Pretty cool, huh?

Compare that to imaging sensors like CCDs, where at very low illumination levels noise overwhelms the actual signal. Even very accurate detectors that can detect single photons reliably (such as photo-multiplier tubes) require large amounts of voltage and usually can’t resolve the difference between one or two photons. Signal-to-noise ratios have been pretty bad so far, too.

Prof. Mohseni’s group at Northwestern has been using nanotechnology fabrication techniques to create a “focalized carrier augmented sensor (FOCUS)” which can transform a single photon event into an output of about 1,000 electrons, at very low voltages. Essentially they have produced something like a rod cell.

Their detector works really well, but nanofabrication is hard and slow. They can’t even image the small, tube-like sensors because they are too delicate; for example, atomic force microscopy — a very gentle technique — destroys the tubes. Nevertheless, this may be the future of things like night vision, positron-emission tomography, and so on.

Yasuyo Ichihara of Kogakuin University presented some research that she and her colleagues performed on what many of us call “color blindness” or “color deficiency.” She would prefer that we say these differently sighted folks have “color confusion,” which I’ll probably try since it’s slightly more evocative than “color deficiency.” And from a PC point of view, it may not be nice to stagmatize roughly 5% of the population as deficient, etc.

Anyway, they presented research of how their lab applied color universal design to remake the map and timetables of the Tokyo subway system to be friendlier for people with both protanopia and deuteranopia. It turns out that the color confusion can be quite simply modeled by placing two iso-chromaticy lines on the standard x-y chromaticity diagram. Along those lines, all colors appear to be the same hue; so it’s not too hard to pick colors that are maximally contrasting by avoiding the iso-lines. They picked four colors, which they claim are suitable for many print applications: black, orange-red, bluish-green, and a more pure blue.

Color universal design doesn’t stop at picking the right colors. Adding non-color information — such as letters, underlining, framing, or other symbols — proved to be very useful aids to comprehension, too.

All of this reminded me that awhile back I implemented a color deficiency simulator in MATLAB based on work by Hans Brettel. I’m making these functions available here in the hopes that they’re useful and that you actually use them when designing something colorful. Simply download the protanopia and deuteranopia simulator M-files and the associated data file. (You will need the Image Processing Toolbox, since these functions use rgb2ind.)

Let’s see what the peppers image I created some years ago looks like to people with color confusion:

rgb = imread('peppers.png');
figure; imshow(rgb); title('Normal color vision')
figure; imshow(deuteranopia(rgb)); title('Deuteranopia')
figure; imshow(protanopia(rgb)); title('Protanopia')

I find this topic really interesting for a few reasons. (1) I’m a photographer who believes that all images contain a seed of untruth but feel ambivalent about what that means in our information age where images form the basis of what most people “know.” (2) Practical applications of image processing are always interesting. (3) It’s fresh in my mind, since I recently helped someone show that an Ethiopian passport was a fraud. (Image processing played only a small part — the content and font on the machine readable section didn’t match the international standard — but showing that parts were pasted in digitally helped create a preponderance of evidence.)

Hany Farid of Dartmouth gave the first plenary session: “Digital Forensics”. Prof. Farid started by quoting a science journal editor’s staggering statistic: 20-30% of submitted journal entries need images resubmitted because of “inappropriate image manipulation.” His lab’s work aims to out these digital forgeries. Here are some techniques that you can use to identify likely forgeries.

JPEG quality tables (Q-Tables) — These 8-by-8 tables of quantization values are stored in each JPEG file and used to decompress the images. Something I did not know was that most camera vendors use unique Q-tables and that they frequently change them when they introduce new camera models. Photoshop, on the other hand, has not changed their Q-tables since version 1. So you can extract these values from a file and see if it has been saved by Photoshop, which might hint at manipulation.

Cloning — Partition the image into blocks, do principal component analysis, lexigraphically sort the results, do region growing, and look for similar regions.

Resampling (shrinking, growing, rotating, etc.) — Use statistical correlation to look for simple interpolation between values.

Included objects frequently have a different color filter array (CFA) pattern than the rest of the image, possibly because of resizing or different in-camera decoding. Also you can create a vector field of the chromatic aberration color fringing throughout the image; inserted parts will likely have vectors pointing the wrong direction.

In addition, the human visual system (HVS) doesn’t easily notice subtle differences in lighting or shadows in parts of a composite image, but Farid laid out some techniques for estimating lighting direction. One impressive method involves determining the direction of the light (in 3-space, mind you) by looking at the specular highlights in the eye. (See Micah Kimo Johnson’s thesis for full details.)

During the Q&A, one wag asked the question on many of our minds. Are there tools or techniques available to mask all of these forgery detection techniques? Apparently yes, since Prof. Farid consults with Adobe on issues of making more realistic photo manipulations.

Update — 10 August 2008: You can see a thirteen minute video with Dr. Farid at PBS’s Nova web site. (Thanks to Jeff Tranberry.)

Fish tacos, my friends. That’s what has brought me to this wonderful place. Batter-fried fish on corn tortillas with salsa, finely cut cabbage, and some secret ingredient that Rubio’s puts in their “Pesky” tacos. With tortilla chips and refried beans on the side. Mmm. . .

It would be wrong to say that I came to California for fish tacos, but I was certainly looking forward to it for several months.

No, I’m in San Jose attending the 20th annual joint SPIE/IS&T Electronic Imaging symposium. Sunday I attended nine hours of short courses: “Color Processing and its Characterization for Digital Photography” and “Perceptual Metrics for Image Quality Evaluation.” Today — before the fish tacos — I sat in on a full day of paper presentations spread over the “Human Vision and Electronic Imaging XIII”, “Image Quality and System Performance V”, and “Digital Photography IV” conferences. (Tomorrow: “Color Imaging XIII: Processing, Hardcopy, and Applications” and “Rocky IV”.) I had hoped to see “Inferring illumination direction estimated from disparate sources in paintings: an investigation into Jan Vermeer’s ‘Girl with a Pearl Earring’”, but I was glad that I heard Daniel Tamburrino’s paper “Digital camera workflow for high-dynamic-range images using a model of retinal processing” instead.

(Supposing that I feel bored motivated, I will try to post some of my notes here in the coming days.)

I’ve only been here two and a half days, but I already feel like I’ve done so much. As I advocate in my book How to Get Rich through Petty Cash, I’m taking the opportunity to enjoy myself while on a business trip. If you travel around the country and only see the inside of your hotel room, you’re wasting your time on this earth. You’ve got to get out of the hotel, out of the high-priced bubble that surrounds any place where convention-goers congregate, and out of town if possible.

The trip from Boston on Saturday was one of the nicest cross-country flights I’ve ever had. It was my first time flying Jet Blue — I like the seats but don’t think I made the most of the seat-back amenities — and I was the only person in my row. Looking out my window I saw the white, snow-covered fields, ice-covered ponds and the sluggish rivers of New England give way to a deep shag of clouds over Minnesota and the corrugated origami of South Dakota and Wyoming. Rapid City, which I’ve never liked at ground-level, was a fine, delicate etched glass trophy on the edge of the blue Black Hills, the gateway to my old flame. Were those spiral holes in the ground and the furrows of overburden south of Gillette there last time? Surely those oil wells near Midwest still make the same unearthly bullfrog croaking I remember when we stopped the car to look at the bright smear of the milky way on a cold winter’s night more than a dozen years ago. And there’s the interstate leading to my city, my river, my mountain, my mother, my long-gone adolescent angst. I press my hand to the window. Clouds and snow fill in the depths of the Wind River Range, the last of the Rockies before the great folds in the earth when we enter Utah and then Terra Incognita and Terra Nullius in Nevada. Lake Tahoe, defiant, is not frozen but a deep black, unlike the muddy water covering fields in the Central Valley. And there’s glorious Point Reyes, unbelievably beautiful in the light of a western sun shining through broken clouds. Finally, the Golden Gate and its fabulous bridge.

After getting my rental car, I immediately headed to the SFMoMA, to see the Jeff Wall exhibit, which closed Sunday while I was in my classes. I like many of Jeff Wall’s photographs, but I’m deeply ambivalent about his work in general. First he has a reputation as the most cerebral living photographer, but I often I feel that the nonstop art historical references — less that a quarter of which I doubt I caught — get in the way of making a photograph that’s pleasing to look at. Should we really let folks like Wolfgang Tillmans, Jürgen Teller (NSFW), and Wall — or me for that matter — revel in elevating every ordinary scene and still claim a fig leaf of art historical pretension? Perhaps if I were more of an insider, I would be less ambivalent; but such is the way with me and all modern art. An-My Lê’s photographs from her Small Wars and 29 Palms series were perfect. And the black-and-white and color work of the Silicon Valley from Gabriele Basilico had me amazed and envious. Finally, I have to admit that despite liking monographs better than surveys, and themed exhibitions better than a hodge-podge of recent acquisitions, I liked Picturing Modernity, a hodge-podge survey of photographs from the museums collection, very loosely grouped around a two word title and including several pictures that (too conveniently) would have fit in recent exhibits at the National Gallery of Art (like this one and that one). Surely it was the luscious, large deadpan photos at the exhibit entrance that enticed me to give it a free pass.

Well that’s all that’s new from the other coast. I wish Lisa were here, and I miss having the cat lie upon my lap while I scratch under his chin. But these small prices must be paid by an international playboy with an expense account a conference to attend.

METACOW: “A Public-Domain, HighResolution, Fully-Digital, Noise-Free, Metameric, Extended-Dynamic-Range, Spectral Test Target for Imaging System Analysis and Simulation”. Wow, that’s hot!

Thanks to Steve for the tip.

Digital does have a lot of advantages, though. It provides instant feedback at the point of making the photograph. It’s also hard to beat the speed and flexibility of a digital workflow. And for 35mm-sized sensors, it’s hard to tell the difference from film.

This article mixes the look of film and the enormous possibilities of digital. It also continues the trend of notes about high dynamic range images that I’ve been posting recently.

Color film is an analog tone reproduction operator. It renders a vast range of illumination values to a particular color image. That is, it maps the high dynamic range world to a lower dynamic range representation. The major factors that influence this tone mapping “function” include the exposure settings (shutter time, aperture, and ISO speed), the film response curves, and development technique. A film response curve for Fuji Velvia, the color film I use, appears below. Notice how the curve is more-or-less flat in the highlights and shadows and that it has an S-shaped curve in between.

If you take some liberties with that S-shaped part of response curve where the real detail in the image is, you might almost say that the curve has a flat part for the shadows, a flat part for the highlights, and a line that connects the two.

When I first started puttering around with high dynamic range images, I began by translating what I knew about how film renders a scene with a lot of tones:

• Film is sensitive across a particular exposure range.
• Parts of the image with less than a certain amount of exposure are rendered as black.
• Overexposed parts of image are rendered as white (technically clear).
• You can’t change the film sensitivity (except with some processing tricks that we’ll ignore).
• You can change the amount of light that is recorded during the exposure.

It wasn’t hard for me to combine this information with the generalization about the film response “curve” to produce a very simple tone-renderer which simulates color slide film in MATLAB. You can download the M-file from my page on MATLAB Central, where you can find some other good stuff. (I’ve also reprinted it below.) I ran it on the HDR image of my office, varying the exposure from 0 EV to 12 EV in half-stop increments, and was very glad to see the same kind of results that I do with film.

(Click for larger image…)

But film is so twentieth-century. So I went on to more interesting forms of tone mapping.

function rgbSimulated = simulateFilm(rgbRadiance, nStops)
%simulateFilm   Perform film-like tone mapping.
%   LDR = simulateFilm(HDR, middleEV) converts the floating-point high
%   dynamic range image HDR to a UINT8 low dynamic range image LDR using a
%   method that recreates the sensitivity of film.  The middleEV value sets
%   the middle exposure value (EV) for the rendering.  Larger EV numbers
%   will generate brighter images; smaller values of middleEV will result
%   in darker images.
%
%   The film sensitivity is set to 6 EV, which is comparable to many
%   late-model transparency (positive) films.
%
%   Example
%   -------
%
%   Render the same HDR image using different "exposure settings."
%   Essentially, simulate exposing the same scene from 0 to 12 EV in 1/2
%   stop steps.
%
%      % Read the HDR image and create a buffer for the rendered images.
%      hdr = hdrread('office.hdr');
%      s = size(hdr);
%      ldr = ones(s(1), s(2), 3, 25, 'uint8');
%
%      % Perform the tone mapping.
%      for p = 0.5:0.5:12.5
%          ldr(:,:,:,p*2) = simulateFilm(hdr, p);
%      end
%      figure; montage(ldr)

% Author: Jeff Mather
% Copyright 2006-2007 The MathWorks, Inc.

% Set the film sensitivity and the mid-tone log-radiance.
sensitivity = 6;
midPoint = 3;

minLogExposure = midPoint - sensitivity/2;
maxLogExposure = midPoint + sensitivity/2;

% Film works in stops.  Convert radiance to base 2 to compute perception.
% Values that are outside the film sensitivity are lost (min or max).
rgbRadiance = rgbRadiance * 2^(nStops);

rgbSimulated = rgbRadiance;
rgbSimulated(rgbRadiance ~= 0) = log2(rgbRadiance(rgbRadiance ~= 0));
rgbSimulated(rgbSimulated < minLogExposure) = minLogExposure;
rgbSimulated(rgbSimulated > maxLogExposure) = maxLogExposure;

% Convert to RGB.
rgbSimulated = uint8(255 * (rgbSimulated - minLogExposure) ./ ...
                     sensitivity);