You can divide the world of image file formats into different kingdoms based on the their structure. There is some overlap between these categories, but for the most part image formats are (1) tag/record-based, (2) structure-like, (3) marker/stream-based, (4) textual, (5) card-like, (6) raw, or (7) opaque.

TIFF, DNG, and DICOM are examples of tag/record-based formats. A unique tag identifies the entity in the file and its meaning. For example, a particular hexadecimal tag might indicate that this is the “photometric interpretation” record. The datatype of this record either explicitly appears after the tag or appears in a data dictionary that’s known to the application developer. Almost always, these records explicitly tell the length of their data, which makes it easy to skip to the tag location of the next record.

Microsoft was (for a time) very fond of making structure-like formats. In these formats, the file looks a lot like the memory representation of a C/C++ data structure. These formats are easy to describe and easy to read if you have the structure definition; simply fread() the data into a variable and reference the data members by name. The problems should be pretty clear. You need to be using a programming language that supports C structs. And you need to know the layout of the struct. And once you define the layout of the struct, it’s fixed. (Well, not exactly. Microsoft changed the data layout in its BMP family of formats with every release of Windows, and used a “magic” value to tell readers which struct to use.) All told, it’s a very brittle kind of format.

JPEG is the prototypical — but certainly not the only — marker-based format. Markers are special combinations of bytes that, like a tag, tell what the data is that’s coming next in the stream. But, very much like struct-based formats and very unlike tagged formats, the data that follows the marker can be heterogeneous. In JPEG, the data that appears after the SOF (Start of Frame) marker is a record, while the data that follows an RSTn marker is just a stream of compressed bytes. The SOI and EOI (Start/End of image) markers don’t even have any bytes that follow them. In marker-based formats, semantics and syntax are rather carelessly jumbled together.

It’s very difficult to quickly parse marker-based formats, because often markers don’t specify how much data appears before the next marker. These are very much “streams” of bytes that you’re forced to read until you come to the next marker. Consequently the number and appearance of markers is very limited and this limitation ripples through to the data that they contain. JPEG markers all begin with the 0xFF byte followed by another byte, which taken together specify which marker it is. Consequently, the appearance of an 0xFF byte in the data of a marker has to be escaped by a NULL byte so that it’s not mistaken for the next marker.

Textual formats, such as XML, have the benefit of being self-describing and readable by both humans and machines. Their main drawbacks are the inflated size of the data they contain (even when represented in a semi-binary CDATA hunk) and the inability to quickly skip through them with binary I/O routines.

FITS is a fairly prototypical “card-like” format. As the name implies, these are fixed-length records like one might have encountered on a punch card. For example in format with 120-character records, the first n characters are reserved for the “variable name” part of the equation, while the remaining 120-n characters are the textual representation of the value of the record. They are frequently text-only for the descriptive part of the format with a binary payload at the end. These are easy to read, but a pain to parse, since the “right hand side” values often have to be interpreted.

Raw and opaque formats aren’t very easy to describe because they’re so varied. In a “raw” format (and there are dozens or hundreds . . . possibly more) all of the bytes are jumbled together in a payload-only file. A separate file may have a header that describes the data and helps a reader/parser make sense of the payload. Or not. These are almost always completely free of any helpful description within the file.

This shouldn’t be confused with opaque files, such as HDF, CDF, or netCDF. These formats are completely defined by their API, which for all intents and purposes, you have to use to access the data within the file. This allows for a lot of richness in handling the data contents, which can be organized in highly optimized ways. The downside is that you’re limited in how you can interact with your data to mechanisms someone else has defined. And data permanence can suffer, since if the tool chain changes (or goes out of existence) you don’t really have a way to get at your data.

Practically, each format style has it’s pros and cons. But tagged formats (which might incorporate features of the record style) are the most durable and easiest for third-parties to work with.

* — Cue awesome “CSI” + “Law and Order” + “House” mashup daydream. *DOINK DOINK*

My naïve idea was to copy the JPEG-encoded to a temporary file and then read that file using the Independent JPEG Group’s libjpeg library. That’s what I did with JPEG data encapsulated in DICOM. This is far too simple an approach for NITF, resulting in incomplete images. Here’s why:

• NITF breaks most images into multiple tiles.
• Each tile is independently compressed into its own image stream.
• NITF uses “block masking,” which prevents storing unimportant tiles.

The idea makes sense on one level. If you’re going to send an image over a low-bandwidth or low-fidelity channel, you want to limit the amount of data that you send, and you want to avoid an all-or-nothing situation during image transmission or reception. But it’s a total pain in the ass for application developers.

Add to this the fact that JPEG is a marker-based format that isn’t very self-describing, and you have a tricky parsing situation.*

Here’s the basic idea behind getting imagery out of a NITF file if it’s been JPEG compressed. (I assume that you already know how to parse a NITF file — see MIL-STD-188-198A if you don’t — and that you have a JPEG codec that you can use to decode the data.)

1. The first two bytes of the compressed stream should be the standard JPEG SOI marker (0xFF 0xD8). This is your sanity check.
2. The next two bytes should be the APP6 marker (0xFF 0xE6). The payload of this marker contains a bunch of useful information about tile sizes and counts, bit depths, etc. Some of this is redundant with what’s inside the NITF file.
3. The remainder of the NITF file should be a bunch of JPEG codestreams delimited by SOI and EOI (0xFF 0xD9) markers. Each delimited stream is one tile in the image; and it’s a completely standalone JPEG stream. It can be extracted to its own file (if necessary) and decompressed. Tiles are stored across the image horizontally and then down.
4. If there’s no block masking, it suffices to read each tile in turn and store it in the appropriate region in the output image.
5. If the NITF file does use block masking, use the values in the BMRnBNDm attribute of the image subheader to find the locations of the blocks that contain actual image data. The masked out blocks will have 0xFFFFFFFF values. The other values — there’s one for each tile — are 0-based offsets pointing to the SOI marker that starts each tile, relative to the start of the JPEG compressed data.

And that’s it. After coding, you should probably test out your parser on the sample NITF files provided by the NGA.

* – You can divide the world of image file formats into different buckets based on the their structure. There is some overlap between these categories, but for the most part image formats are (1) tag/record-based, (2) structure-like, (3) marker/stream-based, (4) textual, (5) card-like, (6) raw, or (7) opaque. I’m going to write more about this in the next post.

It’s time to clear some of them out. Here are some daytime-themed links. (If I had a Tumblr account, I’d just post there. But I don’t need another website no one reads.)

Parallel Computing
James Reinders of Intel TBB fame estimates that datasets (images, videos, etc.) have grown 10x larger over the last five years. Sequential systems are just too dang slow to process this amount of data. In the near future, “parallel programming” will just become “programming.” (I’m still hoping for better language support so that state synchronization and multicore memory issues are as easy to get right as the sequential aspects of programming are now.)

New memory models would certainly help. A new paper, DMP: Deterministic Shared Memory Multiprocessing by Joseph Devietti, Brandon Lucia, Luis Ceze, and Mark Oskin presents some of the problems with the current memory model and provides one possible solution.

Another post on SoftTalk (sponsored by Intel) describes some of the ways that Intel plans to make parallel programming easier this year: Parallel Studio 2010, a Cilk-based offering for task parallelism, “a data-parallel centric model with safety guarantee,” new SIMD instructions, and new array notations.

You might also want a high-level view of how Intel’s offerings work together.

Herb Sutter, who really knows his stuff, wrote an article for Dr. Dobbs a couple years ago about understanding parallel performance. It’s one of a series of articles about multicore/multithreaded programming, and this helps set expectations about what’s possible and gives pointers on where to start making changes.

Miscellaneous
Visual Studio + Time Machine = IntelliTrace. Don’t just move up and down the call stack; now you can move forward and backward in time, too.

Everything you ever wanted to know about floating-point representation: Floating Point Guide. (Everything, that is, unless you work where I do. Then you just have to go down the hall to get that last 2%. Of course, you’ll be drinking from the fire hose. . . .)

What does Microsoft think are the key trends in software development? Cloud computing, the web as a platform, parallel computing, proliferation of devices (with their own capabilities, IO paradigms, etc.), agile development processes, distributed development. Nothing terribly futuristic here, and comments want to know why “mobility” (i.e., phones) isn’t on the list.

The C++0x “final draft” revision is ready for “final” comments. Here’s your chance to see the significant changes to C++ that will be part of the standard next year (they hope).

But our checklists have gotten a little long and messy, which raises the risk that people won’t use them at all. Part of my job is to improve our team best practices and checklists, so I’m working out how to make all of those checklist-bound countermeasures fresher and more accessible.

So I’m very hopeful that Atul Gawande’s The Checklist Manifesto: How to Get Things Right, which arrived on my desk today, will give me some in-the-trenches perspective. I’ll keep y’all posted on what I learn.

Query the file /proc/sys/kernel/random/uuid for a globally unique value. It produces a new value on each read. For a globally unique value that’s persistent between machine startup, query /proc/sys/kernel/random/boot_id.

[earth:/]120 % cat /proc/sys/kernel/random/uuid
78345a49-875f-48d3-aa80-130db5e57991

[earth:/]121 % cat /proc/sys/kernel/random/uuid
d258123d-f2bd-49c0-89d6-e4db5d377776

[earth:/]122 % cat /proc/sys/kernel/random/boot_id
e58a255a-0f6c-418f-ac0b-0fac81a5ff4a

[earth:/]123 % cat /proc/sys/kernel/random/boot_id
e58a255a-0f6c-418f-ac0b-0fac81a5ff4a

Several years ago I read Greenspun’s Tenth Rule of Programming: “Any sufficiently complicated C or Fortran program contains an ad-hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp.” Basically it’s a joke about feature creep in software. I was reminded of this quotation today when I started reading Joshua Kerievsky’s Refactoring to Patterns. He describes the perils of over-engineering, which is basically the same as wasting money.

You see, about seven or eight years ago, I added a Lisp parser to a feature in MATLAB’s Image Processing Toolbox. If you have the toolbox, just take a look in toolbox/images/iptformats/private/dicom_create_IOD.m. There you’ll see a part of the code that looks like this:

function tf = check_condition(expr, metadata)
%CHECK_CONDITION  Determine whether a particular condition is true.
%
%   Conditions are LISP-style cell arrays.  The first element is a
%   conditional operator, remaining cells are arguments to the operator.
%
%   Conditions can be nested.  Each cell array in expr indicates a new
%   conditional expression.

[snip]

%
% Process conditional expressions recursively.
%
switch (lower(operator))
case 'and'

    % This AND short circuits.
    for p = 1:numel(operands)
        tf = check_condition(operands{p}, metadata);

        if (~tf)
            return
        end
    end

[snip]

So why did I do this? The DICOM file format has conditional elements, which only show up in the file if other elements are present, are absent, or have a particular value. I wanted to create a general-purpose, reusable, extensible mechanism for describing these conditions so that I didn’t actually have to write code for all of those conditionals. Moreover, I wanted our customers to be able to write the conditions in a nice tabular form rather than writing code, because — you know — one day they would want to write more of the dozens of kinds of DICOM information objects from scratch than we supported right out of the box.

You probably see where this is going. We ended up extending the product in a different direction after we learned more about how our customers actually wanted to use the DICOM export functionality. So, if you need a Lisp-like algebraic Boolean logic parser in MATLAB, just go look in the previously mentioned file. (Also take a look at the dicom_modules.m file in the same directory to see how to express those conditions.) But for g-d’s sake, ask yourself if you really, really need that, or if you aren’t just fooling yourself into over-engineering your code.

AMD Performance counter resources:

• Doc #26094, Chapter 10 (very detailed!)
• Doc #22007, Appendix D (part of the Optimization Guide)
• Doc #24593, Chapter 13 (part of the AMD64 Architecture Programmer’s Manual)
• Doc #41256, Section 3.14 (somewhat older, but still good)
• OProfile documentation (a list of CPU performance events)

Less techie posts to follow . . . honestly.

In January, I went to the Electronic Imaging symposium in San Jose, promised to write about what I learned and then did not. In February, I went on a “business trip” to Cozumel — via cruise ship — and then posted some of the photos I made on my Holga camera. Then in March, Lisa and I went to Paris to celebrate my graduation. I could talk your ear off all day about how much I love Paris; and if you’re Lisa, you’re probably sick of hearing me walk around the house sighing and saying “I miss Paris.” (Sorry!) Most recently, I promised to post pictures from my backpacking/camping/hiking trip to Utah. I did put some of them into a photoset on Flickr, but I haven’t gotten them here yet. Tomorrow I’m off to Iowa for a family reunion.

“But what about the other 15-or-so weeks of the year? Why no posts?”

It isn’t that I haven’t had any ideas. Here are some of the things I jotted down that I should write about:

Physical architecture vs. software architecture, with respect to the visibility and communication of design — I actually started writing a bit about this after some lengthy conversations with coworkers. Basically, I yearn for the day when software engineers can have books like this, this, and this. We need a shared visual grammar, shared materials, and shared processes so that we too can have coffee-table books that inspire our cohort.

A response to Jeff Atwood on (not) reinventing the wheel again . . . and again — Given what I just wrote above, it’s no surprise that I would be disappointed after reading an article that advocates waste. I hope one day to live in a software engineering world which is more like a real engineering world, a world in which there are two kinds of software businesses: (1) the businesses where almost everyone buys off-the-shelf or special-order parts and uses them to build incredible things, innovating very broadly and (2) the businesses that specialize in making those off-the-shelf and special-order parts, making them well, and innovating very deeply.

Design blogs — I’ve been making a little list of design blogs that I like. . . . I’m a total tease, though, so I’ll share them later.

My notebooks — One of those afore(un)mentioned design web sites has an occasional series looking inside designers’ notebooks. (For example.) I keep most of my thoughts (and some designs) worth remembering in journals and took some beauty shots of them. I’ve actually downloaded the photos off my camera into Lightroom.

French music — The French are different. Especially in their musical tastes. I like some of it and thought about shilling for folks like Mylène Farmer, Mara Tremblay, A.D.N., Abd al Malik, Karna, MC Solaar, Cœur de Pirate, Alain Bashung, Dumas, Karkwa, Noir Désir, Prototypes, Vulgaires Machins, and Indochine.

Women in Software Engineering — Though I’m not part of the Ruby/Rails communities I was disappointed to see the kerfuffle over a recent presentation which reminded many of us how few women there are in our profession, how they’re often (subconsciously) typecast, and how their accomplishments don’t often get enough attention. So here are some belated congratulations to Barbara Liskov, winner of the most recent Turing Award. And I like Liz Keogh’s web site.

My grad program — In a nutshell, I enjoyed my time at Brandeis as I worked toward my master of software engineering (M.S.E.) degree. I learned a lot of practical things related to software construction, tooling, and process (just as I hoped that I would).

And finally, Lent and my year-long cleaning project — After graduating I looked around my office/library and was shocked at how cluttered it was. So just before the new year, I resolved to get through all of the magazines, papers, articles, etc. in my big cardboard “in-box” — seriously, it’s big — by the end of 2009 and recycle whatever is left on January 1st, 2010. My progress is slow. (I’m being very generous to myself right there.) So for Lent I decided to better myself by working on my reading pile. In particular, I forbade myself to add more things to it. Basically, I’ve gotten to that point in my life where I need to put myself on an information diet or two. Which I’ve done, although I find myself snacking a lot.

Anyway, that’s what I’ve been doing: reading from the big in-box, watching some of the 499 films in my Netflix queue, reading other people’s web sites*, writing in my own journal, photographing, gardening, carpooling, and more-or-less not writing here. More to come, eventually. . . .

*   Thanks for nothing, Facebook and Twitter! Just kidding. You know I love you.

The major goals of using object-oriented design are to facilitate the maintenance and extension of software systems by reducing the complexity of software at the class and system level. Successful OO designs are resilient to change, largely because they manage interclass dependencies. In such a system, changes to one part of the system are localized and do not cause a chain of modifications through the system.

The major techniques that OO designers have at their disposal are abstraction, encapsulation, inheritance, polymorphism, and composition. But how does an engineer apply these techniques to create a good design? What are the main ways that sets of classes can be structured and interact to maximize the chance of a successful design? Over the last twenty years, numerous OO practitioners have developed a mature set of rules-of-thumb and best practices to use when constructing and evaluating OO design.

If you celebrate it, enjoy your Thanksgiving tomorrow. Don’t do any homework if you can help it. I won’t!

Design patterns give a shared vocabulary for talking about common software design features, present how and when to use a design, and lay out the trade-offs of using a particular design. They aren’t as widely used as they should be in a profession that claims to value reuse. I think that’s a shame.

As part of the design patterns course, I must create an object model for a real-world problem. And along the way, I’ll be learning about most of the two-dozen OO design patterns in the Gang Of Four book. (Usually I don’t present solutions to my homework in public because I feel it could help the next semesters’ students cheat; but this semester we’re each doing our own thing.)

Since I’m learning about them, I’ll do my best to write about them over the coming weeks. If all goes well, I’ll have an outlet to reinforce what I’m learning; and you, my dear readers, will have a gentle on-ramp to design patterns, too.

Here’s the description I submitted last night of the software system I will design:

So keep tuning in, and we’ll take a tour through design patterns for object-oriented software reuse. And please post comments about where you’ve been able to use design patterns successfully, if you see anything you would have done differently, or if you just want to leave your two cents.