Monthly Archives: May 2010

Collecting

Sometimes, you just have to collect stuff.

Vials of insulin and magazines
Click for larger . . .

Posted in Diabetes, Hoarding, Photography | Leave a comment

Memorial Day

Today is Memorial Day, the day that Americans commemorate lives lost in war and combat. As in previous years, someone from Holliston has placed markers and flags along the main roads in town. Each hand-lettered sign has the name and age of the soldiers, marines and sailors who were recently killed in action in Afghanistan and Iraq, as well as the state or country where they lived.

It’s quite an affecting thing to see the miles of signs, flags, names and ages as I drive home from work. The names don’t leave much of an impression usually. But the ages definitely do. So many of them are so young. And this year more than in many years past — because of the surge of violence in Afghanistan — the dead were from many different countries.

It seemed to me the only way to really convey the experience of seeing all of those signs was to walk along parts of Routes 16 and 126 in Holliston and photograph each of the signs that I saw along the way. Here are 100 or so, as well as a few random scenes along the way to give a sense of what Holliston is like the rest of the year. (You can click any photo for a larger version.)

If you’re in the military, thank you for your service. And please stay safe.

Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010 Memorial Day Weekend, Holliston, Mass., 2010
Memorial Day Weekend, Holliston, Mass., 2010
Posted in Burying Grounds, Central Asia, Photography, This is who we are | 1 Comment

Friday Links: Oily Pancreases and Time Machines

I’m going to try something new, posting a small group of diverse links on a (more-or-less) weekly schedule. Hopefully this will help with my hoarding problem.

Living with Diabetes: Sarah has a really great piece on her site about growing up with the “bad kind” of diabetes. At least that’s how people differentiated type 1 and 2 while she was growing up. But really, all diabetes sucks, especially if you try to ignore it.

Software Development: Keith Swenson’s article 26 Hints for Successful Agile Development is full of good advice about how to do software development effectively — even if you’re not really doing Agile development. (via Infoq)

Functional Programming: Here’s a really l-o-n-g article about functional programming. It’s good, but . . . damn!

Risk and Oil Spills: You would think that a company like BP, whose contractors deal with potentially deadly situations on a daily basis, would have a better handle on risk. Even if BP engaged in neutral cost-benefit analysis, as this NY Times article suggests, it should have chosen the option that lowered its risk exposure. Remember: risk is cost of vulnerability times likelihood of vulnerability. In the case of deep water drilling in the Gulf of Mexico, we’re seeing that the cost of an oil spill is astronomical. The probability of failure would have to be completely zero to make it worth choosing a less expensive option.

Time Machines: Stephen Hawking tells you how to build one using wormholes. He also advises against creating paradoxes where you kill yourself.

The Artificial Pancreas: So what’s this “artificial pancreas” that people with type 1 diabetes keep talking about? Let Wired magazine or Aaron Kowalski tell you. It’s not a cure, but (if done correctly) it will hopefully lower a lot of the variability that we see in our blood glucose levels. Basically, it’s an expert system built into a pump plus continuous glucose monitoring combo. It’s also a bundle of assumptions and heuristics. I find it somewhere between amazing and hella scary.

Posted in Diabetes, General, Hoarding, Life Lessons, Software Engineering, Uncategorized, Worthy Feeds | 1 Comment

Australian Weather

Here are the five day forecasts for where we’ll be in Australia. As Lisa wants to know, “How do you pack for a month of this weather?”

  • Sydney — Highs: 60-68°F, Lows: 51-55°F, rainy, cloudy
  • Darwin — Highs: 85-90°F, Lows: 70-75°F, sunny
  • Alice Springs — Highs: 67-70°F, Lows: 35-47°F, sunny and showers
  • Watarrka NP — Highs: 68-75°F, Lows: 42-50°F, sunny
  • Cairns — Highs: 78-81°F, Lows: 68-71°F, sunny, cloudy, stormy

Visit obama-weather.com for more personalities and week forecast Visit obama-weather.com for more personalities and week forecast

Posted in Australia, Travel | Leave a comment

Separated at Birth

It’s finally time to write that continent-sized post that I’ve been mulling for quite a while. After all, we leave for Australia in just 10 days.

I have a hypothesis — I’m full of them, by the way — that Australia and the United States are fraternal twins separated at birth. We (the U.S.) are the headstrong child who left home in a rage in our teenage years and forged a life of power and wealth. Australia is the marginally younger child who stayed close to the parents, even at a great distance — both physical and emotional.*

So how are our two countries similar?

We both speak English — or something like it — and have funny accents. (Well, Wisconsinites do, anyway.) Hugo Weaving and Nicole Kidman can pull off both of our accents very well. I’m fairly convinced, though, that no American can really do an Australian accent without sounding like an idiot.**

We’re approximately the same age in terms of European settlement, and we’ve each had our British colonial experience. Each nation had its own foundational myth that used the legal fiction of terra nullius to dispossess the native population.

We’re approximately the same size (if you leave out Alaska). And we’re both coastal nations, with the majority of our populations living within a few hundred miles of the ocean. And each country/continent has dry, sparsely populated regions full of ranches and deserts — the outback, if you will — where we like to engage in extractive industries and (occasionally) blow up nuclear weapons.

The U.S. and Australia are both “first world,” late capitalist, market-based, bourgeois democracies. We each have relatively low opinions of our governments. Each country has recently experienced troubles with its healthcare system. We even have our own versions of Medicare, too.

We love prisons and “football” and surfing. We both use dollars, which are currently moving away from parity in our favor as travelers. We each have an ABC television network. And we both like off-beat humor and alcohol.

Finally, we’re demographically similar. And each country has a sizable population that worries quite a bit about new immigrants and “boat people.”

Oh yeah, and the sun also rises in the east.

So let’s talk about differences.

Australia and the U.S. are mirror images. As antipodal pairs, we’re in different hemispheres, no matter how you look at it: east-west or north-south. The sun rises in the east and sets in the west; but at midday it will be in the northern sky when we’re in the southern hemisphere. I personally don’t care how water goes down a drain*** — that’s not going to help me navigate anywhere — but the sun being in the wrong place, that’s going to take some getting used to. Then throw in the whole driving on the opposite side of the road. . . .

I’m really, really looking forward to seeing a different set of stars, though.

We each have our own language quirks. Australia has wonky animals, all of which want to kill you. In America, it’s mostly the people who want to kill you. American football uses pads to dull the pain; Australian footballers use nothing more than toughness and alcohol, it would seem.

Now let’s get down to the big, big differences. Australia can go to the Commonwealth Games. We had this little revolution that got us permanently kicked out of the club. Australia stuck around; so HRH Elizabeth II is technically still the leader. I think she might be on some of their money, too.

Thinking of money, the GDP of the U.S. is roughly 15-20 times larger than our sibling’s. This — along with our youthful rebelliousness — has given us a particular swagger. The United States is an imperial superpower. We can do things that almost all other nations cannot do. (And probably would not do for that matter.)

It will be interesting to see how these differences and similarities appear from the opposite side of the globe. Stay tuned.


* — And then there’s Canada. I love Canada dearly, so I’ll just leave them be for now.

** — I’ve watched “Strictly Ballroom” and “The Adventures of Priscilla Queen of the Desert” probably a dozen times each and the only things I can say with any kind of convincing accent are “Hard?! You think this is hard?” and “bogo pogo.” I’ve decided that the best thing I can do in Australia is to talk like John Wayne.

*** — I’ve finally decided that the whole water down the drain thing isn’t (completely) an urban myth. Hey, precession happens. But the effect is so slight that you can say whatever you want.

Posted in Australia, This is who we are, Travel, USA | 3 Comments

Back in the Saddle Again

Just FYI, I went for a ride this afternoon. Nothing special — just my typical 16-mile, hour-long route up and down the hills of Milford and Upton. My bike seems to be fine and not sluggish at all, strongly suggesting that my problems on Saturday were, in fact related to nutrition and conditioning and just not having a good day.

I think part of my problem might be that I’m bored with my training route. I’m very, very accustomed to it. Perhaps it’s time to make a few changes.

Oh, and I feel very close to having my insulin and food worked out for an afternoon ride with happy BG readings beforehand and afterward. :^)

Posted in Cycling, Diabetes, Life Lessons | Leave a comment

The Anchor on My Ship of Fun

There’s a bumper-sticker out there — you’ve probably seen it — that says “A bad day golfing is better than a good day working.”* Variants of “golfing” include “fishing,” “shopping,” whatever. What you can’t replace it with is “cycling.”

A great day of cycling can’t be beat, but a bad day sucks. Because a bad day of cycling is work. Hard work. Possibly even work in the rain.

No rain today, but I was just not feeling it. I had intended to ride 90 miles, leisurely going over some mountains on a beautiful day.** Long but doable, I thought. After all, a month ago I did an 80-mile loop from my house to the top of Mount Wachusett and back, and that felt really good. But I ended up turning around after 25 miles.

A map of today's ride
Click for larger

Don’t get me wrong, the ride had its good parts. Hardly anyone was driving up or down the mountain. It didn’t take as long to ascend as last time, when there was snow and ice on the road. And the descent down Notch Road was smooth and fast. (I learned that if you run into a bumblebee while descending at close to 40mph***, you can see it coming and it kinda stings hurts as it thuds off your chest.) And — even though it killed me a little to see it — I smiled when I saw someone had painted “HILL 1/2 WAY :) ” on the shoulder of the big climb a little before where I turned around to trade the last 65 miles of my ride for 15. And at the Mount Greylock SP visitor center, I saw a fellow rider with a Team Type 1 jersey.

But something about today just didn’t work. Maybe it’s because I started by climbing a mountain. Maybe it’s because the route was either all uphill or all downhill with few flat sections. Maybe I got too warm on the way up. Maybe I should have had more breakfast before heading out. Maybe something mechanical was wrong. (I even stopped on the way up the mountain to adjust my brakes, since I thought I could feel them rubbing the rear tire.)

Maybe it was one of those things. Maybe it was nothing but my overall lack of skills. Maybe it was all in my head. Whatever the reason, I felt like I had an anchor dragging behind my bike. It felt like the energy I was putting into the bike didn’t move it forward as much as it should, and when I was going slowly (which was often) the bike seemed to slow down on its own.

If the bike was misbehaving — and I’m not saying that it was, although it definitely needs a tune-up — it was mostly me. I just had no energy. I ate and drank, but it didn’t seem to matter. I had cracked. In fact, I contemplated walking my bike the last half mile up the 10% stretch back to the car. I’ve never walked a bike since I first got one with multiple gears. Like I said, it was a tough outing; and I consoled myself with a strawberry shake.

Tomorrow is another day. And since I didn’t go very far today, it might just be tomorrow.


* — This bolsters my (admittedly very biased) assertion that golf is not a sport. At least not any more than billiards or darts. And yet golf is going to be an Olympic sport in 2016, while cricket continues to be excluded. Hmm.

** — I think this ride is jinxed. Two weeks ago, there was a gale/nor’easter that kept me at home. Last week I forgot about a dinner date. And then today. But I declare this: “Taconic Range, I will make you my bitch successfully cross over you.”

*** — I know, I know. The speed limit was only 25. I don’t have a computer/speedometer, so I’m guesstimating. BTW, I’m sure the bee was fine. . . .

Posted in Cycling, Life Lessons | 2 Comments

Happy Fifth Birthday, Dispatches!

Happy fifth birthday, Dispatches!

A lot has happened in my life in the five years since that first post just before our trip to India. I thought you were a goner during my last year of grad school, when I didn’t post anything for more than a month, but National Blog Post Month last November seems to have restarted a more regular rhythm.

I never really knew what you were going to be about. That’s fine with me. I’m a multifaceted individual, who is easily distracted. So, instead of mining the same vein of subjects to gather a steady stream of loyal readers, I’ve written about anything and not quite everything. For a little while I thought about getting you, my weblog, a little brother or sister, turning one of you into a single-subject weblog, and keeping the other one for random stuff. But I never did, because I know you like being an only child, and I don’t have enough time to devote to two.

As a result, you, my little weblog, are always meeting lots of new readers who are drawn in by the Google, stay but a brief while, and then move on. In fact, well over half of your visitors come from Google searches. You’re most popular when you’re unique and nerdy. Here are the top 10 most popular pages over the last five years:

  1. The JPEG Family Circus (2008)
  2. Fifteen is an Eternity in Photography Years (2007)
  3. Ask Dr. Color’s Assistant: Tone-mapping in MATLAB (2007)
  4. The Cognitive Style of Microsoft Project (2007)
  5. Book Notes: The Looming Tower (2006)
  6. Four Days in London (2007)
  7. Deconstructing an Image (2007)
  8. How a Digital Camera Works (2006)
  9. Grandes Expectations, a.k.a. Four Days in Paris (2009)
  10. Tractors (2008)

Okay, I don’t get that last one either. I guess there are a lot of people who, like me, enjoy 1/64-scale tractors. And almost 80% of the views of the JPEG article happened in the first week it was published, when Steve posted a link to it, which got a couple hundred views and was then Stumbled, garnering 30 times more readers. You, my little weblog, were almost famous.

But those things aren’t really what you’re about. Over the last five years, we’ve traveled a lot, started to talk a bit about diabetes, visited many cemeteries, thought about software engineering during and after grad school, played with a large format camera, worried about health care, learned many lessons, taken and posted tons of photographs, and tried to deconstruct the American experience. (Lisa, who is perpetually awesome, helped with some of the posts and many of the photos.)

So what next? What will happen in the next five years?

Given the randomness of posts over the last five years, it’s dangerous to guess, but I bet it looks like the last five years. Without a doubt there will be more travel: In two weeks we’re going a Australia for a month; next year, my mom and I plan to go cycling in Provence; and in 2012, we’re going to England and France with my in-laws. Unless amazing things happen, I’ll still have diabetes and will continue to write about that. No doubt, I’ll also visit some additional technical subjects, which will appeal mostly to the long tail.

You, my little weblog, were born near the beginning of an online historical moment when it seemed everyone was getting a “blog.”* A lot of people moved on — to MySpace and Facebook and Twitter — and let their online journaling end. Meanwhile the idea of the weblog became the basis for a lot of mainstream media and corporate sites. The weblog became the scaffolding for interactive, moderated, medium-to-long-form medium.

I’m excited to see the re-emergence of “microblogging” sites like Tumblr, where people post short things: videos, links to other pages, excerpts from articles with reactions, etc. It’s bringing the social back into “social media.” Now, instead of thinking about getting you a weblog sibling, I’m trying to figure out the right way to integrate shorter snippits with my regular fare.

Because what I really want is to have something like a magazine, with its mixture of time-relevant mini-articles and long-form features: something that combines what has traditionally appeared here with some of the stuff that I’ve offloaded to Delicious or Facebook or Twitter. But that’s all in the future.

Once again, happy fifth birthday, weblog!

p.s. — I haven’t gotten you a present yet, but I know you want a new theme so that you can look a little more hip. And I think I heard you say that you want better comment management, too. I’ll see what I can do.


* — Five years later, I still can’t stand that word “blog.” It’s just too ugly sounding. Like “atheist,” there’s just no happy-sounding, value-neutral way to say it. Of course, you who don’t have my hangups can call this site whatever you’d like. :^)

Posted in General, MetaBlogging | 1 Comment

The “Blow Stuff Up” Conference

In case you wondered what that post from earlier today was all about, perhaps a picture will help:

'Blow Shit Up' conference announcement
Click for larger . . .

This envelope came in the mail yesterday. I don’t know who put me onto this mailing list, but I’m pretty sure it’s related to the work I’ve done over the last few years supporting the NITF file format, whose users are an interesting lot. They don’t really like to talk about what they do or what they keep in their files: secret stuff mostly.

I’m not one to judge. I’ll just say that I’m very glad that I was also responsible for adding support for the DICOM medical imaging format to MATLAB.

Posted in File Formats, General, This is who we are | Leave a comment

A Menagerie of Image File Formats

This is a follow-up to my recent post on parsing NITF files that contain JPEG data. It’s basically a crash course into the organization of the guts of image file formats. If I were ever asked to be an expert witness in a trial, it would probably be about file formats.* This is the area of my expertise.

You can divide the world of image file formats into different kingdoms based on the their structure. There is some overlap between these categories, but for the most part image formats are (1) tag/record-based, (2) structure-like, (3) marker/stream-based, (4) textual, (5) card-like, (6) raw, or (7) opaque.

TIFF, DNG, and DICOM are examples of tag/record-based formats. A unique tag identifies the entity in the file and its meaning. For example, a particular hexadecimal tag might indicate that this is the “photometric interpretation” record. The datatype of this record either explicitly appears after the tag or appears in a data dictionary that’s known to the application developer. Almost always, these records explicitly tell the length of their data, which makes it easy to skip to the tag location of the next record.

Microsoft was (for a time) very fond of making structure-like formats. In these formats, the file looks a lot like the memory representation of a C/C++ data structure. These formats are easy to describe and easy to read if you have the structure definition; simply fread() the data into a variable and reference the data members by name. The problems should be pretty clear. You need to be using a programming language that supports C structs. And you need to know the layout of the struct. And once you define the layout of the struct, it’s fixed. (Well, not exactly. Microsoft changed the data layout in its BMP family of formats with every release of Windows, and used a “magic” value to tell readers which struct to use.) All told, it’s a very brittle kind of format.

JPEG is the prototypical — but certainly not the only — marker-based format. Markers are special combinations of bytes that, like a tag, tell what the data is that’s coming next in the stream. But, very much like struct-based formats and very unlike tagged formats, the data that follows the marker can be heterogeneous. In JPEG, the data that appears after the SOF (Start of Frame) marker is a record, while the data that follows an RSTn marker is just a stream of compressed bytes. The SOI and EOI (Start/End of image) markers don’t even have any bytes that follow them. In marker-based formats, semantics and syntax are rather carelessly jumbled together.

It’s very difficult to quickly parse marker-based formats, because often markers don’t specify how much data appears before the next marker. These are very much “streams” of bytes that you’re forced to read until you come to the next marker. Consequently the number and appearance of markers is very limited and this limitation ripples through to the data that they contain. JPEG markers all begin with the 0xFF byte followed by another byte, which taken together specify which marker it is. Consequently, the appearance of an 0xFF byte in the data of a marker has to be escaped by a NULL byte so that it’s not mistaken for the next marker.

Textual formats, such as XML, have the benefit of being self-describing and readable by both humans and machines. Their main drawbacks are the inflated size of the data they contain (even when represented in a semi-binary CDATA hunk) and the inability to quickly skip through them with binary I/O routines.

FITS is a fairly prototypical “card-like” format. As the name implies, these are fixed-length records like one might have encountered on a punch card. For example in format with 120-character records, the first n characters are reserved for the “variable name” part of the equation, while the remaining 120-n characters are the textual representation of the value of the record. They are frequently text-only for the descriptive part of the format with a binary payload at the end. These are easy to read, but a pain to parse, since the “right hand side” values often have to be interpreted.

Raw and opaque formats aren’t very easy to describe because they’re so varied. In a “raw” format (and there are dozens or hundreds . . . possibly more) all of the bytes are jumbled together in a payload-only file. A separate file may have a header that describes the data and helps a reader/parser make sense of the payload. Or not. These are almost always completely free of any helpful description within the file.

This shouldn’t be confused with opaque files, such as HDF, CDF, or netCDF. These formats are completely defined by their API, which for all intents and purposes, you have to use to access the data within the file. This allows for a lot of richness in handling the data contents, which can be organized in highly optimized ways. The downside is that you’re limited in how you can interact with your data to mechanisms someone else has defined. And data permanence can suffer, since if the tool chain changes (or goes out of existence) you don’t really have a way to get at your data.

Practically, each format style has it’s pros and cons. But tagged formats (which might incorporate features of the record style) are the most durable and easiest for third-parties to work with.


* — Cue awesome “CSI” + “Law and Order” + “House” mashup daydream. *DOINK DOINK*

Posted in Computing, File Formats, Fodder for Techno-weenies, Life Lessons | Leave a comment

NITF + JPEG

I’ve recently been working with streams of JPEG data inside of NITF files. Given my experience supporting I/O involving DICOM files that contain JPEG-compressed imagery, I was extremely surprised to learn how difficult it is to read JPEG from “National Imagery Transmission Format” files. This post exists to help the next person who needs to read JPEG data embedded in NITF or another file format.

My naïve idea was to copy the JPEG-encoded to a temporary file and then read that file using the Independent JPEG Group‘s libjpeg library. That’s what I did with JPEG data encapsulated in DICOM. This is far too simple an approach for NITF, resulting in incomplete images. Here’s why:

  • NITF breaks most images into multiple tiles.
  • Each tile is independently compressed into its own image stream.
  • NITF uses “block masking,” which prevents storing unimportant tiles.

The idea makes sense on one level. If you’re going to send an image over a low-bandwidth or low-fidelity channel, you want to limit the amount of data that you send, and you want to avoid an all-or-nothing situation during image transmission or reception. But it’s a total pain in the ass for application developers.

Add to this the fact that JPEG is a marker-based format that isn’t very self-describing, and you have a tricky parsing situation.*

Here’s the basic idea behind getting imagery out of a NITF file if it’s been JPEG compressed. (I assume that you already know how to parse a NITF file — see MIL-STD-188-198A if you don’t — and that you have a JPEG codec that you can use to decode the data.)

  1. The first two bytes of the compressed stream should be the standard JPEG SOI marker (0xFF 0xD8). This is your sanity check.
  2. The next two bytes should be the APP6 marker (0xFF 0xE6). The payload of this marker contains a bunch of useful information about tile sizes and counts, bit depths, etc. Some of this is redundant with what’s inside the NITF file.
  3. The remainder of the NITF file should be a bunch of JPEG codestreams delimited by SOI and EOI (0xFF 0xD9) markers. Each delimited stream is one tile in the image; and it’s a completely standalone JPEG stream. It can be extracted to its own file (if necessary) and decompressed. Tiles are stored across the image horizontally and then down.
  4. If there’s no block masking, it suffices to read each tile in turn and store it in the appropriate region in the output image.
  5. If the NITF file does use block masking, use the values in the BMRnBNDm attribute of the image subheader to find the locations of the blocks that contain actual image data. The masked out blocks will have 0xFFFFFFFF values. The other values — there’s one for each tile — are 0-based offsets pointing to the SOI marker that starts each tile, relative to the start of the JPEG compressed data.

And that’s it. After coding, you should probably test out your parser on the sample NITF files provided by the NGA.


* – You can divide the world of image file formats into different buckets based on the their structure. There is some overlap between these categories, but for the most part image formats are (1) tag/record-based, (2) structure-like, (3) marker/stream-based, (4) textual, (5) card-like, (6) raw, or (7) opaque. I’m going to write more about this in the next post.

Posted in Computing, File Formats, Fodder for Techno-weenies, Life Lessons | 1 Comment

Diabetes Blog Week Wrap-up (Plus a Warning Shot)

Diabetes Blog Week finished Sunday. I really enjoyed being part of this, writing my own posts and (especially) reading other people’s contributions. I’m giving a huge “Thanks!” to Karen for coming up with the idea for this village green experience and for providing the essential scaffolding that the rest of us were able to use. It really made participating easier.

It’s wonderful to have a place — virtual though it may be — where people understand what I’m talking about on a deep level.* Because so many of us were conversing on the same subject on a daily basis, I found it much less isolating than the usual “I’m going to talk about diabetes now” experience. And I especially enjoyed the honesty that was possible when we were writing for each other; I didn’t feel like I had to have all my shit together before talking about my disease. (It often feels easier to talk to people with working pancreases, if I just pretend that I do — whether or not that’s the case.)

In fact, I think I finally get the real value in the diabetes online community (DOC). In the past I’ve tend to visit the DOC to get information that I felt could help me manage my illness better. But having read so many posts over the last week, I find myself thinking about it as a place that’s primarily about sharing other people’s experiences, talking about my own, and just being present. Certainly, I’m going to keep trying to learn as much as I can from the same sources as before, but being able to get extra perspectives has a ton of value in itself.

I wonder if you feel the same way, too. And I hope that if you’ve been reading me for the first time over the last week, you’ll stick around. Big things are going to happen in the coming weeks that you might want to read about — for example, we’re going to Australia for a month — and I’ve been enjoying all of your comments.

Now here’s the scary part (for me, at least). I want all y’all from the DOC to stick around, but I’m not primarily a “diabetes blogger.” I have a lot of stuff on my mind: diabetes, the US healthcare system, travel, photography, New England cemeteries, and a bunch of stuff that’s more like work than fun. (That last chunk of stuff actually comprises the majority of the traffic to my site, if not the majority of what I write about. . . . Go figure.) And I use this space to post on all of those things. Unless you’re actually me, some of my regular topics aren’t going to be your cup of tea.

So let’s make a deal. If you stick around and ignore the non-diabetes posts that don’t interest you, I promise to give you something worth reading more often than not. Just add me to your RSS feed reader or stop by regularly. Deal? Good.

. . . And now it’s time to put this new relationship to the test. I’ve been sitting on the next two posts (people’s exhibits a and b) for the last week so that they wouldn’t get in the way of your Diabetes Blog Week reading. It was enjoyable, but my! wasn’t there a lot of reading?! And I just didn’t want to add to that.

See y’all again soon!


* — I’m reminded of the time that I accidentally happened upon a little people’s convention in the late 80s in Des Moines, Iowa. Lots of people who are usually “the other” were running around doing their own thing in a supportive community. I felt like the odd one out. In the DOC, we’re on the inside.

Posted in Diabetes, Diabetes Blog Week, MetaBlogging | 3 Comments

A Life Without Diabetes

Here it is, the last day of Diabetes Blog Week. Thank you so much, Karen, for proposing this online extravaganza. Today’s topic is a fitting valediction: Life after a cure.

Diabetes Blog Week banner

I’ll just get this out of the way up front: I’m of two minds on a cure.

I very much want a cure for diabetes. It’s a complete pain in the ass. It affects millions of people worldwide. Even in the most developed nations, it’s a costly illness that shortens life expectancy and makes so many things more difficult. But it’s a disease that knows no borders; and despite being reported in the press as “a disease of prosperity,” it affects the rich and the poor alike. Throughout Diabetes Blog Week, I’ve read a lot posts by parents of children with diabetes; even more for them than for myself, I want a cure.

And yet, I know that patients who accept the facts of their chronic illnesses tend to have better health outcomes than those who cling to hope for a cure. The absence of a cure is the way things are, and the best that I can do for myself and those who care about (and for) me is to do my best everyday with no expectation of ever being free of diabetes. It’s a harsh, pessimistic mindset that I can’t completely buy into, but on a certain level it’s the way I have to think. Yet, I’m new enough to this disease — only eleven years — to occasionally become deliriously hopeful when I hear about some promising-sounding results from JDRF, Joslin, and other research groups.

The Cure
And then, of course, there’s The Cure.

Given my ambivalence, how do I address Karen’s hypothetical proposal?

We are all given a tiny little pill to swallow and *poof* our pancreases are back in working order. No side effects. No more insulin resistance. No more diabetes. Tell us what your life is now like. Or take us through your first day celebrating life without the Big D. Blog about how you imagine you would feel if you no longer were a Person With Diabetes.

I hope that being free of diabetes wouldn’t change me in fundamental ways. I think I have sufficiently internalized the lesson over the last year that eating a reasonable amount and exercising regularly are good parts of our family’s lifestyle. I hope that I would still keep in touch with all of the people I’ve met in the diabetes online community. And I don’t feel like there’s anything now that I can’t do.

But I do think things will be much easier. I will be able do things (like going for a ride or a run. . . or to the dentist) without worrying about how much insulin is floating around and whether my blood sugar is high or going low. It will also be nice to be free of concern over complications — supposing that I haven’t already done the damage. And I’d love to banish those ever-present concerns about whether I guessed the right amount of bolus, whether my basals are right, whether I’m going to go low overnight, etc. After eleven years, I wonder if it will even be possible to get rid of those feelings; perhaps they’re an indelible part of my personality now.

Whatever happens, I can promise you this: I will continue to do everything I can to make sure that everybody worldwide has access to that magic pill.


Update: Oh! I would also go to Kelly’s beach party.

Posted in Diabetes, Diabetes Blog Week | 3 Comments

A Day Late, A Tube Short

Yesterday was day #6 of Diabetes Blog Week. I managed to miss it because we were kinda busy. So I’m gonna make up for it today with two posts. First, some diabetes snapshots.

Diabetes Blog Week banner

Before the pictures, a little story. Remember that on Friday I wrote that I was going to do 90 mile ride in the Taconic Range today? Turns out, I forgot about an evening obligation, so I decided to delay the ride until next weekend and do a similarly sized ride starting at home but without any mountains.

About two hours into my ride through Massachusetts, Rhode Island and Connecticut — before I really even had a chance to get bored — I got a flat. After a year of riding, I was due, but it could have happened in a more convenient place, instead of halfway across the West Thompson Dam. My first thought was a hope that I could just raise my hand like they do in professional races and summon the neutral service vehicle for a quick wheel change. Oh, delusions!

After walking myself back to a place with a shoulder, I made a rookie mistake, breaking the head off the valve of my flat tube as I took it off the wheel. Had I been wiser, I would have also brought an extra tube with me. Like I said: rookie mistakes. Nothing to do after that but pack it in and call Lisa to pick me up. She’s a sweetheart, that girl.

Next week, I’ll be more sensible when I finally do that ride in the mountains.

Here are some pictures from the past couple days:

Eating Palak Paneer
Lunch of palak paneer and chicken korma

What?!
What?!

Testing
How did we do SWAGging lunch? Uh . . . coulda been better.

Bloggin'
Reading all y’all’s blogs

A movable feast
A movable feast

Bike in the graveyard
Stopping by the cemetery in Burrillville, Rhode Island

Joslin marker
Lots of Joslins in this part of Rhode Island and Connecticut

Dr. Joslin, I presume?
I thought at first this might be the guy we PWDs owe a debt of gratitude, but he seems to be an uncle of some distance.

Thompson, CT
Waiting for the cavalry in Thompson, CT

Waiting
I barely worked hard enough to muss my hair

Posted in Burying Grounds, Cycling, Diabetes, Diabetes Blog Week, Life Lessons, USA | 1 Comment

It’s ALL about the Bike

It’s day #5 of Diabetes Blog Week. Today we’re talking exercise.

Diabetes Blog Week banner

I have a bike. I like love to ride it all over.*

Is that exercise? I guess that depends on whether you think it’s “exercise” to do the thing that you love.

On one level, the answer is undeniable: Yes. I have to carve out time from my daily schedule to do it. Sometimes I have to convince myself to get going, especially when I’ve had a tough day and I want to veg out. And it burns a lot of calories, which was part of my initial motivation. According to the computerized bean-counters at MapMyRide.com, I’ve burned more than 130,000 Calories over the last 10 months by running, bicycling, and swimming,** which helps explain why I lost about 25 pounds over the same period.

I don’t think about it as exercise, though. In fact, what I do after work feels more like training. I “train” on weekdays so that I can ride longer distances with more ease on the weekends. I wear myself out repeatedly riding up long hills so that I can feel badass when it comes time to ride up an actual mountain. I go out in the winter and in the rain to put the miles in the bank, so that they’re there when I need to draw on them in the fourth or fifth hour of a ride. While I’m out training I have mental image of my idealized self. “I’m climbing like Andy Schleck. I’m grinding away on the flats like Fabian Cancellara. I’m spinning easily like all those other people in the peloton, waiting for the breakaway to wear itself out.”***

Whatever I call it, cycling is something that I love and that I think about way too much while I’m at work. I live for the long ride on the weekend. This Sunday, I hope to do the 90-mile ride that I was going to do last weekend before the jet stream shifted and changed my plans: Up and over Mt. Greylock in western Mass. before heading into the Taconic Range that divides New York from New England.


View Larger Map

As with all things diabetes, it’s not as easy as just putting in the miles and showing up. There’s day-to-day planning that has to happen, too. I find it easiest to ride in the morning before the day’s first bolus: Just lower the basal about 50% an hour or two before starting and eat frequently along the way, testing every hour or so. I put Clif bars, bananas and string cheese in my jersey pockets and fill up my bottles with Gatorade. And on the weekends that’s what I do.

But weekdays I ride after work, so it’s more challenging. I hate seeing the high numbers, but I build up a bit of a blood sugar cushion by snacking without bolusing along with lowering my basal. And I drink Gatorade throughout my hour-long workout. I’ll keep tweaking everything until I get it right — until my BG levels don’t drop 50-100 mg/dL in an hour — and then I’ll lock it in until diabetes decides to change how my rules work (again).

Thanks to diabetes, I always carry three things with me when I ride****. (1) A tube of glucose tablets, which I occasionally need to use. (2) My phone, which I fortunately have not had to use except to snap the occasional picture. And (3) about $15 dollars in small bills in case I need to stop for an emergency snack or to bribe someone.

But to paraphrase Lance Armstrong, it’s not about the diabetes. I love to ride, and diabetes can come along if it promises to keep up. When I actually get on the bike to ride, that’s the time when I feel like I’m beyond diabetes. I put my pump in the pocket of my Team Type 1 jersey to represent for my PWDs and because I’m so damn proud and inspired by what that professional team does; but cycling connects me to a time before I had diabetes, and it’s my way of being as free from it as possible.


* — I’ve also been known to run, walk, hike, and backpack. And, yes, I’ve even started to enjoy swimming — though, I still suck at it.

** — Seriously, I’m not thinking about competing in a triathlon. I’m not.

*** — I have no delusions about my abilities, though. I’m just a guy with diabetes on a bike, after all.

**** — That’s in addition to the Boy Scout stuff that always stays with the bike: fix-it tools, patch kit, tire levers, etc.

Posted in Cycling, Diabetes, Diabetes Blog Week, Life Lessons, Running, This is who we are | 4 Comments