A half-decade after Herb Sutter wrote that the “free lunch” of Moore’s Law is over, he’s back with his prophet’s wisdom about where we’re going in his January Dr. Dobbs article, “Welcome to the Jungle”. I’ll give you a moment to decide whether to get the Guns N’ Roses song out of your head or use it as a backdrop for this juicy quotation:

If hardware designers merely use Moore’s Law to deliver more big fat cores, on-device hardware parallelism will stay in double digits for the next decade, which is very roughly when Moore’s Law is due to sputter, give or take about a half decade. If hardware follows Niagara’s and MIC’s lead to go back to simpler cores, we’ll see a one-time jump and then stay in triple digits. If we all learn to leverage GPUs, we already have 1,500-way parallelism in modern graphics cards (I’ll say “cores” for convenience, though that word means something a little different on GPUs) and likely reach five digits in the decade timeframe.

But all of that is eclipsed by the scalability of the cloud, whose growth line is already steeper than Moore’s Law because we’re better at quickly deploying and using cost-effective networked machines than we’ve been at quickly jam-packing and harnessing cost-effective transistors. It’s hard to get data on the current largest cloud deployments because many projects are private, but the largest documented public cloud apps (which don’t use GPUs) are already harnessing over 30,000 cores for a single computation. I wouldn’t be surprised if some projects are exceeding 100,000 cores today. And that’s general-purpose cores; if you add GPU-capable nodes to the mix, add two more zeroes.

The big takeaway for software engineers like me is that we’d best be learning how to develop solutions using the emerging APIs so that we can harness all of those extra orders of magnitude of scalability. That involves figuring out how to . . .

• Deal with the processor axis’ lower section [of Sutter's chart] by supporting compute cores with different performance (big/fast, slow/small).
• Deal with the processor axis’ upper section by supporting language subsets, to allow for cores with different capabilities including that not all fully support mainstream language features.
• Deal with the memory axis for computation, by providing distributed algorithms that can scale not just locally but also across a compute cloud.
• Deal with the memory axis for data, by providing distributed data containers, which can be spread across many nodes.
• Enable a unified programming model that can handle the entire [memory/locality/processor] chart with the same source code.

Perhaps our most difficult mental adjustment, however, will be to learn to think of the cloud as part of the mainstream machine — to view all these local and non-local cores as being equally part of the target machine that executes our application, where the network is just another bus that connects us to more cores. That is, in a few years we will write code for mainstream machines assuming that they have million-way parallelism, of which only thousand-way parallelism is guaranteed to always be available (when out of WiFi range). . . .

If you haven’t done so already, now is the time to take a hard look at the design of your applications, determine what existing features — or better still, what potential and currently unimaginable demanding new features — are CPU-sensitive now or are likely to become so soon, and identify how those places could benefit from local and distributed parallelism. Now is also the time for you and your team to grok the requirements, pitfalls, styles, and idioms of hetero-parallel (e.g., GPGPU) and cloud programming (e.g., Amazon Web Services, Microsoft Azure, Google App Engine).

p.s. — I can’t believe that it’s been almost four years since I took a course with Herb out in Washington. That was some hard-core learnin’.

(From How to Think about Parallel Programming: Not!)

• When optimizing code for GPU, FPGA, or CPU, definitely focus on pipelining and overall throughput, not just local optimizations.
• There’s a trade-off between “faster” and “sooner.” It’s not always worth saving a few seconds (or even a few minutes) if the kernels take hours or days to compile. (Then again, sometimes it is.)
• Try to reduce dependence on the language/compiler “stack” that removes inefficiencies. The optimizer does good work, but you can do things to help it. Think about the hardware or architecture format. It’s not a sin to reduce the amount of abstraction in the service of performance. Pay attention to things that affect processor pipelining and cache movement.
• BTW, some languages and technologies exist to provide higher level programming that’s close to the hardware, but they’re proprietary, secret, or still in R&D.
• Use algorithmic optimization techniques. Step back and find the shortest-time computation.
• Avoid using if statements. The goto construct is considered harmful, but if is basically the same thing. Instead think about state machines and polymorphism. There’s no branch-prediction penalty to pay, since the system “just is” in the state it’s supposed to be in. The logic is clearer, because there are no switches, making it easier to test, too.
• Don’t always assume that floating-point values are necessary. Integers can often be creatively used and are far faster for math than double-precision numbers.
• Of course, there’s a compromise between speedy/efficient and readable/maintainable.
• Aim to structure programs as “symbolic intent.” Mathematical descriptions are bad ways of expressing programs. Think about functional programming models instead of procedural.

If you want to know more, you should definitely watch the half-hour interview. And if your reaction was more along the lines of “Yes, yes; that’s all true, and it’s how I design my image processing code,” then I definitely hope you’ll consider applying for the GPU/multicore engineering position we have open.

We need someone with GPU and multicore programming skills. We’re looking for experience with CUDA, OpenCL, OpenMP, Intel’s Threading Building Blocks, or similar technologies. If you’re into making algorithms run wicked fast, you should definitely apply.

The other position focuses on image processing and code generation. If you like implementing image processing algorithms and converting MATLAB code to C code, then this is the job for you.

I’ve been at The MathWorks for almost fourteen years now, and it’s a really great company with an excellent corporate culture, competitive compensation, fantastic benefits, and lots of perks. Because everyone uses MATLAB and because we’ve made some very sensible business decisions over the last 28 years, it’s a very stable company to work for. (Did I mention that we’re putting up our fourth building in our Natick campus? And I think I also mentioned that the entire worldwide staff went on a cruise a few years ago.)

If image processing isn’t your thing, we have dozens of other positions open. Everything from web development to legal department work. Human resources to customer service. Technical writing to application engineering and consulting. Marketing to program management. QE, sales, usability, and more software development positions than you can shake a stick at.

Come, help us accelerate the pace of engineering and science worldwide. And if you do apply, tell them I sent you.

This is another one of those posts. Every year between Christmas and New Years Day, I try to use the quiet week to get stuff done and tie up loose ends. Last year, I cleared out a bunch of notes. This year, I’m looking at presentations and slides from the QCon SF 2011 conference (wrap-up). Its focus on software architecture and project management is about 75% of my job, so many of the presentations seemed tailor-made for me. Here’s some of what I learned.

Erik Doernenburg. “Software Quality: You Know It When You See It” has a really good slide deck that got me thinking about some projects I might want to set up. It’s full of practical, usable suggestions:

• View the code at the 1,000 view, rather than ground-level or 30,000 feet.
• Look at the test-to-code ratio, not just code coverage.
• Graph the change of metrics between versions and revisions, compare across different parts of the code, and look at them relative to industry standards.
• Measure the “toxicity” of code by rolling up various quality metrics about a bunch of modules into stacked bar charts.

We should pose these questions during design and code reviews:

• Is the software/change of value to its users?
• How appropriate is the design?
• How easy is the code/design to understand and extend?
• How maintainable is the software?

It was full of some really great links to things like Metrics tree maps (a.k.a., pretty heatmaps for source code) as well as a few tools: SourceMonitor, iPlasma, and using Moose to visualize quality.

Joshua Kerievsky. “Refactoring to Patterns” — some notes:

• Refactoring is like algebra’s equivalence-preserving manipulations. “Design patterns are the word problems of the programming world; refactoring is its algebra.”
• Understanding the refactoring thought process is more important than remembering individual techniques or tool support.
• Code smells have multiple refactoring options and often benefit from composite refactorings.
• Look for automatable refactorings first. Consider changing the client of smelly code before the smelly code itself.

Guilherme Silveira. “How To Stop Writing Next Year’s Unsustainable Piece Of Code” was pithy and thought-provoking.

• There is no value for architecture or design without implementation. That’s just interpretation of the software.
• “New language. New mindset. new idiomatic usage. Same mistakes.”
• Complexity and composition are natural and good, but if they’re invisible, they’re evil.
• Start with a mess and refactor right away. Starting “right” is hard (and misguided thinking). Refactor for better, not just prettier.
• Make complexity easier to understand and see.
• Hiding complexity in concision hurts testability, since no one knows the complexity is there. Furthermore, if it’s hard to test, it’s also hard to use correctly.
• “Model rules. Do not model models.”

Michael Feathers. “Software Naturalism: Embracing The Real Behind The Ideal” is a presentation that I would like to see/hear, since the slides seemed full of information but weren’t self-explanatory. Here are two things I could glean: 80% of software defects in large projects were in 20% of the files. In general, the more churn in a file, the more complex it tends to be.

Panel: “Objects on Trial” was perhaps the most unusual presentation, since it was a mock-trial. I use objects all the time . . . some of them are good . . . some demonstrably so. Even so, I never latched onto the idea of object-oriented (OO) design versus objects as types. The four panelists, in one way or another, basically said, “That’s the problem.”

One of the panelists drew an extended analogy between the space program and OO. The space shuttle (which we all love) was fixated on reuse but basically was a waste of heavy lifting; people don’t reuse the right stuff. In software, object reuse is largely accomplished by cut-and-paste copying of boilerplate code that does close to what you want. Of course, the panelist acknowledged that we do reuse the ideas in OO via design patterns, and no one seems to have much of a problem with that. Ironically, having a rich pattern language means that software engineers are in a better place than ever before to use objects correctly.

A key problem with our approach to objects is that we’ve failed (generally in software engineering) to handle complexity well, which was supposed to be the point of OO design. A conflation of beauty and OO design makes things worse. Internally, software is ugly, and beauty shouldn’t be a goal. Making a fetish of beauty makes code inflexible because people don’t want to extend the beautiful thing that works.

For other panelists, objects weren’t the problem at all. For them it’s static typing in “OO languages,” such as C++, Java, and C#. We’re at a place now where all of the good things about OO have been lost in an attempt to make OO languages as fast as C. This runs counter to the goal of having “ordinary,” understandable code. Generic programming using strongly typed (possibly template heavy) languages just makes everything complicated.

For me, it’s moot. C++ is what I use, and I don’t have a large proprietary object system that I can tap into for reuse. I’m in the camp that uses C++ objects to generate new types for data hiding and aggregation, as well as (to a lesser extent) reuse. But some of these types are generic, template classes that are hard to understand. I plead “no contest.”

• Brilliantly smart-ass responses to completely well-meaning signs (via John Nack)
• It’s been a very long two weeks. Get prepared. — A hilarious gem from the Bloggess that includes (what’s this?) Nathan Fillion and twine.
• What Fuels the Athlete With Type 1 Diabetes? — A really great article from medical journalist Miriam Tucker. Buy me a Diet Coke and I’ll tell you my own story.
• New Zealand — Kris Freeman is still taking names and kicking ass.
• An Absence of Advocacy — Chris thinks he’s not a good diabetes advocate. Sometimes being present is enough, man. Oh, and having a kick-ass podcast certainly bolsters one’s D cred.
• Fuck Yeah Diabetic Cat — . . . I mean even this is advocacy, Chris.
• Observing demos hurts learning, and confusion is a sign of understanding — Let this article blow your mind. (via Software Carpentry)
• JAMA: The Hazards of Evidence Based Medicine — As with everything it’s about what and how you measure.
• Want To Keep (And Motivate) Your Best Employees? It’s Not About The Money — Relationship-driven leadership.
• Radiology Wife: How To Read A Chest Radiograph — This is pretty much the approach I take when looking at DICOM files our customers send us.
• Nine Deadly Sins of Project Planning — The worst? “Not Learning from Past Planning Sins”
• Scientific photography: Escher in water, eggs exploding, & more — Oooh! Shiny!

• Ultramarathon — The always hilarious author of The Oatmeal describes his long-ass run.
• Inbox 10,000 — E-mail sucks . . . and is mostly worthless. Am I right?
• The Joy of Cycling — A poster.
• Getting Bin Laden — What happened that night in Abbottabad
• The unavoidable power of compromise — The issue of raising the debt ceiling may no longer be a hot topic, but Scott Berkun’s take on the value of compromise is evergreen.
• The Growing User and the Perennial Beginner — A mental model for anyone who creates software interfaces for people of different competency levels.
• Yeah. We know. — You hate needles. We do, too.
• A pair of articles from Victoria Cumbow. In “It’s Political, and It’s Personal” she hopes you’ll encourage President Obama to attend the UN summit on noncommunicable diseases; and “A Gentle Reminder” reminds us why we need to advocate for better education and attitudes for health care professionals. (This kind of experience is why I stopped answering the phone when my insurance company’s nurses called. Bad me.)
• O Is For Outrage — Go sign the petition to President Obama, please.
• Kelly Kunik says, “I’m Not Afraid Of My Pump Being Hacked, Because Like You, I Am The Only User On My Personal Pump Network.”
• Five Ways To Solve The Diabetes Crisis With Data-Driven Design — Just FYI, Fast Company, “Solve” might be over-promising just a wee bit.
• Semifinalists in “Data Design Diabetes Challenge” tackle a major health issue in ingenious ways — The iMedicalApps people have the tone a little better.
• It’s Broken — By “It” many of us mean “the entire system around software patents.”

Not only is that great for me when working with my own data, it will make developing my app easier. People may still need to take the extra step of downloading a CSV file containing their data, but at least they’ll be able to do it on their platform. Not perfect, but better.

In an ideal world — the one that I would have advocated for at pump/CGM HQ — third-party app developers (like me) would be able to ask the online CareLink database for a person’s diabetes data via an application programming interface (API). Mobile app developers could then hold on to that data for offline or mobile use without ever needing to talk directly to the medical devices themselves. Frankly, writing code to connect directly to a life-preserving medical device is quite risky and something I would like to avoid; it’s also the kind of thing that requires rigorous, time-consuming, expensive FDA approval. Not very appealing when all I want to be is a data consumer.

I’m hoping that Medtronic provides a mechanism to open up this data soon, because I’m getting close to being able to benefit from it. And when I say “this data,” I mean “our data” because it really is ours. We’re the ones who generated the data through our self-managment decisions, and we’re the ones who will benefit the most from using that data to make decisions. All I’m really asking for is a way to log in to CareLink without using a web browser and to retrieve data securely.

I’ve been working on my pump+CGM data visualizer a lot recently — most evenings in fact. On my Mac, I can extract events from a comma-separated value (CSV) file generated on the CareLink website, and I can pick out “interesting” events that are relevant for self-management. Now I’m working on being able to store those interesting events in a form that I can send to my iPod. (Then there are the tasks related to visualizing the data, but I’m starting small.)

It’s taking me longer than I expected to build this application. Objective-C isn’t hard, but learning the ins and outs of any new framework library is always a bit involved. (Turns out I’ve been using a lot more C++ than I had expected . . . not that there’s anything wrong with that.) And I realized that I actually need to build two applications: one part that sits on a “traditional computer” that can talk to CareLink and the other that visualizes the data on an iPhone, iPod, iPad.

Here’s a little example of the raw data that I will eventually use to generate graphs and an annotatable logbook:

3/30/11|16:20:00|GlucoseSensorData|AMOUNT=106, ISIG=10.2
3/30/11|16:25:00|GlucoseSensorData|AMOUNT=98, ISIG=9.71
3/30/11|16:30:00|GlucoseSensorData|AMOUNT=98, ISIG=10.59
3/30/11|16:35:00|GlucoseSensorData|AMOUNT=100, ISIG=10.66
3/30/11|16:40:00|GlucoseSensorData|AMOUNT=102, ISIG=10.94
3/30/11|16:45:00|GlucoseSensorData|AMOUNT=102, ISIG=10.6
3/30/11|16:50:00|GlucoseSensorData|AMOUNT=102, ISIG=10.56
. . .
3/30/11|18:14:01|BolusWizardBolusEstimate|BG_INPUT=195, BG_UNITS=mg dl,
  CARB_INPUT=0, CARB_UNITS=grams, CARB_RATIO=8, INSULIN_SENSITIVITY=50,
  BG_TARGET_LOW=110, BG_TARGET_HIGH=110, BOLUS_ESTIMATE=1.7,
  CORRECTION_ESTIMATE=1.7, FOOD_ESTIMATE=0, UNABSORBED_INSULIN_TOTAL=0,
  UNABSORBED_INSULIN_COUNT=2, ACTION_REQUESTOR=pump
3/30/11|18:14:01|BolusNormal|AMOUNT=1.7, CONCENTRATION=null,
  PROGRAMMED_AMOUNT=1.7

Isn’t it about time to put aside my “iPhone developer impostor” feelings and just start coding? Yes it is. I’m not going to build this app unless I start writing it.

ABC: Always Be Coding.

Since last Saturday I’ve learned a lot. I’ve picked up the syntax of Objective-C, which is causing this C++ programmer to “think differently.” I like what I’ve seen from it so far, but we’ll see what I think after building something real. I’ve made a couple “Hello, World!” applications, just enough to get a few basic skills using Xcode and Interface Builder.

Now the hard work begins.

I’ve made a list of requirements for the first couple of (internal) versions of the app; so I know what needs I plan to satisfy. I’ve picked an external library to plot the CGM data. And I’ve started working on the functional design, sketching a few different views that people will use to interact with their data. (I usually hate graphical user interface design, but something about the UIKit components seem to be amplifying my scanty abilities with interaction design.) I still have to figure out the data model — that is to say, the architecture — but I think that should follow from the views I create, which of course is supposed to visualize items in the pump/CGM data model.

Tomorrow I’ll try to put a few components together. Stay tuned!