The business practice of brainstorming has been around with us so long that it seems like unadorned common sense: If you want a rash of new ideas, you get a group of people in a room, have them shout things out, and make sure not to criticize, because that sort of self-censoring is sure to kill the flow of new thoughts. . . .

[Alex Osborn, the 1940s ad man and inventor of brainstorming] thought, quite reasonably, that creativity was both brittle and fickle: In the presence of criticism, it simply couldn’t wring itself free from our own minds. We could only call our muses if judgments didn’t drag us down. Osborn claimed that this very brainstorming process was the secret to BBDO’s durable creativity, allowing his ad guys to produce as many as 87 ideas in 90 minutes—a veritable avalanche. “The brainstorm had turned his employees into imagination machines,” writes Jonah Lehrer in a long, excellent article in The New Yorker. But as Lehrer argues, the only problem with all this is that brainstorming is total bullshit. . . .

• You’re More Creative Working Alone: “Putting people into big groups doesn’t actually increase the flow of ideas. Group dynamics themselves—rather than overt criticism—work to stifle each person’s potential.”
• Criticism Improves the Brainstorming Process: “Usually, inventions often begin when an inventor spots a problem. Good ideas usually don’t hang by themselves, unattached. They come about as solutions. Thus, allowing criticism into a room full of people trying to brainstorm allows them to refine and redefine a problem.”
• Creativity Is About Happenstance, Not Planning: “Too much familiarity bred groupthink. Too little meant that they didn’t have enough chemistry to challenge each other. The most productive groups were those with a baseline of familiarity but just enough fresh blood to make things interesting. . . . Studies have shown that the most successful groups of scientists also work in extremely close physical proximity. Just being around another creative person is vital to the process . . .”

A half-decade after Herb Sutter wrote that the “free lunch” of Moore’s Law is over, he’s back with his prophet’s wisdom about where we’re going in his January Dr. Dobbs article, “Welcome to the Jungle”. I’ll give you a moment to decide whether to get the Guns N’ Roses song out of your head or use it as a backdrop for this juicy quotation:

If hardware designers merely use Moore’s Law to deliver more big fat cores, on-device hardware parallelism will stay in double digits for the next decade, which is very roughly when Moore’s Law is due to sputter, give or take about a half decade. If hardware follows Niagara’s and MIC’s lead to go back to simpler cores, we’ll see a one-time jump and then stay in triple digits. If we all learn to leverage GPUs, we already have 1,500-way parallelism in modern graphics cards (I’ll say “cores” for convenience, though that word means something a little different on GPUs) and likely reach five digits in the decade timeframe.

But all of that is eclipsed by the scalability of the cloud, whose growth line is already steeper than Moore’s Law because we’re better at quickly deploying and using cost-effective networked machines than we’ve been at quickly jam-packing and harnessing cost-effective transistors. It’s hard to get data on the current largest cloud deployments because many projects are private, but the largest documented public cloud apps (which don’t use GPUs) are already harnessing over 30,000 cores for a single computation. I wouldn’t be surprised if some projects are exceeding 100,000 cores today. And that’s general-purpose cores; if you add GPU-capable nodes to the mix, add two more zeroes.

The big takeaway for software engineers like me is that we’d best be learning how to develop solutions using the emerging APIs so that we can harness all of those extra orders of magnitude of scalability. That involves figuring out how to . . .

• Deal with the processor axis’ lower section [of Sutter's chart] by supporting compute cores with different performance (big/fast, slow/small).
• Deal with the processor axis’ upper section by supporting language subsets, to allow for cores with different capabilities including that not all fully support mainstream language features.
• Deal with the memory axis for computation, by providing distributed algorithms that can scale not just locally but also across a compute cloud.
• Deal with the memory axis for data, by providing distributed data containers, which can be spread across many nodes.
• Enable a unified programming model that can handle the entire [memory/locality/processor] chart with the same source code.

Perhaps our most difficult mental adjustment, however, will be to learn to think of the cloud as part of the mainstream machine — to view all these local and non-local cores as being equally part of the target machine that executes our application, where the network is just another bus that connects us to more cores. That is, in a few years we will write code for mainstream machines assuming that they have million-way parallelism, of which only thousand-way parallelism is guaranteed to always be available (when out of WiFi range). . . .

If you haven’t done so already, now is the time to take a hard look at the design of your applications, determine what existing features — or better still, what potential and currently unimaginable demanding new features — are CPU-sensitive now or are likely to become so soon, and identify how those places could benefit from local and distributed parallelism. Now is also the time for you and your team to grok the requirements, pitfalls, styles, and idioms of hetero-parallel (e.g., GPGPU) and cloud programming (e.g., Amazon Web Services, Microsoft Azure, Google App Engine).

p.s. — I can’t believe that it’s been almost four years since I took a course with Herb out in Washington. That was some hard-core learnin’.

(From How to Think about Parallel Programming: Not!)

• When optimizing code for GPU, FPGA, or CPU, definitely focus on pipelining and overall throughput, not just local optimizations.
• There’s a trade-off between “faster” and “sooner.” It’s not always worth saving a few seconds (or even a few minutes) if the kernels take hours or days to compile. (Then again, sometimes it is.)
• Try to reduce dependence on the language/compiler “stack” that removes inefficiencies. The optimizer does good work, but you can do things to help it. Think about the hardware or architecture format. It’s not a sin to reduce the amount of abstraction in the service of performance. Pay attention to things that affect processor pipelining and cache movement.
• BTW, some languages and technologies exist to provide higher level programming that’s close to the hardware, but they’re proprietary, secret, or still in R&D.
• Use algorithmic optimization techniques. Step back and find the shortest-time computation.
• Avoid using if statements. The goto construct is considered harmful, but if is basically the same thing. Instead think about state machines and polymorphism. There’s no branch-prediction penalty to pay, since the system “just is” in the state it’s supposed to be in. The logic is clearer, because there are no switches, making it easier to test, too.
• Don’t always assume that floating-point values are necessary. Integers can often be creatively used and are far faster for math than double-precision numbers.
• Of course, there’s a compromise between speedy/efficient and readable/maintainable.
• Aim to structure programs as “symbolic intent.” Mathematical descriptions are bad ways of expressing programs. Think about functional programming models instead of procedural.

If you want to know more, you should definitely watch the half-hour interview. And if your reaction was more along the lines of “Yes, yes; that’s all true, and it’s how I design my image processing code,” then I definitely hope you’ll consider applying for the GPU/multicore engineering position we have open.

We need someone with GPU and multicore programming skills. We’re looking for experience with CUDA, OpenCL, OpenMP, Intel’s Threading Building Blocks, or similar technologies. If you’re into making algorithms run wicked fast, you should definitely apply.

The other position focuses on image processing and code generation. If you like implementing image processing algorithms and converting MATLAB code to C code, then this is the job for you.

I’ve been at The MathWorks for almost fourteen years now, and it’s a really great company with an excellent corporate culture, competitive compensation, fantastic benefits, and lots of perks. Because everyone uses MATLAB and because we’ve made some very sensible business decisions over the last 28 years, it’s a very stable company to work for. (Did I mention that we’re putting up our fourth building in our Natick campus? And I think I also mentioned that the entire worldwide staff went on a cruise a few years ago.)

If image processing isn’t your thing, we have dozens of other positions open. Everything from web development to legal department work. Human resources to customer service. Technical writing to application engineering and consulting. Marketing to program management. QE, sales, usability, and more software development positions than you can shake a stick at.

Come, help us accelerate the pace of engineering and science worldwide. And if you do apply, tell them I sent you.

Now, our love has finally reached the age of majority. It can vote, form contracts, go to the casino, and buy lottery tickets and cigarettes legally. It no longer has the luxury of being sent to juvie for minor infractions. In New Hampshire it can ride a motorcycle without a helmet. In Puerto Rico and the U.S. Virgin Islands, it can buy its own booze.

It’s a brave new world for our relationship.

It’s true I’ve never watched any of your shows or tried any of your recipes, but I saw your picture on a magazine cover or two at the supermarket. Now that you’ve announced that you have type-2 diabetes, I feel like we’re definitely on a first-name basis.

Type-1, type-2, LADA, gestational—no matter the kind, diabetes sucks. I’m sad to hear that you joined our club. It’s a bummer, but there’s a really big supportive group of people online and in real life who are here for you.

I know there’s a brouhaha brewing about the how/why/when of your announcement, but I frankly don’t care. . . .

Except I will say this: You owe us.

You have a high profile because of your pre-diabetes life. And you have partnered with Novo Nordisk to promote pharmaceuticals, putting you squarely in the diabetes community. You best be using your influence to help people with diabetes. Here are some ways:

• Promote understanding of the differences between type-2 (which you have) and type-1 (which I have), since so many people in the US think there’s just one kind.
• Encourage healthy lifestyle choices for everyone, whether touched by diabetes or not. T-2 is more than diet and exercise, but we have to be honest about their role. There’s no guarantee one way or the other, but every little thing we do (within reason) makes a difference.
• Help people with diabetes in your audience understand that they are more than their disease, that there will be better days and worse days, and that they can do this.
• Work with CDEs (at Novo and elsewhere) to develop a message of empowerment that people with diabetes can use to improve their own self-management as they make choices and work with their family and healthcare providers.
• Stress that there’s more to diabetes than Novo’s drug-of-the-day. Be holistic.

Remember that a bunch of people are watching you. The diabetes online community is watching, too. We’re nice people, but we look out for our own. Based on what I’ve seen in the past, we will cut you like a piece of pecan pie if we think you’re using diabetes for your own benefit and aren’t giving back.

Diabetically yours!

Starting around Christmas time I start to feel a bit overwhelmed. New prezzies (usually) means new books to add to my reading list. Extra time off work means more opportunities to clean up the detritus of the previous year (or longer). That’s a mixed blessing: freeing up space in my brain to concentrate on the right things without actually getting to spend the time doing those things. I’m being much more ruthless about just chucking stuff this year than in the past, and I think I’ll be done soon.

Almost being done is very good, because I have goals. (I don’t go in for New Year’s resolutions. Anything worth doing is worth starting at any point in the year. Why wait for a particular date to have a clean slate?) I tend to keep my goals to myself, but I’m willing to say that one of them involves trying to pimp-slap my out-of-control bookshelf by reading a certain number of pages each week. I figure that even an incredibly slow reader such as myself should be able to average 15 pages/day.

This goal-thinking was (is?) getting me a little down this year. So much of what I want to do in 2012 involves feats of athletic prowess, but my feet were threatening to get in the way of those feats. Lisa, the awesome exercise psychologist of my dreams, is (slowly) helping me see that I am more than my goals and accomplishments, but I still missed running because I really like it.

The week before my injury, I had a wonderful 12+ mile run that took me to the end of one branch of our local rail-trail and then past it into the exurban farmland and acreages of the neighboring towns before picking up the start of the other branch of the trail and following it home. I am eager to get back to that.

For sure, I was was also stressing that not doing these long training runs might leave me ill-prepared for the Around the Bay 30K in late March . . . or possibly incapable of running it at all. Eventually I told myself that I had to stop worrying about whether or not I would be able to do ATB—or the NYC Tri in July or the half-Ironman in August—and just concentrate on getting well. I could still ride my shiny new bike in the basement, there’s always plenty of swimming to do, and on the last day of work in 2011 I got a personalized weight-training program, which I started last week.

Sometimes I need to be reminded to look at the “big picture.”

By the middle of last week my foot didn’t really hurt very much, although I noticed twinges now and again, especially when I moved my foot in particular ways. It kinda sounded like plantar fasciitis, and it kinda didn’t. Everyone I talked to about it had horror stories about how PF messed up a fellow runner for months or years on end, so I was determined to find out what was actually wrong with me before doing anything stupid. I also wanted to find out the right way to start back up when the time was right. I didn’t want to rush into anything, but I could feel myself losing the exercise-every-day-after-work-and-go-to-the-pool-a-few-mornings-each-week habit that I had developed by the beginning of December.

On Friday, I went to my podiatrist, who said (again), “Boy, your feet are eff’ed the fuck up . .  all loosey-goosey and flat and shit.” And then he went on to say, “You don’t have plantar fasciitis, but you’ve gone and slightly fucked up the long tendon that connects your calf to your big toe via your heel. It’s amazing you’ve been able to get way without this kind of shit for so long. You need expensive orthotic shoe-inserts to keep this from happening again. Now, let me teach you some calf stretches and recovery techniques. You should start popping Aleve like a fiend, too. I’ll tape up your foot, and you can go running tonight if you want. But don’t go for any PRs or bullshit like that for a little while.” (I’m paraphrasing just a wee bit here.)

So I’m quite relieved. I’ve gone running twice since visiting my not-at-all-potty-mouthed podiatrist. Each run felt good, foot-wise. The left one isn’t 100% in the hours afterward, but it’s 10x better than the days after I injured myself. The runs also felt shorter and more difficult than I remember them being a month ago. Even so, these short, difficult runs were awesome.

Speaking of amazing things. I’ve been out on my road bike twice this new year already, and each time I wore shorts. New Years Day was the first time I’d been out since early October, and the lingering chill on the thawing roads couldn’t bring me down. Saturday morning’s sunny, 50°F, 25-mile ride had no chill at all. By way of contrast, at this point last year we had more than 30 inches of snow on the ground, and we were in for 60 more.

So I guess there’s that, too.

Oh, and there’s swimming! The Friday before Christmas I got up super-early despite not needing to go to the office. The pool was open, and I had the chance to get a full hour-and-a-half swim, instead of my typical 40-or-so minutes. The last time I had this opportunity, I swam two miles, and I wanted to give it another go, testing my blood glucose along the way. The results were very much like last time—better actually. My BG stayed almost constant; my 250-yard split times were fairly consistent throughout; and I swam a quarter mile farther in the same amount of time.

Now that I’ve written this, I’m reminded how fickle I can be. Yes, winter can be a cold, dark, lonely, depressing, snowy, stir-crazy-making time of the year. But it seems that all I need is a good report from the doctor, a run or two, an outdoor bike ride, a nice swim, and the constant loving support of Lisa for me to feel like a good spring is just around the corner.

p.s. I guess I should add that last night Lisa and I watched a documentary about U.S. athletes in the Beijing Olympics. It wasn’t the best thing ever, but it sure looked beautiful on our new high-def TV. I can barely wait to see this year’s games. Hurry summer don’t be late.

This is another one of those posts. Every year between Christmas and New Years Day, I try to use the quiet week to get stuff done and tie up loose ends. Last year, I cleared out a bunch of notes. This year, I’m looking at presentations and slides from the QCon SF 2011 conference (wrap-up). Its focus on software architecture and project management is about 75% of my job, so many of the presentations seemed tailor-made for me. Here’s some of what I learned.

Erik Doernenburg. “Software Quality: You Know It When You See It” has a really good slide deck that got me thinking about some projects I might want to set up. It’s full of practical, usable suggestions:

• View the code at the 1,000 view, rather than ground-level or 30,000 feet.
• Look at the test-to-code ratio, not just code coverage.
• Graph the change of metrics between versions and revisions, compare across different parts of the code, and look at them relative to industry standards.
• Measure the “toxicity” of code by rolling up various quality metrics about a bunch of modules into stacked bar charts.

We should pose these questions during design and code reviews:

• Is the software/change of value to its users?
• How appropriate is the design?
• How easy is the code/design to understand and extend?
• How maintainable is the software?

It was full of some really great links to things like Metrics tree maps (a.k.a., pretty heatmaps for source code) as well as a few tools: SourceMonitor, iPlasma, and using Moose to visualize quality.

Joshua Kerievsky. “Refactoring to Patterns” — some notes:

• Refactoring is like algebra’s equivalence-preserving manipulations. “Design patterns are the word problems of the programming world; refactoring is its algebra.”
• Understanding the refactoring thought process is more important than remembering individual techniques or tool support.
• Code smells have multiple refactoring options and often benefit from composite refactorings.
• Look for automatable refactorings first. Consider changing the client of smelly code before the smelly code itself.

Guilherme Silveira. “How To Stop Writing Next Year’s Unsustainable Piece Of Code” was pithy and thought-provoking.

• There is no value for architecture or design without implementation. That’s just interpretation of the software.
• “New language. New mindset. new idiomatic usage. Same mistakes.”
• Complexity and composition are natural and good, but if they’re invisible, they’re evil.
• Start with a mess and refactor right away. Starting “right” is hard (and misguided thinking). Refactor for better, not just prettier.
• Make complexity easier to understand and see.
• Hiding complexity in concision hurts testability, since no one knows the complexity is there. Furthermore, if it’s hard to test, it’s also hard to use correctly.
• “Model rules. Do not model models.”

Michael Feathers. “Software Naturalism: Embracing The Real Behind The Ideal” is a presentation that I would like to see/hear, since the slides seemed full of information but weren’t self-explanatory. Here are two things I could glean: 80% of software defects in large projects were in 20% of the files. In general, the more churn in a file, the more complex it tends to be.

Panel: “Objects on Trial” was perhaps the most unusual presentation, since it was a mock-trial. I use objects all the time . . . some of them are good . . . some demonstrably so. Even so, I never latched onto the idea of object-oriented (OO) design versus objects as types. The four panelists, in one way or another, basically said, “That’s the problem.”

One of the panelists drew an extended analogy between the space program and OO. The space shuttle (which we all love) was fixated on reuse but basically was a waste of heavy lifting; people don’t reuse the right stuff. In software, object reuse is largely accomplished by cut-and-paste copying of boilerplate code that does close to what you want. Of course, the panelist acknowledged that we do reuse the ideas in OO via design patterns, and no one seems to have much of a problem with that. Ironically, having a rich pattern language means that software engineers are in a better place than ever before to use objects correctly.

A key problem with our approach to objects is that we’ve failed (generally in software engineering) to handle complexity well, which was supposed to be the point of OO design. A conflation of beauty and OO design makes things worse. Internally, software is ugly, and beauty shouldn’t be a goal. Making a fetish of beauty makes code inflexible because people don’t want to extend the beautiful thing that works.

For other panelists, objects weren’t the problem at all. For them it’s static typing in “OO languages,” such as C++, Java, and C#. We’re at a place now where all of the good things about OO have been lost in an attempt to make OO languages as fast as C. This runs counter to the goal of having “ordinary,” understandable code. Generic programming using strongly typed (possibly template heavy) languages just makes everything complicated.

For me, it’s moot. C++ is what I use, and I don’t have a large proprietary object system that I can tap into for reuse. I’m in the camp that uses C++ objects to generate new types for data hiding and aggregation, as well as (to a lesser extent) reuse. But some of these types are generic, template classes that are hard to understand. I plead “no contest.”

sigh

Anyway, here’s a few pictures and some updates . . . bullet-point style!

• Last Saturday Lisa and I went to New York for the day to visit a few galleries. The “Calder 1941″ exhibit at Pace’s 57th gallery was amazing! And Nan Goldin’s “Scopophilia” show at Matthew Marks is worth a trip to Chelsea. Our day-trip occurred 52 weeks after the trip where we met Kim, Gina, Caroline, and Allison. Time flies!
• Sunday we traveled into Cambridge to see “Tinker Tailor Soldier Spy.” Lisa had been looking forward to it for months, and it didn’t disappoint. It was our second trip into Cambridge in as many weeks. The previous weekend we attended an alumni event there, and I got a shout-out from the new president of the college. Evidently, we engineers from liberal arts schools are rare beasts.
• I haven’t gone for a run since last Thursday, when I tested the waters with an easy three-mile treadmill workout. The next day my foot was a little cranky again, so I’m taking some more time off running. I’m still riding and swimming, but I miss my long runs and my speedwork sessions.
• Speaking of swimming, I got a bit depressed Monday and yesterday when I realized that the “really good” triathletes in my races cover the same distance in half the time it takes me. So I talked to my sports psychologist (Lisa) who helped me with some perspective: I’m not a super-fast swimmer—as long as Dara is at the pool, I’ll never be the fastest—but I shouldn’t worry so much as long as I’m still making progress. If I put too much pressure on myself, then I won’t have any fun. And, even though it’s really hard for me to seek assistance, I need to ask some of my of peeps and/or a coach to look at what I’m doing and give me some pointers. (I find it difficult to work at something for a long time and not be as good at it as I believe I can be. It’s good that it keeps me motivated, but I’m trying to work on managing frustration.)
• When I went to the pool this morning, I decided I was just going to swim without worrying about times or how much progress I am (not) making or other people’s abilities. Part of this involved changing the way that I talk to myself while swimming; if I can’t make the voices in my head say positive things, perhaps I can give them something else to talk about. My inner boatswain kept me going with this conversation: “We’re going to do three things today: stop dropping my glide arm so much after entry; roll from side to side better during the stroke; and pull through the whole stroke farther. Bup bup bup!” That seemed to work. Even though I wasn’t worrying about times, I was encouraged by the splits I saw. Turns out, I swam the fastest ever by almost a minute per mile. Yay!

What’s new with you?