Category Archives: Software Engineering

Performance Programming Links

Posted on October 22, 2010 by Jeff Mather

Faithful readers: This dispatch falls into the bucket of “stuff that’s about work and not diabetes.” (Not that diabetes isn’t its own kind of work.) This is what I do for my 9-5. It’s the stuff I do between thinking about diabetes, my next bike outing, where I want to travel someday, and all those other things. Don’t worry, we’ll get back to all that “good stuff” soon enough; but this post is for the lurking coworkers and people searching on the Google.

Here are a few performance-related articles and presentations that I stumbled upon recently:

Igor Ostrovsky at Microsoft wrote a very useful description of various processor cache effects. Here are the notes I took while reading it:

  • Array access is slower than math
  • Data alignment can determine how many cache lines are touched
  • Keep modifications/accesses within the same cache level
  • Be aware of instruction level dependencies and parallelism opportunities
  • Avoid touching too many memory locations from the same cache line, causes contention
  • There’s a trade-off between the size of what you’re putting in the cache and the number of elements you touch in it. — What does that mean, Jeff? Not exactly sure, but cache assoc. is usually not a huge deal compared to other issues.
  • Because cores have their own caches and because memory needs to remain consistent, avoid letting two threads modify data on the same cache line.
  • Even when you think you know what you’re doing, there’s other crazy stuff going on.

Isn’t that last one the sad truth about performance optimization?

Igor also has an excellent article about branch prediction. Basically, if you structure your code so that if branches are predictably true (or predictably false) the CPU can start walking down that code path until it’s proved wrong. But if it’s close to random, you’ll see performance hits.

Joe Duffy, also at Microsoft, debunks the “premature optimization is evil” myth. Joe summarizes the article himself quite well, so I’ll just quote him.

In this short article, we’ll look at some important principles that are counter to what many people erroneously believe this [“avoid premature optimization”] statement to be saying. To save you time and suspense, I will summarize the main conclusions: I do not advocate contorting oneself in order to achieve a perceived minor performance gain. Even the best performance architects, when following their intuition, are wrong 9 times out of 10 about what matters. (Or maybe 97 times out of 100, based on Knuth’s numbers.) What I do advocate is thoughtful and intentional performance tradeoffs being made as every line of code is written. Always understand the order of magnitude that matters, why it matters, and where it matters. And measure regularly! I am a big believer in statistics, so if a programmer sitting in his or her office writing code thinks just a little bit more about the performance implications of every line of code that is written, he or she will save an entire team that time and then some down the road. Given the choice between two ways of writing a line of code, both with similar readability, writability, and maintainability properties, and yet interestingly different performance profiles, don’t be a bozo: choose the performant approach. Eschew redundant work, and poorly written code. And lastly, avoid gratuitously abstract, generalized, and allocation-heavy code, when slimmer, more precise code will do the trick.

Follow these suggestions and your code will just about always win in both maintainability and performance.

If you do anything that falls under the labels of “multicore,” “multithreaded,” or “multi-processor,” then you should definitely add Multicore Info and Intel’s software blogs to your RSS feed aggregator.

One recent Intel offering is a recording of “Introducing Intel Parallel Building Blocks”. The Intel folks discusses Intel Cilk Plus, Threading Building Blocks (TBB), and Array Building Blocks. Here are some notes from the 45 minute presentation:

  • Intel Cilk Plus is a lower-level way to expose parallelism and the potential for vectorization to C++ code. It currently only works with the Intel compiler.
  • TBB is awesome. That’s me saying that. I use it to easily exploit data parallelism in our existing codebase. (Because I use it already, I didn’t take many notes on it. Sorry.)
  • Array Building Block (ArBB) provide the highest level of abstraction presenting the ability to specify array data containers (1-D to N-D, including nested containers and [soon] sparse data) and then do vector operations on the data in the containers. Sounds familiar.
  • Cilk Plus includes reducers to prevent data races, has #pragma simd to empower the compiler to make vector assumptions, and has a mechanism for specifying a range of vector data.
  • Parallel Inspector and Parallel Amplifier are alleged to help squeeze performance out of code that uses TBB.
  • ArBB uses a two-part compilation process, one of which is a runtime JIT.
  • There are collisions between the schedulers in Intel MKL and Cilk (and TBB, etc.). Beware.

And finally, there’s a recording of a webinar about Intel Parallel Advisor that may be of interest. It’s next in my queue of things to watch.

Posted in Computing, Fodder for Techno-weenies, Software Engineering, Worthy Feeds | Leave a comment

An Invitation to Diabetes Researchers

Posted on October 17, 2010 by Jeff Mather

I just finished reading Michael Bliss’s excellent The Discovery of Insulin about the amazing work at the University of Toronto in 1921-1923 by Banting, Best, Collip, and Macloed. For most readers, it’s surely a story about discovery and rivalry and collaboration in medicine, culminating in the first effective treatment of diabetes and (very quickly thereafter) the Nobel Prize for Medicine.

For me it was also a history of what I avoided by being diagnosed with type 1 diabetes after 1922. Bliss includes a small sample of the lives of some of the people afflicted with diabetes before the discovery of insulin. I was truly inspired by those who were fortunate enough to receive this “miracle drug,” but I was heartbroken by all of the people of that age who didn’t make it because the only treatment was to survive on a meager 500-or-so calories for intolerable months until slipping into the coma of ketoacidosis and then (eventually) death.

It’s been almost 89 years since the first successful clinical use of insulin, but we still don’t have a cure. At best, insulin is the key part of a hormone replacement therapy where people with diabetes try to mimic a pancreas. At worst, insulin is a fickle treatment that is difficult to use, expensive, and out of reach of millions of people worldwide.

While I’m so grateful for what I have — a treatment that gives life — what we need now is a cure . . . even if it’s only useful for people who are not yet sick or are newly diagnosed.

If you’re working on a cure or a new therapy for diabetes, the rest of this post is for you.

I will help you with any technical software aspects of your research that you need. If you have questions about MATLAB or C++, I will do my very best to answer them. If you need help with software design or object-oriented analysis/design, I can help you structure your solution. If you require some help with the non-development aspects of software development — such as project management or using tools — I will help you there, too.

My particular areas of expertise are MATLAB, C++, medical imaging formats (such as DICOM), some familiarity with image processing, and multithreaded programming using Intel’s TBB. I also know my fair share about life with diabetes. I can’t promise that I’ll have all of the answers to your particular question, but I’ll work with you as much as I can. And I’ll connect you with my coworkers as much as I can, if it comes down to that. Finding a cure that’s safe, effective, and universally accessible isn’t my day job, but moving into the third era of diabetes where no one dies from it and no one has to impersonate a pancreas is something I’d love to be a small (even anonymous) part of.

You can e-mail me at work or home, and you can hit me up on Twitter, too.

Posted in 101 in 1001, Book Notes, Diabetes, MATLAB, Software Engineering | Leave a comment

Friday Links: Oily Pancreases and Time Machines

Posted on May 28, 2010 by Jeff Mather

I’m going to try something new, posting a small group of diverse links on a (more-or-less) weekly schedule. Hopefully this will help with my hoarding problem.

Living with Diabetes: Sarah has a really great piece on her site about growing up with the “bad kind” of diabetes. At least that’s how people differentiated type 1 and 2 while she was growing up. But really, all diabetes sucks, especially if you try to ignore it.

Software Development: Keith Swenson’s article 26 Hints for Successful Agile Development is full of good advice about how to do software development effectively — even if you’re not really doing Agile development. (via Infoq)

Functional Programming: Here’s a really l-o-n-g article about functional programming. It’s good, but . . . damn!

Risk and Oil Spills: You would think that a company like BP, whose contractors deal with potentially deadly situations on a daily basis, would have a better handle on risk. Even if BP engaged in neutral cost-benefit analysis, as this NY Times article suggests, it should have chosen the option that lowered its risk exposure. Remember: risk is cost of vulnerability times likelihood of vulnerability. In the case of deep water drilling in the Gulf of Mexico, we’re seeing that the cost of an oil spill is astronomical. The probability of failure would have to be completely zero to make it worth choosing a less expensive option.

Time Machines: Stephen Hawking tells you how to build one using wormholes. He also advises against creating paradoxes where you kill yourself.

The Artificial Pancreas: So what’s this “artificial pancreas” that people with type 1 diabetes keep talking about? Let Wired magazine or Aaron Kowalski tell you. It’s not a cure, but (if done correctly) it will hopefully lower a lot of the variability that we see in our blood glucose levels. Basically, it’s an expert system built into a pump plus continuous glucose monitoring combo. It’s also a bundle of assumptions and heuristics. I find it somewhere between amazing and hella scary.

Posted in Diabetes, General, Hoarding, Life Lessons, Software Engineering, Uncategorized, Worthy Feeds | 1 Comment

Buckshot o’ Links – Software Development Edition

Posted on April 16, 2010 by Jeff Mather

I’m a hoarder. I may not have lived through the Great Depression like my grandmother did, but I seem to have inherited the gene that led her to keep dozens of plastic Cool Whip tubs in her attic “just in case she needed them.” My grandfather kept used bolts and nails for reuse. Me, I keep articles about things I think I should know some day. From smart people on Twitter and e-mail lists and coworkers I gather links to articles, PowerPoint presentations, blog posts, and videos. And they hang out in my browser tabs — forever.

It’s time to clear some of them out. Here are some daytime-themed links. (If I had a Tumblr account, I’d just post there. But I don’t need another website no one reads.)

Parallel Computing
James Reinders of Intel TBB fame estimates that datasets (images, videos, etc.) have grown 10x larger over the last five years. Sequential systems are just too dang slow to process this amount of data. In the near future, “parallel programming” will just become “programming.” (I’m still hoping for better language support so that state synchronization and multicore memory issues are as easy to get right as the sequential aspects of programming are now.)

New memory models would certainly help. A new paper, DMP: Deterministic Shared Memory Multiprocessing by Joseph Devietti, Brandon Lucia, Luis Ceze, and Mark Oskin presents some of the problems with the current memory model and provides one possible solution.

Another post on SoftTalk (sponsored by Intel) describes some of the ways that Intel plans to make parallel programming easier this year: Parallel Studio 2010, a Cilk-based offering for task parallelism, “a data-parallel centric model with safety guarantee,” new SIMD instructions, and new array notations.

You might also want a high-level view of how Intel’s offerings work together.

Herb Sutter, who really knows his stuff, wrote an article for Dr. Dobbs a couple years ago about understanding parallel performance. It’s one of a series of articles about multicore/multithreaded programming, and this helps set expectations about what’s possible and gives pointers on where to start making changes.

Miscellaneous
Visual Studio + Time Machine = IntelliTrace. Don’t just move up and down the call stack; now you can move forward and backward in time, too.

Everything you ever wanted to know about floating-point representation: Floating Point Guide. (Everything, that is, unless you work where I do. Then you just have to go down the hall to get that last 2%. Of course, you’ll be drinking from the fire hose. . . .)

What does Microsoft think are the key trends in software development? Cloud computing, the web as a platform, parallel computing, proliferation of devices (with their own capabilities, IO paradigms, etc.), agile development processes, distributed development. Nothing terribly futuristic here, and comments want to know why “mobility” (i.e., phones) isn’t on the list.

The C++0x “final draft” revision is ready for “final” comments. Here’s your chance to see the significant changes to C++ that will be part of the standard next year (they hope).

Posted in C, Computing, Fodder for Techno-weenies, Hoarding, Software Engineering | Leave a comment

The Checklist Manifesto

Posted on March 24, 2010 by Jeff Mather

How can you prevent mistakes? Some mistakes have extraordinary costs: airplane crashes, surgical infections, building collapses, nuclear power-plant explosions. Even the mistakes that don’t kill people — like software defects and leaky roofs — can slow you down by adding waste to a process, forcing you go back and spend time (and money) to fix a problem. In either case, we don’t choose to make these mistakes. So how do we prevent them?

Atul Gawande proposes a solution for all sorts of endeavors in his book The Checklist Manifesto: How to Get Things Right. It’s a short, engaging read, and I recommend it for anyone who has to apply knowledge to complete a task. That’s most of us.

We use checklists and recipes in some of our software development processes, and I’m in the process of applying what I’ve learned to improve them. I hope to share some of the results here in coming months — supposing that the final product isn’t too site-specific — but in the meantime, here are my more-or-less raw notes from Gawande’s book, which isn’t specific to any particular industry, although it was written from a surgeon’s perspective.

  • Checklists are all about managing complexity, providing a “cognitive net” against “flaws of memory and attention and thoroughness.” They are “forcing functions . . . straightforward solutions that force the necessary behavior.” A good checklist should help its users “get the stupid stuff right.”
  • The project plan is a kind of checklist. And the communication (submittal) schedule is a complexity manager. The idea is to communicate what needs to happen when in a complicated process (like building a skyscraper, writing software, or operating on a patient) and having a process in place to ensure that all of the parties in the project have shared all of the information about changing requirements and problems available at specific times.
  • It’s possible/advisable to use tools to manage complexity, conflicts, and information integration. Sometimes the result of using the tool looks like a checklist, but not always. Sometimes it’s a Gantt chart or a cook’s recipe.
  • The checklist steward: Anybody can change a checklist, but it has an owner who feeds and waters it.
  • Complex situations don’t (usually) require detailed instruction. They do require high-level goals and lots of communication. (Gawande gives the fascinating case study of Wal-Mart’s response to hurricane Katrina in 2005.) Solutions should be simple, measurable, transmissible. They should encourage team interaction and engagement. Project owners should facilitate communication for complex tasks.
  • The team huddle helps coordination, and it can help with keeping commitments. It’s important to communicate risks and issues early and often.
  • Communication should happen (at the very minimum) during specified “pause points” between transitions in the process. In the operating room, these points might be just before administering the anesthesia, before closing up the patient, etc. In an airplane cockpit, they are before starting the engines, before leaving the gate, before take-off, before landing, and so on. (Figuring out what these are in a software development process is something I’ve already started considering.)
  • Aviation uses lots of small checklists. A “normal situation” checklist should be very short. An exceptional situation should be very brief, readable, and actionable, too.
  • Good checklists are made by practitioners, usable, available, put into use, about 5-9 items long, tested, and completable in about 60-90 seconds (or less).
  • Bad checklists are long, imprecise, vague, hard-to-use, or impractical.
  • Cockpit crew have created two categories of checklists: DO-CONFIRM and READ-DO. “With a DO-CONFIRM checklist . . . team members perform their jobs from memory and experience, often separately. But then they stop. They pause to run the checklist and confirm that everything that was supposed to be done was done. With a READ-DO checklist, on the other hand, people carry out the tasks as they check them off — it’s more like a recipe.”
  • Things that are “never” forgotten by a normal practitioner don’t need to be on a list.
  • After they’re put into use, checklists need continuous improvement. They must be revisited and refined. It’s a good idea to put a publication date on them.
  • Most people — doctors, financiers, software engineers, etc. — don’t like to use checklists. They consider them neither fun nor in keeping with the “heroic” nature of their role. They feel checklists are “beneath them.”
  • Ideally they should be usable and helpful for both novices and old-hands.
  • Checklists should not be rigid, creativity- or team-killing exercises. They’re designed to “get the dumb stuff out of the way” and provide the leeway to be creative on the hard/sexy stuff. They’re frameworks for self-discipline and productivity.

Other people’s notes about the book:

The Daily Show With Jon Stewart Mon – Thurs 11p / 10c
Atul Gawande
www.thedailyshow.com
Daily Show Full Episodes Political Humor Health Care Reform

Do you use checklists? How well do they work for you? What do you like and dislike about them? Feel free to leave your feedback in the comments.

Posted in Book Notes, From the Yellow Notepad, Health Care, Life Lessons, Software Engineering | Leave a comment

Ignore the Code

Posted on March 18, 2010 by Jeff Mather

I recently discovered Ignore the Code, a well-written, nontechnical software/design weblog by Lukas Mathis. It’s really good. Go check it out.

Posted in Software Engineering, Worthy Feeds | 1 Comment

Checklists

Posted on February 18, 2010 by Jeff Mather

We use checklists a lot at work. They help us reduce waste and ensure a high quality product. If we’ve run into a problem before, we’re likely to run into it again, so we might as well go down the checklist of “Did you think about this?” and “Did you do that?” items before submitting code into the repository.

But our checklists have gotten a little long and messy, which raises the risk that people won’t use them at all. Part of my job is to improve our team best practices and checklists, so I’m working out how to make all of those checklist-bound countermeasures fresher and more accessible.

So I’m very hopeful that Atul Gawande’s The Checklist Manifesto: How to Get Things Right, which arrived on my desk today, will give me some in-the-trenches perspective. I’ll keep y’all posted on what I learn.

Posted in Book Notes, Computing, Life Lessons, Software Engineering | Leave a comment

Implementing Lean Software Development – Part 1

Posted on February 11, 2010 by Jeff Mather

At midday yesterday my carpool buddy and I looked at the weather map, saw a scary blob of impending frozen doom, and decided to heed the Governor’s advice to go home. But the snow that was forecast — the blizzard that was supposed to mess up the evening commute — never really materialized here in the Commonwealth. Oh well. Only 1-2″ to shovel.

While working away the afternoon at home, I started reading Mary and Tom Poppendieck‘s Implementing Lean Software Development. I guess “rereading” would be more accurate, this time with an eye toward actually implementing the ideas. Here are my notes from the first few chapters.


What is Lean Production?

  • Use just-in-time (JIT) flow of work products.
  • Stop the line to fix problems as soon as they are discovered.
  • Eliminate stockpiles of in-process inventory. Create small batches.
  • Make mistakes impossible from the beginning.

Product development is knowledge creation via exploration of designs, experimentation with prototypes, and integration meetings that evaluate designs and make decisions.

Schedules have synchronization points, but let expert workers function autonomously between them.

Explore multiple options and make competing prototypes. Defer the decision about which to actually build until the right time (usually later than you think).

Software should be easy to change. Employ all of those high-quality best practices and metrics during construction.

The Toyota Product Development System — upon which the Poppendiecks base their book — is very empirical. The final idea/design/product emerges during development. At the outset, define the goals to meet, not the technology or architecture you’re going to use to satisfy it.

1: Eliminate Waste — Minimize the time from request to fulfillment

  • Waste is everything that doesn’t actually add value to a customer feature. Focus on value.
  • Reduce the amount of partially done work (inventory). [Basically, don't work on what you haven't been asked for, and complete what you start. Get it written, tested, checked-in, and reviewed.]
  • Reduce the time between requirements, coding and testing. This helps reduce requirements churn. Use test-driven development (TDD). Shun “big bang” integration.
  • Avoid extra features and scope bloat.

2: Build Quality In

  • Control the conditions for creating quality components, preventing defects.
  • Write tests first. Use TDD.
  • Continuously integrate. Integrate code and tests together.
  • Write less code. Keep it clean and simple. Don’t duplicate. Refactor.

3: Create Knowledge

  • Go from a sketch to detailed design during construction.
  • Release a minimal set of functionality to customer for feedback.
  • Create daily builds and get feedback (knowledge) from integration tests.
  • Have an experienced team. Pour learning back into the team.
  • Build a modular architecture that supports adding/changing features as you learn more.
  • Generate new knowledge through disciplined experimentation. Codify that knowledge and make it available to the larger organization. Build a knowledge base. Encourage long-term thinking.
  • Continuously improve processes.

4: Defer Commitment

  • Try to make most design decisions reversible/undoable/nonbinding.
  • Schedule irreversible decisions for the last possible responsible moment.
  • Always move toward the concrete, and identify where change is likely to occur.
  • Experimentation reduces the risk of making the wrong decision.
  • Planning is an exercise, but plans are overrated. Avoid “plan-driven methods.”

5: Deliver Fast — Compete on time

  • Be faster than your customers’ ability to change their minds.
  • Requires low defect rates, high quality processes, good understanding of the customer.

6: Respect People

  • Favor ownership at the worker level, not top-down.
  • An entrepreneurial leader should own the product and foster engagement.
  • Develop and nurture technical expertise as a competitive advantage.
  • Give general plans with reasonable goals. Then let the group self-organize to meet them.
  • There is no “one best way.” There is continuous improvement, though.

7: Optimize the Whole

  • Optimize the whole value stream, that is, not just local processes.
  • Pay particular attention to where products,code and processes transition between departments, teams, organizations or physical locations.

The product development timeline:

  • Concept — What’s the unmet customer need or desire?
  • Feasibility — Test feasibility by experimentation. Build stuff. Ship betas. Design systems. Investigate the major features of the business process, key hardware modules, interfaces, boundary behaviors, software architecture and constraints. Can the product really be built? Will it work?
  • Pilot — Work with your customers. Show off the product; collect data; iterate. Let people choose from multiple options by playing and “voting with their attention.” Run multiple pilots to refine the design.

You have to go way beyond meeting basic expectations, even beyond adding new features or improving performance. You have to identify needs customers don’t know they have and then delight them by meeting those needs. This requires developing a deep understanding of the customers’ world.


That’s all for now. I’ll post more as I continue my way through the book — perhaps the next time it “snows.”

Posted in Book Notes, From the Yellow Notepad, Software Engineering | Leave a comment

Why I Love My Job

Posted on January 13, 2010 by Jeff Mather

So you already know that I ♥ my pancreas. I hope that soon I’ll be able to ♥ both my real and “artificial” pancreases. Kerri at Six Until Me has a great write-up of an announcement between JDRF, Johnson & Johnson, and DexCom which should speed my new love’s arrival. (Update: The Diabetes Mine has even more details and an interview.)

And I ♥ my job. I won’t try to take credit for any of the hard work that JDRF-funded scientists and industry R&D folks are doing to make an artificial pancreas a reality. They deserve all of the credit and so much more. But I know for certain that many people at those institutions and others are using our tools as part of the day-to-day toil of making this a reality. (MATLAB’s graphics have a very *ahem* distinctive appearance, which I’ve seen in presentations about the artificial pancreas and diabetes self-management findings.)

The MathWorks‘ mission statement is that we “accelerate the pace of engineering and science worldwide.” Although it might just sound like nice words on a web site or T-shirt, at times like this it makes me very proud to know that we really do make great things happen faster. And knowing that — even as a peripheral actor — I help with progress, well, that inspires me to do my job that much better, by putting a little more care and polish into my features (especially those related to medical imaging), by rooting out all of the hidden software defects that I might otherwise overlook, and by working hard to get as many of those customer-requested features into the product that I can. That may sound a little corny, but I know that everything I do to accelerate the development of our products helps other people accelerate the creation of their products. And I’m not the only one in the office who feels this way.

Now, back to (indirectly) helping other people fight the good fight. . . .

Posted in Diabetes, MATLAB, Software Engineering, This is who we are | 1 Comment

Implementing Lean Software Production

Posted on November 12, 2009 by Jeff Mather

Just a short post today, dear readers, seeing that it’s a bit later than I’d expected.

A coworker and I decided today that we really want to try our hand at lean software engineering. Our team has been moving this direction for a while in fits and starts — thinking about waste, preventing defects, and doing root cause analysis — but this is our first time at getting to the heart of truly “lean” production. Fortunately, we have good management and team members, who aren’t afraid to optimize processes; and it wasn’t hard for us to get the approval to investigate how we work.

We’ve never taken a really hard look at the “value stream” of our product development before. The first step involves examining all of the things that we do to get a product out the door: from the time that we decide to start working on a feature until it’s in our customers’ hands. In the next step, we’ll identify all of the things that don’t actually contribute to the feature from a customer’s point of view (i.e., the parts that aren’t “value”) and try to eliminate or at least minimize the time that we spend on those tasks.

We want to make this a whole team activity, but right now my coworker and I are just starting to read The Toyota Way and Implementing Lean Software Development so that we can lead the process effectively.

I’ll keep updating you on our progress. And if you have any advice, please leave a comment.

Posted in NaBloPoMo, NaBloPoMo 2009, Software Engineering | Leave a comment

Diabetes Application – A Sample Report

Posted on August 24, 2009 by Jeff Mather

A couple weeks ago, I presented a teaser of my diabetes application. I’ve been working on it a lot lately, creating GUIs to enter most of the data on my insulin pump and blood glucose meter. But mostly I’ve been creating charts and reports. (Putting data in is much less interesting than analyzing it, in an effort to improve my “control.”)

Here’s one of the reports. It has a bit of everything, including the kitchen sink. The one I’m taking with me to my endocrinologist tomorrow is much more focused. Visualizing all of the data associated with diabetes is kind of tricky, so I’m working on that. More features to come. . . .

(Please, don’t judge me based on my readings. :^) Right now I’m using food to cover my insulin, which is the opposite of how it should work — it seems I never really got switched from the NPH mindset. Getting past that is one of the main reasons why I’m doing in this project.)

I’m committed to releasing everything under some kind of open source license. Everything, that is, except the data and anything that would involve a diagnosis or recommend a particular kind of treatment. I don’t want to be on the hook for FDA approval and all that involves.

Posted in Data-betes, Diabetes, Historical Record, Life Lessons, MATLAB, Software Engineering | Leave a comment

What I’m Doing Now

Posted on August 9, 2009 by Jeff Mather

Blood glucose readings plot

More details to follow . . . eventually.

Posted in Data-betes, Diabetes, Life Lessons, Software Engineering | 2 Comments

MATLAB and Greenspun’s 10th Rule

Posted on June 26, 2009 by Jeff Mather

Just to get this disclaimer out of the way, let’s remind everyone that this is my personal website. Even though I may write some of the articles at work about things I’ve done at work, they aren’t in anyway supervised or endorsed by my employer. I’m only including this disclaimer because this article is meant to be self-deprecating in that “I had the best of intentions, but now I feel a little silly” way.

Several years ago I read Greenspun’s Tenth Rule of Programming: “Any sufficiently complicated C or Fortran program contains an ad-hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp.” Basically it’s a joke about feature creep in software. I was reminded of this quotation today when I started reading Joshua Kerievsky’s Refactoring to Patterns. He describes the perils of over-engineering, which is basically the same as wasting money.

You see, about seven or eight years ago, I added a Lisp parser to a feature in MATLAB’s Image Processing Toolbox. If you have the toolbox, just take a look in toolbox/images/iptformats/private/dicom_create_IOD.m. There you’ll see a part of the code that looks like this:



function tf = check_condition(expr, metadata)
%CHECK_CONDITION  Determine whether a condition is true.
%
%   Conditions are LISP-style cell arrays.  The first element
%   is a conditional operator, remaining cells are arguments
%   to the operator.
%
%   Conditions can be nested.  Each cell array in expr indicates
%   a new conditional expression.

%[snip]

%
% Process conditional expressions recursively.
%
switch (lower(operator))
case 'and'
    
    % This AND short circuits.
    for p = 1:numel(operands)
        tf = check_condition(operands{p}, metadata);
        
        if (~tf)
            return
        end
    end

[snip]

So why did I do this? The DICOM file format has conditional elements, which only show up in the file if other elements are present, are absent, or have a particular value. I wanted to create a general-purpose, reusable, extensible mechanism for describing these conditions so that I didn’t actually have to write code for all of those conditionals. Moreover, I wanted our customers to be able to write the conditions in a nice tabular form rather than writing code, because — you know — one day they would want to write more of the dozens of kinds of DICOM information objects from scratch than we supported right out of the box.

You probably see where this is going. We ended up extending the product in a different direction after we learned more about how our customers actually wanted to use the DICOM export functionality. So, if you need a Lisp-like algebraic Boolean logic parser in MATLAB, just go look in the previously mentioned file. (Also take a look at the dicom_modules.m file in the same directory to see how to express those conditions.) But for g-d’s sake, ask yourself if you really, really need that, or if you aren’t just fooling yourself into over-engineering your code.

Posted in Computing, Life Lessons, MATLAB, Software Engineering | Leave a comment

AMD performance counters

Posted on May 27, 2009 by Jeff Mather

Here’s a little reminder for myself and for anyone on the web who’s searching for information about AMD Athlon and/or Opteron performance counters.

AMD Performance counter resources:

Less techie posts to follow . . . honestly.

Posted in Computing, Fodder for Techno-weenies, From the Yellow Notepad, Software Engineering | Leave a comment

Rememberance of things past

Posted on May 21, 2009 by Jeff Mather

This has been the busiest year ever. True, last year I was in school and (consequently) working twice as much. But this year, I’ve actually gone places and done things that have generally kept me away from posting here.

In January, I went to the Electronic Imaging symposium in San Jose, promised to write about what I learned and then did not. In February, I went on a “business trip” to Cozumel — via cruise ship — and then posted some of the photos I made on my Holga camera. Then in March, Lisa and I went to Paris to celebrate my graduation. I could talk your ear off all day about how much I love Paris; and if you’re Lisa, you’re probably sick of hearing me walk around the house sighing and saying “I miss Paris.” (Sorry!) Most recently, I promised to post pictures from my backpacking/camping/hiking trip to Utah. I did put some of them into a photoset on Flickr, but I haven’t gotten them here yet. Tomorrow I’m off to Iowa for a family reunion.

“But what about the other 15-or-so weeks of the year? Why no posts?”

It isn’t that I haven’t had any ideas. Here are some of the things I jotted down that I should write about:

Physical architecture vs. software architecture, with respect to the visibility and communication of design — I actually started writing a bit about this after some lengthy conversations with coworkers. Basically, I yearn for the day when software engineers can have books like this, this, and this. We need a shared visual grammar, shared materials, and shared processes so that we too can have coffee-table books that inspire our cohort.

A response to Jeff Atwood on (not) reinventing the wheel again . . . and again — Given what I just wrote above, it’s no surprise that I would be disappointed after reading an article that advocates waste. I hope one day to live in a software engineering world which is more like a real engineering world, a world in which there are two kinds of software businesses: (1) the businesses where almost everyone buys off-the-shelf or special-order parts and uses them to build incredible things, innovating very broadly and (2) the businesses that specialize in making those off-the-shelf and special-order parts, making them well, and innovating very deeply.

Design blogs — I’ve been making a little list of design blogs that I like. . . . I’m a total tease, though, so I’ll share them later.

My notebooks — One of those afore(un)mentioned design web sites has an occasional series looking inside designers’ notebooks. (For example.) I keep most of my thoughts (and some designs) worth remembering in journals and took some beauty shots of them. I’ve actually downloaded the photos off my camera into Lightroom.

French music — The French are different. Especially in their musical tastes. I like some of it and thought about shilling for folks like Mylène Farmer, Mara Tremblay, A.D.N., Abd al Malik, Karna, MC Solaar, Cœur de Pirate, Alain Bashung, Dumas, Karkwa, Noir Désir, Prototypes, Vulgaires Machins, and Indochine.

Women in Software Engineering — Though I’m not part of the Ruby/Rails communities I was disappointed to see the kerfuffle over a recent presentation which reminded many of us how few women there are in our profession, how they’re often (subconsciously) typecast, and how their accomplishments don’t often get enough attention. So here are some belated congratulations to Barbara Liskov, winner of the most recent Turing Award. And I like Liz Keogh’s web site.

My grad program — In a nutshell, I enjoyed my time at Brandeis as I worked toward my master of software engineering (M.S.E.) degree. I learned a lot of practical things related to software construction, tooling, and process (just as I hoped that I would).

And finally, Lent and my year-long cleaning project — After graduating I looked around my office/library and was shocked at how cluttered it was. So just before the new year, I resolved to get through all of the magazines, papers, articles, etc. in my big cardboard “in-box” — seriously, it’s big — by the end of 2009 and recycle whatever is left on January 1st, 2010. My progress is slow. (I’m being very generous to myself right there.) So for Lent I decided to better myself by working on my reading pile. In particular, I forbade myself to add more things to it. Basically, I’ve gotten to that point in my life where I need to put myself on an information diet or two. Which I’ve done, although I find myself snacking a lot.

Anyway, that’s what I’ve been doing: reading from the big in-box, watching some of the 499 films in my Netflix queue, reading other people’s web sites*, writing in my own journal, photographing, gardening, carpooling, and more-or-less not writing here. More to come, eventually. . . .

*   Thanks for nothing, Facebook and Twitter! Just kidding. You know I love you.

Posted in Computing, General, Software Engineering, Travel, Worthy Feeds | Leave a comment