At midday yesterday my carpool buddy and I looked at the weather map, saw a scary blob of impending frozen doom, and decided to heed the Governor’s advice to go home. But the snow that was forecast — the blizzard that was supposed to mess up the evening commute — never really materialized here in the Commonwealth. Oh well. Only 1-2″ to shovel.

While working away the afternoon at home, I started reading Mary and Tom Poppendieck’s Implementing Lean Software Development. I guess “rereading” would be more accurate, this time with an eye toward actually implementing the ideas. Here are my notes from the first few chapters.

What is Lean Production?

• Use just-in-time (JIT) flow of work products.
• Stop the line to fix problems as soon as they are discovered.
• Eliminate stockpiles of in-process inventory. Create small batches.
• Make mistakes impossible from the beginning.

Product development is knowledge creation via exploration of designs, experimentation with prototypes, and integration meetings that evaluate designs and make decisions.

Schedules have synchronization points, but let expert workers function autonomously between them.

Explore multiple options and make competing prototypes. Defer the decision about which to actually build until the right time (usually later than you think).

Software should be easy to change. Employ all of those high-quality best practices and metrics during construction.

The Toyota Product Development System — upon which the Poppendiecks base their book — is very empirical. The final idea/design/product emerges during development. At the outset, define the goals to meet, not the technology or architecture you’re going to use to satisfy it.

1: Eliminate Waste — Minimize the time from request to fulfillment

• Waste is everything that doesn’t actually add value to a customer feature. Focus on value.
• Reduce the amount of partially done work (inventory). [Basically, don't work on what you haven't been asked for, and complete what you start. Get it written, tested, checked-in, and reviewed.]
• Reduce the time between requirements, coding and testing. This helps reduce requirements churn. Use test-driven development (TDD). Shun “big bang” integration.
• Avoid extra features and scope bloat.

2: Build Quality In

• Control the conditions for creating quality components, preventing defects.
• Write tests first. Use TDD.
• Continuously integrate. Integrate code and tests together.
• Write less code. Keep it clean and simple. Don’t duplicate. Refactor.

3: Create Knowledge

• Go from a sketch to detailed design during construction.
• Release a minimal set of functionality to customer for feedback.
• Create daily builds and get feedback (knowledge) from integration tests.
• Have an experienced team. Pour learning back into the team.
• Build a modular architecture that supports adding/changing features as you learn more.
• Generate new knowledge through disciplined experimentation. Codify that knowledge and make it available to the larger organization. Build a knowledge base. Encourage long-term thinking.
• Continuously improve processes.

4: Defer Commitment

• Try to make most design decisions reversible/undoable/nonbinding.
• Schedule irreversible decisions for the last possible responsible moment.
• Always move toward the concrete, and identify where change is likely to occur.
• Experimentation reduces the risk of making the wrong decision.
• Planning is an exercise, but plans are overrated. Avoid “plan-driven methods.”

5: Deliver Fast — Compete on time

• Be faster than your customers’ ability to change their minds.
• Requires low defect rates, high quality processes, good understanding of the customer.

6: Respect People

• Favor ownership at the worker level, not top-down.
• An entrepreneurial leader should own the product and foster engagement.
• Develop and nurture technical expertise as a competitive advantage.
• Give general plans with reasonable goals. Then let the group self-organize to meet them.
• There is no “one best way.” There is continuous improvement, though.

7: Optimize the Whole

• Optimize the whole value stream, that is, not just local processes.
• Pay particular attention to where products,code and processes transition between departments, teams, organizations or physical locations.

The product development timeline:

• Concept — What’s the unmet customer need or desire?
• Feasibility — Test feasibility by experimentation. Build stuff. Ship betas. Design systems. Investigate the major features of the business process, key hardware modules, interfaces, boundary behaviors, software architecture and constraints. Can the product really be built? Will it work?
• Pilot — Work with your customers. Show off the product; collect data; iterate. Let people choose from multiple options by playing and “voting with their attention.” Run multiple pilots to refine the design.

You have to go way beyond meeting basic expectations, even beyond adding new features or improving performance. You have to identify needs customers don’t know they have and then delight them by meeting those needs. This requires developing a deep understanding of the customers’ world.

That’s all for now. I’ll post more as I continue my way through the book — perhaps the next time it “snows.”

read more from this topic.....

In my most recent post I mentioned the matter of an on-going debate over the proper target for HbA1c values. I went back and listened again to the debate at this year’s EASD conference on the “right” A1c target. (You can find it by starting at the EASD webcast page and searching for “Berger”.)

The talks by Drs. Irl B. Hirsch and Andrea Siebenhofer-Kroitzsch are chock full of information, and I’ll summarize them here. I’ve also taken the liberty of including some of their slides. Any mistakes in recapping these presentation are entirely due to my limited knowledge of epidemiology and shouldn’t reflect on the good doctors arguments. Also, there’s a lot of statistics in these presentations; and while I studied mathematics as an undergraduate, I went the abstract math route and am rubbish with statistical jargon.

First, Dr. Hirsch advocated for a target of 6.5%.

• An A1c of 6.5% works out to an average of 144 mg/dL or 7.8 mmol.
• But 6.5% actually has a fairly large overlap with 7.0% if you use the 95% confidence intervals of each.
• So, there’s not that much of a difference between 6.5 and 7.0%.
• 6.5% is protective. There’s a small but significant reduction in the risk of complications with an A1c below 7.0%.
• And there’s an “exaggerated increased risk of hypoglycemia” below 7.0% for type 1.
• The DCCT (which suggested that there was increased risk of severe hypoglycemia with better A1c) was reported in 1993. It’s a different time now. JDRF continuous glucose monitor (CGM) studies show that risk is way down at all levels of A1c to the same as very high A1c.
• Lower A1c values are protective for type 2 as well, but treatment is a “moving target.”
• No matter what you target, the actual is going to be higher (often by as much as 0.5%). — So why not aim lower?
• Furthermore, doctors have been told “additional action” is required if the A1c is more than 1.0% over the target — so people with diabetes (PWDs) often aren’t getting told to change until well into the danger zone.
• Does the ACCORD study indicate that trying to achieve normal A1C results in increased mortality? [Yikes!]
• As in other trials, a higher average A1c in ACCORD is associated with a higher risk of death: +22% for each 1% of A1c.
• And with intensive treatment, the risk of death continues to decrease below 7.0% — Not so with “standard treatment strategy.” [I believe the "intensive treatment" group aimed at lower A1c.]
• Moreover, the people with the highest risk of mortality in the study, were the people with the lowest decline in A1c.
• The causes of death in ACCORD weren’t differentiated, though.
• Of course, the target value is individualized based on other factors. “Clinical judgement always trumps guidelines.” “If you can’t get there, don’t push it.”

Dr. Andrea Siebenhofer-Kroitzsch used meta-analysis of several different diabetes trials to argue that we can’t tell definitively whether improving glycemic control (as measured by a lower A1c) actually helps.

• There are many other factors that along with A1c confound what causes improved/worse outcomes. Factors such as social status.
• So far, there are only seven randomized, controlled trials on A1c (DCCT is by far largest with 1441 patients).
• The age of people in the DCCT were quite young, so it was hard to see macrovascular complications (such as heart disease) directly from the study.
• And, the ACCORD study had to be stopped because intensive treatment had a higher mortality for some groups. [Seriously?]
• Some studies showed more hypoglycemia and more weight gain with aggressive treatment.
• Other changes are more effective in preventing complications: blood pressure, cholesterol reduction.
• Don’t confuse intensive self-management with A1c.
• Tight glycemic control is very beneficial for young people with type 1. Admits that people with type 1 need tight glycemic control because of length of treatment. [But type 2 is occurring much earlier in age, one audience member reminded us.]
• Lots of open questions about the value of glycemic control.
• Individualized treatment is more important, almost suggesting letting patients to set their own targets as long as a minimum level of outcomes are met.

Here are my reactions and some questions I had while listening to the session:

• It’s good to see that the increased use of CGM technology can dramatically reduce the number of severe hypoglycemia episodes. Given what it costs to go to the ER, there’s another argument in favor of covering CGMs and supplies more widely by insurance carriers.
• The meta-analysis was very . . . meta. Consequently it pushed very hard on the difference between “proof” and “connection.” A lower A1c may suggest better outcomes, but it doesn’t prove that the lower A1c caused it. I get this, but the data are compelling to me.
• Almost all of Dr. Siebenhofer-Kroitzsch charts suggest favoring more intensive treatment, although that seemed to contradict the core of the presentation. Perhaps I don’t know how to read them correctly? [Mo meta, Mo problems.]
• Does the fact that clinical trials never met their A1c targets mean that people with diabetes (PWDs) are going to feel even worse about not meeting A1C targets if the target is lowered to 6.5%?

Ultimately, based on the questions and comments from the audience, the debate sounded like a draw. I think I’m in Dr. Hirsch’s camp, and I would ultimately like the lowest A1c I can safely get; but I don’t think I’ve ever really been close to breaking below 6.5%. So maybe Dr. Siebenhofer-Kroitzsch’s camp and the ADA have a point worth considering carefully when looking for a number to cover all of us.

Some of Dr. Hirsch’s slides:

Some of Dr. Siebenhofer-Kroitzsch’s slides:

read more from this topic.....

This post is inspired by Anna in Montréal, who is adjusting her basal rates.

A Side-effect of Exercise

I have a bike. This shouldn’t be a big surprise, as I’ve written about my border-to-border and Quabbin Reservoir rides recently.

I love riding my bike, and during the warm months with lots of sunshine I rode it almost everyday. I like the sensation of rolling along, with the wind whistling in my ears and the scenery blurring by. I don’t even mind the hills anymore, even though the wind stops whistling as I crawl along.

All of that riding has been great at lowering my blood glucose reading. In fact, it’s worked a little too well. Insulin — the enzyme that helps transport glucose into your cells for energy — becomes much more effective when you exercise. And when your muscle cells slide past each other, they basically act as pumps and can work (almost) without insulin. (But not completely without.)

When I ride, I carry a waterbottle full of sports drink — Gatorade if you care to know — but I was still having hypoglycemia more often than I’d like. If you don’t have diabetes, this is commonly referred to as “cracking,” “bonking,” or “hitting the wall.” Your muscles have spent their glycogen (a form of glucose usually found inside of muscles), and your body can’t convert enough fatty acids into energy. You feel tired and find it hard to keep going.

If you have diabetes, hypoglycemia has all of these same attributes. Of course, for us it also means that our brains don’t get enough glucose either. So we’re not just tired, we can also pass out. And stopping for a little while doesn’t fix the problem. We have to replace it right away. So I try really hard not to have hypoglycemia. And as someone trying to take off a few pounds — which is going quite well, thank you very much — I want my body to turn fat into fatty acids and leave my blood glucose more or less alone. And I don’t like to drink a lot when I run, which I’m doing now that it’s dark in the evening. But, as I said on Halloween, hypos scare me.

Time to Fix the Basals

So I pulled out my copy of Smart Pumping. “Oh look! I should probably fix my basal rates as a first step.” Well, it was time to do that anyway.

For those of us with insulin pumps, basal insulin is the continuous trickle of background insulin that keeps our blood glucose in the happy/normal range of 80-150 mg/dL, counteracting the steady release of blood glucose from the liver. Why the liver does this, I don’t know. I read in my book that basal insulin should be about 40-50% of your average total daily dose (basal + food/correction boluses). Mine was about 55-60%. A sign of problems. Fixing these rates requires skipping meals and testing every couple hours. Skipping meals is hard. And if one reading is too much different than the one before, you have stop, make adjustments, and start another day.

But I wanted exercise to be easier (and last longer). And (more importantly) I was coming up on my 10-year anniversary with diabetes and had told myself that I wasn’t going to go five or ten more years just “getting by.” I was starting to be quite unhappy about all of my hypos and high readings. And I had a supportive, active new endocrinologist who wanted to help me improve my readings and my ability to predict them.

After figuring out my morning basal rates — which involved about four or five skipped breakfasts — I began my afternoon tests in late September. When I started, I was getting 22.0 units per day. (A unit is 1/100 of a mL. A vial of insulin has 1000 units.)

Four weeks, seven tests and seven adjustments later, I think I’ve figured it out. I have just one more afternoon test to go — I hope — but I think my new rates are correct. For the curious people with diabetes out there, here’s a bit of the data.

Numbers and stuff

On 26 September, my basal rates were

00:00 - 07:00 = 0.9 u/hr
07:00 - 09:00 = 1.0
09:00 - 20:00 = 0.9
20:00 - 00:00 = 1.0
(22 units per day)

The first or second test:

6 October 2009
Last basal: 4.0u at 7:42 (active insulin at 12:30 is 0.4u)
11:39 - 143
12:54 - 79
Stopped

8 October 2009 - Attempt #2 or 3
Last basal: 4.3u at 7:22
11:20 - 155
12:56 - 97
Stopped

I was checking more often than required in those early tests, because I could actually feel my blood sugar moving. I kid you not, and I had suspicious that I was going a scary place.

By 12 October, I had changed my rates to

00:00 - 0.9 u/hr
07:00 - 1.0
09:00 - 0.7
15:00 - 0.8
20:00 - 1.0

13 October 2009 - Attempt #4
11:38 - 133
13:00 - 93
14:00 - 78
Stopped

19 October 2009 - Attempt #5
11:27 - 216
12:54 - 188
14:45 - 138
15:47 - 111
Stopped

After this, I was feeling close to being there, and before my last test had already knocked off a lot of insulin.

00:00 - 0.9 u/hr
07:00 - 1.0
09:00 - 0.7
11:00 - 0.5
15:00 - 0.6

27 October 2009 - Attempt #7
Previous bolus - 07:51
10:44 - 256
11:53 - 234 (Active insulin 1.1u)
13:36 - 226
16:01 - 180
18:01 - 153

I know that last test started out with high readings, but I just wanted to get the damned thing done, and I knew that having higher readings would give me a good cushion for seven hours of testing.

Currently, here’s where I am:

00:00 - 0.9 u/hr
07:00 - 1.0
08:00 - 0.8
09:00 - 0.7
11:00 - 0.5
20:00 - 1.0
(18 units per day)

These basal tests are all about gradual refinements. If your BG readings change by more than 30 mg/dL up or down over any two hour period, adjust +/- 0.1 unit/hour starting 2-3 hours before the dip/bump. The Pumping Insulin authors also recommend redoing all of your basal tests (overnight, morning, afternoon, and evening) whenever you change exercise patterns, your weight changes 5-10%, or you suspect they’re wrong because of fasting highs or lows.

read more from this topic.....

Here’s a little reminder for myself and for anyone on the web who’s searching for information about AMD Athlon and/or Opteron performance counters.

AMD Performance counter resources:

• Doc #26094, Chapter 10 (very detailed!)
• Doc #22007, Appendix D (part of the Optimization Guide)
• Doc #24593, Chapter 13 (part of the AMD64 Architecture Programmer’s Manual)
• Doc #41256, Section 3.14 (somewhat older, but still good)
• OProfile documentation (a list of CPU performance events)

Less techie posts to follow . . . honestly.

read more from this topic.....

Just a short post informing you about a survey paper I wrote for my object-oriented (OO) design class: Surveying Quality in Object-Oriented Design. In it, I look at the major books and articles concerning best practices for OO design. If you do OO programming, you’ll probably find something interesting. (It’s language-neutral, too.)

The major goals of using object-oriented design are to facilitate the maintenance and extension of software systems by reducing the complexity of software at the class and system level. Successful OO designs are resilient to change, largely because they manage interclass dependencies. In such a system, changes to one part of the system are localized and do not cause a chain of modifications through the system.

The major techniques that OO designers have at their disposal are abstraction, encapsulation, inheritance, polymorphism, and composition. But how does an engineer apply these techniques to create a good design? What are the main ways that sets of classes can be structured and interact to maximize the chance of a successful design? Over the last twenty years, numerous OO practitioners have developed a mature set of rules-of-thumb and best practices to use when constructing and evaluating OO design.

If you celebrate it, enjoy your Thanksgiving tomorrow. Don’t do any homework if you can help it. I won’t!

read more from this topic.....

The first week of my final semester of grad school is in the bag. This go-round is all about software design, both low-level (“RSEG 109: Object-Oriented Design“) and high-level (“RSEG 290: Special Topics: Design Patterns”). When you add in the reading group we’re starting at work, this could also be called the “UML semester.”

Design patterns give a shared vocabulary for talking about common software design features, present how and when to use a design, and lay out the trade-offs of using a particular design. They aren’t as widely used as they should be in a profession that claims to value reuse. I think that’s a shame.

As part of the design patterns course, I must create an object model for a real-world problem. And along the way, I’ll be learning about most of the two-dozen OO design patterns in the Gang Of Four book. (Usually I don’t present solutions to my homework in public because I feel it could help the next semesters’ students cheat; but this semester we’re each doing our own thing.)

Since I’m learning about them, I’ll do my best to write about them over the coming weeks. If all goes well, I’ll have an outlet to reinforce what I’m learning; and you, my dear readers, will have a gentle on-ramp to design patterns, too.

Here’s the description I submitted last night of the software system I will design:

So keep tuning in, and we’ll take a tour through design patterns for object-oriented software reuse. And please post comments about where you’ve been able to use design patterns successfully, if you see anything you would have done differently, or if you just want to leave your two cents.

read more from this topic.....

As promised before, here are some more notes from the classes that I’ve taken as part of my soon-to-be-completed Master of Software Engineering degree. This time: (software) project management. Most of this information comes from Effective Project Management by Robert K. Wysocki.

FYI, this was one of the few classes where most of my in-class notes weren’t about the course material at all but were reflections about how we do project management where I work. Once I discovered that I was a project manager, I realized that I had best become better at doing it. Funny how obvious that seems in retrospect.

Basics

Project = “A sequence of unique, complex, and connected activities having one goal or purpose and that must be completed by a specific time, within budget, and according to specification.” (Wysocki, 4)

Program = A collection of projects with multiple goals.

Most “interesting” software projects involve some degree of unclear requirements or unknown solution. These projects should ALWAYS use an adaptive/agile or iterative approach.

• Examples: Evolutionary waterfall (for low risk/easy projects), SCRUM, Rational-Unified Process (RUP), Dynamic Systems Development Method.
• These methods separate high-level and detailed planning. Each must be done, but the detailed planning is not done all “up-front.”
• (These are not iterative approaches: Pure Waterfall, Rapid/Parallel Development, Staged Delivery.)

Continuous quality managment and process quality improvements appear as keys to successful projects.

What every project should have . . . to some extent or another

Linear/waterfall, iterative/agile/adaptive, and extreme project management techniques all have the same phases, they just appear in different ways. They are:

• Define the project: Take the problem, proposed solution, and objectives and make a project charter and scope document
• Develop detailed plan(s) — preferably iteratively and just-in-time
• Launch the plan(s)
• Monitor and control project progress: Reporting, change control, problem escalation, revising plans
• Close out the project — Acceptance, installation, party! Seriously, you must party.

Risk Management

The major responsibility of the project manager is to manage risk in the project.

• Identify risks:
• Quality and performance with respect to technology
• Resource allocation
• Planning process
• Organizational support
• Changing legal and regulatory requirements/availability
• Suppliers and contractors
• Assess risks:
• Separate risk, magnitude, and probability
• Exposure = Probability of loss times cost of loss
• Consider using a risk matrix (high-medium-low cost v. high-medium-low probability) to track exposure
• Consider whether solution costs more than the loss
• Assess risks at each project phase/iteration
• Respond to risks:
• Accept — Do nothing
• Avoid — Don’t do that part of the project
• Contingency planning — Reframe the plan to deal with risk
• Mitigate — Reduce the probability and/or the magnitude of loss
• Transfer — Outsource the risky part to someone more capable of handling it
• Monitor and control risks:
• Make a risk log.
• Review risks at status meetings.
• Add triggers to risks so that countermeasures are taken at the appropriate time.

Project estimation

The average worker efficiency in IT is 50-65%. That’s the amount of time actually devoted to project work. That doesn’t include ad hoc interruptions, which takes another 33% of so of the workday. And there’s a lot of variation in duration for the same task, since everyone works at different capacities. So . . . It’s best to think in terms of task size and not the time that it takes to complete a particular time.

Methods for estimating task size:

• Similarity to other activities already done — Usually a very good predictor
• Historical data — Usually very objective and concrete
• Expert advice — Be weary of using just this
• Delphi technique — Iterative planning poker. Result is the average of the third round (or consensus)
• Three point technique — E = (Optimistic + 4*Most Likely + Pessimistic) / 6 for however you define those three terms
• Wide-band Delphi — Delphi technique with three point computation instead of a simple average

You can (and should) determine duration from the effort values and from that cost.

Project task management

Having a work breakdown structure (WBS) does not mean that the project must be managed like a waterfall, with all of the tasks defined to a fine precision before implementation can start (though some tools make this more likely).

Parts of the WBS can (and should) be high-level at the start. The plan gets more detailed with each iteration. Instead, treat a WBS as a represention of the functional/modular breakdown of the system. It’s useful for visually thinking about the project, designing the architecture, planning and estimating, and reporting status.

The network diagram is more useful in actually planning the project than a Gantt Chart, which is good because Gantt Charts suck. Then network diagram contains sequencing information, and you can use it to find the critical path of tasks that define the full project duration.

Random thoughts

This stuff — plus copious amounts of Hindi and Arabic scribbling — filled the spaces between my notes from reading Wysocki.

• If I had a time machine and could redo parts of a project, when would I go to add more value or lower costs?
• Lucky + smug = ?
• Consider keeping a historical journal for estimation: size of project, time, resources, technology.
• Engineers are creative, problem-solving people. Rule-following is not a creative act and implies a solved problem. If software engineers are going to do project management, the project management techniques must not get in the way of actually solving the project’s problems or it just won’t happen.
• I am the very model of a modern major general.
• How well does agile planning and development scale? Can you do critical path analysis with it? Is it even worth trying to do that?
• Measure quality, productivity, maintenance work v. feature work, time to market. Measure when starting, when passing milestones, when encountering defects.
• I like postage stamps.
• Iterative development should have deliverables that can actually be met at the end of each iteration. The iterations should be tied to deliverables. Milestones shouldn’t just be mile markers; that’s what a calendar is for.
• Do risk management at every stage of the project.
• Use a pull system for features with insertion for bugs and technical support assistance. Translate “do interruptions now” to “do next.” Finish up what’s in progress if it’s worth doing.
• Product != Project != Program != Product

read more from this topic.....

Here are a few notes from the yellow notepad I’ve been using for school over the last six months. This first installment touches on C. Later I’ll add some UNIX programming and C++ notes. Basically these are the things that struck me as I was quickly reading the “refresher” material for my spring class: “Advanced C Programming for UNIX.” After more than a decade with C, I feel I know the core language pretty well; and here are the things that struck me as worth noting.

Is there a portable way to refer precisely to datatypes of a particular size? Yes, in C99 #include <stdint.h>.

Use signed integer types larger than char with getc() or getchar() to accomodate EOF (-1).

A good running average is avg += (x - avg) / n; It reduces the chance of overflow, is fast, and doesn’t require you to keep track of all values.

Each comment collapses to a single blank character during parsing.

Many system names start with an underscore ( “_” ). Avoid creating new identifiers that start that way.

All functions have extern storage, just like variables defined outside of a function.

Use volatile when it’s possible that the variable’s value will change from outside the normal flow of control. Technically this prevents compiler and processor instruction reordering optimizations around the use of the variable.

The register storage class is for small, fast-changing variables like loop indices (not for bigger things like arrays). And you can’t do &variable for register-declared storage.

Array names are const pointers, so an array name is not a valid l-value.

It’s not possible to take an address of a literal value, like 3.14 or "I am the very model of a modern major general.". So . . .

Because C-strings are either arrays (char s[]) or pointers to memory areas (char *s), it’s illegal to reassign a value to a string variable directly (e.g., s = "new string"; /* won't compile */).

When passing multidimensional arrays to a function, the function’s argument list must give the size of all but the slowest changing dimension.

Function pointers: int (*f)(int, double) is the same as int f(int, double). The parentheses are necessary to ensure the * binds to f and not int. To use the function pointer f either of these is acceptable: (*f)(37, 3.14) or f(37, 3.14). (This is “declaration follows usage.”)

const int *p is a pointer to a constant int.
int const *p is a constant pointer to an int.

“LP64″ scheme means long ints and pointers are 64-bit values.

Some compilers use precompiled headers to speed up compilation. This can make the inclusion order of header files significant. Also, some compilers (like Xcode) use “prefix headers,” which are always included in a project.

Defining NDEBUG turns the assert() macro into a no-op.

read more from this topic.....

It’s Friday here at The Metrowest Homeopathic Imaging and File Rehabilitation Center, the day when I usually spend part of my time on 20% projects. [1] But I’ve basically been working on the results of my last such project full time, so perhaps it’s time to take a moment and look at the world rushing by and catch up on a few things before getting too far behind.

I moved offices for about the tenth time since I started working last week. That’s not a bad thing; we’ve been growing consistently over the last 9 1/2 years, and the company likes to keep work groups physically co-located without stuffing people into cubicles. While packing my office, I decided to purge my files, getting rid of a lot of old, old papers. (The process was actually a lot like Merlin Mann’s War on Clutter.)

It was during this time that I realized how much information I have tied up in paper notes. The same goes for e-mails. (Over those 9 1/2 years I’ve managed to keep more than 13,000 7,000 messages.) That information isn’t doing anyone other than me any good at all . . . coworkers, amateur color enthusiasts, no one! So expect to see some random color-related stuff coming your way.

[1] – A 20% project is a self-directed project that’s “off the books.” There are two main reasons for doing projects like this. It gives people freedom to explore and dive into topics that are risky or somewhat out-of-model. These projects also add slack to a system, preventing workers from being overutilized. Studies show that beyond a certain point the more you use any resource — whether it’s an engineer, a highway, a drillbit, whatever — you get lower productivity.

read more from this topic.....

There are two things you should know about me: I am not a good person, and I hate being bored or having my time wasted. Those two things combined pretty much say everything you might need to know about me, not unlike Dostoevsky’s underground notetaker:

I am a sick man. . . . I am a spiteful man. An unattractive man. I think that my liver hurts. But actually I don’t know a damn thing about my illness. I am not even sure what it is that hurts. . . .

I am quick to judge and wrong as often as not. Sometimes it takes me months or years to realize the folly of my first impressions. An artifact of my Iowan-ness, a genetic marker from my place of origin, it’s my curse for being extroverted and open-minded. Occasionally I publicly call out those I mistake for what they’re not, but more often I nurture these mistakes quietly on the sweet milk of my incorrectness. Knowing that I’m prone to being wrong doesn’t make me less likely to be wrong, but it does keep me quieter . . . in general — my recent excoriation of Martin Amis notwithstanding.

Unfortunately, naming my demons doesn’t give me complete control over them — I’m not one who thinks that analyzing the progenitors of my peevishness gives me total will-to-power over my future — but I have been trying to say “hmm . . . how interesting” more. Fortunately, I’m rather good-natured despite my wickedness and, by and large, get along with almost anyone.

Which brings me ’round to that other thing and the inspiration of this dispatch: Lectures are hard, nigh on unbearable sometimes.

You see, I do the required reading before class and (usually) the optional readings, too. I am a slow reader, mostly because I hear the words in my head and because my inner voices engage the author’s disembodied voice in spirited discussion. I deliberate and I question. And I either suspend disbelief when I don’t know much on the subject or fill in gaps or deconstruct assumptions (mine or the author’s) when I do. I read and I edit and I rewrite books and articles in my head, which is probably why I remember most of what I read and less of what I see or hear.

I suspect few other people read quite so pathologically, but I got spoiled at Grinnell by everyone actually having read the assigned material in advance. I chafe when I sit down in class and see the same material from the readings presented on overheads as Powerpoint bullets: tiny, clipped bons mots struggling to break free in search of true pith and vim. My same inner voices that discussed the work with the authors wait anxiously for a nibble of something new, but all I hear from them is . . . well, let’s take a look at my notes from tonight’s class.

• αβγδεζηθικλμνξοπρστυφχψω
• ΑΒΓΔΕΖΗΘΙΚΛΜΝΞΟΠΡΣΤΥΦΧΨΩ
• ῷ μοι κακαδαιμων. εγενετω τθφλος. χαλεπως εστι ό βιος.
• वीर-ज़ारा
• ॐ
• Lots of people talk. [We have a class of 25 people graded on participation.]
• Pair programming is a bit Stepford Wives-ish.
• People think XP is just about pair programming.
• [Instructor F]’s 3 + 1 rules — Lame.*
• [Instructor F] wanted a semester-long class on requirements!
• How to get the most out of these classes? Zen-like and let it wash over me? Pick out the good stuff as it floats bye like Wracker Quoyle? “Hmm . . . how intersting” for the rest?
• Other people are real people. I like books more than lectures, but we do learn from others experiences.
• How do people [like me] with partial (but real & sometimes deep) experience [from the workplace] learn [how to do what we already do better]?
• What should be our goals? How do [high-level] Aquinas-like books [based on distinguishing between this and that] like our text by Sommerville fit in?
• Why don’t I trust instructors? Do I unjustly expect infallibility? Why? Is it my Old Testament upbringing? Perhaps it fits with my Iowa-ness. Perhaps I’m just mean. Probably . . .
• I don’t think [Instructor F] really knows the difference between functional and nonfunctional requirements, but I don’t think that’s important. (See above) But we are wasting a lot of time trying to distinguish them . . . poorly.

Welcome to the purgatorial life of a night school philosopher-engineer. At least I’m getting therapy as well as an education.

* — “#1 – You are responsible. #2 – ‘Stuff’ happens. #3 – If #2 happens see #1.” #4 was slightly more helpful, but I won’t bore you with it. [Instructor F] presented these individually via Powerpoint with a long dramatic pause after the first. So, seeing “You are responsible.” on the screen with a pregnant pause led me to ask “Responsible for what?” thinking it might be some sort of ploy to get us to a sort of uber-insight into Software Development Methodologies. “For yourselves!” was the reply. Giggles all around.

What are we, seven years old? Funny story there. The first week of class we had to sing “Happy Birthday” to [Instructor F]’s first-grader at the course break. “Today is my son’s birthday. Since I can’t be there with him, it would mean a lot . . .”

read more from this topic.....

Going through some old e-mail at work, I found this review of Steve McConnell’s book Professional Software Development: Shorter Schedules, Higher Quality Products, More Successful Projects, Enhanced Careers that I wrote for my group a couple years ago. It changed my perspective on my career a bit and helps explain why I’m in a software engineering master’s program.

McConnell’s central point is that software development is not living up to its full potential. Anyone who has ever read any part of Code Complete shouldn’t be surprised to find that McConnell blames improper software construction and project management practices for some of this. Professional Software Development goes further in arguing that the software profession lacks the level of rigor that other professional careers have and that it needs to get it. In the book’s four sections, McConnell first surveys the problem and then focuses on individual, organizational, and industry-wide ways to build a better software engineering profession.

In the first part, “Software Tar Pits,” McConnell examines (with data) why software projects fail. The most frequent cause of software project failure is requirements problems. Beware of “last-minute syndrome”; continual progress or tool-building is necessary at all times, and project leaders should look daily at how much progress was made. Beware of not preparing and doing “code and fix” development. Changing requirements is the most common source of cost/schedule problems because “software isn’t soft.”

For McConnell the reason why software developers get into these “tar pits” is that most code creators (often despite their other skills) lack formal software engineering skills. The use of the word “engineering” is intentional and is intended to reflect both a training methodology and a level of professionalism. Computer science is research-oriented, with a goal of advancing knowledge; software engineering, on the other hand, is application-oriented, with a goal of applying knowledge and problem solving. Software engineering principles deal with the “essential difficulties” of software development; software is complex, changeable, must conform to messy reality, and is difficult to visualize. Fortunately, the “stable core” of software engineering’s professional knowledge is pretty well advanced in dealing with these complexities. Information doesn’t get shared around enough, and we just haven’t been well trained in this core knowledge:

• SW requirements
• SW design
• SW construction
• SW maintenance
• SW configuration management
• SW quality
• SW engineering management
• SW engineering tools and methods
• SW engineering process

According to McConnell, these are the core competencies that software engineers must have (to varying degrees depending on their job).

In part two, “Individual Professionalism,” McConnell discusses what individual software engineers should do to improve their proficiencies. Much of this is unremarkable: start with professional education and develop skills in a practical setting. But to be a full professional requires more controversial steps: certification and licensing. (It’s worth noting earlier than McConnell does that professionalism and high-quality aren’t necessarily synonymous and that most software developers won’t require the stringent requirements of certification or licensing. As the risks to the public and the novelty of the applications increase, McConnell feels the need for professional accountability–along with legal accountability–also increases.)

The characteristics of great designers include

• A large set of standard patterns to apply to problems
• Mastery of tools
• Being drawn to complexity
• Curiosity
• Seeking out criticism
• Experiencing and learning from failed projects
• Not being afraid to use brute force during discovery, and
• A desire to create and apply knowledge.

In the third part “Organizational Professionalism” McConnell uses data from the literature to suggest that team cohesiveness and an organization’s use of best practices are bigger contributors to team success than individual qualities. Having good developers is important, but the biggest schedule/cost improvements come from merging principles with creativity. The best organizations know the rules aren’t one-size-fits-all and have the confidence to break them and trade certain principles in favor of others.

Projects need specialization in the core knowledge areas listed above as much as companies do. The larger the project, the more specialization is needed in the areas of construction, design, planning and tracking, business management, QA, and requirements. Even ad hoc, this leads to better follow through and ownership.

Some interesting statistics from this section of the book include the gem that the return on investment of improving practices has been measured at 500-900%. That doesn’t include harder to quantify benefits such as faster time to market and improved customer confidence. Another one is that the typical effort in months for a project is 2.94 * KSLOC ^ 1.1, where KSLOC is number of lines of code (in thousands) that a project will require. NASA’s Software Engineering Lab, which is one of the best, does much better 1.27 * KSLOC ^ 0.986. They’re actual more efficient with larger projects.

It’s possible to measure the level of maturity of a software organization using the SW-CMM model. Companies fall into one of these levels:

1. Initial: Chaotic, underperforming, unshared knowledge, “heroic” or code-and-fix development. This is the default level of most organizations.
2. Repeatable: Basic program management practices vary project-by-project, shared responsibility, falter with novel problems.
3. Defined: Standardized PM practices across the organization, process group, training, projects are on-time/budget, more advanced than code-and-fix.
4. Managed: Highly predictable results, stable processes, database of project data for effectiveness analysis, organization-wide processes for all projects.
5. Optimizing: Continuous/proactive process improvements, results are measurable and measured, QA focus on defect prevention, root-cause analyses.

Research shows that the SW-CMM method is cost-effective, allows risks, doesn’t lead to extra bureaucracy, but does take a while to implement (on average 3 years per level) and is costly.

In chapter 16 McConnell lays out an interesting, laddered professional development plan for software engineers that organizations can use. The system is the one his company uses, and it measures engineers’ proficiencies in the knowledge areas listed above (putting them into five broad competencies); has plans for training, mentoring, and public recognition of improvement; is tied to salary; and presents a roadmap for becoming a “professional.”

McConnell revisits that idea of a software engineering profession in the final part of the book, “Industry Professionalism.” It seems to be the most controversial section, though it really shouldn’t be. According to McConnell’s thinking, software development is already on the way to becoming an engineering field, despite having a ways to go. When it finally arrives at the professional stage–which comes after commerce is wed with science to produce a high quality, dependable end-product–certification and licensing will be the recognition of the status quo. Customers will expect important software to behave well and will be able to hold one or more licensed individuals who worked on the product civilly or criminally liable. That might sound extreme but the potential of some software to do harm (or good for that matter) is just as great as the “products” of other professionals (doctors, lawyers, civil engineers, etc.).

Certification is voluntary and shows proficiency in a field. Licensing is not mandatory or universal but might be required by some jurisdictions when accountability is required. (Consider that your friend can do your own taxes and you can blame no one if things go wrong, or you can use a CPA and have legal recourse in the case of negligence or ethical violations). McConnell estimates that only 1-5% of all software professionals will need licensing.

Beyond that, the industry as a whole has an enormous responsibility to the public trust to act in a professional, ethical manner. Moreover, the industry should do a better job applying computer science and software engineering knowledge that was developed many years ago and is currently unused.

On the whole, Professional Software Development is a compelling, well-argued read. However you feel about its most controversial parts, if you construct, test, or manage software, it is well worth being familiar with the issues that will likely affect your career in the next decade or so. Beyond that, I found it contained a very compelling argument for seeking out and fixing the inefficiencies in individual projects and throughout the whole organization.

read more from this topic.....