My group at work has just started reading Implementing Lean Software Development by Mary and Tom Poppendieck. As with any other reading group, the communal experience encourages us to read something that we might not pick up on our own. Because the books are (hopefully) relevant to our vocation, discussing them together helps us find ways to apply what we learn. We haven’t started discussing this book yet, but already I am intrigued.
For the uninitiated, Lean Software Development applies the core principles of lean manufacturing (as embodied in the Toyota Production Sytem) to software development. In particular, the process attempts to eliminate waste, build quality into a product, and continuously improve the development process. The authors admit that software development isn’t the same as manufacturing automobiles or clock radios, but I’ve managed to suspend my disbelief and see how well the principles apply. Perhaps at a later time I will recap what I’ve learned from the book.
But for now I would like to go off in a tangential direction inspired by the authors’ brief history of organized manufacturing from its birthpangs in 18th century Europe, through the interchangeable parts revolution that exemplified the American System of manufacturing, continuing on to the Ford that used interchangeable semiskilled labor and parts, to lean manufacturing at Toyota. But it’s that earlier, pre-manufacturing period that interests me now and seems so applicable to software development: the craft economy.
Imagine a craft economy. It’s hard to do. Everything is made and then made again but slightly differently. Two craftsmen build two things using their own unique toolsets, and the results are often incompatible or just different enough to require learning how to use it and make it costly to fix when broken. When the craftsman needs a new tool, she typically goes to her own machine shop and builds it specifically for the purpose.
I’m not disparaging craft economies. Some — most? — of the world’s superlative things were produced by craft economies: the pyramids, Chartres Cathedral, illuminated manuscripts, Sputnik. And there’s something charming and old-timey about one-of-a-kind goods. But craft work is incompatible with modern production; primarily because it’s expensive to produce lots of individual things.
Is software development in the “craft economy” milieu?
By and large, every company is doing things differently. There are few standard practices, and what standard practices do exist — such as agile methodologies — are implemented quite deifferently. Software developers also have a dearth of tools to use; common wisdom says that it’s often easier to build your own toolset than to adapt existing offerings to the problem at hand. To be sure there are a few basic off-the-shelf tools that everyone uses — compilers, standard libraries, and code editors — but not every software group uses other essential tools: revision control systems (SCMs), rapid prototyping languages, automated testing systems, etc.
Most companies build components in-house for their own products but few commoditize these tools in ways that other development groups can use them. Of course, Java, COM, and vendor foundation classes do exactly this, but these are just a drop in the bucket of all the functionality a software developer needs to solve a particular task. There is a dearth of reusable code for purchase or free download.
Contrast software development with the manufacture of physical goods. Toyota and GM have a large number of vendors who build components that are to a great extent reusable by anyone: sheet metal, bolts, spark plugs, tires, radios, electronic control units, etc. Despite being rather sophisticated, radios from different vendors fit in the same hole (compilation), can easily be wired into any car (APIs), and have similar user interfaces. While there are specialized tools for specific tasks, for the most part, a wrench set is a wrench set is a wrench set. Tools vary in terms of price, durability, and the part they fit, but they are usally applicable to a wide variety of tasks.
In lean production, manufacturers avoid holding inventory, and parts arrive “just in time” for use. In software development and other knowledge work, teams are prized for their depth of knowledge and the expertise that they can apply to problems — not necessarily just novel problems. Groups need this experience because someday they may need to call upon it to resolve a problem almost identical to one they have already solved. Often extra frameworks are built into APIs needlessly because the engineer has a hunch what may be useful in the future, essentially storing knowledge in the product long before it is needed.
These thoughts should not be considered an indictment of software development. Clearly adopting some aspects of mass production (lean or otherwise) will translate into lower development costs; but should software engineering really be more like mass production than craft? Craft guild members experienced significant hardships when interchangeable parts gave birth to assembly lines, even rioting. Does the concept of a “software supply chain” make sense right now? What we call “software engineering” is still largely invention; the necessary conditions for software mass production may still be years down the road.
All things considered, I think the emergence of software mass production based on truly reusable components and a just-in-time supply chain is closer than most expect. Two factors will likely propel us there: open source projects and the net effect of business process outsourcing.
In my job I use a lot of open source software libraries . . . sometimes incorrectly called free software. (Free software definitely has lots of hidden costs: time spent learning an API’s syntax and capabilities, time spent interfacing two different products, time spent reviewing license agreements, etc.) These open source projects run the quality gamut, but the best provide first-class code that implements 90% of the functionality anyone would need for a given task. For my group that means reading and writing image and scientific data formats. Because using a library can easily save $100,000 in development costs (not to mention opportunity savings and the ability to bring new products to market quickly) the for-profit world is embracing open source, even if components aren’t really interchangeable yet. Some large companies, such as Adobe and Apple have placed discrete parts of their products into open source projects partly to encourage others to develop for their “platforms” and engender trust in their products.
Software project outsourcing currently accounts for a very small portion of overall software development. As software becomes more modular, it will be possible to shift even more small tasks to places where skilled or semi-skilled can do it more cheaply. (This is part of Thomas Friedman’s thesis in The World is Flat.) Furthermore, if companies like Infosys, Wipro, and Tata can hold onto the intellectual property from BPO projects, they have a perfect opportunity to create interchangeable software parts. Lots of small to medium-sized interchangeable components available at low or fixed cost: that sounds a lot like a supply chain to me.
What are your thoughts? Is there a software craft economy? Is mass production of software around the corner? Is software development too closely tied to invention or too different from making widgets to be comparable to manufacturing?
