Going through some old e-mail at work, I found this review of Steve McConnell’s book Professional Software Development: Shorter Schedules, Higher Quality Products, More Successful Projects, Enhanced Careers that I wrote for my group a couple years ago. It changed my perspective on my career a bit and helps explain why I’m in a software engineering master’s program.
McConnell’s central point is that software development is not living up to its full potential. Anyone who has ever read any part of Code Complete shouldn’t be surprised to find that McConnell blames improper software construction and project management practices for some of this. Professional Software Development goes further in arguing that the software profession lacks the level of rigor that other professional careers have and that it needs to get it. In the book’s four sections, McConnell first surveys the problem and then focuses on individual, organizational, and industry-wide ways to build a better software engineering profession.
In the first part, “Software Tar Pits,” McConnell examines (with data) why software projects fail. The most frequent cause of software project failure is requirements problems. Beware of “last-minute syndrome”; continual progress or tool-building is necessary at all times, and project leaders should look daily at how much progress was made. Beware of not preparing and doing “code and fix” development. Changing requirements is the most common source of cost/schedule problems because “software isn’t soft.”
For McConnell the reason why software developers get into these “tar pits” is that most code creators (often despite their other skills) lack formal software engineering skills. The use of the word “engineering” is intentional and is intended to reflect both a training methodology and a level of professionalism. Computer science is research-oriented, with a goal of advancing knowledge; software engineering, on the other hand, is application-oriented, with a goal of applying knowledge and problem solving. Software engineering principles deal with the “essential difficulties” of software development; software is complex, changeable, must conform to messy reality, and is difficult to visualize. Fortunately, the “stable core” of software engineering’s professional knowledge is pretty well advanced in dealing with these complexities. Information doesn’t get shared around enough, and we just haven’t been well trained in this core knowledge:
- SW requirements
- SW design
- SW construction
- SW maintenance
- SW configuration management
- SW quality
- SW engineering management
- SW engineering tools and methods
- SW engineering process
According to McConnell, these are the core competencies that software engineers must have (to varying degrees depending on their job).
In part two, “Individual Professionalism,” McConnell discusses what individual software engineers should do to improve their proficiencies. Much of this is unremarkable: start with professional education and develop skills in a practical setting. But to be a full professional requires more controversial steps: certification and licensing. (It’s worth noting earlier than McConnell does that professionalism and high-quality aren’t necessarily synonymous and that most software developers won’t require the stringent requirements of certification or licensing. As the risks to the public and the novelty of the applications increase, McConnell feels the need for professional accountability–along with legal accountability–also increases.)
The characteristics of great designers include
- A large set of standard patterns to apply to problems
- Mastery of tools
- Being drawn to complexity
- Curiosity
- Seeking out criticism
- Experiencing and learning from failed projects
- Not being afraid to use brute force during discovery, and
- A desire to create and apply knowledge.
In the third part “Organizational Professionalism” McConnell uses data from the literature to suggest that team cohesiveness and an organization’s use of best practices are bigger contributors to team success than individual qualities. Having good developers is important, but the biggest schedule/cost improvements come from merging principles with creativity. The best organizations know the rules aren’t one-size-fits-all and have the confidence to break them and trade certain principles in favor of others.
Projects need specialization in the core knowledge areas listed above as much as companies do. The larger the project, the more specialization is needed in the areas of construction, design, planning and tracking, business management, QA, and requirements. Even ad hoc, this leads to better follow through and ownership.
Some interesting statistics from this section of the book include the gem that the return on investment of improving practices has been measured at 500-900%. That doesn’t include harder to quantify benefits such as faster time to market and improved customer confidence. Another one is that the typical effort in months for a project is 2.94 * KSLOC ^ 1.1, where KSLOC is number of lines of code (in thousands) that a project will require. NASA’s Software Engineering Lab, which is one of the best, does much better 1.27 * KSLOC ^ 0.986. They’re actual more efficient with larger projects.
It’s possible to measure the level of maturity of a software organization using the SW-CMM model. Companies fall into one of these levels:
- Initial: Chaotic, underperforming, unshared knowledge, “heroic” or code-and-fix development. This is the default level of most organizations.
- Repeatable: Basic program management practices vary project-by-project, shared responsibility, falter with novel problems.
- Defined: Standardized PM practices across the organization, process group, training, projects are on-time/budget, more advanced than code-and-fix.
- Managed: Highly predictable results, stable processes, database of project data for effectiveness analysis, organization-wide processes for all projects.
- Optimizing: Continuous/proactive process improvements, results are measurable and measured, QA focus on defect prevention, root-cause analyses.
Research shows that the SW-CMM method is cost-effective, allows risks, doesn’t lead to extra bureaucracy, but does take a while to implement (on average 3 years per level) and is costly.
In chapter 16 McConnell lays out an interesting, laddered professional development plan for software engineers that organizations can use. The system is the one his company uses, and it measures engineers’ proficiencies in the knowledge areas listed above (putting them into five broad competencies); has plans for training, mentoring, and public recognition of improvement; is tied to salary; and presents a roadmap for becoming a “professional.”
McConnell revisits that idea of a software engineering profession in the final part of the book, “Industry Professionalism.” It seems to be the most controversial section, though it really shouldn’t be. According to McConnell’s thinking, software development is already on the way to becoming an engineering field, despite having a ways to go. When it finally arrives at the professional stage–which comes after commerce is wed with science to produce a high quality, dependable end-product–certification and licensing will be the recognition of the status quo. Customers will expect important software to behave well and will be able to hold one or more licensed individuals who worked on the product civilly or criminally liable. That might sound extreme but the potential of some software to do harm (or good for that matter) is just as great as the “products” of other professionals (doctors, lawyers, civil engineers, etc.).
Certification is voluntary and shows proficiency in a field. Licensing is not mandatory or universal but might be required by some jurisdictions when accountability is required. (Consider that your friend can do your own taxes and you can blame no one if things go wrong, or you can use a CPA and have legal recourse in the case of negligence or ethical violations). McConnell estimates that only 1-5% of all software professionals will need licensing.
Beyond that, the industry as a whole has an enormous responsibility to the public trust to act in a professional, ethical manner. Moreover, the industry should do a better job applying computer science and software engineering knowledge that was developed many years ago and is currently unused.
On the whole, Professional Software Development is a compelling, well-argued read. However you feel about its most controversial parts, if you construct, test, or manage software, it is well worth being familiar with the issues that will likely affect your career in the next decade or so. Beyond that, I found it contained a very compelling argument for seeking out and fixing the inefficiencies in individual projects and throughout the whole organization.
