I gave my second presentation tonight in my Software Testing Techniques course. Two weeks ago, I discussed software defects with a synopsis of the Coding Horror article Making Developers Cry Since 1995. दीप्ती-जी, my QE, hasn’t made me cry — yet. This week: defect tracking systems. You know, those applications where testers (and everyone else) log the defects that bring us to tears.

Here’s a synopsis:

Purpose of Defect Tracking

  • Ensure that all issues get fixed or closed.
  • Facilitate communication related to defects.
  • Provide a sensible workflow that makes fixing defects easier.

Major Features

  • Issue entry
  • Workflow modeling (ensure high quality process)
  • Change Notification
  • Search and reports
  • Multiuser

How They Differ

  • Who can enter and change issues?
  • Customizability (workflow, reports, fields)
  • Underlying technology — most use RDBMS
  • Who can create a report?
  • Public v. Private reporting — Does every issue get its own URL?
  • Client-server, web-based, standalone
  • Features, cost
  • Logging / audit trail — can help with government compliance
  • Security

Extras

  • Project tracking (requirements, tasks, estimation, dependencies etc.)
  • Source control (SCM) - Link checkins with issues (bidirectionally)
  • Customer interactions (CRM)
  • Help desk tickets
  • Release systems: Build, test, integration systems
  • Web publishing (defects and solutions)
  • LDAP / Active Directory
  • Automatic reporting from applications
  • Documentation systems (release notes)
  • APIs — SQL, RSS, XML-RPC, etc.

A Sampling of Reports and Queries

  • Records by owner
  • Records by component
  • Records by release
  • Custom record list
  • Find/fix rate
  • Aging
  • Dashboards / scorecards
  • Release readiness

Some Defect Tracking Software

I looked at several defect tracking solutions while researching and compared what I read to my experiences with my company’s in-house problem tracking tool. Once a system has covered the basics of defect reporting, tracking, and notification (that is, company-wide communication about changes in the state of a defect), it’s worth asking how else a defect tracking system can add value to a software organization. The amount of extra “value” available from defect tracking software appears to vary directly with the size of the budget one can set aside for buying an off-the-shelf package (e.g., JIRA), configuring an open source product (such as Bugzilla), or building a solution from scratch (like what we have at the office).

Depending on its size, a software organization may benefit from a number of extra features that aren’t directly on the defect-resolution path. Many organizations want to share the state of externally reported defects with their customers. Similarly, companies often wish to integrate their development databases (containing defects and customer-reported enhancements) with their documentation and marketing departments when it comes time to create release notes and “what’s new” documents. Integrating source control and build mechanisms with defect tracking makes verification easier, facilitates compliance, and aids managers in determining release readiness. Many of the software products currently available support all of these tasks and have customizable workflows that model high-quality software development processes (such as those in the Capability Maturity Model).

Also, defect tracking systems can handle more than defects. Software enhancements and maintenance tasks usually follow the same pattern as bug fixing — triage, development, integration, and verification — though they differ in scope and often require additional process modeling. But, it can be hard to bolt these extra features onto a product that wasn’t designed to handle them; in addition to picking a configurable, modifiable system, choose one whose defect workflow also makes sense for projects.

Coming up: a defect/feature tracking system workflow. Plus, more photography and whatnot.