Tuesday, November 16, 2010

Cost of Defects in Requirements

     Tom Gilb, Jeff Sutherland, Kai Gilb

Requirement defects cause rework. Every defect is a bug that causes extra work. So it is important to check your requirements for defects.

Tom Gilb has an agile one hour workshop that will take a random page of your specifications and identify the number of defects. You can then multiple the defects discovered on this page by the total number of pages of requirements to estimate the total number of requirement defects that you are planning to implement.

Defects are things like:

1. The requirement is not clear to the developer implementing it.
2. The requirement specifies technical details instead of the user requirement.
3. The requirement is not testable.

In a recent paper, Tom describes a one hour workshop with management in a large company. They identified 33 defects in an hour. He has data from many decades of work showing that the first time through you will only identify 1/3 of the defects. If you spend another hour you will find many more defects.

So in one page, many companies have 100 defects. Any major defect has been shown to take about 10 hours of extra work. In one of the examples in Tom's paper he tells the management they have 2000 hours of rework ahead of them. They are surprised and ask him how he knows that. The project team has been working for a year and was supposed to be finished after about 2000 hours of project time. However, they still have another year of work.

So take a look. 100 defects for every page of product backlog items may be slowing your projects down. Tom Gilb. Agile Specification Quality Control: Shifting Emphasis from Cleaning to Sampling Defects. TE Testing Experience. March, 2009. Here is a link to Tom's paper.

1 comment:

Joe Little said...

And we can say more broadly that 'bad news does not get better with age'. Meaning: Any defect in thinking (and thinking in the sky always has defects) must be discovered quickly and fixed quickly in our work. And our work is only about knowledge creation (aka 'thinking').

So, always and everywhere we are asking: "How do we stop the bad news getting worse with age?" It is a hard problem, and it is down mainly to the people to solve it. Getting the spec 'right' is only one key aspect of the problem.

As one example, if the spec has no defects but nonetheless is not the highest value stuff that the customer wants, that is 'bad news' that we must find and fix. As quickly as possible.


PS. These are not ideas that are new to Jeff Sutherland at all. But thoughts about extending this particular blog post...for others.