Some fix time data

October 13, 2005

Someone at work wanted some info on estimating, so I went to my trusty bookshelf and pulled out a copy of Watts Humphrey’s “A discipline for software engineering”. It’s got some info on how to better estimate based on historical estimate vs. actuals.

But that’s not what this post is about. One little gem I came across is on page 275. It’s about the cost of resolving a defect found during the different phases of the development process. As follows:

Some Fix Time Data. There are not many published data on the time required to identify software defects. Following are some that are available:

  • IBM: An unpublished IBM rule of thumb for the relative costs to identify software defects: during design, 1.5; prior to coding, 1; during coding, 1.5; prior to test, 10; during test, 60; in field use, 100.
  • TRW: The relative times to identify defects: during requirements, 1: during design, 3 to 6; during coding, 10; in development test, 15 to 40; in acceptance test, 30 to 70; during operation, 40 to 1000 [Boehm 81]
  • IBM: The relative time to identify defects: during design reviews, 1; during code inspections, 20; during machine test, 82 [Remus]
  • JPL: Bush reports an average cost per defect: $90 to $120 in inspections and $10,000 in test [Bush] {Note-published 1995}
  • Freedman and Weinberg: They report that projects that used reviews and inspections had tenfold reduction in the number of defects found in test and a 50 percent to 80 percent reduction in test costs, including the costs of the reviews and inspections.

Clearly, defect identification costs are higher during test and use. Thus anyone who seeks to reduce development cost or time should focus on preventing or removing defects before starting test.