Defect Age & Defect Spoilage
The later we discover a bug, the greater harm it does and the more it costs to fix.
Defect Age is another metric that can be used to measure the test effectiveness, which assigns a numerical value to the fault depending on the phase in which it is discovered.
Below table shows a scale for measuring defect age. Notice that this scale may have to be modified to reflect the phases in your own software development lifecycle and the number and names of your test levels. For example, a requirement defect discovered during a high-level design review would be assigned a PhAge of 1. If this defect had not been found until the Pilot, it would have been assigned a PhAge of 8.
Below table shows an example of the distribution of defects on one project by phase created and phase discovered. In this example, there were 8 requirements defects found in high-level design, 4 during detailed design, 1 in coding, 5 in system testing, 6 in acceptance testing, 2 in pilot, and 1 in production. If you've never analyzed bugs to determine when they were introduced, you may be surprised how difficult a job this is.
Defect Spoilage
Defect Spoilage is a metric that uses Phase Age and distribution of defects to measure the effectiveness of defect removal activities.
Below table shows the defect spoilage values for a particular project, based on the number of defects found weighted by defect age. During acceptance testing, for example, 9 defects were discovered. Of these 9 defects, 6 were attributed to defects created during the requirements phase of this project. Since the defects that were found during acceptance testing could have been found in any of the seven previous phases, the requirements defects that remained hidden until the acceptance testing phase were given a weighting of 7. The weighted number of requirements defects found during acceptance testing is 42 (i.e., 6 x 7 = 42).
| Phase Discovered | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Phase Created | Req. | HL Design | Det. Design | Coding | Unit Testing | Intg. Testing | System Testing | Accep. Testing | Pilot | Prod. | Spoilage= Weight/Total Defects |
| Req. | 0 | 8 | 8 | 3 | 0 | 0 | 30 | 42 | 16 | 9 | 116/27=4.3 |
| HL Design | - | 0 | 9 | 6 | 0 | 4 | 15 | 6 | 14 | 8 | 62/20=2.1 |
| Detailed Design | - | - | 0 | 15 | 6 | 12 | 0 | 0 | 6 | 42 | 81/31=2.6 |
| Coding | - | - | - | 0 | 62 | 32 | 18 | 8 | 15 | 120 | 255/109=2.3 |
| Summary | - | - | - | - | - | - | - | - | - | - | 514/187=2.7 |
Generally speaking, lower values for spoilage indicate more effective defect discovery processes (the optimal value is 1). As an absolute value, the spoilage has little meaning. However, it becomes valuable when used to measure a long-term trend of test effectiveness.
[Source: Systematic Software Testing by Rick D. Craig and Stefan P. Jaskiel]
Another Example of Defect Age & Defect Spoilage
The following table defines the scale according to phases. So, for instance, requirement defects, if found in the design phase, have a scale of 1, and the same defect, if propagated until the production phase, goes up to a scale of 4.
Above table: Scale of Defect Age
Above table: Defect Spoilage
Once the scale is decided now we can find the defect spoilage. Defect spoilage is defects from the previous phase multiplied by the scale. For instance, in the above table we have found 8 defects in the design phase from which 4 defects are propagated from the requirement phase. So we multiply the 4 defects with the scale defined in the previous table, so we get the value of 4. In the same fashion we calculate for all the phases. The following is the spoilage formula. It’s the ratio of the sum of defects passed from the previous phase multiplied by the discovered phase then finally divided by the total number of defects. For instance, the first row shows that total defects are 27 and the sum of passed on defects multiplied by their factor is 8 (4 X 1=4 + 2X 2= 4). In this way we calculate for all phases and finally the total. The optimal value is 1. A lower value of spoilage indicates a more effective defect discovery process.
[Source: Software Testing Interview Questions by S. Koirala & S. Sheikh]