Defect Seeding is a technique that was developed to estimate the number of bugs resident in a piece of software.
Conceptually, a piece of software is "seeded" with bugs and then the test set is run to find out how many of the seeded bugs were discovered, how many were not discovered, and how many new (unseeded) bugs were found. It's then possible to use a simple mathematical formula to predict the number of bugs remaining.
[If an organization inserted 100 seed bugs and later were only able to locate 75 of the seeded bugs, their seed ratio would be 0.75 (or 75%). If the organization had already discovered 450 "real" defects, then using the results from the seeding experiment, it would be possible to extrapolate that the 450 "real" defects represented only 75% of all of the real defects present. Then, the total number of real defects would be estimated to be 600. Since only 450 of the potential 600 real defects have been found, it appears that the product still has 150 "real" defects waiting to be discovered plus 25 seed bugs that still exist in the code. Don't forget to remove the seed bugs!]
| Estimated Number of Real Defects Still Present | = | Estimated Total Number of Real Defects | - | Number of Real Defects Found |
Seed Ratio = Number of Seed Bugs Found / Total Number of Seed Bugs
Number of Seed Bugs Found = 75
Total Number of Seed Bugs = 100
Seed Ratio = 75 / 100 = 0.75 or 75%
| Estimated Total Number of Real Defects | = | Number of Real Defects Found Seed Ratio |
Number of Real Defects Found = 450
| Estimated Total Number Of Real Defects | = | 450/.75 = 600 |
Estimated Number Of Real Defects Still Present = 600 - 450 = 150
[Source: Systematic Software Testing by Rick D. Craig and Stefan P. Jaskiel]
Drawbacks of Defect Seeding
A common problem with this type of technique is forgetting to remove the errors deliberately inserted. [Source]
Another common problem is that removing the seeded defects introduces new errors. To prevent these problems, be sure to remove all seeded defects prior to final system testing and product release. A useful implementation standard for seeded errors is to require them to be implemented only by adding one or two lines of code that create the error; this standard assures that you can remove the seeded errors safely by simply removing the erroneous lines of code. [Source]
A simple Question on Defect Seeding:
A defect-seeding program inserts 81 defects into an application. Inspections and testing found 5,832 defects. Of these, 72 were seeded defects. How many errors or defects are predicted to remain in the application?
523
640
648
729
Correct Answer: 729
To determine the total number of defects, use the following formula:
| 72(discovered seeded defects) 81(total seeded defects) | = | 5832(total discovered defects) X(total defects) |
Then, solve for X:
72X=5832(81)
X= 5832(81)
72
X=6561
Take the total number of defects minus the total discovered defects to determine the amount of defects predicted to remain in the program:
6561 – 5832 = 729
[Source]
Another way of defining Defect Seeding:
Defect seeding is a practice in which defects are intentionally inserted into a program by one group for detection by another group. The ratio of the number of seeded defects detected to the total number of defects seeded provides a rough idea of the total number of unseeded defects that have been detected.
Suppose on GigaTron 3.0 that you intentionally seeded the program with 50 errors. For best effect, the seeded errors should cover the full breadth of the product’s functionality and the full range of severities—ranging from crashing errors to cosmetic errors.
Suppose that at a point in the project when you believe testing to be almost complete you look at the seeded defect report. You find that 31 seeded defects and 600 indigenous defects have been reported. You can estimate the total number of defects with the formula:
IndigenousDefects[Total] = ( SeededDefects[Planted] / SeededDefects[Found] ) * IndigenousDefects[Found]
This technique suggests that GigaTron 3.0 has approximately 50 / 31 * 600 = 967 total defects.
To use defect seeding, you must seed the defects prior to the beginning of the tests whose effectiveness you want to ascertain. If your testing uses manual methods and has no systematic way of covering the same testing ground twice, you should seed defects before that testing begins. If your testing uses fully automated regression tests, you can seed defects virtually any time to ascertain the effectiveness of the automated tests.
[Source]
Another worth reading article on Defect Seeding
Defect and Fault Seeding In Dependability Benchmarking