If the same kinds of tests(TestSet) are repeated again and again, eventually the same set of test cases will no longer be able to find any new defects/bugs.To overcome this “Pesticide Paradox”, it is really very important to review the test cases regularly and new and different tests need to be written to exercise different parts of the software or system to potentially find more defects.
What is Defect Clustering?
When a small number of modules contains most of the bugs detected or show the most operational failures. Defect Clustering basically based on Pareto Principle, also known as 80-20 Rule which states that approximately 80% of the problems are caused by 20% of modules.
What is AUT Hop Spot?
When we are testing a new software against the user requirements,defects will be finding at a large numbers in certain flow or area of the code which is more complex or critical. When the same software or code is to be tested again & again,after some changes or modifications,we will find the defects in initial few iterations of testing by identifying & concentrating on the 'Hot spot'.
Importance of Hop Spot?
Tester can focus the same area in order to find the more number of defects. It will reduce the time & cost of finding defects as by focusing on certain areas more bugs can be find in order to improve quality of the software.
Parasite Paradox?
But after certain number of iterations of testing,as the testing improves we see the defect numbers dropping, the most of the bugs will be fixed & the 'Hot spot' area will be cleaned up. Developers will be extra careful in a places where testers found more defects & might not look in other areas. Hence by executing the same test cases will no longer help to find the more defects. The test cases need to be revised, and new and different tests need to be written to exercise different parts of the software or system to potentially find more defects.
Now we have two choices.
1.To write whole new set of test cases to exercise different parts of the software.
2.To prepare new test cases & add them to the existing test cases.
Parasite Paradox and Defect Clustering
In the first case,we will be finding more potential defects in an area where we did not focused earlier or the area at which developer was not extra cautious as the tester was not raising the defects from these areas. But by neglecting earlier identified defect cluster, we are certainly taking a risk of giving less importance to the area which used be very critical in order to find the more defects in earlier iterations of testing.
In the second case,we can find the new potential defects in the new area as well as we can focus on the earlier identified defect cluster. But on the other hand the number of test case will become so large that it will increase the testing time & in turn increase the cost of testing. Hence too many useless tests may be an overhead.
To deal with this we can approach the middle path for balancing the disadvantages of both the listed choices. We can identify & remove the test cases that are not much important & have failed to detect any defect in certain number of test cycles.
For example, if you have 10 tests that cover the same area and none of them have detected bugs in number of cycles (for example,5 test cycles), then we should review them and reduce the number of test cases.
If test has not reported a bug in the last 5 runs, review it and verify its importance and weighting whether we should keep or archive it.
In this way we can keep the check on number of useless test cases without compromising with the quality as all the test cases covering important areas of the software will be retained. We have to follow this method regularly & keep updating/modifying our test cases whenever there is a change/modification occurs in the software.
