While writing test cases, how much detail is required?
How much detail is required to write test cases is dependent on many factors. Below you can have a look at what different people say about this:
1. Justin has a great answer for this on Stackexchange: If tester's experience level/level of Exploratory Testing skills is high then test cases require relatively little detail and if the experience is low then enough detail should be there as to avoid tester confusion. He also takes the case of importance of keeping detailed records, if importance is high then very detailed test cases else again enough detail should be there as to avoid tester confusion.
Image Credits
2. user246 says - you need to calibrate(adjust) the level of detail with the capabilities of your tester. I think you should always describe the intent (purpose) of the test case, i.e. the requirement you are testing. For someone with more experience, you can leave the rest to the tester's imagination. For someone with less experience, you may need to spell things out, with the accompanying maintenance costs.
Afterwards he says that specific details of how you test something may change more quickly than the intent of the test case. For example, it may be important for an individual's first name to be up to 20 characters in length, but the way you navigate to the relevant dialog box will change over time.
3. In Suchit Parikh's point of view the time spent by a tester in documenting too much is a time wasted, as the current testers will be well aware of the basic interaction with the system, and the new hire should be given appropriate training of the system and hence, every single step need not be documented in a test case.
One thing is for sure that detailed test cases are time consuming to develop and are harder to maintain.
4. I also like the points made by ljeanwilkin "add some time in the schedule for ad hoc testing, if that's even necessary. Good testers play around regardless of whether that time appears in the schedule."
5. Point made by thekid is also worth noting "Write them at a level of detail that if the UI were to change that would not require you to have to rewrite all the steps to recreate."
6. bfoote also makes a point worth noticing - "Design the cases for someone who knows how to test but has limited knowledge of the product."
7. Heiberg says, I think you should go for the test case level which is good enough for the test executers to be able to do their job. If you discover some cases which are not detailed enough - add some more details.
More detail => More maintenance work when system changes, more work when you want to use the test cases for another project.
Less detail => test cases will not be executed exactly the same way each time (for example different test data might be used). So you might not be able to reproduce a problem found by the test case just by re-running it once.
A real life story about this (since I have one - not sure if it adds anything useful):
In my previous job (testing mobile phone SW) we had test cases which were not detailed. This was OK when the test case writers were also the test executers and had experience and knowledge about the SW. Then test execution was moved to another team (in another country), which did not have the same knowledge and experience. After this move the test cases (of course) needed to be much more detailed and all the things not mentioned because they were common knowledge had to be added.
8. JakeBrake says:
Based upon all thus far, it would seem in reading between the lines that at a minimum:
1) it depends upon the nature and the criticality of the application or system,
2) the familiarity of the tester with the application-under-test,
3) the schedule,
4) level of complacency in a job as related to tenure, and
5) exclusive of all others - contractual requirements.
Example for 1): In a system that lands aircraft automatically aboard an aircraft carrier, one can ill-afford to have non-detailed test cases.
Example for 2) through 4): A newbie at a new employer? There are a myriad (countless) of scenarios here depending on how much project time is allocated to test development.
9. Matt Archer has written a detailed post on How Much Detail Should I Write In My Test Case?
Factors as to why we may consider writing a more detailed Test Cases like Evidence of Test Coverage, Future Automation, Necessary Precision, Control, Repeatability and Factors as to why we may consider writing a less detailed Test Cases like Application Knowledge, Other Documentation, Test Analysis Experience, Goal Focused, Bug Reporting Experience, Available Time and Resource.
10. Goldsmith did an experiment where test cases are described with 5 different levels of details. Interestingly, when the results were compiled, no one level of detail stood out as the “right” level of detail. He further writes "Requirements need to be driven down to detail—first so developers can develop the proper products, and second so testers can confirm they’ve done so. The survey example exercise can be used just as effectively to guide meaningful communication and practices regarding interpretation of how many requirements are represented."
11. Linda Westfall in her book [The Certified Software Quality Engineer Handbook] writes that the level of detail needed in a documented test case will vary based on the needs of the project and testing staff. For example, automated test cases may need less documentation than those run manually, unit level test cases may need less documentation than system test cases, test cases for a system used to search for library books may need less rigor than those for airplane navigation systems. The diagram illustrates two examples of test case documentation: one simple example for unit test cases and one more rigorously documented example for a system test case.
Unit Test Case Example
System Test Case Example
How much detail is required to write test cases is dependent on many factors. Below you can have a look at what different people say about this:
1. Justin has a great answer for this on Stackexchange: If tester's experience level/level of Exploratory Testing skills is high then test cases require relatively little detail and if the experience is low then enough detail should be there as to avoid tester confusion. He also takes the case of importance of keeping detailed records, if importance is high then very detailed test cases else again enough detail should be there as to avoid tester confusion.
Image Credits
2. user246 says - you need to calibrate(adjust) the level of detail with the capabilities of your tester. I think you should always describe the intent (purpose) of the test case, i.e. the requirement you are testing. For someone with more experience, you can leave the rest to the tester's imagination. For someone with less experience, you may need to spell things out, with the accompanying maintenance costs.
Afterwards he says that specific details of how you test something may change more quickly than the intent of the test case. For example, it may be important for an individual's first name to be up to 20 characters in length, but the way you navigate to the relevant dialog box will change over time.
3. In Suchit Parikh's point of view the time spent by a tester in documenting too much is a time wasted, as the current testers will be well aware of the basic interaction with the system, and the new hire should be given appropriate training of the system and hence, every single step need not be documented in a test case.
One thing is for sure that detailed test cases are time consuming to develop and are harder to maintain.
4. I also like the points made by ljeanwilkin "add some time in the schedule for ad hoc testing, if that's even necessary. Good testers play around regardless of whether that time appears in the schedule."
5. Point made by thekid is also worth noting "Write them at a level of detail that if the UI were to change that would not require you to have to rewrite all the steps to recreate."
6. bfoote also makes a point worth noticing - "Design the cases for someone who knows how to test but has limited knowledge of the product."
7. Heiberg says, I think you should go for the test case level which is good enough for the test executers to be able to do their job. If you discover some cases which are not detailed enough - add some more details.
More detail => More maintenance work when system changes, more work when you want to use the test cases for another project.
Less detail => test cases will not be executed exactly the same way each time (for example different test data might be used). So you might not be able to reproduce a problem found by the test case just by re-running it once.
A real life story about this (since I have one - not sure if it adds anything useful):
In my previous job (testing mobile phone SW) we had test cases which were not detailed. This was OK when the test case writers were also the test executers and had experience and knowledge about the SW. Then test execution was moved to another team (in another country), which did not have the same knowledge and experience. After this move the test cases (of course) needed to be much more detailed and all the things not mentioned because they were common knowledge had to be added.
8. JakeBrake says:
Based upon all thus far, it would seem in reading between the lines that at a minimum:
1) it depends upon the nature and the criticality of the application or system,
2) the familiarity of the tester with the application-under-test,
3) the schedule,
4) level of complacency in a job as related to tenure, and
5) exclusive of all others - contractual requirements.
Example for 1): In a system that lands aircraft automatically aboard an aircraft carrier, one can ill-afford to have non-detailed test cases.
Example for 2) through 4): A newbie at a new employer? There are a myriad (countless) of scenarios here depending on how much project time is allocated to test development.
9. Matt Archer has written a detailed post on How Much Detail Should I Write In My Test Case?
Factors as to why we may consider writing a more detailed Test Cases like Evidence of Test Coverage, Future Automation, Necessary Precision, Control, Repeatability and Factors as to why we may consider writing a less detailed Test Cases like Application Knowledge, Other Documentation, Test Analysis Experience, Goal Focused, Bug Reporting Experience, Available Time and Resource.
10. Goldsmith did an experiment where test cases are described with 5 different levels of details. Interestingly, when the results were compiled, no one level of detail stood out as the “right” level of detail. He further writes "Requirements need to be driven down to detail—first so developers can develop the proper products, and second so testers can confirm they’ve done so. The survey example exercise can be used just as effectively to guide meaningful communication and practices regarding interpretation of how many requirements are represented."
11. Linda Westfall in her book [The Certified Software Quality Engineer Handbook] writes that the level of detail needed in a documented test case will vary based on the needs of the project and testing staff. For example, automated test cases may need less documentation than those run manually, unit level test cases may need less documentation than system test cases, test cases for a system used to search for library books may need less rigor than those for airplane navigation systems. The diagram illustrates two examples of test case documentation: one simple example for unit test cases and one more rigorously documented example for a system test case.
Unit Test Case Example
System Test Case Example
