Test automation has come a long way and in many ways is comparable to a software project. Therefore my proposed metric actually tailors a software cost estimation model to test automation. Here is how it is done. Software involves code. Test automation involves scripting. Both code as well as test scripts need to implemented, maintained, tracked and version controlled. In addition, test scripts include almost all coding constructs like IF conditions, FOR loops etc. making them applicable to code cost estimation techniques. However these estimation techniques have to be altered accordingly to produce the best results. This is what I will be attempting to do.
Cost estimation involving software generally measure one of the following two attributes:
1. LOC (Lines of Code)
2. Function Points
[Source]
The costs related of automated testing include:
cost of software tools.
cost of hardware.
cost of salaries for automated test engineers.
cost of training people.
percents from the pay off for equipments.
and software tools used in testing process.
maintenance expenses for equipments and tools.
expenses for analysis the behavior of the product based on the test data generated or on the results of the previous tests.
Compared with manual testing, the cost of automated testing is higher, especially at the beginning of the automation process.
[Source]
Examples of fixed automation costs:
Hardware (additional or upgrades to existing).
Testware software licenses.
Testware software support.
Per seat licenses.
Automation environment design, implementation.
Automation environment maintenance.
Scripting tools.
Tool and environment licenses.
Tool training.
Tool introduction and ramp up.
Examples of variable automation cost factors:
Test case design for automation.
Test case implementation.
Test maintenance.
Oracle creation.
Test case execution.
Test results analysis.
Defect reporting.
Test results reporting.
Test execution savings.
After-hours testing by systems (not people).
[Source]
The cost of automated testing is comprised of several parts:
The cost of developing the automation.
The cost of operating the automated tests.
The cost of maintaining the automation as the product changes.
The cost of any other new tasks necessitated by the automation.
This must be weighed against the cost of any remaining manual testing, which will probably be quite a lot. In fact, I've never experienced automation that reduced the need for manual testing to such an extent that the manual testers ended up with less work to do.
How these costs work out depend on a lot of factors, including the technology being tested, the test tools used, the skill of the test developers, and the quality of the test suite.
How about the ongoing maintenance cost?
Most analysis of the cost of test automation completely ignore the special new tasks that must be done just because of the automation:
Test cases must be documented carefully.
The automation itself must be tested and documented.
Each time the suite is executed someone must carefully pore over the results to tell the false negatives from real bugs.
Radical changes in the product to be tested must be reviewed to evaluate their impact on the test suite, and new test code may have to be written to cope with them.
If the test suite is shared, meetings must be held to coordinate the development, maintenance, and operation of the suite.
The headache of porting the tests must be endured, if the product being tested is subsequently ported to a new platform, or even to a new version of the same platform.
I know of many test suites that were blown away by hurricane Win95, and I'm sure
many will also be wiped out by its sister storm, Windows 2000.
These new tasks make a significant dent in a tester's day.
[Source:Test Automation Snake Oil]