Here are a few questions to consider when choosing a software test automation tool:
What are you trying to accomplish with the automation tool? Get specific, break down your goals, and make sure the tool you're considering for purchase can handle your needs.
How old is the tool, what version number is the tool on? Is the product stable? And will it likely continue to be on the market?
Does the tool have a major upgrade coming? Is this the right time to buy?
What training classes or books or online forums are available for education?
What does the ramp-up learning curve look like for you and your team?
Does your staff have the needed technical background to make use of the tool?
You have also to consider Compatibility Issues:
Any testing tool you choose will need to be compatible with:
The operating system(s) your application supports.
The development environment(s) used to create your application.
Third party software with which your application integrates.
Budget Constraints:
When you and your management set the initial automated test tool budget, consider these questions:
How much can we afford?
How much is this worth?
You will also want to consider these questions:
What benefit does the organization hope to get from the tool?
Is the benefit quantifiable and measurable?
It is also reasonable to ask the sales person how their tool compares to their competition:
How can I compare your product with other products on the market?
Under what situations is your tool the best choice?
Under what situations is your tool probably not the best choice?
What features differentiate your tool from the competition?
What don't you like about your tool?
While investigating the tools on the market, you probably discovered additional requirements you either didn't know about when you created your initial list or that you forgot to include. Add them now.
It is certainly possible that there is only one tool on the market that meets your requirements. If not then, use your list of requirements to rank the tools you investigated earlier.
Evaluate the Finalists.
During the evaluation, it is a good idea to model the entire process that your group intends to follow. For a GUI automation tool, that means you should:
Create the test.
Check the test into source control.
Run the test.
Evaluate the results.
Pretend the user interface changed and you need to modify the test accordingly.
Also See:
searchsoftwarequality
stickyminds
2005_Criteria_for_Software_Testing