Sunday, October 5, 2008

5 Generations of Test Automation Framework

The testware development process is a succession of generations, which have become increasingly advanced and sophisticated over the time.

First Generation

Capture/playback tools

The main advantage was it allowed tests to be run unattended for several hours each day.Also it made automation of regression testing possible.

Tests were easily broken because even a minor change to the AUT required that the test to be recorded again manually. It was overall expensive at the time of first execution. Maintenance overhead was much more.

Second Generation

It was the addition of Scripting to the Capture/Playback tools. These scripts were highly customized or modified. Most people were of the view that testing has become software development project as to create a new test process meant to create a new script. It required considerable technical knowledge, expertise and support. Maintenance of ever-increasing libraries of automation scripts became a great challenge.

Third Generation

Data-driven testing

In this approach Test data is held separately. The test library is the test scripts. Capture replay may (or may not) be used as an input to the script-building process but more use is now of driver scripts. One advantage is that the data can be set ready by testers without depending on automation engineers. Now data can be created in huge amounts (as per the requirements) and in many possible variations to test the AUT robustly. These previous points greatly divide the problem into two parts so that it can be resolved amicably but maintenance issues are still there not now only of the scripts but also of data.

Fourth Generation

Smart Framework engines (Keyword driven) are used in this case to replace scripting for the major part. With this the maintenance needed due to revisions in the test is greatly reduced. The cost and time of test design, automation, and execution is also greatly reduced with this approach.

Fifth Generation

Hybrid testing tools

Most frameworks evolve into this over time and multiple projects. Smart Framework engines too are used here but somewhat refined to offer what may be called business rules-based test design and “scriptless” automation through the use of GUI views, templates etc.

Also See:

Keyword Driven Framework

Functional Decomposition Framework

Choose Your Test Automation Framework