Test automation should be seen as an addition, not a replacement, to manual testing.
Challenge:
What to automate - To often organizations purchase an automation tool and adopt the attitude that an "entire" test project should be automated. This is seldom, if ever, realistic. Know what to automate and what to leave manual. Time can be wasted trying to automate test cases which will only be executed once or twice more. [Source]
When analyzing what to automate, keep in mind that the test program should not duplicate the AUT's program logic. One rule of thumb to consider is that if it takes as much or more effort to automate a test script for a specific test requirement as it did to code the function, a new testing approach is required. [Source: Automated software testing By Elfriede Dustin, Jeff Rashka, John Paul]
If testing only needs to happen a small number of times, it should be done manually. If it needs to be run regularly--daily or even every milestone--it should be automated. Finding bugs right away while development still has the code in their minds is best. Do thorough exploratory testing of each feature immediately. Afterwards, automate the major cases. Automating everything is too costly. The project won't wait for all that testing to be written. [Source]
Decide a test selection criterion for automation to improve automation effectiveness
Test case selection criterion to automate:
Is the test case repeatable?
Does this test case require manual intervention?
Has the test case passed manual verification?
Are all the preconditions for the test case taken care-of?
Are the execution steps very clear?
Do we have test data for this test case?
Is the expected result clear enough to decide the test case status (PASS & FAIL)?
Will the test case survive the functional changes around it?
Is the test case straightforward for automation?
Can I trust this script to really test this part of the feature?
[Source]