To Automate or Not to Automate

In automated testing, Quality Assurance teams use testing tools to run detailed, repetitive, and data-intensive tests. Tools such as Testworthy allow teams to test more code – faster and more accurately, freeing up QA engineers to focus on tests that require manual attention and unique human skills. For an optimal testing process, it should be decided beforehand during the planning phase about which tests to automate and which to leave for manual testing. Following are some guidelines.

When to automate

1.  If the test run is repetitive and/or is run frequently, it should be automated. This will save precious time and resources, and will also yield more accurate results each time.

2.  Automate test cases which test critical paths – features/user flows that if fail, cause a considerable damage to the business.

3.   Inputting lots of data can be exhausting and error-prone if done by a human. This is why tests which require large data sets should be automated.

4.   There are three test cases that should always be automated: regression test, smoke test and sanity test, as they need to be run against every build/release of the app. Tests that need to be run against multiple configurations (different browsers/OS) should also be automated.

5.  It is better to automate performance testing, such as load testing and stress testing. Similarly, if a case requires an overnight/long-hours devotion, they should also be automated.

When to choose manual testing

1.   If you have a test case that is not yet thoroughly explored, manual testing is a better option.

2.   Tests that you will run only once. However, if this one-time test has a huge set of data, it is better to automate it, since it just might be used later on.

3.   Tests that require a user to respond in some unique ways.

4.   Tests that require ad hoc/random testing based on domain knowledge/expertise (exploratory).

5.   Tests which do not have predictable results. To get the best value out of automation the tests must produce predictable and reliable results in order to produce pass and fail conditions.

Automated test cases, when done well, provide fast feedback for developers, alert testers to problems well before they reach production, and free up testers to do more exploratory testing. Using test management tools such as Testworthy further eases the process, by automating test cases, analyzing data, generating useful reports, and storing test cases for future customization and use. However, as stated earlier, careful planning and deliberation are a must for a successful testing process.