There is an increasing pressure for QA testers to automate their processes. This is due to the assumption that automation is an easy way to cut costs and speed up testing — which can often be true. However, depending on the application being tested, automation isn’t always the right call. In some cases, it might not be worth the initial time and financial investment. So, what are the pros and cons of automation, and how can you find the right balance?
What is automation in QA testing?
When a QA engineer tests software manually, they’ll interact with it as if they were a real end-user. They will log any bugs and implement a fix. Increasingly, QA engineers are utilising automation tools, which involve running numerous pre-scripted test suites. These tools include AI bots, which explore an application as a person would (in theory), testing functions and generating data about the process.
What are the pros?
Speed: automated testing is often faster than manual testing because thousands of tests can be run simultaneously, and the data can be generated quickly. In large scale projects, it’s essential to automate at least part of your QA testing process due to the sheer number of manual testers you would need to meet turnaround times.
Reduce Human error: people make mistakes — especially on repetitive and monotonous tasks — so automating tests that fall into this category is good for everyone. According to research, organisations that utilise automation in their processes are more likely to be both highly productive and ‘human-friendly’ workplaces.
Efficiency: repetitive tasks such as smoke tests and regression tests are ideal for automation. There’s no real advantage to having a person do these manually, and in the long run, this will free up the tester’s time to focus on complex tasks, data interpretation and software fixes.
What are the cons?
Set-up costs: Automation can be expensive at the beginning. Specialist QA testers with a knowledge of automation command a higher salary, and you’ll also have to consider the cost of purchasing and maintaining automation tools. However, in the long-term automation should be cost-saving, as fewer engineers are needed to complete everyday testing tasks.
Adaptability: If your product frequently changes, that means the scripts will also need to be updated. Therefore, it may not be worth the hassle of creating automated scripts because you’ll miss out on the time and cost-saving benefits.
Context: Automation is simply inappropriate for many QA tests. For example, automated programs like Selenium can’t recognise video controls, so they’re not useful for testing video streaming. In addition, more subjective tasks like UX testing require a creative human input that can’t be automated.
How do you find the right balance?
Considering how the pros and cons of automation apply to your application will help establish the right balance of automation in your testing process. For example, if your product is relatively stable then the automated test script is likely to have a long shelf life, meaning that’s it's going to be worth the initial investment.
Experienced software tester, Shilpa Chatterjee Roy, suggests that selecting appropriate test cases and calculating return on investment (ROI) are key to adopting automation in your testing process. How you do this will depend on your application, but she suggests considering attributes such as the purchasing and licensing cost of the tool, time to develop the scripts, time to maintain the scripts, time to analyse the results manually and automatically, time and cost to train the resources, and management overheads.
To summarise, automation provides a host of time and cost-saving benefits, but it’s not a sure-fire way to improve software. Automation is not an all or nothing proposition, but a tool to maximise efficiency and allow QA testers to focus on the more complex tasks. Ultimately, you should evaluate your tests on a case by case basis and make the decision to automate on a micro-level.