By Stephen Beck, Automation Architect, Inspired Testing
3E® is a complex application with simple interfaces. There are countless screens and endpoints to test which makes it arduous to get full coverage through manual processes. The first thought that any quality assurance specialist will have is; can we automate this? Automation has plenty of advantages:
- On-demand tests with quick results – no need to schedule a testing cycle for automation
- Exact test repetition (consistency) – no human input to cause deviance in the test results
- More tests can be run per test cycle – the human testers can do different tests to the automation
- Automation can cover more complex tests leaning toward statistics and long calculations
- Code is created in reusable ”modules” which allows us to reuse parts of the tests in other tests
- Once function libraries are created, they can be reused without the same time investment
Automated tests can enable types of testing which are impossible with manual testing. An example of this is from the insurance industry with regard to quotations. I have seen a manual test which validated a scheme level quote calculation to a single percentage of tolerance. This had been the method of testing since the application’s inception, as member level validations would have required over one hundred and seventy thousand cells of data to be checked—not possible for humans.
Automation was applied as a check and discovered multiple errors that failed the tolerance check. Automation enabled these errors to be discovered, and furthermore it was able to perform these checks in less than one minute.
Using automated tests enables us to create test suites which take a manual testing cycle, convert it to automation and do it in an nth of the time. To cap it off, automation testing doesn’t require people for initiation and can be part of the deployment process. What’s not to love?
If we think about 3E, there are over one hundred workflows which are part of the base version of the application, all of which require testing. Once a law firm purchases 3E, they will then naturally start customizing it to meet specific business needs which causes the backlog of workflows to increase. For a law firm to start automating, that becomes a daunting task; there are hundreds of screens and processes which need to be mapped out with acceptance criteria and checks put in place to ensure quality. Luckily, because of the nature of 3E and the time since the Angular version was released, we are starting to see testing companies build accelerator packs.
- These packs work by taking the base installation of 3E and building an automated suite off the vanilla requirements
- This can reduce the backlog of effort to get started with automation significantly, as it scopes the automation effort to the customizations made by the law firm and
- It reduces the effort required to test a product not developed by the law firm
As we start to see the mountain of automated scripts developing, we run into a new problem: testing is complicated and test scripts are complex to write and require effort to maintain— automation scripts need to be kept current, as flows and locators on an application can change due to development and business requirements.
This leads us to a concept which more quality assurance teams are starting to adopt to address the growing automated testing effort: Automate your risk, not your application.
A focus on automating everything will eventually require your automation team to regularly grow if they are to keep up with maintenance requirements while still creating new scripts. It makes more sense to apply types of testing based on a cost-effort ratio. Automation should be used to save time and money and not for the sake of automation.
What should we automate then?
- High risk processes
- Tests that require calculations
- Tests that require thousands of validations
- Tests that require random sampling
- Tests that check against databases
- Tests that need to be run repeatedly