Menu
Backlog automation – QA Team required bandwidth, in order to create an automated release cycle.
As the company was growing fast, the QA teams needed to create bandwidth in order to build an automated release process, which would help them keep up with the release velocity of the products. The bandwidth available for automation was being consumed by other products in the pipeline. Also the estimated time needed for automation of HRE was 4-6 months of 2 -3 dedicated automation engineers time. This would only cover the build verification test. Also since the product was undergoing rapid changes, majority of the features were assumed to also impact the user interface, which would mean that maintaining the test scripts should not take too much
Once Nogrunt was signed up, the customer was able to supply a document that listed all of the test scenarios that needed to be automated. The first challenge for the automation team was to understand the product. The team spent a day running the test scenarios manually to get to know the product better. During this time, it was noted that , some test steps did not match the application. Then the team discussed, the design, structure and sequence of the test scenarios and that also helped decide the sequence in which the scenarios would be automated. All of the creation scenarios were to be automated first and would also run first as part of the test runs. All configurations that turned on/off features, would be tested separately in their own test run and would not be mixed with the main run (for instance, one test required turning off centers, which would affect the behaviour of a large portion of the tests ). All reports scenarios would be run at the end.
The first Monday after the tests were built, the pass percentage had fallen to about 72%. On digging further the automation engineer realized that they had not been informed about UI changes that were released to the QA environment. By the end of the day all of the scenarios that had UI changes had been reauthored and the pass percentage was back up at 95% at which point the tests were handed over to the customer to own and maintain.
Armed with the clarifications, the design and the sequence of the scenarios, the single Nogrunt automation engineer was able to automate all of the tests within 6 days. At the end of 6 days the tests were all organized into appropriate suites and had been executed as part of the suite. The pass percentage was around 60%. Post another round of understanding the tests and clarifications from the customer, by Day 8 the team had gotten to a 90% pass percentage. The rest 10% failures were due to configurations needed to be changed.
Though the Nogrunt test automation platform has enabled the customer to transform how they direct their bandwidth, it is the capability of the Nogrunt platform that really made a difference The QA team can now focus on the task of identifying the quality leaks and defects in the system, while Nogrunt takes care of rapid automation and execution of the tests. The added benefit of the Nogrunt platform is that the QA teams now no longer need to maintain test environments, they can just focus on scenarios and coverage while Nogrunt addresses the grunt work of maintaining environments, authoring tests and executing them.
Eventually the Nogrunt platform was able to deliver in 6 days, something that was estimated to take 18 man months worth of effort using traditional automation approach.