Automatic regression tests in a CI/CD environmentPublished on: Author: Team Communication Category: Test-driven development
Test Automation Engineer Niels van der Meer is part of one of our development teams at Rijkswaterstaat (the Department of Waterways and Public Works). He successfully deploys automatic regression tests on the ‘Better Served’ project. By deploying tests early in the development process, bugs in the application are found sooner and solved more quickly. And if the test succeeds, it is an immediate confirmation for developers that the quality of their code is good.
The ‘Better Served’ project has to improve the coordination of operating times of locks and bridges on demand. How? Through efficient coordination, fast and safe handling of shipping traffic, and by providing current and uniform information. Qualogy helps Rijkswaterstaat with the development of web forms that organize the coordination of locks and bridges in a uniform manner.
Automatic regression tests
All application functionalities are tested with automatic regression tests after a change in the code. Niels: “It’s actually simply a simulation of the manual regression test. This repetitive work is often experienced as boring. By automating the test, I immediately eliminate error sensitivity of manual testing. If we deploy something in the test environment, we know for sure that it works.
I think it’s important to involve the business in the process. That is why I use the Robot Framework to build my automatic regression tests. It’s a keyword-based framework, which makes it understandable to the business as well.”
Automatic regression tests in a CI/CD environment
Qualogy’s software development process is set up to go into production as quickly and efficiently as possible. Niels: “We shift the test phase to the left in our CI/CD pipeline with automatic regression tests. We call this the ‘shift left’ principle. Tests are made at the same time as writing the code. Bugs in the application are therefore found and fixed earlier. The result: high quality code in less time. Meaning, less costs.”
Costs versus benefits
Building an automatic regression test is time-consuming. Niels explains: “You have to take every scenario into account: the happy flows and the not-so-happy flows. The test has to be as comprehensive as possible. The investment at the start of the process is therefore high, but because of its repetitive nature, it yields a lot at the end. For example, in the ‘Better Served’ project, we test every push to GitLab. That sometimes amounts to 20 times a day.
Not everything is testable. But that’s okay. It ensures that we stay focused in areas that aren’t tested with automatic regression tests.”
Trust the test
Developers can blindly trust the test. Because if the test succeeds, they know their code is good. Niels: “Rijkswaterstaat is very impressed. Normally, between 60 and 70 bugs are reported in their test environments. The application test environment only produced one bug in a month and a half. We at Qualogy are quite proud of that.”
For more information on the possibilities of automatic regression tests, please contact us at firstname.lastname@example.org or 070 319 5000.