Testing from a Developer’s point of viewPublished on: Author: Bram Dekker Category: IT development and operations
Testing is not always the most fun part of our job. Some even see it as a necessity, but I think we can all agree that testing will save us all a lot of time and because of that the customer’s money.
If you start building an application and the team has a shared view of the application, tests are not always needed to be sure that you are in control of the code and system behavior. However, when the team starts to change (which is not an if, but a when), and the system starts increasing in scope and complexity, we could start losing grip of having full control over the quality of the system. Changing the system might then lead to a breaking change in another part of the system that we just did not think of. At that point, running a test-suite that was enhanced over time during the development of the system might be a savior.
On the other hand, having to extend a legacy system with functionality might also be a good case for a test-suite. Given that we do not have control over the system and we can consider it as a black box, we could start building a test-suite around it by checking the output values with given inputs. If we then extend the system, we can at least be (almost) sure that the old behavior will not break because of our changes.
Cover all cases
Running a test-suite in these cases and seeing all tests turn up green is of course a relief. However, you can never be sure that the tests that were written cover all cases. Below I will point out some ideas and common pitfalls of writing tests that might help to build a more stable and understandable set of tests that covers more system functionality and will decrease the maintenance nightmare.
The ideas and tips below focus mainly on functional system tests, but some of these ideas could be applied in other layers of the testing stack (integration, unit, ...). This is because testing to me is a multidisciplinary effort that should reflect business value and system behavior.
Shared Idea of System Quality
Most people within a project team do agree that testing is important but might not agree on what level testing should happen: system, integration, unit and what it should encompass: only external or internal dependencies, full system meltdown triggers, is not always clear. To start testing it is always preferred to get a shared understanding of all team members, their view on what should be tested and why. When you are working on a project with dependencies on other systems or teams it is wise to manage expectations between your team and the external dependency and with that avoid finger pointing later on. Preferably this shared idea is written down and reflected upon to avoid confusion.
Data generation using builders
Test-cases related to the same subject use similar data but slightly different from each other. For example a User with or without an email address. To avoid writing a lot of boilerplate code, the builder pattern might help you to generate slightly different version of the same objects for testing purposes.
Mob test definition sessions
Alone we might be able to come up with most key examples of a feature that we want to build. However, by talking about it with others and discussing about it we can find the edge-cases. Because of this, defining tests should also be a group effort. In an agile context for example, the product owner together with the tester could come up with the basic definition and then enhance the definitions with the rest of the team during a refinement session. This multidisciplinary approach also increases the robustness because a multitude of perspectives are involved.
Measure test change
To see if the your tests are written with the right level of abstraction, you can measure the rate of changes within your test-suite. If your tests are version controlled together with your code, this can be easily done. If your product is getting towards a stable release, the amount of test changes should also decrease. If this is not the case, it would be good to take another good look at the abstraction level of your tests.
Test coverage thresholds
In projects that enforce thresholds on test coverage, it is common that a false sense of quality creeps in. Setting thresholds means that teams will try to focus on getting above the threshold instead of thinking about the actual benefit of the test. This might lead to tests that do not have any benefit and do increase the maintenance of the test-suite. My advice would be to think about the cost versus benefit. Try to do a risk analysis and see what risk you as a team and the business think is feasible.
Feel responsible for testing
In companies with dedicated teams, testing and defining test-cases is often delegated to specific developers or a different group (testers). In my view this is a shared responsibility that should also be done by developers. Discussing the various test-cases and writing the tests makes us more aware of what is important within the system and helps us to explore the edge-cases better. Next to that, it will lead to a common testing toolset (everyone uses the same tools). This way, a tester’s role is not only to test the system but also to improve the testing awareness of developers and come up with better ways to test. This means that their job will also include coaching and guidance.
Conciseness, coherence, completeness
Tests are useful in multiple ways. They could be used as acceptance criteria for a feature, as a specification of the system or as documentation for later reference. These scenarios require a different level of detail. To be useful for these cases try to keep the following in mind:
- Always add an introductory text to the tests
- Start off with simple cases then write down the more complex examples that test the boundaries of the feature
- Group related examples together
- Highlight key examples of a feature
- Organize more comprehensive tests and technical tests, which are related to the feature at hand, separately to keep a clean overview1
1 Fifty Quick Ideas To Improve Your Tests, p. 62 Balance Three Competing Forces
Tests writing tips
Teams approach testing in a very different way. Some teams write no tests, some a few and some too many. If you decided that you want to write tests, a good rule of thumb is to start with the key examples. After that see if you need to extend the test-suite by changing variables and exploring boundaries.
In the ‘to improve’ part of the example above, it is not clear what is tested and what influences the result. The title says: withdrawal from bank account but what variables make this a valid withdrawal (amount, origin, currency, private)? Try to focus on group variation (amount, origin, currency) and split these into different tests. This makes it easier to read and invites to explore edge cases.
Arrange, Act, Assert (AAA)
From TDD (Test Driven Development) we know that we should structure our tests in the following way:
These terms improve the readability of a test and separates the: parameters, action and result check, which makes the test easier to read. These terms are reflected in different frameworks in a different way. In the example below use the ‘Given’, ‘When’, ‘Then’ terminology.
The ‘to improve’ case has multiple Given’s and When’s which makes it hard to see what is actually being tested. What parameters are relevant (name, account number) and what action is tested (is selecting the account details needed)? In the ‘good’ example we can see that it is actually the name of the account which makes the payment to fail (not taken into account the account number). Try to structure your code as follows:
- Set up your parameters (arange)
- Do the action (act)
- Check if the action yielded the appropriate result (assert).
Describe what, not how
Tests that describe ‘how’ something is done are hard to write, subject to a lot of change, often time-related and do not test what you want to capture.
If we were to change the login button identifier or the layout of the page, we would have to rewrite the ‘to improve’ set. If we have a network delay and the button would not have rendered on time, the example will have also failed. The ‘good’ set will continue to work and does not have any of these constraints.
We as developers try to be as concrete as possible when writing our software. This sometimes reflects back in our tests. However, this might be convenient, it does not provide more information to someone looking at the test than he/she would when looking at the code. Defining tests is also about exploring boundaries. Using mathematical formulas abstract away these boundaries and let people think less about the boundaries.
If I were to write down the ‘to improve’ case, during a discussion with a stakeholder we could agree that this is a valid test case. However, it could be February which has less days than 30 will lead to a different validity status. Writing this down explicit makes a deeper discussion about validity constraints more probable.
Want to know more?
I hope that these tips might help you with your testing endeavours. Not all topics that were mentioned above might apply to your project, because testing approaches, tools and context are very different between projects. If you want to know more about this topic I would recommend to read the book: “Fifty Quick Ideas To Write Tests” by Gojko Adzic, David Evans and Tom Roden. It’s a very interesting read (disclaimer: I am not affiliated with neither this book nor the authors in any possible way).