Some time ago, I saw this statement passing by on Twitter.
My first thought was, why? Why are tests useless when they don’t fail. I disagree with that. Test is about gathering qualitative information. Test is about questioning the product to observe it, to learn from it. Test is about making informed decisions.
If a test does not fail, it tells us that the result meets our expectation, based on the requirements we have. With this input, we expected this output. Check. That is valuable information. When all our tests meet our expectations, we can say that we have confidence in the feature we have developed. It is ready to be shipped.
You must know that the most important question a CEO of a company always ask is, “Can it be shipped?”. Yes, it can based on the information we have collected.
When a test doesn’t fail, it also has value. That is why we automate tests. A test can be run over and over again and every time it delivers the expected output (We hope). We are happy, because it does not fail. So, if all my automated tests never fail, does it means that all my automated tests are useless. No, they are not. Tests have value when they don’t fail, automated or not.
When tests are failing they also have value, but of a different context. With this information, we start different activities to deal with the failing test.
To conclude, whether a test fails or not, it is always useful information.