Who is testing your unit tests?

A week ago there was a discussion on one of the Dev channels I follow on Facebook about Test-Driven Development (TDD). I am a bit afraid of these as they frequently become “Someone is wrong on the Internet”. This time I enjoyed it, as it was a great opportunity to put into words how I feel about the practice and thus understand it better.

One of the topics that came up was this – “TDD is flawed by default, as nobody is testing your unit tests”. I was certain this is just plain wrong, but it made me think. It is an argument I have heard before, so perhaps it was time to explore on this topic and find a good answer. Do I care about that? Who is testing your unit tests? What does TDD has to offer in that regard?

Why should we care?

Should we actually care about unit tests? Lets try to make an analogy to explain this better.

How about a clock? We all have some – on our hand, in our home, in our laptops, mobile phone and other devices. To have a clock is usually easy – you need to change or charge the battery from time and thats it. Sometimes you also need to verify that clock time is indeed the correct one. There is automatic time sync, but not for all clocks. If only there was an automatic service to check this for me and tell me – your microwave time is wrong, fix it!

This exactly what test automation will do with your code.

So your clock time is wrong. Where do you find a clock that you know its correct? Depending on how precise you want to be, it gets progressively harder. Your watch has stopped, as you forgot to wind it up. You can ask a friend what is the time or use any of the many clocks around you. They are not precise, but are also infinitely better than your stopped watch.

When your code is broken, even a few imperfect tests are better than nothing.

Nowadays many clocks you can buy come with some kind of auto sync feature, so you do not have to do anything. They are not that simple and more expensive. But life is easier with them, as they will always show good time. Maybe you will still have to verify the time, but this will be rare and in case of events like going to another timezone. The clock in your mobile phone and computer is also syncing like this. Easy!

This is what working with test automation feels like. Except you have to set it up yourself…

Suppose that you want to be very precise. Perhaps you are one of the few people on this planet that actually needs this in your work. Or maybe you are just that kind of person? Your watch needs to have perfect time, period. So you verify it against the absolutely most precise watch known to men – the Atomic Clock. Or, even better – you build your own Atomic clock! Doing this requires significant investment of resources, but hey, your clock is now 100% precise! But is your time really correct, at last?

Who is verifying your atomic clock time?

Nobody.

And its still not perfect. We just do not have better option yet.

It is the same with your code.

This is software development – there is always “a better way”. Always something more you can be do – different types and levels of test automation, code reviews, code analyzers, manual testing, etc… Yet your code will never reach 100% correctness. Unless you code and test only Hello Worlds and simple prototypes, of course.

And this is totally fine. Most projects do not need 100% correctness anyway.

Try to be pragmatic about it. Nobody cares about perfect timing – a few seconds or a minute difference is OK. 5 picoseconds? Who cares about that? 15 minutes? You better fix your clock timing right now…

Unit tests will help you a lot in your projects and will definitely raise the quality bar of your software. They are recommended for any project, except simple and experimental ones. So if your website has a simple registration form and a few pages, don’t bother.

But what about bugs in tests?

Yes, unit tests have bugs, like every other piece of code. Having tests will reduce the total amount of bugs in your project. So what if some unit tests have bugs? You have less bugs in your project, so its a win.

How is this possible? First – test are simple. Ideally they are few lines long. Simple code -> less errors -> less bugs. Of course there are cases when tests are and will be more complex than that. But it is highly unlikely for them to be more complex then production code – unless something very wrong is happening with your code of course… And if your test is alright it will help you make the implementation alright.

But what if your test is indeed wrong?

This is an issue, as tests can lead you do some false sense of security. “I wrote a test for that, its working!”

Here are the most probable reasons for wrong tests:
* You did not understand the problem, so wrote a bad test in the first place
* Your implementation has a bug and your test is written to verify it

Writing tests or not – if you do not understand the problem, bug is inevitable, so nothing that can be done to prevent it once that happens. But what about the second case? This is one of the reasons I personally dislike writing tests after the “production” code. Temptation to write a test that will just assert whatever the method I test returns is just too big sometimes, especially when deadline is approaching.

To sum it up:

• You get less bugs as tests are simpler than your production code
• Test should test your understanding of the problem, not your implementation

How TDD helps?

TDD is not required, but helps. Even before you write any code it will make you think about how your code will be used, instead of what your code is supposed to do. It is also harder to get your implementation wrong when you follow the Red-Green-Refactor cycle. When you start with a failing test (Red) you are verifying it will actually fail on error, so you know it works. Thus writing test first is a process that helps you make less errors.

Is testing your unit tests even possible?

Of course it is! Cool tools and techniques like Mutation testing exists, if you need such a thing. This is an advanced territory, so not for everybody and definitely not a topic for this post 🙂

So, who is testing your unit tests?

You. Your “production” code. Nobody 🙂

In a way all the above answers are correct.

It’s you, when you try to understand the problem you are trying to solve and apply this knowledge in code.

It’s your “production” code, when you run your test. One of test and “production” code can be buggy, but chances for both being wrong and test passing are small.

And finally it’s nobody, because there is no need to test more.

Thank you for reading.