The Cult Of Unit Testing

by zen master

There are many things that sound great in principle, and sometimes a few of those things can even work out in practice. These rare gems should be shared with others right? People like to spread the good word. In time though the idea gets jumbled with other things. In its new corrupted form things stop working “in practice”. However the old stories live on, an affirmation that it is still a great idea. A cult is born and in the presence of this cult even the thought of questioning it will unleash wrath from the gates of conformity.

Programming in one sense is the art of controlling complexity, the human brain are limited. It is very easy to take a simple task and add more complexity to it. If we are not careful we can get killed by the blast of complexity explosion. Once the complexity exceeds our capacity to comprehend it, that is when system instability are inevitable.

Debugging a problem is an attempt to understanding state behavior  Fixing a bug are catering for a system state or edge case that was not envisioned before.  It is thus not surprising the less you know about a system, the less you can predict its various states and behaviors.

Unit testing like riding with the training wheels on

Unit testing like riding with the training wheels on

.

In an effort to stop the rise of system instability, unit testing has been seen as a way to fight it. An elegant solution in principle: write code that verifies the behavior of a “unit” of code. Once a set of “unit tests” are written the process of code verification is thus automated. Code changes can be made with total confidence. Its like riding a bike with the training wheels on, you are never going to fall over.

There are a long list of arguments against unit testing, with those arguments there are counter arguments. However none of those arguments deal with the deep rooted failure of the principle itself.

What is that failure? The failure is the base assumption that the unit test can encompass ahead of time the full set of system states and edge cases. This is contradictory because the reason bugs occur are because we are not able to envision ahead of time all cases. How can we then begin to write tests for things we don’t even know exist?

Does this mean unit tests are useless? No unit tests can be used in very special cases I believe. An example of a special case would be an algorithm, in this case it makes sense because you can know what to expect and thus test against it. 

OK if unit tests can not fight the rise of system instability what then are the alternative solutions?

In my humble view I would suggest that we go back to the root, the failure to predict state and behavior  This will never be solved (bugs will thus always occur) however the more we can understand a system the rarer those cases become.

Instead of endless unit tests, start by reducing system complexity, refactor mercilessly even rebuild from the ground up when needed. This is not a perfect solution (there are no silver bullets) think more of “water resistant” than “water-proof”.

Conclusion

Unit tests are pushed in an almost cultist way. Among the cult those that do not use unit tests are seen as knuckle dragging neanderthals or unenlightened. If you want increased stability you will need to first control complexity, to do that you need to reduce it. Unit test are thus band aids trying to work with complexity instead of reducing it.

Beauty is more important in computing than anywhere else in technology because software is so complicated. Beauty is the ultimate defense against complexity. (David Gelernter)