I was watching a recorded webinar on QTest, the unit testing framework that comes with Qt, and the first question the presenter asked was who runs their unit tests before committing their code changes. Apparently only one hand went up and the presenter snickered saying that it was hard to believe that that one person really does run the unit tests before each commit. That floored me, I personally can’t fathom not running my unit tests not only before I commit but each time I compile my code while I’m developing it; That is the point of unit tests after all. You run your unit tests often in order to detect changes to your codes behaviour while your modifying your code. The sooner you find out you unexpectedly changed your codes behaviour the easier it will be to determine which modification caused the behaviour to change.

Unit tests are your safety net that allow you to not fear change.

At first I thought this was a symptom of a complacent environment after all the presenter was using QtCreator on Linux which simply doesn’t seem to have the tools one comes to expect from say VisualStudio and Windows (understandably so). That is VisualStudio comes with a built in test runner that automatically run all your unit tests and displays the results graphically each and every time you compile. However blaming the tools for not being as polished as you might like is just a cop-out; it might not be as nice as VisualStudio but there is no reason why your QtCreator project can not run your unit tests as a post build step thus ensuring your unit tests are always run each time you compile your code and thus are run every time before you commit. This way you get constant fast feed back while you work.

In the end I think that what I saw in the webinar was the result of not understanding what the heck unit tests are and why they are important.

What the heck is a Unit Test

A Unit Test is a tool for software developers to detect changes to the behaviour of the code.

They do not test for correctness they test for change.

What do I mean by my codes behaviour? I mean without looking at the code what do I expect my code to do. If I give a method input A I expect an output of B. I don’t particularity care how it generates B just that it does. That is its behaviour and I can test for that.

If I have a unit test in place that alerts me if my method stops outputting B when I give an input of A I am free to change the how with confidence that my changes wont negatively effect consumers of my method because the behaviour is proved not to have changed. This most certainly does not mean that I can’t change the behaviour of the code; however, it forces me to acknowledge that I’m conscientiously changing the behaviour. Unit Tests make me conscientiously accept changes to code behaviour by forcing me to update my unit tests. This might sound to some like a knock against unit tests being that when I do want to change my codes behaviour I have extra work to do but its really another glowing example of why unit tests are important. That extra work (i.e. updating your unit tests to match your codes new behaviour) is the difference between making a design decision to change the codes behaviour and introducing a bug by unintentionally changing the codes behaviour. By having tests that check for change I can be more confident that my code is doing what I meant it to do.

If Unit Tests are suppose to detect change why are they called Unit Tests vs. Change or Behaviour Tests?

Ah well that is because Unit Tests are designed to achieve two things:

1. Detect Change
2. Locate Change

Not only should your unit tests tell you something changed they should tell you where that change is located. That is the big difference between an Unit Test and an Integration Test; The larger the area under test is the harder it is to pin point what is causing the unexpected change. The unit in Unit Test is stating that the test should be of a single isolated unit of code.

Above when I was saying that I have a method that takes in A and outputs B, that method might be part of a class. The unit test I write to detect any changes to that method should only exercise the logic within that method and care very little about any of the logic in the rest of the parent class. In fact care needs to be given to ensure that the method under test does not depend on any other methods in the class and if so they need to be snubbed out so as to make sure that any bugs in the depending methods don’t skew the results of the test of our method in question.

So why is it so important that I run my unit tests often verses letting my build server do it?

Much like a smoke detector with a dead battery; a unit test you don’t run isn’t helping anyone.

Think about how you develop today, if while your developing your code if you notice a bug in what your currently working on its pretty easy to fix right then and there however if 6 months later when QA comes to you with the bug its a little more difficult to figure out whats causing it since its been a while since you looked at that code. If you wait until your done all your modifications to run your unit tests your just going to cause loop backs that are more costly then if you were checking your work as you went.

Remember Unit Tests don’t check for correctness they check for change. If you fail a unit test we’re not saying you did a bad thing or that your a bad programmer; We’re saying you changed the expected behaviour and are asking if you meant to do that. The answer could be yes so you just have to update the test to match the new expectation, no harm no foul. Its all about the conversation and being aware of change.

Yes this means that you should be writing your tests as you go if not before you start developing your code. But if your developing with testing in mind those nasty dependencies that you have to figure out how to break won’t need breaking because you worked them into your design.

But if I’m running unit tests locally won’t this mean I’ll be spending more time waiting?

Yes but only a few extra milliseconds

Seriously, that’s because of the other important definition of a unit test; they are fast.

Its not uncommon to have up to hundreds of unit tests if not thousands depending on the size of your application. If each test took even a second to run you’ll be waiting for hours for your test suite to complete (5000 tests at 1 second a test equals more then an hour to run) and then we’ll be looking at those costly loop backs again. Its because of the requirement of providing fast feed back to developers that unit tests need to be fast and when I say fast I mean that a single test should take no longer then 10 milliseconds at the most; Ideally a single test should only take a single millisecond to complete. Extrapolate that out to the thousands of unit test scenario your now only waiting a few seconds for the test suite to complete (5000 tests at 1ms a test equals 5 seconds to run).

So since they only take a second or two to run you shouldn’t even notice them unless you broke one in which case you either accept the change and update the test or you update your still in progress code to match the expected behaviour.

Detecting change so you don’t have to fear it

In summary what the heck is a unit test?

1. A unit test is a tool for detecting and localizing change
2. A unit test is fast < 10ms
3. A unit test is small
4. A unit test is used so we don’t have to fear change

I hope this helps to bring some better understanding as to what a unit test is and the motivations behind them.

Until next time think imaginatively and design creatively