How to test your tests

One of the benefits of writing your tests first is that you will always see the test fail. This is important, because you can’t trust a test you haven’t seen fail. Think about a line of code like this: assert a = 3

Of course, you meant to write a == 3, but you may not realize that if it’s in a test that you wrote to verify already-working code. It would pass, and you’d assume it passed because the code it’s testing really did set a to 3. But if you wrote and ran the test first (or commented out the working code to see a test failure), you’d notice that the test was passing when it shouldn’t, and fix the bug.

Watching a test fail is one way to test your tests.

But don’t just see a red/failing test and run off to make it pass. Pay attention to the failure message. You could have a different bug in your test that’s causing the wrong failure. Maybe due to a syntax or logic error. So if you don’t get the failure you expect, that’s another sign that your test may have a bug.

Now when you see a failure you expect, only write just enough code to fix that specific failure. Is your “makes a equal to 3″ test failing because the module is missing? Don’t implement the entire module, just create it. Then watch it fail because the function is missing from the module. Now don’t implement the entire function, just declare it. And so on. Keep fixing only the immediate failure until you hit green. Does the implementation feel complete? If not, you need another test.

It may feel silly at first, but if you train yourself to always take these micro steps, you can be sure that every line of your code is actually being tested. If you take large steps, the chances increase that untested – or even superfluous – code sneaks into your system. Fixing only the current failure tests your tests for completeness.

So while you’re in the “red” step of “red, green, refactor”, remember to keep an eye out that you’re red for the right reason, and don’t try to jump straight to green, just fix whatever is making you red right now. Eventually you’ll get there, and you’ll feel super confident in your code.

Leave a Reply (markdown is supported)