Enough with the crappy tests already

May 19, 2023 4:31 pm

Happy Friday,

I was on a call with a client recently, and they were showing the tests they wrote for some user stories.

I often ask to look at tests with teams new to writing automated tests consistently.

Oddly, that seems to be everyone.

Months ago, this client might write one test that would be the simplest happy-path test they could come up with. Often that test would involve mocking that would simply say they called their function instead of confirming they got the desired result.

Fast-forward to now, the team often has many tests for everything, but we've gone too far in the other direction.

We have tests just because we can have tests.

What's an example? How about a test that exists to prove a log entry happened? How about a test that a setter or constructor works?

Neither extreme is good, but it is something that seems fairly universal. Most developers never really develop their skills when it comes to writing tests, and as a result, many test suites are somewhere between laughably bad and dangerously bad.

This newsletter isn't going to be one that tries to solve this whole problem because there's a lot that goes into writing useful tests.

Don't worry, though; I am going to leave you with two very useful things to help move things forward:

  • Run your tests more often and in a random order
  • Make refactoring tests part of code reviews

Running your tests more often and in a random order does a world of good for folks who treat tests as a chore. Running them more often moves tests from being a distant concern to something you live with. Randomizing the order helps teach teams the importance of test isolation. Let me assure you, test isolation is one of the biggest things that cripple engineering groups. Test isolation gives you confidence that the result you get is trustworthy. Without it, you never know if the tests will pass or fail, which means they're not useful.

As for refactoring tests, this is something that should be obvious, but for most groups, tests are not considered part of the code and are similarly ignored as a part of reviews. Or put it another way. Most groups don't care about the quality of their tests.

So, by saying that test cleanup and refactoring are part of code reviews, it puts the team in the position to look at tests and ask, "How can I make this better?" Sadly, most will not have much of an answer to this question, but this journey of improvement starts with them at least asking.

I'll give you a hint, though, refactoring tests looks quite a bit different than refactoring normal code, so be patient. A good thing to look for is duplication across suites.

Bad tests slow teams down and cause issues. Good tests accelerate teams and allow for rapid change. So, enough with the bad tests already!



PS: I've been putting my money where my mouth is and doing TDD to build my reko.day app.