Happy Friday,

Today's email is going to get a bit technical. You see, one of the things I wind up teaching teams as they go from good to great is how to test their code better. It turns out that most developers have spent a fraction of energy learning the skills of automated testing compared to writing "Good" code.

It turns out their "Good" code is buggy and hard to maintain more often than not. So I teach them how to test it, which forces them to unlearn a lot of habits around writing code, but also teaches them about writing good tests. I want to get into an element of what I consider to be good tests, but it is admittedly hard for me to explain.

Let me use a scenario that should be pretty easy to conceptualize—adding a new API response.

Avoid Large Assertions

One approach to testing this would be to write a test with an assertion that says the response object matches the expected object. This would be instinctive for folks newer to testing, but it's not quite what I'd recommend.

While this test is easy to write, it's typically not quite clear when it fails. A perspective to maintain for folks is that when you write tests, you want the failures to be instructive and informative. So, ambiguous names or assertions leave someone looking at a failure, unsure of what it means, how it happened, or what to do next other than start debugging.

Want to know what most people do when confronted with that? They just make the test pass or delete it.

When the large assertion fails, it is all or nothing. You don't have much context about what is really being tested or if the failure is a regression or expected. While you might see the difference between what was expected or not in a literal sense, you don't have a lot of information to tell you what step you should take, and that's what we can fix.

Favor Small Descriptive Assertions

Okay, so the alternative is to take one large assertion and ask yourself what you'd like to know when the test fails, and then break it down.

So, for the example of a new API response, we can take one large assertion and say that we actually care about two different things. One is that the basic format of the response is what we expect, and the other is that the actual data is calculated correctly.

The first test for the format or contract being valid will fail when someone changes the shape of the response. This one test informs you that a contract has changed, and it is way easier to assess what the next step is. If you didn't intend to alter the contract, you have a regression.

Now, for data correctness, each field could get its own set of tests. Consider one field that is based on a calculation. In the large test, it could fail because the calculation changed, the contract changed, or anything, and it would be just one failure. If you have a test for the one bit of data being calculated, then when it fails, you know exactly what is going on. Your test will tell you about the expected behavior around the calculation, which will also help inform you if you have a regression.

But...

The most common objections I get to this approach is how many more tests it is to write, and the concern that it will lead to more failures to maintain over time.

It turns out that the cost to write these tests is pretty negligible. Testing is one of those odd areas where copying and pasting is useful and encouraged. This makes writing these tests take only minutes.

The other concern around dealing with multiple failures over time is also not what people think it is. Good tests fail for expected reasons. When the reason is clear and obvious for failure, addressing the failure is quick and obvious. When you have a large single assertion, you have to investigate, debug, and then very likely guess what the right action to take is. That is slower and more error-prone than a small obvious test failing for a clear reason.

Because most folks write the large test, they are used to those test taking a lot of time, and being hard to debug when it fails. Their concerns, ironically, say a lot more about the need to move away from the instinct to have these large assertions than any fault with the smaller ones I advocate for.

If you're technically inclined, I put together a GitHub repo as an example so you can make your own judgement call.

Here's my weekly update for October 4th, 2024...

🗒️ What Makes Good Teams Great

Making great teams takes a lot, but in this article I want to dive into three ingredients.

A great team knows they have an impact on decisions, a strong sense of team, and they are relentless in pursuing mastery.

Click here to read more

Enjoy,

Ryan Latta