Happy Friday!
One thing I've been trying to do more of is read research papers on topics that I am passionate about, have data on, or deal with regularly at my clients. A specific topic I like to focus on is testing.
I remember early in my development career treating testing the way most developers do, as a painful chore that I would try to minimize. Eventually, a switch kind of flipped for me, and I got tired of cleaning up bugs and stupid code. That led to me abandoning conventional development wisdom and doing the things that everyone said were wrong, like testing more and doing TDD.
As it turns out, that was one of the better decisions in my development career. My career was fast-tracked, I stopped getting support calls, and I started shipping bug-free code faster than my peers.
Still, over a decade later of replicating these results across companies and teams my findings are considered controversial.
Here's a 2015 study that focused on examining what developers actually do in their IDEs in relation to testing. It matches what I've observed, but might surprise a number of you. The really short answer is that developers overemphasize the burden of testing while doing very little of it.
Now, let me break down some specifics. The study went for 2.5 years and had over 2000 developers participate by allowing data collection in their IDE. The data collected focused on the time spent doing specific activities like running tests, modifying tests, modifying production code, etc. Here is what they found:
- 43% of projects had tests at all, and were used in only 8% of the coding sessions.
- Of the projects with tests, the tests were never run or touched 57% of the time
- 12% of the tests were considered non-deterministic (Flaky)
- It took under 15 minutes to fix a broken test 75% of the time
- 1.7% of developers follow TDD
- 1 developer did TDD more than 20% of the time
- Developers spent 1/4 of their time working on tests, but often estimate it's half their time
Now, I'd understand if one of your first reactions is to say that a 10-year-old study isn't relevant, but I want to temper that reaction. The overwhelming sentiment among developers that I've worked with is that testing sucks and should be minimized and that preference appears in the high defect rates, broken and brittle tests, padding coverage numbers, and faulty "spot-checking".
Sincerely,
Ryan
PS: Here is a quick assessment—look at the ratio of bugs/defects created to work items completed. This is your defect rate or change failure rate. Want to talk about practical ways to lower that number? Give me a call.
0 comments