Archive for November, 2012

Why it doesn’t pay to write unit tests

21 November 2012 5 comments

I’ve unfortunately had to do a lot of traveling in the last couple days and I’ve been reading No one makes you shop at Wal-Mart on the plane. Among other things, the book describes the economic model underlying  the idea of a ‘public good’.

A public good, as opposed to a private good, is basically a resource that is enjoyed freely by a group. A clean environment or a quiet neighborhood is a good example of this.

A private good, in this example, might be the right to play your music as hard as you want to. However, if everyone starts doing that, the public good of a quiet neighborhood will soon disappear. In this scenario, everyone in the neighborhood has to pay the cost of setting a limit on how hard and when you can play your music in order to preserve the public good of a quiet neighborhood. This idea is related to the Prisoner’s Dilemma, for those of you curious about that.

With regards to software development, a set of well-running unit tests is a public good while the act of writing a  unit test is actually a private cost. I think this is self-explanatory to most of the developers reading this,  but I’ll explain it just to be sure.

Writing a unit test that is of good quality is not advantageous to a developer writing or modifying code for the following reasons:

The productivity of a developer is measured based on 2 things:

  1. The number and quality of features the she produces
  2. The number of bugs she fixes

The first measurement, the number of features produced, is weighted more heavily: that is to say that creating a feature is, in general, seen as more productive than fixing a bug.

However, writing a good unit test does not directly contribute to either the creation of features or bug fixes.

While writing a unit test might be helpful when creating or modifying a feature, it’s not necessary. Every decent developer out there is perfectly capable of writing software without having to write a unit test. In fact, writing a unit test costs time which a developer might otherwise have spent on writing more features or fixing bugs! The better the unit test is, the more time that a developer will have needed to spend on it, making good unit tests more “expensive” than lower quality unit tests.

Thus, from an individual developer’s perspective, it does not pay to write good unit tests, especially in the short term.

Furthermore, the unfortunate thing about “quality”, is that the quality of a feature (or any piece of code) is something that can only be measured in relation to how long the code has existed. In other words, the quality of code is never immediately apparent and frequently only apparent after a significant period of time and use. Often, by the time the (lesser) quality of the code becomes apparent, no one can remember or determine exacty who created the feature and it’s not productive to search for that information.

But it does always pays to have a high quality suite of unit tests. A well-written suite of unit tests does 3 things:

  • Most importantly, it will alert the developer to any problems caused by new or changed code.
  • Because an existing unit test will obviously use the existing API, it will alert the developer to problems with backwards compatibility if the developer changes the existing API.
  • Lastly, unit tests are functional examples of code use: they document how existing code should and can be used.

All of these benefits help a developer to write better quality features (in less time) and help not only with fixing bugs, but also with preventing bugs!

But, in a situation in which there are no external pressures in how a developer writes his or her code, there are no immediate reasons for a developer to write unit tests. This is especially true in situations in which the developer will only be working on a project for a relatively short period of time.

Of course, some developers might feel that writing tests helped them develop features more quickly — or that it might help them fix bugs more quickly. However, if at a certain point they have to justify their use of time to a superior (the project lead, project manager, etc.) and they explain that they were writing unit tests instead of writing new features or fixing bugs, they will get in trouble, especially if there’s less value placed on unit tests or refactoring.

At a company, obviously, this is where a project manager, project lead or even a CTO comes in. While it may be in the interest of the developer to create new features and fix bugs as quickly as possible, it’s probably equally important to the CTO and other managers that the quality of the software created meets certain standards. Otherwise, users of the software might become so disgruntled with the software that they’ll complain, leading to a negative reputation of the software, which may lead to the company going bankrupt!

It’s in the interests of the CTO and other managers to require software developers to write unit tests that are of a certain level of quality: namely, the unit tests should be good enough to assure that the software retains a positive reputation among it’s customer base. This is often a difficult limit to quantify but luckily often easier to qualify, I think.

Open source software is, however, a different story. There is no CTO and the highest authority in an open source project is often the lead of the project.

The question, then, is what determines the quality of a suite of unit tests in an open source project? To a large degree, the answer is obviously the attitude of the lead of the project. Attitude is often very hard to measure, unfortunately: it’s easy enough to say one thing and do another. If you ask the lead what she thinks, it’s hard to say if the answer she gives you represents the attitude that she communicates to the rest of her project.

Realistically, one of the most decisive factors determining the quality of the features of an open source project is simply the example set by the lead in the code and tests that he or she writes.

Categories: Other

Openshift at Devoxx 2012

14 November 2012 1 comment

Good morning! If you’re at Devoxx today, feel free to come by and visit me at 14.00: I’m giving a talk on OpenShift. I’ve got to say this: I’ve rarely had so much fun preparing a talk.

The thing is, OpenShift just works. Sure, now and then you have to figure a few things out, but given all of the work it does for you — or rather, given all of the work you no longer need to do, it.. it rocks!

The talk is called “Openshift: State of the Union” and it’s a quick primer followed by a couple of demos. Now, I just need to hope they don’t randomly decide to do maintenance during my talk. ;D

I’ve just uploaded my slides for those curious: you can find them on slideshare.

See you at the talk!

Categories: Other