« Project Manager or Project Coper: Errata | Main | Stupidity +/- 100% »

December 25, 2004

Is your build broken, Broken or BROKEN?

As with most Agile development teams, we use a token to villify the most recent team member to have caused our CI build to fail. In our case, it's a rather Academy Award-looking figurine of one of the Teenage Mutant Ninja Turtles (not sure which one), and the ceremonial handing over of the token upon a new build breakage is something we take rather more pleasure in than we probably should.

Our record on clean builds tends to vary quite significantly; we'll occasionally run green for most of an iteration, but we also have extended periods of almost perpetually broken builds, which causes no end of constenation to our long suffering build manager.

Recently, there have been a few debates over the semantics of a "broken build". Quasi-hypothetical scenarios along the lines of "what if I break the build, realise it and then correct the problems before Cruise Control starts another cycle... does that count?" are put forth to determine whether the token is able to be passed to a new owner or not.

I tend to stay somewhat tangential to these discussions because, to my mind, the underlying issues with a broken build are three-fold:

  1. In the case of a failure in some part of the compile/deploy cycle, the broken build can immediately and completely stop a developer from continuing to work.

  2. In the case of regression as seen through test failures, the broken build can prevent the application being released at the end of an iteration.

  3. In the case of a stylistic failure as reported by Checkstyle, Clover or the like, the broken build can be an indication of a deeper issue around some area of the codebase.

Obviously, the first of these cases is the most immediate and costly. The worst case in this scenario is a broken compile caused by (for example) a new file that was missed in a commit, coupled with the developer leaving the office for some period (e.g., heading home early, holidays, etc).

The effects of the second case can range from severe to mildly annoying, depending on the proximity of the iteration close and/or the amount of overlap in the areas of the codebase covered by different developers. Occasionally, a test failure initiated by one developer can hide a problem in another story, if that story is in the same neighbourhood of the codebase.

To my mind, the third case (i.e., stylistic problems), although it can indicate deeper problems in the system's design, is by far the least worriesome of the three categories and therefore should be considered "the lesser of three evils" in this company. I know of some people who move Checkstyle and similar such checks to the end of their CI build cycle, thereby preventing such problems from overriding the more pressing issues of syntactic and semantic correctness, as gauged via the build process itself and the test suite respectively. Although such a move does violate the "fail fast" doctrine, it does appeal to me from a triage perspective.

Posted by Andy Marks at December 25, 2004 09:41 AM

Comments

Post a comment

Thanks for signing in, . Now you can comment. (sign out)

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)


Remember me?