« Risk if you do, risk if you don't | Main | The Curse of the Early Adopters »

April 30, 2005

Joel versus the Agilisti

Joel On Software is one of the several blogs I try and stay on top of - mostly because I really relate to his pragmatic approach to software development and the issues on it's periphery. So it was with some interest that I read The Joel Test, a 12-step process for getting a handle on how good a software team is. Now, apart the obvious bias towards developing a product which is interleaved throughout the 12 generally sensible questions, I was struck by how the Agile community might rate on his Test. On behalf of the teams I've worked with over the last couple of years, here's how we rank:

  1. Do you use source control? Hell yeah!
  2. Can you make a build in one step? Ditto
  3. Do you make daily builds? Honestly Joel - only every day? I'm surprised! But the test was devised in 2000 and his reasoning is solid, so I'll forgive the slackness :-)
  4. Do you have a bug database? Yep... not FogBugz, but not everyone's perfect
  5. Do you fix bugs before writing new code? Depends - is that what the customer wants? Or do they want more functionality and are willing to leave the bugfixes for now? I guess I only get a half mark for that one :-(
  6. Do you have an up-to-date schedule? If I'm doing Adaptive Planning, then my schedule is up-to-date as of my last iteration - guess that's a yes
  7. Do you have a spec? Nope, or at least not in the glossy, ring bound phone book sense that's being aluded to here. Damm - crashed and burned on that one.
  8. Do programmers have quiet working conditions? Bzzzzt again - now I'm in a rut!
  9. Do you use the best tools money can buy? Never have, never expect to - ahhh, three in a row!!!
  10. Do you have testers? Yep - generally supplied by the kind client, but yep. Whew - back in the black again.
  11. Do new candidates write code during their interview? Shoots... and scores! Yes again.
  12. Do you do hallway usability testing? I think I'll interpolate here and give myself a full mark here. Customer sign-off during/after the iteration is driving at the same point

So I came out with an overall score of 8.5/12, which plants me firmly in "serious problems" territory! Am I particularly fussed by this??? Not in the least. Why? Because I think Joel's got his wires crossed in a couple of respects towards the focus of this test and what the most important contributer to a successful project team actually is.

Firstly, I think the questions he poses are all excellent ways of determining the quality of a software development environment, however this is a necessary but not sufficient measure of the quality of the team(s) working in this environment. Put simply, a partially enlightened software manager could automate the build, add version control, unhook the phones etc (i.e., address all the points hinted at in the test) and then put the Larry, Curly and Moe of the development world into this shiny, quiet environment with brand spanking new machines... and no good would come of it at all!

Until a valid means of quantifying the quality of the developers themselves is found, everything else is merely diverting noise. I'll take a bunch of high quality developers over a high quality environment any day of the week and twice on Sunday. Good teams are the closest thing to a first-order determinant of a project success that I can think of, and they will naturally create the sort of environment Joel treasures in order to do their work effectively, but I wouldn't want to induce the presence of such a team by the existence of the environment itself.

Unfortunately, effectively measuring developer quality is a slippery slope and the metrics thrown at the problem in the past (line of code per day, defects per line of code, etc) are well wide of the mark. And given the anecdotal evidence of the best developers being an order of magnitude better than average developers (can someone please point me to the original reference of this fact - is it Yourdon?), I suspect those developers on the left side of the bell curve and the majority of employers are silently happy and grateful for this fact: imagine attempting to justify a 50% salary band across a development team when some can quantitatively demonstrate they are almost 10 times more productive than others! Come to think of it, I don't think the gun developers would be overly happy in this knowledge either, irrespective of how internally driven they may be - perhaps ignorance is truly bliss in this case?

But back to The Test! Even if a more Agile-friendly version of this were to be devised, I still wouldn't be particularly ready to automatically class every 12/12 team as "high quality". Joel finishes the article with four perspectives from which the test can be administered: of these, I would only consider the last two (from a potential employee or investor) to be valid uses of the test. The final phrase of the article is probably the best summation I can provide "...this test can provide a quick rule of thumb" - nothing more.

Posted by Andy Marks at April 30, 2005 01:32 PM

Comments

I think the workarea can count as quiet if the only people making the noise are the programmers in it. :)

In my experience, the only way I'll ever have a quiet workspace is if I'm the only one there...

Posted by: Robert Watkins at May 1, 2005 07:45 PM

Post a comment

Thanks for signing in, . Now you can comment. (sign out)

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)


Remember me?