« And the winner is... | Main | A Personal Retrospective »

October 27, 2005

Conveying intent in tests

Although I cannot remember who it was who first made this comment about my general approach to unit testing, but I'm indebted to them. The comment was something along the lines of "Andy, you have good test coverage, but what are the tests actually doing?". In short, what is the intent of the tests. From that time forward, I've paid a great deal of attention to how I name my tests better in order to convey the raison d'etre for the test: no more do my tests take the form of testValidateAddress1, testValidateAddress2 when they can be testValidateAddressFailsWithUnknownStreetType or testValidateAddressFailsWithMissingStreetName.

And over time, I've discovered a test's name is not the only way to convey the intent of the test...

I also use other general good naming practices to help those poor sods with the unfortunate task of following in the footsteps of my code and tests. In short, I look to convey my test's intent in the following four places, depending on which is applicable for the test at hand:

  • Test method name. 'Nuff said.
  • Extracted method names. Test sufficiently big to require the extraction of new methods? If so (and the test cannot be refactored into smaller self-contained tests), make sure these method names are also sufficient to convey the intent of each new method
  • Local variable names. Even if it's just something along the lines of "expectedFoo", it can make a assertEquals(expectedFoo, actualFoo); much easier on the eyes
  • Exception content. Recently I've found myself revisiting those times where I've been doing negative testing and asserting an exception thrown from a particular method. Up until now, I've normally just used the caught exception as proof of the test success, but I've started to experiment with a further check for the text of the exception message purely to assist in the test clarity. Something along the lines of:
public void testValidateAddressFailsWithMissingStreetName() {
  Address addressWithoutStreetName = constructFullyPopulatedAddress();
  addressWithoutStreetName.setStreetName(null);

  try {
    command.validateAddress(addressWithoutStreetName);
  } catch (ValidationException expected) {
    assertEquals("Address must have a street name", expected.getMessage());
  }
}

Posted by Andy Marks at October 27, 2005 09:16 PM

Comments

Hey Andy,

this is certainly something I've done at various times in the past. James once commented that it was slightly odd as I was making structure from strings which, in general, ain't such a great idea but for exceptions I'm happy to make, well, an exception :)

Posted by: Simon Harris at October 27, 2005 05:36 PM

Which is just more evidence that we should use the same level of design guidelines for test code as we do for app code.

Posted by: Jason Yip at October 27, 2005 07:25 PM

Don't test the exception's message. Test the exception's _state_ - something like this:

assertEquals(ValidationProblem.missingField, error.getProblem())
assertEquals("StreetName", error.getProblemField())

By embedding the message into the test, you are forcing the decision of how to display the error into the test, when this is a UI concern. In contrast, by testing the state, you ensure that the information needed to work with the exception is available.

Naturally, you can of course have tests for the Exception itself - here, you can test that the message is constructed appropriately for the right state. Or you can move those tests to an appropriate exception handler for the UI.

Posted by: Robert Watkins at October 27, 2005 09:46 PM

I have come to the same conclusion with regards to exception testing. I generally assert* on the cause of the exception as well, be it null or otherwise, to make it clear whether or not another exception is the cause and taking care not to swallow it.

Posted by: Brandon at October 28, 2005 06:42 AM

I'm a great believer in intent revealing names. Since I've read Martin's book on Refactoring, I've been on a quest to create better names for methods and objects. People don't seem to get the importance of well named methods or variables and seem to think that comments should compensate for poorly naming.

I've posted the other day on guidelines for naming test methods. Maybe you should have a look, if you are interested.

http://exceptionz.blogspot.com/2005/10/test-list-and-naming-test.html

Posted by: Maruis Marais at October 28, 2005 07:15 AM

Andy, having worked with you a bit, the other thing I noticed was that you religiously include a message in your asserts. It's something I'm often too lazy to do, but they certainly make your tests very understandable.

Posted by: Mike Williams at October 28, 2005 09:29 AM

Thanks for reminding me Mike, I'd forgotten about that oft-overlooked and underused part of JUnit: the assertion message. I tend to use them on everything but "assertEquals" methods, where the "expected foo but got bar" tends to suffice for me.

Posted by: Andy at November 4, 2005 09:56 AM

Post a comment

Thanks for signing in, . Now you can comment. (sign out)

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)


Remember me?