Re: JavaDocs vs. Tests (was: Interview)
On Sun, 4 Oct 2009, Kenneth P. Turvey wrote:
On Sat, 03 Oct 2009 21:57:58 -0400, Eric Sosman wrote:
When I'm writing a new method, I start with a nebulous
idea of what the thing is supposed to to (if I had no such notion, I
wouldn't have a reason to write the method at all). Step One is to write
a rough copy of what will eventually be the Javadoc; during the process
I discover the foggy patches in my nebulous idea, shine light into the
corner cases, and so on. Then I code the method (possibly coming up
with yet a few more tweaks along the way), and finally I turn the rough
draft into real Javadoc. (This is why even my private methods have doc
comments: They're my explanation to me of whatever folly I had in mind
at the time of coding.)
A friend of mine and I have been exchanging thoughts on just this topic
over the last couple of nights. When I deliver source code my clients
expect some level of documentation in the code, but my friend is certain
that comments should really be left out and instead use the unit tests as
documentation for what the method actually does.
This makes sense in a couple of different ways. A developer will get
more useful information from a unit test than he will from the docs. In
addition the unit tests will stay current and don't take any extra effort
to maintain if you are going to write them anyway.
[warning - long and rambling post ahead]
The official party line where i work is that the tests are the
documentation, and we don't write javadoc comments at all (except when we
do - see below). I'm dubious about it, though. Tests always indicate what
a method does in a specific situation, but don't necessarily communicate
what the method does in general. For instance, for a method
Utils.calculate, the test:
assertEquals(4, Utils.calculate(2));
assertEquals(9, Utils.calculate(3));
Doesn't tell you much. But writing it like this:
assertEquals(2 * 2, Utils.calculate(2));
assertEquals(3 * 3, Utils.calculate(3));
Is more indicative that what calculate is really doing is computing a
square. The problem is that people don't always write tests with an eye on
explanatory power; they (quite reasonably!) focus on the primary use of
tests, ie testing.
And how about:
@Test(expected=IllegalArgumentException.class)
testHBF4() {
Utils.process("HBF4");
}
Which tells us that "HBF4" is not a legit input to process, but doesn't
give any clue as to why. If you'd called the method
testThatStringsOverThreeCharactersAreRejected, or
testThatStringsWithNumbersAreRejected, then it'd be clear. Very few of our
test methods have names like that.
At least one of my colleagues insists on naming tests in a way that
actively conceals this information. Say we've got a function (or a web
service or something) formatName which takes a Person object and gives you
a string like "J. Robert Dobbs". To test this, we'd make a JSP page called
userList.jsp which lists users, formatting their names using this method.
We'd then write a test that goes to this page and checks that the names
are as expected. I'd call that test method testFormatName, because the
*purpose* of it is to test the formatName method. This colleague would
call it testUserList, because that's what it's *actually* doing. He's
quite right that that is what it's doing, but it completely misses the
point of what the test is about.
Either way, to be honest, someone looking for documentation of what
formatName does is going to have a pretty hard time. This is largely a
consequence of our approach of doing all our tests through the web
interface to a running server, rather than directly on objects - but this
is an approach we take because so much of what we do can't be isolated
from the server it lives in without herculean effort (it's mixed up with a
third-party framework; mocking would mean mocking most of the framework,
which is a non-starter). But perhaps this is me being defensive; perhaps
we should man up and isolate our objects, so they could be tested
directly.
I don't have a firm opinion on this. I know my clients are going to
want JavaDocs. I'm not sure where I stand on whether this is reasonable
or not. For libraries, it certainly makes sense, but for applications..
I'm not sure it does.
We're currently in the process of releasing a library, rather than an app,
and that does indeed have javadocs. What's a bit weird is that in its
released form, it *doesn't* have tests - the tests for it are actually
tests for a demo app written on top of the library. We're not shipping the
demo app as part of the library, so we're not shipping the tests! This may
end up changing; the demo app was written with being a reference app in
mind, so at some point, we probably will bundle that, and its tests, with
the library too.
Anyway, this sort of brings me to what is mercifully my final point:
tests-as-documentation work okay if you also have access to the people who
wrote the tests. I can shout out "What does this formatName method do?",
and someone in the office will tell me "Oh, look in testUserList". If
there's anything there i don't understand, i can ask again about that.
Because we're an Extreme-ish team, that knowledge is widely spread - there
will be at least two people who wrote that test and that method, and
likely more who've revisited it for one reason or another. Thus, really,
the true documentation of the system is a sort of folk knowledge. That's
fine if you're a small team building an app which no outsiders will need
to understand the innards of. It's useless if you're building a library.
Here endeth the ramble.
tom
--
The best way to predict the future is to invent it. -- Alan Kay