Monday, March 06, 2023
Another attempt at a “unit test”
Or, “What is a ‘unit test,’ part III”
The reactions to my previous post were interesting—it wasn't a “unit test.” At best, it might have been an “integration test” but because it involved actual work (i.e. interaction with the outside world via nasty nasty side effects, aka I/O) it immediately disqualified it as a “unit test.” And to be honest, I was expecting that type of reaction—it appears to me that most unit test proponents tend to avoid such “entanglements” when writing their “battle tested” code (but I'm also willing to admit that's the cynical side of me talking). There were also comments about how 100% code coverage was “unrealistic.”
Sigh.
One respondent even quoted me out of context—“… that we as programmers are not trusted to write code without tests …” and cut the rest of the sentence: “… yet we're trusted to write a ton of code untested as long as such code is testing code.” Which was my cynical take that the “unit tests” (or the code that implements “unit tests” ) are, themselves, not subjected to “unit tests.” Something I kept trying to impart to my former manager, “stop taking the unit tests as gospel! I don't even trust them!” (mainly because the business logic of the project was convoluted and marketing kept using different terms from engineering, at least engineering in my department)
But when I left off, I said there was one final function that should fit as a “unit,” and thus, perfect for “unit testing.” Again, it's from my blog engine and the function in question deals with parsing a request, like “2001/10/02.2-11/03.3” (which requests all blog posts starting from the second post on October 2nd to the third post of November 3rd, 2001). or “2001/11/04.2” (the second post from November 4th, 2001).
The function tumbler_new() does no I/O
(that is—no disk, network or console I/O),
touches no global variables,
only works with the data given to it
and does some covoluted parsing of the input data—if this isn't a good candidate for “unit tests” then I don't know what is.
The tests were straightforward—a bunch of failure cases:
tap_assert(!tumbler_new(&tumbler,"foo/12/04.1",&first,&last),"non-numberic year"); tap_assert(!tumbler_new(&tumbler,"1999/foo/04.1",&first,&last),"non-numeric month"); tap_assert(!tumbler_new(&tumbler,"1999/12/foo.1",&first,&last),"non-numeric day"); tap_assert(!tumbler_new(&tumbler,"1999/12/04.foo",&first,&last),"non-numeric part"); tap_assert(!tumbler_new(&tumbler,"1998",&first,&last),"before the start year"); tap_assert(!tumbler_new(&tumbler,"1999/11",&first,&last),"before the start month"); tap_assert(!tumbler_new(&tumbler,"1999/12/03",&first,&last),"before the start day"); tap_assert(!tumbler_new(&tumbler,"1999/12/04.0",&first,&last),"part number of 0"); tap_assert(!tumbler_new(&tumbler,"2023",&first,&last),"after the end year"); tap_assert(!tumbler_new(&tumbler,"2022/11",&first,&last),"after the end month"); tap_assert(!tumbler_new(&tumbler,"2022/10/07",&first,&last),"after the end day"); tap_assert(!tumbler_new(&tumbler,"2022/10/06.21",&first,&last),"after the end part"); tap_assert(!tumbler_new(&tumbler,"1999/00/04.1",&first,&last),"month of 0"); tap_assert(!tumbler_new(&tumbler,"1999/13/04.1",&first,&last),"month of 13"); tap_assert(!tumbler_new(&tumbler,"1999/12/00.1",&first,&last),"day of 0"); tap_assert(!tumbler_new(&tumbler,"1999/12/32.1",&first,&last),"day of 32"); tap_assert(!tumbler_new(&tumbler,"1999/12/04.0",&first,&last),"part of 0"); tap_assert(!tumbler_new(&tumbler,"1999/12/04.24",&first,&last),"part of 24"); tap_assert(!tumbler_new(&tumbler,"2010/07/01-04/boom.jpg",&first,&last),"file with range"); tap_assert(!tumbler_new(&tumbler,"2010/7/1-4/boom.jpg",&first,&last),"file with redirectable range");
Plus a bunch of tests that should pass:
test("first entry","1999/12/04.1",&(tumbler__s) {
.start = { .year = 1999 , .month = 12 , .day = 4 , .part = 1 },
.stop = { .year = 1999 , .month = 12 , .day = 4 , .part = 1 },
.ustart = UNIT_PART,
.ustop = UNIT_PART,
.segments = 0,
.file = false,
.redirect = false,
.range = false,
.filename = ""
});
test("some mid entry","2010/07/04.15",&(tumbler__s) {
.start = { .year = 2010 , .month = 7 , .day = 4 , .part = 15 },
.stop = { .year = 2010 , .month = 7 , .day = 4 , .part = 15 },
.ustart = UNIT_PART,
.ustop = UNIT_PART,
.segments = 0,
.file = false,
.redirect = false,
.range = false,
.filename = ""
});
test("last entry","2022/10/06.20",&(tumbler__s) {
.start = { .year = 2022 , .month = 10 , .day = 6 , .part = 20 },
.stop = { .year = 2022 , .month = 10 , .day = 6 , .part = 20 },
.ustart = UNIT_PART,
.ustop = UNIT_PART,
.segments = 0,
.file = false,
.redirect = false,
.range = false,
.filename = ""
});
test("requesting a file","2010/07/04/boom.jpg",&(tumbler__s) {
.start = { .year = 2010 , .month = 7 , .day = 4 , .part = 1 },
.stop = { .year = 2010 , .month = 7 , .day = 4 , .part = 23 },
.ustart = UNIT_DAY,
.ustop = UNIT_DAY,
.segments = 0,
.file = true,
.redirect = false,
.range = false,
.filename = "boom.jpg",
});
/* ... other tests ... */
With this function checking the results:
static void test(char const *tag,char const *tum,tumbler__s const *result)
{
tumbler__s tumbler;
assert(tag != NULL);
assert(tum != NULL);
assert(result != NULL);
tap_plan(10,"%s: %s",tag,tum);
tap_assert(tumbler_new(&tumbler,tum,&first,&last),"create");
tap_assert(btm_cmp(&tumbler.start,&result->start) == 0,"start date");
tap_assert(btm_cmp(&tumbler.stop,&result->stop) == 0,"stop date");
tap_assert(tumbler.ustart == result->ustart,"segment of start");
tap_assert(tumbler.ustop == result->ustop,"segment of stop");
tap_assert(tumbler.segments == result->segments,"number of segments");
tap_assert(tumbler.file == result->file,"file flag");
tap_assert(tumbler.redirect == result->redirect,"redirect flag");
tap_assert(tumbler.range == result->range,"range flag");
tap_assert(strcmp(tumbler.filename,result->filename) == 0,"file name");
tap_done();
}
I ended up with a total of 328 tests and of the three attempts I made,
this one feels like the only one that was worth the effort—it's a moderately long function
[Moderately long? It's 450 lines long! —Editor]
[But it does one thing, and one thing well—it parses a request! —Sean]
[450 lines! —Editor]
[I'd like to see you write it then! —Sean]
[… Okay, I'll shut up now. —Editor]
that implements some tricky logic and deal with some weird edge cases.
If I ever go back to rework this code
(and I've only revised this code once in the 23 years it's been used,
way back in 2015—it was a full rewrite of the function)
the tests could be useful
(if I'm honest with myself,
and the API/structure doesn't change).
And from looking over the test cases,
I can see that I could get rid if (Seems I was
wrong---the .segments from the structure,
so there's that..segments field is needed for
tumbler_canonical())
Overall, I'm still not entirely sure about this “unit test” stuff, especially since “unit” doesn't have a well defined meaning. In my opinion, I think it works best for functions that do pure processing (no interaction with the outside world) and that implement some complex logic, like parsing or business logic. Back when I was at The Enterprise, had we but one function (or entry point) that just implemented all the business logic from data gathered from the request, it would have have made testing so much easier. But “unit tests” for all functions? Or modules? Or whatever the XXXX a “unit” is? No. Not for obvious code, or for for code that interacts with external systems (unless required because human lives are on the line). I'm not saying no to tests entirely, but to the slavish adherence to testing for its own sake.
Or maybe, instead of having AI write code for us, have it write the test cases for us, intead of the future I'm cynically seeing—where we write the test cases for AI written code.
