The Boston Diaries

The ongoing saga of a programmer who doesn't live in Boston, nor does he even like Boston, but yet named his weblog/journal “The Boston Diaries.”

Go figure.

Monday, March 06, 2023

Another attempt at a “unit test”

Or, “What is a ‘unit test,’ part III

The reactions to my previous post were interesting—it wasn't a “unit test.” At best, it might have been an “integration test” but because it involved actual work (i.e. interaction with the outside world via nasty nasty side effects, aka I/O) it immediately disqualified it as a “unit test.” And to be honest, I was expecting that type of reaction—it appears to me that most unit test proponents tend to avoid such “entanglements” when writing their “battle tested” code (but I'm also willing to admit that's the cynical side of me talking). There were also comments about how 100% code coverage was “unrealistic.”


One respondent even quoted me out of context—“… that we as programmers are not trusted to write code without tests …” and cut the rest of the sentence: “… yet we're trusted to write a ton of code untested as long as such code is testing code.” Which was my cynical take that the “unit tests” (or the code that implements “unit tests” ) are, themselves, not subjected to “unit tests.” Something I kept trying to impart to my former manager, “stop taking the unit tests as gospel! I don't even trust them!” (mainly because the business logic of the project was convoluted and marketing kept using different terms from engineering, at least engineering in my department)

But when I left off, I said there was one final function that should fit as a “unit,” and thus, perfect for “unit testing.” Again, it's from my blog engine and the function in question deals with parsing a request, like “2001/10/02.2-11/03.3” (which requests all blog posts starting from the second post on October 2nd to the third post of November 3rd, 2001). or “2001/11/04.2” (the second post from November 4th, 2001).

The function tumbler_new() does no I/O (that is—no disk, network or console I/O), touches no global variables, only works with the data given to it and does some covoluted parsing of the input data—if this isn't a good candidate for “unit tests” then I don't know what is.

The tests were straightforward—a bunch of failure cases:

  tap_assert(!tumbler_new(&tumbler,"foo/12/04.1",&first,&last),"non-numberic year");
  tap_assert(!tumbler_new(&tumbler,"1999/foo/04.1",&first,&last),"non-numeric month");
  tap_assert(!tumbler_new(&tumbler,"1999/12/foo.1",&first,&last),"non-numeric day");
  tap_assert(!tumbler_new(&tumbler,"1999/12/",&first,&last),"non-numeric part");
  tap_assert(!tumbler_new(&tumbler,"1998",&first,&last),"before the start year");
  tap_assert(!tumbler_new(&tumbler,"1999/11",&first,&last),"before the start month");
  tap_assert(!tumbler_new(&tumbler,"1999/12/03",&first,&last),"before the start day");
  tap_assert(!tumbler_new(&tumbler,"1999/12/04.0",&first,&last),"part number of 0");
  tap_assert(!tumbler_new(&tumbler,"2023",&first,&last),"after the end year");
  tap_assert(!tumbler_new(&tumbler,"2022/11",&first,&last),"after the end month");
  tap_assert(!tumbler_new(&tumbler,"2022/10/07",&first,&last),"after the end day");
  tap_assert(!tumbler_new(&tumbler,"2022/10/06.21",&first,&last),"after the end part");
  tap_assert(!tumbler_new(&tumbler,"1999/00/04.1",&first,&last),"month of 0");
  tap_assert(!tumbler_new(&tumbler,"1999/13/04.1",&first,&last),"month of 13");
  tap_assert(!tumbler_new(&tumbler,"1999/12/00.1",&first,&last),"day of 0");
  tap_assert(!tumbler_new(&tumbler,"1999/12/32.1",&first,&last),"day of 32");
  tap_assert(!tumbler_new(&tumbler,"1999/12/04.0",&first,&last),"part of 0");
  tap_assert(!tumbler_new(&tumbler,"1999/12/04.24",&first,&last),"part of 24");
  tap_assert(!tumbler_new(&tumbler,"2010/07/01-04/boom.jpg",&first,&last),"file with range");
  tap_assert(!tumbler_new(&tumbler,"2010/7/1-4/boom.jpg",&first,&last),"file with redirectable range");

Plus a bunch of tests that should pass:

  test("first entry","1999/12/04.1",&(tumbler__s) {
    .start    = { .year = 1999 , .month = 12 , .day = 4 , .part = 1 },
    .stop     = { .year = 1999 , .month = 12 , .day = 4 , .part = 1 },
    .ustart   = UNIT_PART,
    .ustop    = UNIT_PART,
    .segments = 0,
    .file     = false,
    .redirect = false,
    .range    = false,
    .filename = ""
  test("some mid entry","2010/07/04.15",&(tumbler__s) {
    .start    = { .year = 2010 , .month = 7 , .day = 4 , .part = 15 },
    .stop     = { .year = 2010 , .month = 7 , .day = 4 , .part = 15 },
    .ustart   = UNIT_PART,
    .ustop    = UNIT_PART,
    .segments = 0,
    .file     = false,
    .redirect = false,
    .range    = false,
    .filename = ""
  test("last entry","2022/10/06.20",&(tumbler__s) {
    .start    = { .year = 2022 , .month = 10 , .day = 6 , .part = 20 },
    .stop     = { .year = 2022 , .month = 10 , .day = 6 , .part = 20 },
    .ustart   = UNIT_PART,
    .ustop    = UNIT_PART,
    .segments = 0,
    .file     = false,
    .redirect = false,
    .range    = false,
    .filename = ""
  test("requesting a file","2010/07/04/boom.jpg",&(tumbler__s) {
    .start    = { .year = 2010 , .month = 7 , .day = 4 , .part =  1 },
    .stop     = { .year = 2010 , .month = 7 , .day = 4 , .part = 23 },
    .ustart   = UNIT_DAY,
    .ustop    = UNIT_DAY,
    .segments = 0,
    .file     = true,
    .redirect = false,
    .range    = false,
    .filename = "boom.jpg",

  /* ... other tests ... */

With this function checking the results:

static void test(char const *tag,char const *tum,tumbler__s const *result)
  tumbler__s  tumbler;
  assert(tag    != NULL);
  assert(tum    != NULL);
  assert(result != NULL);

  tap_plan(10,"%s: %s",tag,tum);
  tap_assert(btm_cmp(&tumbler.start,&result->start) == 0,"start date");
  tap_assert(btm_cmp(&tumbler.stop,&result->stop) == 0,"stop date");
  tap_assert(tumbler.ustart == result->ustart,"segment of start");
  tap_assert(tumbler.ustop == result->ustop,"segment of stop");
  tap_assert(tumbler.segments == result->segments,"number of segments");
  tap_assert(tumbler.file == result->file,"file flag");
  tap_assert(tumbler.redirect == result->redirect,"redirect flag");
  tap_assert(tumbler.range == result->range,"range flag");
  tap_assert(strcmp(tumbler.filename,result->filename) == 0,"file name");

I ended up with a total of 328 tests and of the three attempts I made, this one feels like the only one that was worth the effort—it's a moderately long function [Moderately long? It's 450 lines long! —Editor] [But it does one thing, and one thing well—it parses a request! —Sean] [450 lines! —Editor] [I'd like to see you write it then! —Sean] [… Okay, I'll shut up now. —Editor] that implements some tricky logic and deal with some weird edge cases. If I ever go back to rework this code (and I've only revised this code once in the 23 years it's been used, way back in 2015—it was a full rewrite of the function) the tests could be useful (if I'm honest with myself, and the API/structure doesn't change). And from looking over the test cases, I can see that I could get rid if .segments from the structure, so there's that.(Seems I was wrong---the .segments field is needed for tumbler_canonical())

Overall, I'm still not entirely sure about this “unit test” stuff, especially since “unit” doesn't have a well defined meaning. In my opinion, I think it works best for functions that do pure processing (no interaction with the outside world) and that implement some complex logic, like parsing or business logic. Back when I was at The Enterprise, had we but one function (or entry point) that just implemented all the business logic from data gathered from the request, it would have have made testing so much easier. But “unit tests” for all functions? Or modules? Or whatever the XXXX a “unit” is? No. Not for obvious code, or for for code that interacts with external systems (unless required because human lives are on the line). I'm not saying no to tests entirely, but to the slavish adherence to testing for its own sake.

Or maybe, instead of having AI write code for us, have it write the test cases for us, intead of the future I'm cynically seeing—where we write the test cases for AI written code.

Discussions about this entry

Obligatory Picture

[It's the most wonderful time of the year!]

Obligatory Contact Info

Obligatory Feeds

Obligatory Links

Obligatory Miscellaneous

You have my permission to link freely to any entry here. Go ahead, I won't bite. I promise.

The dates are the permanent links to that day's entries (or entry, if there is only one entry). The titles are the permanent links to that entry only. The format for the links are simple: Start with the base link for this site:, then add the date you are interested in, say 2000/08/01, so that would make the final URL:

You can also specify the entire month by leaving off the day portion. You can even select an arbitrary portion of time.

You may also note subtle shading of the links and that's intentional: the “closer” the link is (relative to the page) the “brighter” it appears. It's an experiment in using color shading to denote the distance a link is from here. If you don't notice it, don't worry; it's not all that important.

It is assumed that every brand name, slogan, corporate name, symbol, design element, et cetera mentioned in these pages is a protected and/or trademarked entity, the sole property of its owner(s), and acknowledgement of this status is implied.

Copyright © 1999-2023 by Sean Conner. All Rights Reserved.