The Boston Diaries

The ongoing saga of a programmer who doesn't live in Boston, nor does he even like Boston, but yet named his weblog/journal “The Boston Diaries.”

Go figure.

Wednesday, June 02, 2021

The point of writing software seems to be writing tests for the software, not in running the software

I've been stuck in testing hell the past few weeks, what with the changing of the old guard, and the new guards crew are heavy into the whole “unit test über alles” religion. I understand the reason behind testing, and unit testing in particular. The few times I've tried unit testing, it has been helpful, but …

  1. I think it works best when you start out with unit testing in mind;
  2. it works best with a module or library;
  3. you test at the boundary of the module or library and not the implementation;
  4. you do have tests for your tests, right?

That last bit might sound a bit glib, but it is a real concern. Even Knuth is quoted as saying, “beware of bugs in the above code; I have only proved it correct, not tried it.” I even joked about having to test our tests with both my current (but soon-to-retire) manager and his (soon-to-me-my) manager, because our current regression test has 15,852 individual tests (not all of them might be valid—hard to say, since the tests are automatically generated).

Project: Lumbergh” was not written with unit tests in mind. Or more specifically, it is a unit. The whole thing. And it's complicated because not only does it implement our business logic (which over ten years has gotten quite complex) but because it has to query multiple databases at the same time across the network (“at the same time” because it has to do its job as a phone call is being made). How does one deterministically test delayed (and/or dropped) queries?

Another issue—“Project: Lumbergh” sends a message to “Project: Cleese” in some circumstances (“Project: Cleese” handles an HTTP notification on behalf of “Project: Lumbergh” because trying to add a TCP connection in a program that deals with UDP would have taken too much engineering time) and we need to check if “Project: Cleese” was notified. I solved that one by mocking the HTTP endpoint that “Project: Cleese” talks to with two interfaces, one for “Project: Cleese” and one for the regression test to query. The issue there is the regression test might ask the mock if it received a request from “Project: Cleese” before “Project: Cleese” gets the notification from “Project: Lumbergh” (a classic race condition). I got it working reliably, but now the regression tests takes over twenty minutes to run, instead of the two-plus minutes (even on the happy path, and I'm not entirely sure why).

It seems like management is more concerned about the tests rather than the product.

Obligatory Picture

[It's the most wonderful time of the year!]

Obligatory Contact Info

Obligatory Feeds

Obligatory Links

Obligatory Miscellaneous

You have my permission to link freely to any entry here. Go ahead, I won't bite. I promise.

The dates are the permanent links to that day's entries (or entry, if there is only one entry). The titles are the permanent links to that entry only. The format for the links are simple: Start with the base link for this site: http://boston.conman.org/, then add the date you are interested in, say 2000/08/01, so that would make the final URL:

http://boston.conman.org/2000/08/01

You can also specify the entire month by leaving off the day portion. You can even select an arbitrary portion of time.

You may also note subtle shading of the links and that's intentional: the “closer” the link is (relative to the page) the “brighter” it appears. It's an experiment in using color shading to denote the distance a link is from here. If you don't notice it, don't worry; it's not all that important.

It is assumed that every brand name, slogan, corporate name, symbol, design element, et cetera mentioned in these pages is a protected and/or trademarked entity, the sole property of its owner(s), and acknowledgement of this status is implied.

Copyright © 1999-2021 by Sean Conner. All Rights Reserved.