The Boston Diaries

The ongoing saga of a programmer who doesn't live in Boston, nor does he even like Boston, but yet named his weblog/journal “The Boston Diaries.”

Go figure.

Wednesday, August 31, 2022

We could have skipped writing a program

A bit of background—“Project: Sippy-Cup” uses data from a single column from a database to do its job. It doesn't query the database directly since we have a tight deadline, so there's a custom binary file that contains around 100,000,000 records, each record having a unique key and a 32-bit value. It doesn't matter what the key or the value is, just that this file exists. So, with that out of the way …

I was at lunch today with some fellow cow-orkers. Talk turned towards a QA engineer who was tasked by my friend TS (a senior QA engineer) to write a program to scan the data file used by “Project: Sippy-Cup” and count each unique value. I had written such a program in Lua (which worked by directly reading the binary file itself—easy enough since “Project: Sippy-Cup” is in Lua and has to read the binary file). TS wrote one in Python to do the work from a text dump of the binary file. The text output is just:

unique-key-1 = value
unique-key-2 = value

It's not hard to parse, it's just that the text dump is 100,000,000 lines long.

The QA engineer in question couldn't get his program to work.

It was only after lunch did I realize that none of us had to write a program. No, all it would have taken was running:

GenericUnixPrompt> dump-proprietary-data -s \
	| awk '{print $3}' \
	| sort \
	| uniq -c \
	| sort -rn \
	> /tmp/report.out


Obligatory Picture

[“I am NOT a number, I am … a Q-CODE!”]

Obligatory Contact Info

Obligatory Feeds

Obligatory Links

Obligatory Miscellaneous

You have my permission to link freely to any entry here. Go ahead, I won't bite. I promise.

The dates are the permanent links to that day's entries (or entry, if there is only one entry). The titles are the permanent links to that entry only. The format for the links are simple: Start with the base link for this site:, then add the date you are interested in, say 2000/08/01, so that would make the final URL:

You can also specify the entire month by leaving off the day portion. You can even select an arbitrary portion of time.

You may also note subtle shading of the links and that's intentional: the “closer” the link is (relative to the page) the “brighter” it appears. It's an experiment in using color shading to denote the distance a link is from here. If you don't notice it, don't worry; it's not all that important.

It is assumed that every brand name, slogan, corporate name, symbol, design element, et cetera mentioned in these pages is a protected and/or trademarked entity, the sole property of its owner(s), and acknowledgement of this status is implied.

Copyright © 1999-2024 by Sean Conner. All Rights Reserved.