The Boston Diaries

Thursday, January 02, 2020

Net of Cards

I just spent the past three days sans email due to an issue with my hosting company. The title of this post isn't to denigrate the hosting company because the situation wasn't entirely their fault. I used to do that type of work so I know how crazy it can get. It also doesn't hurt that I know the owner, and I was able to provide some key information so they could fix the problem. The initial issue was fixed, but there was still an issue where it was only a few virtual servers (and mine was one of them) were still cut off from the Internet. It was my describing the situation in detail to Bunny that I had that “… and then this over here … wait a second … that's it!” moment that sovled the issue.

I then told the owner of the company, “all you need to do is reroute the tachyon emitter through the shield generator and transport Wesley 25 meters outside the ship to get this working.” Said technobabble was applied and my server was back online.

No, the title of this post is just a bit of commentary on the utter fragility of modern software (link via Lobsters) that this situation reminds me of. The fact that it took several engineers several days to get a virtual server back online is telling. That there are so many layers of abstraction that it's often hard to determine the root cause of the issue. That I suffered for nearly three days sans email!

Anyway … Happy New Year everybody! I'm just sad that this didn't happen (link via Kirk Israel). It would have been awesome!

Friday, January 03, 2020

The New Age East Asian meditation neurohacking golf club store

The last items of note in Dad's storage unit are golf clubs—everything else has been cleaned out. Today is pretty much the last day we have to clean out the unit before we're stuck with a full month's rent. Bunny had done the work of organizing the clubs, now we just had to get them out of there.

[Four golf club bags, plus a few loose drivers. If only this was the extent of the problem. If only … ]

[Bundles of drivers. Woods, that is. And plenty of chances of duffed shots into the weeds.]

[More drivers. More bags. Irons. Oh, we aren't even close to being done here.]

[Irons, a bit of scrap, and putters. Ha! Ha ha ha ha! Ha ha ha ha ha ha ha ha! There's enough here to outfit the Scottish army. Sob.]

[Empty golf bags. We do have enough to open up a store, don't we? Thank God that's the last of it. Sheesh!]

Several trips later, and we don't have to worry about the storage unit any more.

Now, anybody want some golf clubs?

Please?

Sunday, January 05, 2020

Star Wars: The Rise of Skywalker

When we last left our heroes, Luke was dead, Kylo didn't shoot first, and Finn was in a pointless B-plot.

And now on with the show …

I know I'm late to the party on this, but as far as remakes of “Star Wars: Return of the Jedi” go, it's not bad. I certainly liked it way more than I did the previous installment, and it was clear that J. J. Abrams ran from Rian Johnson's direction of “Star Wars: The Last Jedi,” for good or ill.

And make no mistake, this is a remake of “Return of the Jedi,” down to a showdown on the forest moon Endor and good ol' Emperor Palpatine pulling his “give in to your hate, strike me down and rule the galaxy” shtick he pulled on Luke. But hey, J. J. Abrams also directed “Star Wars: The Force Awakens” which was a remake of “Star Wars: A New Hope,” so I'm not terribly surprised by it either.

The movie, like all Star Wars movies, is visually beautiful, but … I think I don't care for modern movie techniques like quick editing and over-reliance on garish special effects (I was surprised by the epileptic warning of flashing lights shown at the beginning of the movie—yes, it's that bad) and this movie is filled with them, to the point I found it distracting during the climax of the movie. Another aspect of the movie I found a bit annoying was the whole “fetch quest” vibe I got from it. The whole “we need to go here to get this MacGuffin that will show us how to get to the next MacGuffin.” I thought I was watching a Star Wars movie based on a role playing game. And due to the MacGuffin hunt, we went from location to location. In the original trilogy, “A New Hope” took place in three primary locations (Tatooine, the Death Star, Yavin 4), “The Empire Strikes Back” takes place in three primary locations (Hoth, Dagobah, Bespin), and “Return of the Jedi” takes place in, you guessed it, three primary locations (Tatooine again, the second Death Star, and the forest moon Endor). I lost track of the number of locations in this movie—I think at least six planets and numerous ships.

And can we get away from the XXXXXXX desert planets already? Sheesh.

Afterwards at dinner, refrigerator logic started to kick in as little details started not making sense. One example: one MacGuffin the characters obtained that, story wise, must have been made after “Return of the Jedi” but before “The Force Awakens.” But as I starting thinking of that particular MacGuffin, I asked myself, who made it? Why was it made? Who was it made for? It didn't make sense. And that's just one MacGuffin—there are others.

If this wasn't a Star Wars movie, it would be a fine popcorn type movie. Decent, but nothing terribly special about it. And that's what's sad about this movie. It's … okay. It wasn't bad, but it's not great. I don't hate it.

So … yeah.

Monday, January 06, 2020

Adding CGI support to my gopher server

Back when I released my gopher server, the only way to generate dynamic output was to add a custom handler to the program. I noticed that other gopher servers all claimed CGI support, but when I was rewriting the gopher server, I felt that CGI support as defined didn't make much sense for gopher, but an email conversation changed my mind on the subject. I thought I would go through how I support CGI for my gopher server.

On a Unix system, the “meta-variables” defined in the specification are passed in as environment variables. So going through them all, we have:

AUTH_TYPE

Only required if the request requires authorization. Since gopher doesn't have that concept, this meta-variable doesn't have to be set. Good. Next.

CONTENT_LENGTH

This is only defined if data is being passed into the CGI script. The gopher protocol doesn't have this concept, so this meta-variable doesn't have to be set.

CONTENT_TYPE

If CONTENT_LENGTH isn't set, then this one doesn't need to be set either.

GATEWAY_INTERFACE

The specification I'm following defines version 1.1 of CGI, so this one is easy—it's just set to “1.1” and we're done.

PATH_INFO

This one is tough, and I had to run a bunch of experiments on my webserver to see how this meta-variable works. As the specification states:

It identifies the resource or sub-resource to be returned by the CGI script, and is derived from the portion of the URI path hierarchy following the part that identifies the script itself.

Basically, if I reference “/script” then PATH_INFO isn't set, but if I reference “/script/data” then PATH_INFO should be “/data”. Because of this meta-variable (and a few others) I had to drastically change how requests are passed around internally, but I got this working.

One issue I had with this was leading slashes. Gopher doesn't have a concept of a “path”—it has the concept of a “selector,” which is an opaque sequence of characters that make up a reference. That, in turn, makes gopher URLs different enough from web URLs. This also means that a gopher “selector” does not have to start with a leading slash, something I had to mention up front on my gopher space (none of the selectors on my gopher site start with a slash). But there are gopher sites out there with selectors that do start with a slash, and I wanted to take both types into account. That was harder than it should have been.

But it also needs the leading portion of the selector upto the script name prepended. For example, if the selector is “Users:spc/script/foobar” then PATH_INFO should be “Users:spc/foobar”.

And this meta-variable is only set if there's a “sub-resource” defined on the selector.

PATH_TRANSLATED

And the beat goes on.

Whereas PATH_INFO is the selector with the script name removed (for the most part), PATH_TRANSLATED is the underlying filesystem location with the script name removed. So, using the example of “Users:spc/script/foobar” then the resulting PATH_TRANSLATED would be “/home/spc/public_gopher/foobar”. Also, if PATH_INFO is not set, then I don't have to deal wit this meta-variable.

Both where a bit tough to get right.

QUERY_STRING

Easy enough—gopher does have the concept of search queries so if a search query is supplied, it's passed in this, otherwise, this is set to the empty string.

The one kicker here is that the specification states that QUERY_STRING is URL-encoded, which is not the case in gopher. I decided against URL-encoding the non-URL-encoded search query, which goes agains the standard, but there are other parts of the standard that don't fit gopher (which I'll get to in a bit).

REMOTE_ADDR

The address of the remote side. Easy enough to provide. Enough said here.

REMOTE_HOST

The standard states:

The server SHOULD set this variable. If the hostname is not available for performance reasons or otherwise, the server MAY substitute the REMOTE_ADDR value.

I'm setting this to the REMOTE_ADDR value. Done! Next!

REMOTE_IDENT

Nobody these days supports ident and the specification states one may use this, so I'm not. Next.

REMOTE_USER

The meta-variable AUTH_TYPE doesn't apply, then this one doesn't apply, so it's not set.

REQUEST_METHOD

This one was tough, and not because I had to go through contortions to generate the value. No, I had to to through mental contortions to come up with what to set this to. The specification is written for the web, and it's expected to be set to some HTTP method like GET or POST or HEAD. But none of those (or really, any of the HTTP methods) apply here. I suppose one could say the GET method applies, since that's semantically what one is doing, “getting” a resource. But the gopher protocol doesn't use any methods—you just specify the selector and it's served up. So after much deliberation, I decided to set this to the empty string.

I suppose the more technical response should be something like “-” (since the specification defines it must be at least one character long) but that's the problem with trying to adapt standards—sometimes they don't quite match.

SCRIPT_NAME

This will typically be the selector echoed back, but the meta-variables PATH_INFO and PATH_TRANSLATED complicate this somewhat. But given that I've calculated those, this one wasn't that much of a problem.

SERVER_NAME

Easy enough to pass through.

SERVER_PORT

Again, easy enough to pass through.

SERVER_PROTOCOL

Unlike the meta-variable REQUEST_METHOD, this one was easy, “GOPHER”.

SERVER_SOFTWARE

Again, easy to set.

The specification also allows protocol-specific meta-variables to be defined, and so I defined a few:

GOPHER_DOCUMENT_ROOT: This is the top level directory where the script resides, and it can change from request to request. My gopher server can support requests to multiple different directories, so the GOPHER_DOCUMENT_ROOT may change depending upon where the script is served from.
GOPHER_SCRIPT_FILENAME: This differs from the meta-variable SCRIPT_NAME as this is the actual location of the script on the filesystem. SCRIPT_NAME is the “name” of the script as a gopher selector.
GOPHER_SELECTOR: The actual selector requested from the network.

And that pretty much covers the input side of things. The output, again was a bit difficult to handle, semantic wise. The standard expects the script to serve up a few headers, like “Status”, “Content-Type” and “Content-Length” but again, gopher doesn't have those concepts. After a bit of thought, I decided that anyone writing a CGI script for a gopher site knows they're writing a CGI script for a gopher site and such things won't need to be generated. And while in theory one could use a CGI script meant for the web on a gopher server, I don't think that will be a common occurance (HTML isn't common on most gopher sites). So at the places where I broke with the standard, that's why I did it. It doesn't make sense for gopher, and strict adherence to the standard will just mean some work done just to be undone.

By this point, I was curious as to how other gopher servers dealt with the CGI interface, so I looked at the implementations of three popular gopher servers, Gophernicus, Motsognir and Bucktooth. Like mine, they don't specify output headers, just the content. But unlike mine, they vary wildly with the meta-variables they defined:

Bucktooth

Defines the least number:

SERVER_HOST
SERVER_PORT

And the following nonstandard meta-variable:

SELECTOR

Motsognir

Defines a few more:

GATEWAY_INTERFACE, which is set to “CGI/1.0” and as far as I can tell, isn't described anywhere.
QUERY_STRING
REMOTE_ADDR
REMOTE_HOST
SCRIPT_NAME
SERVER_PORT
SERVER_SOFTWARE

And the following nonstandard meta-variables:

QUERY_STRING_SEARCH
QUERY_STRING_URL, which appears to be the same as QUERY_STRING_SEARCH

Gophernicus

Which defines the most (even more than I do):

GATEWAY_INTERFACE, which is set to “CGI/1.1”
QUERY_STRING
REMOTE_ADDR
REQUEST_METHOD, which is set to “GET”
SCRIPT_NAME
SERVER_NAME
SERVER_PORT
SERVER_PROTOCOL, which is set to either “HTTP/0.9” or “RFC1436”
SERVER_SOFTWARE

And the nonstandard meta-variables:

COLUMNS
CONTENT_LENGTH, which is set to 0
DOCUMENT_ROOT
GOPHER_CHARSET
GOPHER_FILETYPE
GOPHER_REFERER
HTTPS
HTTP_ACCEPT_CHARSET
HTTP_REFERER
LOCAL_ADDR
PATH
REQUEST
SCRIPT_FILENAME
SEARCHREQUEST
SERVER_ARCH
SERVER_CODENAME
SERVER_DESCRIPTION
SERVER_TLS_PORT
SERVER_VERSION
SESSION_ID
TLS

Gophernicus seems the most interesting. It seems they support running gopher over TLS, even though it doesn't make much sense (in my opinion), and try to make their CGI implementation appear most like a webserver.

What this says to me is that not many CGI scripts for gopher even look at the meta-variables all that much. But at least I can say I (mostly) support the CGI standard (somewhat—if you squint).

Tuesday, January 07, 2020

The Heisenberg Notification Center of Windows 10

So it's been a few months since I received the Corporate Overlord's mandated managed Windows 10 laptop and among some of the annoying aspects of it are these little notifications that briefly pop up and then disappear. I'll have it on so it can do its thing (if I don't have it “phone home” at least once every three weeks, the Corporate Overlords will assume it's been stolen and have it remotely wiped next time its turned on) and I'm doing my thing when this little chime rings out. I then have to stop what I'm doing as this small box slides into the lower right corner of the screen, and by the time I get my gaze to the box, it slides quickly out of view, only leaving me with a glimpse of something or other updating or needing updating or I need to update or who knows what. I can't read it in time before it slides away into the bit bucket.

And as far as I can tell, there's no real way to recall them, nor any way to configure the notification to remain on the screen longer than it takes me to glance at the lower right corner of the screen (as if I could modify anything on this “managed” laptop anyway).

Sometimes, I question whether I'm seeing things or not. I'm finding the whole experience a bit unsettling.

It could be an omen, or it could be that someone just forgot to water the poinsettia

Just outside my office at the Ft. Lauderdale Office of the Corporation is this rather sad looking poinsettia:

[A picture of a nearly (if not outright) dead poinsettia. During Christmas vacation, my aunt on MyFaceMeLinkedSpaceBookWeIn linked to an article about keeping poinsettias around because they can be used to filter the toxins out of the air. I hate to think that this is what is happening to this poor poinsettia (never mind us!).]

I'm just hoping it's someone just forgot to water the poor thing.

Wednesday, January 08, 2020

Chiabacca

I turn my back for one second and BAM! a Chiabacca just appeared out of nowhere on my desk!

[Rrraaaaaaaawwwwwwwwwrrrrrrrrrrr aaaaaaarrrrrrrraaaaaaarrrrrrr!]

This means something …

Monday, January 13, 2020

“Security may indeed be a large sailboat with Krugerrands for ballast”

Going through the last box of Dad's papers, Bunny found an extensive collection of notes and drafts of a book Dad was trying to write back in 1982, titled Running Away To Sea. It seems the election of one Ronald Reagan to the Presidency spooked Dad a bit and he started researching how to survive “the badness” (as he called it) not by living off-grid in a shack in the middle of Montana, but by living off-grid in a boat in the middle of the Pacific.

From the time one slips his mooring lines, he begins to put an insurmountable distance between himself and those who might try to take that which he has set aside for his family's survival. Once the ocean is reached, safety from the problems on land ceases to be an immediate consideration.

Until the pirates show up.

Okay, to be fair, I did find references and draft material covering the problem of pirates, but I found his stance on a 12 gauge shotgun to be “more accurate” than a hand gun to be questionable at best. “Accuracy” on a rolling, pitching boat in the open water is going to be questionable, regardless of choice of firearm.

There is correspondence with yatch manufactuers, blue prints, price breakdowns (nearly $300,000 in 1982 dollars, making it nearly $800,000 in today's dollars—ouch!) and scores of articles on everything related to sailing. It also appears that Dad was trying to invent a new type of sail, as there were drawings he did and correspondence with an engineering firm. I'm not sure what I'll do with it all, but the blueprints are cool.

Tuesday, January 14, 2020

Notes on an Overheard Conversation About a Bank Account

“Hello, this is XXXXXX of [a bank in California]. How may I help you?”

“Yes, I'm calling about the closing out of an account. I received the affidavit form and I've filled it out except for one line—I don't know how much is in the account.”

“I'm sorry, but I can neither confirm nor deny he has an account with us.”

“What?”

“I can neither confirm nor deny he has an account with us.”

“You sent me the affidavit!”

“Yes, but I can neither confirm nor deny his account with us.”

“I have his account number in front of me!”

“I can neither—oh … um … please hold … <mumble mumble mumble>”

“I'm sorry, I didn't hear that.”

“Sorry, I can't speak any louder than this—what is the account number?”

“XXXXXXXXX”

“Okay, the amount left in the account is $45.71”

“Seriously?”

“Seriously.”

“You're only telling me because it's a rounding error to you guys, right?”

“Shhhhhhhhh!”

“Okay! Anyway, thank you.”

“Anything else I can do for you?”

“Yes, your direct phone number in your email? When I called it, I got a health clinic in Encino.”

“Seriously?”

“Seriously.”

Friday, January 17, 2020

Mysterious packets in the night

For about a decade, I've been monitoring syslog traffic in real time. It makes for an interesting background screen. For instance, I've noticed over the years Mac OS-X getting more and more paranoid about running executables. that one UPS needs a new battery, and just how many DNS requests Firefox makes. Just stuff I notice out of the corner of my eye.

So it was rather alarming when I just saw the following pop out:

iTunes  tcp_connection_destination_perform_socket_connect 140 connectx to 2600:1403:b:19a::2a1.443@0 failed: [65] No route to host 
iTunes  tcp_connection_destination_perform_socket_connect 140 connectx to 2600:1403:b:1a8::2a1.443@0 failed: [65] No route to host 
iTunes  tcp_connection_destination_perform_socket_connect 140 connectx to 2600:1403:b:1ab::2a1.443@0 failed: [65] No route to host 
iTunes  tcp_connection_destination_perform_socket_connect 140 connectx to 2600:1403:b:1ac::2a1.443@0 failed: [65] No route to host 
iTunes  tcp_connection_destination_perform_socket_connect 140 connectx to 2600:1403:b:185::2a1.443@0 failed: [65] No route to host 
iTunes  tcp_connection_destination_perform_socket_connect 141 connectx to 2600:1403:b:1ac::2a1.443@0 failed: [65] No route to host 
iTunes  tcp_connection_destination_perform_socket_connect 141 connectx to 2600:1403:b:185::2a1.443@0 failed: [65] No route to host 
iTunes  tcp_connection_destination_perform_socket_connect 141 connectx to 2600:1403:b:19a::2a1.443@0 failed: [65] No route to host 
iTunes  tcp_connection_destination_perform_socket_connect 141 connectx to 2600:1403:b:1a8::2a1.443@0 failed: [65] No route to host 
iTunes  tcp_connection_destination_perform_socket_connect 141 connectx to 2600:1403:b:1ab::2a1.443@0 failed: [65] No route to host 
iTunes  tcp_connection_destination_perform_socket_connect 143 connectx to 2600:1403:b:188::2a1.443@0 failed: [65] No route to host 
iTunes  tcp_connection_destination_perform_socket_connect 143 connectx to 2600:1403:b:18c::2a1.443@0 failed: [65] No route to host 
iTunes  tcp_connection_destination_perform_socket_connect 143 connectx to 2600:1403:b:18d::2a1.443@0 failed: [65] No route to host 
iTunes  tcp_connection_destination_perform_socket_connect 143 connectx to 2600:1403:b:1a1::2a1.443@0 failed: [65] No route to host 
iTunes  tcp_connection_destination_perform_socket_connect 143 connectx to 2600:1403:b:1ad::2a1.443@0 failed: [65] No route to host

Now, I have IPv6 enabled on my Mac to play around with the technology, but my main connection out to the Intarwebs is still plain IPv4. So that explains the error. But the question is why is iTunes trying to connect to some machine on the Intarwebs? I have iTunes running, but doing nothing at the moment.

So then I look into that IPv6 address. First, it's assigned to Europe, which is odd, because I'm not in Europe. Second, it seems it belongs to Akamai Technologies. So the bigger question now is, what is iTunes trying to get from Europe? Is my computer trying to snitch on me? Checking for updates? Is iTunes feeling neglected?

I don't know … and that is bothering me.

Tuesday, January 28, 2020

I feel the earth move under my feet

I'm sitting at my desk at the Ft. Lauderdate Office of the Corporation when I get this weird feeling the building is moving. How, the Tri-Rail runs along side the Ft. Lauderdale Office of the Corporation, and the rail itself is shared with freight trains. When a freight train goes by, you can feel it in the building. But this movement doesn't feel the same. First, I can't hear the train (which I can from my desk since it's on that side of the building). Second, it's a longer, slower, swaying motion rather than than the short back-and-forth type movement typical of freight trains blowing past the building.

“Hey, TS1,” I said, “do you feel the building moving?”

“No.”

“Oh.” So I'm left with the thought that 1) I'm going crazy or 2) I'm experiencing some major medical event. The sensation subsides after a minute or so, I'm not on the floor writhing in pain or unconscious, so I stop thinking about it and go back to work.

I'm at home when Bunny says, “Did you hear about the 7.7 earthquake between Cuba and Jamaca today?”

“Really? When did it happen?”

“Um … looks like this afternoon. It was felt as far north as Miami.”

“No, I didn't hear about it, but I think I felt it.”

It's one thing to worry about earthquakes in Brevard, North Carolina but now I have to worry about them here? La-la-la-la-la-la-la-la-la-la!

“Sean, why are you running around with your fingers in your ears?”

Wednesday, January 29, 2020

“This is an invalid protocol because I can't open a file”

TS2 comes to my desk at the Ft. Lauderdale Office of the Corporation. “I'm running a load test of ‘Project: Cleese’ and it's not functioning. If I run a normal test, it runs fine.”

“Hmm … let me take a look.” I head back to TS2's desk. Sure enough, “Project: Cleese” is crashing under load. Well, not a hard crash—it is written in Lua and what's crashing are individual coroutines due to an uncaught error (the Lua equivalent of exceptions) where the only information being reported is “invalid protocol.” I have TS2 send me copies of the data files and script he's using to load test, and I'm able to reproduce the issue. It's an odd problem, because it appears to be crashing on this line of code:

local sock,err = net.socket(addr.family,'tcp')

I dive in, and isolate the issue to this bit of C code that's part of the net.socket() function:

if (getprotobyname_r(proto,&result,tmp,sizeof(tmp),&presult) != 0)
  return luaL_error(L,"invalid protocol");

Odd, “tcp” is a valid protocol, so I shouldn't be getting ENOENT, and the buffer used to store data is large enough (because normally it works fine) so I don't think I'm getting ERANGE. And that covers the errors that getprotobyname_r() is documented to return. I add some logging to see what error I'm actually getting.

I'm getting “Too many open files” and it suddenly all makes sense.

getprotobyname_r() is using some data file (probably /etc/protocols) to translate “tcp” to the actual protocol value but it can't open the file because the program is out of available file descriptors. “Project: Cleese” is out of file descriptors because each network connection counts as a file descriptor, and the test systems (Linux in this case) only allow 1,024 descriptors per process. It's easy enough to up that to some higher value (I did 65,536) and sure enough, the “Too many open files” error starts showing up where I expect it to.

On the plus side, it's not my code. On the minus side, you have to love those leaky abstractions (and perhaps relying upon documentation a bit too much).