The Boston Diaries

Monday, Debtember 27, 2004

Notes from a drive through window

The woman in front of me pulls her car out of the drive through lane into a U-turn and starts to head the other way, then stops, and parks across five parking spots. This isn't boding well.

“Um … hello?”

“Brrzzzzzckorder please?”

“Yes, I'd like a number two with tomato—”

“Bzzzzzick drink with that?”

“—um … Coke.”

“BzzzzzzzIs that all?”

“Um … A number two with tomato, lettuce and—”

“Bzzzzzzzzzclkwizzzzlezzrrrrrrrrrrrrrrp”

“Forget it! I'll come inside!”

“Brrrrzzzzzzk?”

“Yes, I'd like a number two with tomato, lettuce—”

“Is that for here or to go?”

“Are you people incapable of letting me finish my order?”

“If we let you finish your order, we'll have to actually listen.”

“Oh.”

I'm waiting for my food. The cashier then asks the girl behind me of she's ready to order.

“Um … like, yeah. I'll … um … take a bacon cheeseburger, fries and a shake.”

“Would you like a combo meal?”

“Um … yeah, sure … okay.”

“Do you want the junior combo meal or the number four?”

The girl has that “deer-caught-in-headlights” look. “Um … you mean … like … I have to think?”

Mercifully my food arrived and I was able to leave.

User expectation

Have I told you how much I hate—

—yes, I think I have.

There are about five different trains of thought running through my head about this right now, but it would take some serious effort, and a lot of writing to explain all the thoughts (touching on such diverse subjects as data storage, data organization, configurations, RISC (or rather, a RISCesque approach to service configurations) and process management, in addition to these XXX XXXXXX XXXXXXX control panels.

But alas, I think I'll try to limit myself for now to just control panels.

Now, it may be that it's just this particular control panel we use here at The Company, but my limited exposure to two other control panel platforms hasn't left me feeling all that confident on their overall operation.

The problem, I think, is one of user expectation. And as a user of this control panel, it doesn't meet my expectation of what the tool should do. Sure, it might be easier for some kid out of college with scant experience with Unix to get up and running, but when anything (and I mean anything) goes wrong, it's damn near impossible to clear up without resorting to a lower level approach, like … the command line! (Oooooooooooooh!) And the problem with dropping down to the command line is that often times, you break the control software, or that whatever you fix breaks again when the control software asserts its control over the system and resets the configurations back to how it thinks they should be.

For instance, up until late last week we've been having an IP address conflict—two servers responding to the same IP address. This is not a good thing. Late last week I was finally able to track down which two machines were fighting over the IP address (and to do this, I had to log into the managed switches, check the ARP cache to see which port each machine was plugged into (found out that a third machine was programmed with the same IP address), then track down the two machines in question—no nice graphical user interface for this, no siree). I finally pinned down which machine was supposed to have the IP (this was a case of a website being moved from one server to another then yet to a third). Removed the site (using the control panels) from the two machines that the site shouldn't be on (figuring that the control panel would do its magic to remove everything it should and not break more stuff) and forced the network switch to clear its ARP cache for that IP address.

Then all was right with the world, and the network, and the site. And lo, the customer was happy because now they could get to their website.

Until today.

And lo, the old server said that it has that IP address, and lo, the switch obeyed and started sending network traffic unto it, even though that server servith not under that IP address. And lo did the customer complaineth.

Book of ARP, Chapter 2, verse 17

Yup, the switch was sending traffic for the IP address in question to the wrong server. I log into that server, A, and run ifconfig and well … the IP address does not show up. I log onto the server that is supposed to have that IP address, run ifconfig on it, and well … the IP address does not show up there either!

I log into a third machine, ping the IP address, then check to see which machine is getting the traffic—it's the old server.

I scour up and down the old server, searching every file under /etc (where all the configuraiton files live) and not finding the IP address anywhere. I double check through the control panel that the IP address is not supposed to be there, and nope, according to the control panel, this server is not supposed to be listening in on that IP address.

Yet it is.

It was suggested that I reboot the old server and see if that fixes the problem.

It does.

The correct server now responds to the IP address.

This is not a Windows box! You're not supposed to fix problems by randomly rebooting the server!

Yet I scour through that server, looking for the IP address and not finding it. I can't figure out how the server was told to listen to that IP address, because ifconfig isn't showing any network interface with that address. But nonetheless, it is.

[I found out, through some research, that this control panel uses the ip command, not ifconfig like every other Unix system I've used. Even if this is a better program for what the control panel is trying to do, why, oh why, is it incompatible with ifconfig? What is up with that?]

Perhaps I'm asking too much of a control panel that can manage to deal with manual tweaks of the system from time to time? Or perhaps I'm just bitter that I have to learn yet another system for the umteenth time, and one that can't even do a simple thing like change an IP address of a website (and funny, but I don't have nearly this level of problem with the other servers I admin—the ones that don't have a control panel installed).