Sunday, January 01, 2006
Happy New Year
At some arbitrary point in time we slip into a new year and thus ends the obligatory “See you next year!” jokes for approximately 360 days.
We also, at this arbitrary point in time, make promises to ourselves to improve this, change that, and to stop doing the other. Such self dellusion usually ends in another 40 days or so as time marches on.
We also hope that the following 365 days, six hours, mine minutes and nine seconds (or 365 days, five hours, 48 minutes and 45 seconds, or 365 days, six hours, 13 minutes and 52 seconds, depending upon how one defines the term “year”) will be much better than the previous 365 days, six hours, mine minutes and nine seconds (or 365 days, five hours, 48 minutes and 45 seconds, or 365 days, six hours, 13 minutes and 52 seconds, depending upon how one defines the term “year”).
So much hopes and dreams and schemes pinned on the flipping of a single second at some arbitrary point in time.
Anyway …
HAPPY NEW YEAR!
Monday, January 02, 2006
Tar pits, legal and otherwise
Now that I more or less run my own DSL connection, I'm able to have more than a single static IP address. Because of that, I've given my main home system a public IP address so that I no longer have to log into my firewall/NAT system to get to it.
One thing about sitting next to a server is that over time, you get used
to the various patterns of noise the harddrive makes. It starts quickly
grinding, I know that a load of spam is coming in via my backup email server
(a common tactic of spammers—send spam to the backup servers which have less
information about valid users so they can “honestly” say they delivered the
email). The grinding that starts at 4:00 am are various cron
jobs rotating logs, clearing out temporary files and what not.
But over the holiday weekend linus
(the machine in question)
started making some disk noises I'm unfamiliar with. A constant “click click”
noise, perhaps a second apart. I start poking around the system and it's a
scripted attack, attempting to log into my system:
Jan 1 13:57:50 linus sshd[24760]: Failed password for root from XXX.XXX.XXX.198 port 43440 ssh2 Jan 1 13:58:05 linus sshd[24784]: Failed password for nobody from XXX.XXX.XXX.198 port 45220 ssh2 Jan 1 13:58:06 linus sshd[24786]: Failed password for root from XXX.XXX.XXX.198 port 45351 ssh2 Jan 1 13:58:14 linus sshd[24799]: Failed password for www from XXX.XXX.XXX.198 port 46207 ssh2 Jan 1 13:58:27 linus sshd[24819]: Failed password for news from XXX.XXX.XXX.198 port 47673 ssh2 Jan 1 13:58:29 linus sshd[24823]: Failed password for games from XXX.XXX.XXX.198 port 47977 ssh2 Jan 1 13:58:33 linus sshd[24829]: Failed password for mail from XXX.XXX.XXX.198 port 48413 ssh2 Jan 1 13:58:34 linus sshd[24831]: Failed password for adm from XXX.XXX.XXX.198 port 48563 ssh2 Jan 1 13:59:03 linus sshd[24877]: Failed password for operator from XXX.XXX.XXX.198 port 51355 ssh2 Jan 1 13:59:12 linus sshd[24891]: Failed password for sshd from XXX.XXX.XXX.198 port 51947 ssh2
Hundreds of attempts.
Now, I could block them on the sever:
GenericUnitRootPrompt# route add -host XXX.XXX.XXX.198 reject
But that would only block them from my one server. And besides, there's this firewall between the internet and the DSL router at the office. Cut off the traffic a bit upstream so they can't scan any of the DSL connections:
GenericUnixRootPrompt# iptables --table filter --insert FORWARD --source XXX.XXX.XXX.198 --jump REJECT
And boom, the traffic stops dead.
For a few hours.
“Click click click click”
Yet another brute force attempt to log into my server. Previous one was from Germany. This one from the Netherlands. Another one a few hours later from Korea. One from Lakeland, Florida!
I ended up stopping eight of these attempts over the weekend—not because my system was particularly vulnerable, but the “click click click” of the disk drive was rather annoying (I should note that the drive itself is not going bad—it's just the sounds the heads make when seeking across the platters).
“Click click click click”
In fact, there's an attempt right now going on as I type this, from Korea— or rather, there was an attack.
And the servers at The Office get scanned constantly. I should know, I get the multithousand line log summaries daily.
So with all that in mind, I'm trying to cobble up a LeBrea Tarpit system at The Office. Legal disclaimers on the site aside, it's still available and in checking the rather loathsome Florida's Super DMCA Act, I think legally we can run it—we're not a “cable operator” (¶ (1)(a) §1 §812.15, F.S.), nor are we a “communications service provider” under the definition given (¶ (1)(e) §1 §812.15, F.S.), nor do we provide a “communications service” under the given definition (¶ (1)(d) §1 §812.15, F.S.). And even if we are classified as a “cable operator” (and really, the definition screams “television broadcasting”), then it falls under ¶ (2)(a) §1 §812.15, F.S.:
(2)(a) A person may not knowingly intercept, receive, decrypt, disrupt, transmit, retransmit, or acquire access to any communications service without the express authorization of the cable operator or other communications service provider, as stated in a contract or otherwise, with the intent to defraud the cable operator or communications service provider, or to knowingly assist others in doing those acts with the intent to defraud the cable operator or other communications provider.
See … we are the communications service provider in that case. We're not out to defraud ourselves! We gave ourselves permission!
And it's not entrapment. LeBrea isn't simulated a poorly administered
system just waiting to be hacked. It just accepts TCP connections and keeps them on hold (as it were)
indefinitely. It's not violating the letter of the TCP specification (it could be argued it's
violating the spirit of the TCP specification, but really, what jury will want to spend
time listening to geeks pontificate about the finer points of dogmatic
religious theology TCP connection semantics?
Tuesday, January 03, 2006
“Little orange houses, for you and me … ”
For Spring, the Orange Houses of Detroit (link via dETROITfUNK).
Yeah … I'm walking kind of funny now …
Due to Hurricane Wilma, then the holiday season, we ended up being just a wee bit behind in our electrical bill. In discussing this last night, it was decided that I would pay the outstanding amount on the way to work today. There's a gas station about a mile or so that one can pay their utility bills. I was expecting this to take only a few minutes or so.
I arrive at said gas station and I'm the only customer. I hand over the bill to the cashier who starts running it through some device (to read the account and amount off the numbers written in magnetic ink along the bottom), then punching lots of buttons, then running it through again, then punching numbers, then marking up the bill with incompresensible hieroglyphics, running it through again, then violently jabbing a pin-infested cloth doll; pins are falling out and bouncing off the counter top. Minutes go by, and the cashier finally turns to me.
“I'm sorry,” the cashier said. “But the bill is being rejected.”
“Rejected?” Oh XXXX, I thought.
They've already cut our account and are releasing the hounds
thugs collection agents as I stand here.
“Yes, we always have problems with Lake Worth Utilities,” said the cashier, glancing towards the cloth doll. “No problems with any other company. But Lake Worth Utilties.”
“Lake Worthless Utilties?” I said.
“Pretty much.”
“Ah.”
“You can pay at the City Hall. Just down Lake Worth Avenue. Down by Dixie Highway.”
“Which side of the street?”
“North side. You can't miss it.”
“Thanks.”
Couldn't miss it because the traffic lined up down the block in front of the Lake Worth City Hall Annex, waiting paitently to pay their bills in the drive-through.
Not wanting to bother with that, I park across the street, and head into the Lake Worth Cith Hall Annex building.
I don't know that it is about government buildings, but every single one is painted in Institutional Paint. It's not a single particular shade of color—sometimes a pale green, other times a jaundiced yellow, or a pale flesh tone—but the shades themselves are never bright, always dull, like it's been there forever, and will remain there forever, eternally dingy. The Lake Worth Cith Hall Annex is a pale flesh color (maybe even a sickly peach color) headed towards a dull beige.
Soul sucking, is what it is.
There's a main hallway that shoots through the Lake Worth City Hall Annex building, and lined up down this hall is a line of people, waiting to get into the Utilties office in the center of the building. A real life Kurgan walks down the hall, deciding he'll come back on Thursday. A cute girl ahead of me in line is wearing a shirt, “I may not be perfect, but parts of me are AWSOME” and yes, parts of her are awsome. At least it keeps my mind of the numbing wait in line.
Half an hour later, I'm actually in the Utilties office, still waiting in line. The cashier window is on the other side of the room where a single person is working. I assume there is at least one other person working the drive though, since I don't see the cashier leave the desk.
Half an hour later, I'm the next in line. At this point, I see they take credit cards, but just past, far enough that it's hard to read is a sheet of paper with the word “VeriCheck” written across the top. The sign nearby that shows that Lake Worth Utilties accepts credit cards also has the word “VeriCheck” across it. I step over to the sheet of paper, keeping my leg stretched out with my toe marking my spot in line, and scan the sheet of paper.
It's a list of fees.
It's a list of fees one will pay for the “convenience” of paying your grossly inflated Lake Worth Utility bill with your credit card (or in my case, a debit card).
What a XXXXXXX racket, I think. The “fee” to “conveniently” pay with your “credit card” is a “measly” 5%.
XXXXXXX XXXXXXXX!
“Do you understand that there is a fee if you pay with your credit card,” said the cashier when it was my turn to pay.
“Not like I have any other choice,” I said. Notice how the signs giving the fee structure was posted next to the cashier, and not, say, out in the hall or near the door, when I might have been able to do something about it. Now, after an hour?
The cashier then mentions the fee. “Are you sure you want to continue paying?”
Not really, but then you're less inclined to continue our electrical supply, so here, let me bend over, I thought. What I said was, “Yes. No other choice really.”
“Splendid,” said the cashier, taking my bill and debit card.
Splendid indeed.
Now, I don't see this as a fault of capitalism. No, this is a fault of a government mandated monopoly that is nothing less than coercion. FPL, for all their faults, is at least working with economies of scale and are able to charge about half that of Lake Worth Utilties.
Although on my way out, I did warn a few people about the fees awaiting them, if they so choose to use plastic to pay. At least one guy started on about taxation without representation, so maybe there's hope yet …
Peekaboo! Oh … you're gone!
I have what may be some bad news for Mark. The club he likes (heck, I liked it too!), the one on Lake Worth Avenue near Congress, is no more. The building has been torn down, and there's new construction on the site (as I found out while driving around today). Given what I saw of the new building, unless the club is going to be two stories (which I doubt), the club is no more.
Sorry Mark.
Rant about my cell phone, Part I
As I was standing around in line, I was kicking myself for not having a pen, so I could make a few notes for the inevitable blog entry. Then I suddenly remembered: I have a cell phone! With a note pad on it!
Sure, it sucks having to type words using a numeric keypad, but it has enough programming to realize that when I hit 4475 it's most likely because I want to type out “girl” than “ggpj” (otherwise, I'd have to type 4 pause 444777555), but the “memo” feature only accepts about 40 cha
Rant about my cell phone, Part II
racters before I have to start a n
Rant about my cell phone, Part III
Stupid software limitations.
Kinda says it all, doesn't it?
Guys, you keep posting that same story about a serious security flaw in Windows.
Windows XP Flaw ‘Extremely Serious’
An anonymous quote at Slashdot about the bazillionth security hole in Windows wherein an image—about as inert as binary data can be—is now a vector for executable code to take over your system.
At this point, can't Microsoft be taken down (as in “corporate death penalty”) as either being completely malicious or completely incompetent? I mean, seriously—infected images?
Sheesh!
Wednesday, January 04, 2006
I just love helpful software
Offered without comment.
Y:\>net share trickout=y:\testfolder The resource named cannot be shared. More help is available by typing NET HELPMSG 3736. Y:\>net helpmsg 3736 The resource named cannot be shared. Y:\>
Slowly working through a tarpit
Since we have this endless supply of Cobalt RaQs at The Office, I used one to set up a LaBrea Tarpit system. Playing around with it I noticed that LaBrea doesn't stop port scans per se, since port scanners tend to do the TCP handshake then drop the connection, whereas LeBrea will put a connection on hold indefinitely (assuming the other side keeps the connection up). It will also (as I ran it) accept connections on every single TCP port, all 65,536 of them, which is something that doesn't happen on a real server, so I may have to limit the number of ports LaBrea responds to.
Remotely recursive
While installing Linux on a new 1U server for a customer, I managed to play around
with our new MegaRAC®
K1, a rather neat KVM that works over TCP/IP. While it can be
hooked up to a Linux system, the remote display software only works under
Windows (of course, and IE—it
demands IE). Annoying, but not
too much of an issue, since I do have rdesktop
installed on my Linux
workstation at the Office.
So just because I can, at home I logged into my workstation at The
Office. Since X Windows
has had a remote desktop feature since it was first written way back in the
mid-80s, it's easy enough for me to run rdesktop
there and have
it displayed on my computer at home (the Mac mini). Then, logging into the
Windows system at The Office, I'm able to run the MegaRAC® K1 program
to get to the console of the new server.
Just because I can (and yes, it's as slow as it sounds).
Thursday, January 05, 2006
Stupid PC tricks
I had so much fun with the MegaRAC® K1 that I
hooked the device up to my workstation at The Office (and you can hook a
keyboard, video and mouse up to it, and still remote desktop into
the system), then from my system used rdesktop
to get to the Windows
box, from which I remote desktopped (um … yeah) back into my
workstation.
The effect was much like pointing a video camera to a TV displaying what the video camera sees.
Just because I can.
Slogging through the 'pit
Today was the first day I ran the Labrea tarpit on the network. I almost didn't leave the office since the results were most interesting. The first hour it ran (from 19:04:31 Eastern to 20:04:31) it “pitted” 9,309 connections from 865 unique IPs. And the ports involved:
Port # | Port description | # connections |
---|---|---|
135 | Microsoft-RPC service | 4,996 |
445 | Microsoft-DS Service | 3,724 |
139 | NetBIOS Session Service | 295 |
22 | Secure Shell Login | 231 |
80 | Hypertext Transport Protocol | 62 |
6348 | unassigned (possible worm?) | 1 |
That Microsoft specific ports are at the top of the list are totally unexpected here.
I did learn a few things about LaBrea though. One, it only works on a single netblock. Unfortunately for us at The Company, we have several network blocks to worry about and that means either a few machines running this, or several instances (and given that LaBrea puts the network interface in promiscuous mode, I'm not sure how multiple instances would react with each other on the same interface) on different interfaces on one box.
Two, the network block does not have to match the network block the actual system is in, which saves an unsused IP address (ha ha).
A puzzling thing though. I got home, it was still running. I checked back a few hours later, and nothing past 20:21:17. LaBrea was still running, but either we captured all that could be captured, or something else was up.
Or down, as it turned out.
The interface that LaBrea was running on just died. I don't know if the switch doesn't like it (unlikely), the network cable is bad (could be—I did make the cable) or the interface just blew up (also a possibility). Even a reboot of the system didn't fix the problem. I'm hoping it's just the cable.
Packets from Mars
Another thing I'm seeing with the LaBrea Tarpit software—wierd ARP packets.
I've got the logging set to “verbose” (in case any problems come up) and one result is that it logs all the ARP requests that it sees, but can't answer (can't answer them because they're outside the normal netblock LaBrea is “answering” for). So I see a bunch for other public IP addresses we have. And the various private IP addresses that I'm not exactly sure what they're being used for, but hey, they're private.
And then … there are the ARP requests that shouldn't exist:
Jan 6 00:55:40 ltp : IP address not in netblock - ARP WHO-HAS 64.76.67.1 TELL 64.76.67.64 Jan 6 00:55:42 ltp : IP address not in netblock - ARP WHO-HAS 64.76.67.1 TELL 64.76.67.248 Jan 6 00:55:43 ltp : IP address not in netblock - ARP WHO-HAS 64.76.67.254 TELL 64.76.67.236 Jan 6 00:55:43 ltp : IP address not in netblock - ARP WHO-HAS 64.76.67.1 TELL 64.76.67.248 Jan 6 00:55:44 ltp : IP address not in netblock - ARP WHO-HAS 64.76.67.1 TELL 66.252.226.46 Jan 6 00:55:44 ltp : IP address not in netblock - ARP WHO-HAS 64.76.67.254 TELL 64.76.67.236 Jan 6 00:55:44 ltp : IP address not in netblock - ARP WHO-HAS 64.76.67.1 TELL 64.76.67.248 Jan 6 00:55:45 ltp : IP address not in netblock - ARP WHO-HAS 64.76.67.254 TELL 64.76.67.236 Jan 6 00:55:47 ltp : IP address not in netblock - ARP WHO-HAS 64.76.67.1 TELL 64.76.67.248
I have no clue where these are coming from.
None, and I mean none of our addresses are from the
64.0.0.0/8
block. That doesn't appear to be any of our
upstream providers. I can traceroute to 64.76.67.248
but why I'm seeing ARP requests from it is beyond me. Especially since
ARP requests
can't be routed! It's something local sending it out, or it's
LaBrea misinterpreting the ARP packet (since it can be used for more than just
IP→MAC address translations).
One thing for sure, I'll have to check the code, and possibly modify LaBrea to log the MAC address so I can track this down. Then again, I have to hit the code anyway, since technically, the MAC address LaBrea uses is wrong (the “global/local” bit in the address is not set, and it should be).
Friday, January 06, 2006
More tarpit stuff
The problem ended up being the cable; nothing a little recrimping couldn't fix.
I did however, run LaBrea on the working port last night, and have a full twelve hours of data, from 00:00:00 (Eastern) to 11:59:59, and the results are rather amusing. 55,331 port connections on hold, from 1,743 unique IP addresses. And the only surprising thing is the low number of scans for SMTP.
Port # | Port description | # connections |
---|---|---|
135 | Microsoft-RPC service | 30,218 |
445 | Microsoft-DS Service | 11,813 |
139 | NetBIOS Session Service | 5,934 |
4899 | Remote Administration | 2,412 |
80 | Hypertext Transport Protocol | 1,692 |
22 | Secure Shell Login | 1,190 |
6129 | Dameware remote administration software | 486 |
1080 | W32.Mydoom.F@mm worm | 404 |
2100 | Oracle XDB FTP Services | 377 |
4444 | W32.Blaster.Worm | 372 |
1433 | Microsoft SQL Server | 258 |
15118 | Dipnet/Oddbob Worm | 140 |
5000 | Microsoft Universal Plug-n-Play | 13 |
2745 | Bagle/Beagle/Tanx viruses | 10 |
25 | Simple Mail Transport Protocol | 7 |
47707 | unknown | 5 |
And it seems, from these results, that simply blocking the ports used by
Microsoft Windows will stop 87% of these scans (and for our particular run,
if I just blocked 216.82.207.49
I would have stopped 35% of all
the scans—that was a particularly persistent computer).
Update on Saturday, January 7th, 2006
I may not have been properly tarpitting the connections.
Saturday, January 07, 2006
Some updates on a tarpit
I played with LaBrea a lot yesterday and today. One of the things I found was that I may have not been actually “tarpitting” any connections on Thursday and early Friday. I had set the maximum bandwidth that LaBrea could use to 4 Kbps and that may have been too low to do any good. I upped the bandwidth to 32 Kbps and that's when I started seeing “Persist Activity” records in the log file. It looks like the bandwidth capped out, so since then I've upped the allowable bandwidth to 64 Kbps.
Another annoying aspect to LaBrea—it's not easy to change the MAC address it uses for tarpitting. As I mentioned, the MAC address it uses is technically incorrect, as it uses a “Global” address, not a “locally assigned” address. It also makes it difficult to run more than one instance of LaBrea on a network (and we may need to do so on our network until I can get all the traffic segmented properly).
I came up with an address to use, DE:CA:FB:AD:00:01
(yes,
that spells “decaf bad”) which is distinctive enough to see in ARP tables. I found the line
of code to change in labrea.h:100
#define ETH_ADDR_BOGUS "\x00\x00\x0f\xff\xff\xff" /* Bogus MAC addr used in IP capture */
Simple enough to change. But doing so made LaBrea unable to actually
tarpit any connections. Changing the MAC address back to 00:00:0F:FF:FF:FF
and it
would work. Which means that LaBrea has hardcoded elsewhere in the
code.
Found it too, in labrea_init.c:248
static char bpf[] = "arp or (ip and ether dst host 00:00:0F:FF:FF:FF)";
Believe me, if there was an option to select the MAC address to use for tarpitting, I would have used it. So I suppose I can now add such a feature, now that I know where to look for it.
So, now that I'm properly tarpitting connections, I've pulled stats from another twelve hour run, which should be more accurate than my last chart. With the additional information, I've been able to calculate the time each connection is “held” and again, it's interesting. This time, the twelve hours resulted in a tarpitting of 27,427 connections across 1,418 unique IPs with an average “hold time” of 48 minutes, 21 seconds (although there were 302 connections held for over 11 hours, presumedly they still are). Again, the ports being scanned hold nothing surprising, with this time 63% of the scans for Microsoft specific ports:
Port # | Port description | # connections |
---|---|---|
Port # | Port description | # connections |
139 | NetBIOS Session Service | 7127 |
1433 | Microsoft SQL Server | 4605 |
80 | Hypertext Transfer Protocol | 4425 |
445 | Microsoft-DS Service | 4361 |
135 | Microsoft-RPC service | 1290 |
3306 | MySQL Server | 1157 |
443 | Hypertext Transfer Protocol over SSL | 978 |
3840 | unknown | 915 |
22 | Secure Shell Login | 892 |
3380 | unknown | 455 |
8080 | Hypertext Transfer Protocol—typical alternative port | 403 |
15118 | Dipnet/Oddbob Worm | 361 |
4899 | Remote Administration | 249 |
1080 | W32.Mydoom.F@mm worm | 79 |
25 | Simple Mail Transfer Protocol | 64 |
21 | File Transfer Protocol command port | 24 |
57 | Mail Transfer Protocol (obsolete) | 18 |
6885 | Bittorrent | 6 |
6129 | Dameware remote administration software | 5 |
34153 | client-port on Red Hat Linux 9.0, Fedora Core 1, Red Hat Enterprise 3 | 4 |
1025 | Microsoft-RPC service (what? Another one?) | 3 |
18448 | unknown | 2 |
524 | Novell Core Protocol request | 1 |
20298 | unknown | 1 |
2008 | Microsoft terminaldb configuration | 1 |
18000 | unknown | 1 |
I may need to write a program to process the logging in real time, since LaBrea is quite versbose, as it looks like it'll generate about 150M per day of logging information.
Sunday, January 08, 2006
Elvis has left the planet
Last month I wrote about the Weekly World News and how I was sad to see it have a disclaimer about the fictious nature of its articles. And today, another thought hit me—one thing you no longer see in taboids are Elvis sightings (this being his 71st birthday and all). Has the tabloid world finally realized that Elvis “The King” Presley has not only left the building, but actually left this plane of existence? That scores of fans and New Age cranks come to grips that Elvis will no longer haunt midwest grocery stores and fried chicken fast food restaurants?
Say it ain't so!
Monday, January 09, 2006
The user expectation is on fire
A recurring theme here is one of user expectation. Or I should say, the expectation I have as an old school Unix administrator (that doesn't really care for Unix administration but can do it) and the current state of the art in Unix—or rather, Linux—administration.
Today's project was a consolidation of DNS—moving all the zone files to a central server and have that one serve the zones to the actual register name servers. That way, we have one location to make changes and from there they are propagaged out. Nothing too wild.
But it's not working. The zone transfers are failing. Big time. I'm reading the Cricket Book and I'm close to really mucking with the configuration files when this little voice in the back of my head goes Hey, check to see if there's a firewall running on the server. See, user expectation—I'm not used to Unix servers running firewall software as a default. I'm not running Windows for crying out loud.
And thankfully, it turns out to be a firewall issue. Otherwise I thought I was going crazy.
Tuesday, January 10, 2006
Monday
To whom it may concern:
It's Tuesday!
This is not Monday!
Tuesday!
Repeat after me: TUESDAY!
If ever there's a day when both DNS servers fail, it's impossible to find a working serial port on a computer, my DSL connection fails, the A/C at The Office isn't functioning properly and the power cord falls out of my workstation for no good reason, it's MONDAY!
Not TUESDAY!
Please try to schedule any Hell Days for Monday.
Thank you.
What a fluke
From: Sean Conner <sean@conman.org>
To: Bunny XXXXXXXX <XXXXXXXXXXXXXXXXXXXXXX>
Subject: Re: The cable
Date: Sun, 8 Jan 2006 05:35:27 -0500 (EST)It was thus said that the Great Bunny XXXXXXXX once stated:
Nice save on that cable. Very intuitive. Most of us would have looked for zebras. I recently changed out an outdoor light fixture (see attached), and when it didn't work, I almost didn't think to try a new lightbulb …
Helps to have a $5,000 network analyzer that can be used to test cables (even tell how long they are!).
Bunny and I (and yes, Bunny is her real name) have been exchanging emails about the network cable I recrimped. She seemed impressed that I would admit to making such a mistake, but hey, as mistakes go, it beats adding an external SCSI drive to your primary business computer without turning it off (a mistake an old boss of mine did).
So I figured she might get a kick out of what a $5,000 network analyzer looks like. And I think it can do more than just measure the continuity and length of network cables, but darned if I've used it for more than that.
Wednesday, January 11, 2006
Cliff hanger
As if yesterday weren't bad enough, today I come home to find The Younger offering much sacrafice to the Great Porcelain God, due to stuff he may have ingested. Wlofie had called Poison Control and while what he ingested normally wouldn't be a problem, since he was making much offerings to the Porcelain God and complaining about abdominal pains, it was suggested we head off to The Emergency Room!
We arrived just before 8:00 pm, with Spring flying to The Emergency Room from work. She arrived around 8:30, and shortly thereafter (after yet another offering to the Great Porcelain God) The Younger, accompanied by Spring, went into the bowels of the hospital.
At 1:30 am (yes, I am pre-dating this, as most of this happened Wednesday, and had there been wireless at The Emergency Room, I would have posted, but alas, there was none) Spring called, saying that The Younger's X-ray was inconclusive and next up was a CAT scan, and that the rest of us should probably head on back home as there was no telling how long The Younger and Spring would be there.
And that, alas, is all the information I have at this time.
So … how was your day?
Update on Thursday, January 12th, 2006 at 3:30 am
Doctors think it's a stomach flu, although why it took a CAT scan to figure that out, I don't know.
Thursday, January 12, 2006
Cliff hanger, Part II
Today was spent at The Hospital.
Not The Hospital whose Emergency Room I dropped The Younger off at. No. That Hospital does not have a pediatric section, so The Younger was transported to a different Hospital during the wee morning hours.
Now, since Spring has already mentioned one reason why he's there, I no longer have to dance about the issue of The Younger swallowing two kitchen magnets.
Magnets.
Yes, plural.
Now, as kitchen magnets go, these are tiny. Think of a push pin, one used to tack notices up to a bulletin board, but instead of a metal pin sticking out of it, it has a small round magnet to hold it to a refridgerator. We're talking something about 1/8″–1/4″ across.
And he only swallowed the magnet part, not the push pin part. Just the magnet part.
It was suggested by the doctors last night that The Younger might have pica which is a persistent craving for non-food like items.
Oh, and he also got a stomach flu, as if things weren't bad enough.
So anyway, today was spent at The Hospital.
The Younger was sent through X-ray enough times that he no longer needs a nightlight and the reason for the multiple X-rays throughout the day was to track the movement of the two magnets. The concern was if the two magnets attract each other in different parts of the intestines and lodge themselves against the intestine wall. Fortunately, the X-rays showed that not only where they progressing down his system, but they had already attracted to each other and travelling as one unit if I understood it correctly.
So, once the magnets are flushed from his system, The Younger can then return back home.
The Great Egress Incident
Since The Younger and Spring were transported to The Hospital from The Emergency Room via ambulance, Spring's car was still at The Emergency Room. I worked out a plan whereby Smirk would pick me up at The Hospital, drive me to The Emergency Room, whereby I would drive Spring's car back to the Hospital, then I would drive back home to get some sleep so I could go to work tomorrow.
Around 11:30 pm or so, Smirk calls me on my cell phone to say he's outside the main entrance to The Hospital. The main entrance to The Hospital is this large revolving door (it has motion detectors, and will start moving automatically). I nearly smack into the thing trying to walk outside.
It's not moving.
I try pushing it on the sign that says “Don't Push” but it won't budge. In fine print, the sign mentions something about the handicapped doors to either side of the revolving door. So I pick the one to the right, and try to go out.
It doesn't budge.
Nearby is a large flat disk with a stylized wheelchair and the words “Open Door” on it (or something to that effect—I don't recall exactly). It's made to be pressed, and I press it. I hear this light buzzing noise, only the door isn't opening. I try pushing the handicapped door, but it still doesn't budge.
Neither does the one to the left of the revolving door.
Through the windows, I can see Smirk's vehicle.
I try all three egresses again, pushing and shoving and otherwise trying to get to the other side of the doors.
Then I call Smirk on the cell phone, to inform him that I'm having some difficulty in leaving The Hospital. Through his laughter I'm told that he's seen my difficulty and suggests meeting him outside this Hospital's Emergency Room.
Friday, January 13, 2006
Monday, Part II
As if Tuesday wasn't bad enough (so was Wednesday and Thursday but that dealt with things outside of work), I arrive at The Office to one machine spewing out a sustained 20Mbps of traffic, and another one not responding at all.
For the one vomitting over the network (a customer's colocated box), I logged in, and saw the following errors:
eth0: Transmit error, Tx status register 82. Probably a duplex mismatch. See Documentation/networking/vortex.txt Flags; bus-master 1, dirty -1756485798(10) current -1756485798(10) Transmit list 00000000 vs. f7ed8480. 0: @f7ed8200 length 80000042 status 00010042 1: @f7ed8240 length 8000002e status 0001002e 2: @f7ed8280 length 800005d6 status 000105d6 3: @f7ed82c0 length 80000548 status 00010548 4: @f7ed8300 length 80000036 status 00010036 5: @f7ed8340 length 8000003e status 0001003e 6: @f7ed8380 length 80000042 status 00010042 7: @f7ed83c0 length 800005ea status 000105ea 8: @f7ed8400 length 800005ea status 000105ea 9: @f7ed8440 length 8000004e status 8001004e 10: @f7ed8480 length 800005d6 status 000105d6 11: @f7ed84c0 length 800005d6 status 000105d6 12: @f7ed8500 length 800005d6 status 000105d6 13: @f7ed8540 length 800005d6 status 000105d6 14: @f7ed8580 length 800005d6 status 000105d6 15: @f7ed85c0 length 80000042 status 00010042
Over and over again, for days—ever since it was rebooted last week. So
I check Documentation/networking/vortex.txt
:
Transmit error, Tx status register 82
This is a common error which is almost always caused by another host on the same network being in full-duplex mode, while this host is in half-duplex mode. You need to find that other host and make it run in half-duplex mode or fix this host to run in full-duplex mode.
As a last resort, you can force the 3c59x driver into full-duplex mode with
options 3c59x full_duplex=1
but this has to be viewed as a workaround for broken network gear and should only really be used for equipment which cannot autonegotiate.
I know Dan the network engineer prefers not to have auto negotiation as
he's seen problems with it, so I call him up, and ask him to set the switch
port the machine is plugged into as 100Mbps auto-negotiate
. Once he did that:
eth0: Setting full-duplex based on MII #24 link partner capability of 4101.
And the traffic immediately cleared up (I suspect that when the box was first plugged in, the switch port was set to auto-negotiate, but some time afterwards, Dan set the switch port not to negotiate, but since the negotiation already happened, no errors were reported. It was only after the box was rebooted last week that the problem cropped up).
The second box was also a colocated machine, and the customer there just told me to power cycle it. Instead of just powercycling the machine, I hooked the crash cart up to it, just to see if any error messages were on the screen.
Only the screen was blank.
Couldn't get it to unblank at all.
The machine must be hosed, so I power cycled it.
And that's when I saw the other video interface.
Sigh.
The rest of the day was pretty much like that—just as I clear one problem, another would pop up. All day I was busy with problems.
It was like Monday Tuesday all over again …
More updates on the tarpit
Labrea is actually logging about half a gig a day. Over a 24 hour period (from about 6 am Thursday to 6 am today) I'm tarpitting 82,359 connections across 2,059 unique IP addresses (24,252 connections from a single IP address). And while the number of network ports being accessed has increased a bit, it's the Microsoft specific ports that are still the most popular targets (with 72% of the scans):
Port # | Port description | # connections |
---|---|---|
Port # | Port description | # connections |
139 | NetBIOS Session Service | 24941 |
445 | Microsoft-DS Service | 23013 |
1433 | Microsoft SQL Server | 6772 |
4899 | Remote Administration | 5620 |
135 | Microsoft-RPC service | 4722 |
80 | Hypertext Transfer Protocol | 3697 |
8080 | Hypertext Transfer Protocol—typical alternative port | 1686 |
7212 | (unknown) | 1683 |
8000 | (unknown) | 1471 |
10000 | (some web based control panels use this port) | 951 |
The program I'm using to generate the stats is written in Perl, and it took about 4 hours to run over a day's worth of data (the machine that does the tarpitting isn't the fastest machine we have, but it's more than enough to dedicate to just running LaBrea). I definitely want to write a program to process LaBrea data in real time.
Saturday, January 14, 2006
The passing of stuff
I'm at the hospital, which unfortunately does not have Internet access (which sucks, and it's obvious that I'm posting this some days after I wrote this).
Good news for The Younger: he seems to have gotten over his stomach flu.
Bad news for The Younger: the magnets have been stuck somewhere in the large intestines for the past 48 hours. And he won't be released until the magnets are released.
Sigh.
Processing realtime data from LaBrea
Since I'm stuck at The Hospital, and The Hospital doesn't have wireless, I might as well work on the real-time LaBrea data processing program. Loaded up the laptop with program fragments and libraries I may need and brought along my copy of Advanced Programming in the Unix® Environment by W. Richard Stevens, and am coding up a storm.
And yes, I'm coding this up in C. Not like I'm going anywhere in the next couple of days.
LaBrea spits out lines like:
1136832897 Initial Connect - tarpitting: 82.240.204.251 3334 -> XXX.XXX.XXX.XXX * 1136832897 Initial Connect - tarpitting: 82.240.204.251 3339 -> XXX.XXX.XXX.XXX 139 1136832897 Persist Trapping: 82.240.204.251 3334 -> XXX.XXX.XXX.XXX 445 * 1136832898 Persist Activity: 216.248.36.242 45285 -> XXX.XXX.XXX.XXX 135 1136832898 Persist Activity: 216.248.36.242 34589 -> XXX.XXX.XXX.XXX 135 * 1136832898 Persist Activity: 216.211.61.158 3862 -> XXX.XXX.XXX.XXX 135 1136832898 Persist Activity: 211.236.205.138 4459 -> XXX.XXX.XXX.XXX 139 *
There's some other stuff that's logged, but I'm primarily looking for lines like the above. Which means the information I can store will look something like:
struct tprecord { IP src; Port sport; IP dest; Port dport; time_t start; /* first time we see src:sport-dest:dport */ time_t last; /* time of last activity of src:sport-dest:dport */ size_t packets; };
Now, since we'll be handling a lot of connections, we need to store these
structures in a way that is easy to search through. Could use a hash table,
and we are keying off the src:sport→dest:dport
tuple, and
that's 10 bytes of pretty much random binary data, which is good for this
type of thing. But it's an open ended hash, not knowing how many entries
we'll be handling. There are several methods to handle overflows in a hash
table, but that still means at some point doing a linear search, and then
there's the overhead of having a hash table.
There's also storing the data in a tree. In fact, I even have code in C to manage a balanced tree (taken from Knuth). But the downside is—I only have code to add to the tree, not to delete from it. And it was hard enough to write as is.
And there's still the overhead of maintaining a tree structure.
So I'm using arrays. Less overhead and straightforward to manage. The only trick is keeping the array sorted in some order, but as long as you compare the structures in a consistent manner, that's not really an issue (and by the way, I'm sorting by source address, port, destination address and port, in that order).
To make it even easier, I have an array of structures (see above), and an array of pointers to said structures, and it's the pointer based array that is sorted (since it's faster to swap pointers than swap whole structures). And once sorted, you can use a binary search to find the record.
The default binary search in C, bsearch()
, returns either a
pointer to the found record, or NULL
if not found. Good for
most uses, but not quite what I want. In addition to either finding the
record or not, I'd also like to know where in the array the record
was found, and if not, where it would appear if it was in the
array. And C's bsearch()
does not return the index.
Why do I want that?
My thinking is that if the search (reguardless of what type of search it is) didn't find the record in a sorted array, it has the information as to where the missing record should be. And with the information as to where, it would be “trivial” to add the record at the right point in the array.
And that beats adding the new record at the end, and restorting (C's
qsort()
is Quick Sort, which on average is O(n lg
n)
running time, but the worse case is O(n2)
and the worst case is then the array is mostly sorted to begin with, which
is why I'd rather not resort after each addition).
And try as I might, I could not get a binary search to work that would also return the index. For now, I replaced the search with a linear search, until I can comb through some reference materials at home.
The other problem I'm having is the array manipulation code doesn't quite work. When the program starts, it has space to store X records. If it runs out of space, it increases the size of the various arrays until it hits some large number of records, then it will go through, purge records fitting some criteria.
Only it's not working properly.
Got a lot of work done on the program though, and it was nice to get into The Zone (even with a headache). At this point, I'm headed home for sleep and what looks to be yet another day at The Hospital.
Sunday, January 15, 2006
And this too, will pass … the only question is when!
Back at The Hospital. The Younger is over definitely over his stomach flu, but the magnets are still stuck.
Normally on Sundays I run a D&D game for the family and The Kids were looking forward to playing this week, but frankly, I don't want to run a session at The Hospital. For one thing, I have a pile of reference material I use during a session and there isn't a good enough surface for me to use. And the smoke breaks (for both Spring and Wlofie smoke) will be longer (since they have to head outside The Hospital instead of just outside Casa New Jersey), leading to less actual gaming time.
Second, I'm still in a programming head space and don't want to loose being in The Zone.
Third, Spring fell asleep as soon as I got there, and Wlofie was home sleeping, and that's half the party. I'd rather not run a session with that many missing party members.
In an effort to get the magnets moving, The Younger is being forced to eat (not only by us, but through The Hospital staff) even though he complains of stomach pains.
I have this feeling that it's going to be a long hospital stay.
Some notes on a binary search implementation
You would not believe how hard it was to write a binary search that returned the correct index for a missing record in an array.
Even with Programming Pearls by Jon Bentley (reference book of the day).
Binary search is a deceptively simple algorithm that is easy to get wrong (be especially careful when searching empty arrays). And of the binary search implementations I've seen, when they do return the index in the array, it's to an element that's actually there, and some other value (like 0 or -1) if the element isn't there. I've yet to see an implementation that returns an index reguardless (plus an indication if the element was found or not).
So I spent several hours getting a binary search to return an index even if the element wasn't found. Before looking at my solution you should at least try coding it (I should note that the code is pulled from the real-time LaBrea data processing program I'm writing, which just enough changes to get a single file to compile and with as little change to the source code as possible, so some of the names may not make that much sense, but the logic should be clear).
Squishing bugs
I've been simplifying the real-time LaBrea data processing program. One simplification: don't try to grow the arrays. If I'm trying to grow the arrays and I'm going to stop at some point anyway, why not just allocate as much as I'll ever use when the program starts? It gives you a definite upper bound (and one that can be set from the command line) and boy, does it simplify the code.
I also got the binary search routine working correctly after a few hours.
And I had to remind myself just how C's qsort()
works, and make sure I know that (in my program at least) I'm getting
pointers to pointers to structures, and not just pointers to structures
(when I fixed that, I started getting correctly sorted arrays).
The only outstanding bug I have right now deals with adding a record after the array has been purged of old records. The problem is that I'm using an index from before the purge. I should be able to get this running on the LaBrea tarpit system in the next day or two.
Monday, January 16, 2006
Sexism is alive and well even today
The Younger has yet to pass the magnets, so he's still in The Hospital.
Sigh.
And so far, despite repeated corrections, most of The Hospital staff still refer to The Kids as “girls.” Okay, understandable given the shoulder length hair both have, and slender bodies.
But repeatedly?
“So, is she with you?”
“Yes, he is.”
“Should I give her a badge?”
“Yes, you can give him a badge.”
“Here you go little girl.”
“Aaaaaaaaaaaaarg!”
(Not to mention that one staff member not only though The Younger was a girl, but still wore diapers! I mean, yes, we're waiting for an event from down there, but The Younger can hit the bathroom for that. Sheesh!)
Didn't we learn that long hair isn't an indication of gender during the 70s?
It's enough to make me want to put my head in my hands and sob uncontrollably.
Stupidity is alive and well today too!
On the way to work (after dropping off Wlofie and The Older at The Hospital), I stopped off at a gas station to fill up Lake Lumina. There are two pumps next to each other, both free. I'm trying to negotiate my way there when an SUV cuts in. But does the soccer mom driving said SUV pull all the way forward?
No, of course not.
She pulls up to the first pump, and stops.
Would it hurt her to pull up to the next pump, free of any vehicular obstruction and fill up there?
Yes, of course it would.
So I have to drive around this bree munching Yuppie soccer mom and circle around to some other free pump.
Then, when said bree munching Yuppie soccer mom finished filling up her SUV (I didn't actually see her get out of the vehicle and pump any gas, although I did see her talking on a cell phone), does she pull forward to circle around the lot to leave?
No, of course not.
Instead, she backs up and tries to negotiate a three point turn right there in front of the pump.
It's at this point that I want to put her head in my hands and shake violently.
Monday, only this time, on a Monday
From: Dan the network engineer <XXXXXXXXXXXXXXXXXX>
To: Sean Conner <sean@conman.org>
Subject:
Date: Mon, 16 Jan 2006 11:12:05 -0500Hey Sean,
XXXXXXXXX's traffic issues have not been solved, and, in fact, are still causing some major issues.
I believe this server has been hacked and needs to be taken off-line …
Okay, see! This is a Monday thang …
Sigh.
Not that bad, as these things go
Well, the server was hacked, but
it looks to be a customer account was compromised, since the executables
where owned by a customer account, the processes were running on
unpriviledged ports, and the server was being used as part of denial of
service attacks, with executables hidden under a hidden directory in
/var/tmp
.
Fortunately, the system hacked is running Linux without module support, so patching system calls to hide activity is impossible without a reboot (which would be noticed).
And as always, it could have been worse.
Tuesday, January 17, 2006
“It comes in two flavors, ‘icky’ and ‘yucky.’”
The magnets are still stuck in The Younger. And this morning he was in pain. Quite a bit of it too. Hopefully the pain means they are on the move, but we won't know until later today after his insides are scanned once again.
If the magnets haven't moved, tomorrow the doctors go in.
Here's hoping the magnets leave his system on their own.
Real-time LaBrea data processing program
I finished writing the real-time LaBrea data processing program (most of it last night at The Hospital) and the final few bugs were real doozies.
The program works by getting data from LaBrea, then it looks up the connection and updates the information accordingly. The code pretty much looks like:
void start_tarpit(time_t stamp,char *line) { struct tprecord *exists; struct tprecord rec; size_t index; read_record(&rec,line); exists = tpr_search(&rec,&index); if (exists == NULL) exists = pull_free_record(); exists->src = rec.src; exists->sport = rec.sport; /* ... */ record_add(index,exists); } /*******************************/ struct tprecord *pull_free_record(void) { if (g_poolnum == g_poolmax) do_forced_garbage_collection(); return(&g_pool[g_poolnum++]); } /*****************************/ void record_add(size_t index,struct tprecord *rec) { if (g_recnum == g_recmax) do_forced_garbage_collection(); memmove( &g_rec[index + 1], &g_rec[index], (g_recnum - index + 1) * sizeof(struct tprecord *) ); g_rec[index] = rec; g_recnum++; }
tpr_search()
is the binary search routine I wrote the other day, and as
you can see, it returns the index to where the record is in the array, or to
where it should be. pull_free_record()
just returns
the next free slot in the structure array, and if there are no slots
available, it does a removes some older records according to some criteria.
And record_add
will add the record to the pointer array,
also removing older records if there is no space left.
Some records are deleted. All remaining records move about. The pointer array is resorted. So between the time
exists = tpr_search(&rec,&index);
and
record_add(index,exists);
index
may not be a valid index anymore!
Oops.
(Never mind the fact that one of the two calls to
do_forced_garbage_collection()
is redundant)
Simple enough to fix once I knew what was going on.
Another bug dealt with named pipes. Instead of directly piping the data
from LaBrea to ltpstat
(what I call the read-time LaBrea data
processing program), I decided to go through a named pipe, which would allow
me to start and stop either one independantly from the other.
Now, I'm testing my program, running cat sample >
labrea.pipe
in one window, and ltpstat --input
labrea.pipe
in another. It's working, until the data in
sample
runs out and cat
closes its side of the
named pipe.
Now, the code that reads in the data is in a library I wrote, and it just
assumes when a read()
returns 0, that it's the end of the file,
and marks it as being closed. ltpstat
ignores the “end of
file” status, and keeps trying to read a now-closed file. We get into a
busy loop and the system load shoots up. Also, if I now try to pump more
data through the named pipe, ltpstat
ignores the data.
Even when I modify the library code to not mark “end of file” when there's
nothing to read does nothing, as from that point on, read()
just returns nothing anyway and seems to be a “feature” of Linux
(or Unix—I didn't have Advanced
Programming in the Unix® Environment with me to look this up), so I
restructure the main loop:
while(1) { in = openinput(); while(!StreamEOF(in)) { process_labrea_output(); } StreamFree(in); }
That was all fine and good, until I threw signals into the mix.
I use signals to tell ltpstat
(what I call the read-time
LaBrea data processing program) to dump various
information—SIGHUP
to print the number of connections and
unique IPs being tracked, SIGUSR1
to do a raw dump of all the
data accumulated so far and SIGUSR2
to generate a more or less
human readable dump.
But signals are basically interrupts. And operating system theory states that one process should never find another process with its pants down (so to speak—and I should warn you—the paper I linked to is way more technical than I've gotten here). Signals and system calls interact (that is, if a process is signaled while making a system call into the kernel) in one of two ways—the system call will simply fail, or it will be restarted automatically. And one can select which method to use.
If I elected to have the system call fail, and ltpstat
would
fail, either to open (a system call) the named pipe, or in reading (another
system call) the named pipe after I signal the program.
If I elected to have the system call restarted, and the signal handlers I set up would never get called (due to the way I handle the signals).
I ended up reimplementing two routines from my library (which is used in more than just this program) for just this program. I select for the “system call fail” method, check to see if the system called failed due to a signal, and if so, check for the signals, and try again.
Again, this took a few hours to track down.
But now the program works, and I can finally get real time statistics from LaBrea.
Last minute update!
I just received a call from Spring—the mangets are in The Younger's stomach! (why this wasn't found out sooner, I don't know). So, because it's in his stomach, he's being moved yet again to another hospital, one that has the equipment that might get the magnets from his stomach.
I'll be heading there once I get word which hospital it is.
Some initial data from a real-time LaBrea data processing program
While I'm waiting a call back, some more on LaBrea.
Yesterday (from January 16 at 06:28:25 to January 17 08:54:50) LaBrea
generated 1.1G of log data, and it took
full five minutes to run grep 'Initial Connect' daemon.log.0 | wc
-l
(255,344 new tarpitting connections by the way).
LaBrea was also running at full speed, maxed out at 64Kbps bandwidth to keep all these connections tarpitted (the maximum I set LaBrea to use, by the way).
That first large dip in the graph (the one around 6:30 in the morning) is
probably due to the system attempting to rotate a 1.1G log file. The second dip, at the right (around
3:00 pm) is when I restarted LaBrea so its logging information would go
through ltpstat
.
After an hour of running:
Start: Tue Jan 17 14:55:59 2006 End: Tue Jan 17 15:55:59 2006 Running time: 1h Pool-max: 1048576 Pool-num: 24322 Rec-max: 1048576 Rec-num: 24322 UIP-max: 1048576 UIP-num: 1282 Reported-bandwidth: 40 (Kb/sec)
And after two hours:
Start: Tue Jan 17 14:55:59 2006 End: Tue Jan 17 16:56:19 2006 Running time: 2h 20s Pool-max: 1048576 Pool-num: 33326 Rec-max: 1048576 Rec-num: 33326 UIP-max: 1048576 UIP-num: 1632 Reported-bandwidth: 40 (Kb/sec)
And right this second:
Start: Tue Jan 17 14:55:59 2006 End: Tue Jan 17 18:37:19 2006 Running time: 3h 41m 20s Pool-max: 1048576 Pool-num: 42931 Rec-max: 1048576 Rec-num: 42931 UIP-max: 1048576 UIP-num: 2148 Reported-bandwidth: 40 (Kb/sec)
Okay, pool-max
and rec-max
are the maximum
sizes for the structure array and pointer array, and both should always be
equal at all times (I'm displaying this number more for debugging purposes
than anything else), while pool-num
and rec-num
(which should also be equal at all times) represent the current number of
connectioned tarpitted. I also keep track of unique IPs, which just now is currently 2,148
(out of 1,048,576 that I can store). I also just found out that IP address 195.130.152.85
has 4,809 connections currently tarpitted (and in the few seconds it took to
do that query, five more connections were tarpitted).
I'll be releasing this code in the next few days, when I can write up some documentation and slap on a license.
This just in …
Just now received a call from Spring—the trip to the other hospital has been postponed until tomorrow morning.
That is all.
Wednesday, January 18, 2006
No news is good news, I suppose
The Younger is waiting to be transported to the next hospital in line. The magnets are still in his stomach waiting to be removed.
So, hurry up and wait …
LaBrea ad-naseum
Just because I was curious, I ran the Perl script that parses the LaBrea logs on the 1.1G logfile from the other day. I knew it would take at least four hours or so to run.
I then created a modifed version of ltpstat
to parse this
file (it's the output from syslogd
, hense some slight parsing
modifications were required) and ran it at the same time.
What I found interesting (during the actual run of both programs) is that my program had a larger virtual memory footprint over the Perl version (easily seven to eight times larger) but the resident set size (the amount of physical memory being used—the rest not physically allocated or shoved off into swap space) of my program was half that of the Perl script. In retrospect this was expected—Perl was growing the data as it was being generated whereas my program allocates the whole thing at once, but my program has less overhead in keeping track of said data.
And ltpstat
is faster than the Perl script even if it isn't
gathering the stats in real-time— 3 hours, 22 minutes and 24 seconds to
run vs. 6 hours, 39 minutes 49 seconds—almost half the time. I didn't
see how much memory the Perl script was using just prior to finishing, but I
can't see how it would be less than ltpstat
.
The instance of ltpstat
I started yesterday is still running:
Start: Tue Jan 17 14:55:59 2006 End: Wed Jan 18 14:55:59 2006 Running time: 1d Pool-max: 1048576 Pool-num: 388929 Rec-max: 1048576 Rec-num: 388929 UIP-max: 1048576 UIP-num: 20298 Reported-bandwidth: 64 (Kb/sec)
Looks like I may break a million connections sooner than I expected.
Thursday, January 19, 2006
This is the stay that never ends
Latest update on The Saga of the Younger: he's been moved to the next hospital in line, and there the doctors discovered that the magnets are stuck in an ulcer in his stomach, and the only way to get them out is surgery. We have no time frame for when said surgery will happen though.
Updates as I get them.
Friday, January 20, 2006
Update
The Younger is out of surgery. That is all I know at this time.
Update a few minutes later …
The magnets did get separated—one ended up in his stomach and the other one ended up in his intestines. They did attract each other and the attraction was enough to cause them to borrow through tissue.
They've been removed (and were quite corroded from what I understand) and the holes sewn up. He's in recovery for the forseeable future. So now it's just waiting around until he heals up.
Saturday, January 21, 2006
Quick game of ketchup
I have internet access for the evening, courtesy of having to come to work. Trying to catch up on the most essential emails and such, in between working cases.
I love you people. Thank you so so so much for the messages of concern, wishes, prayers, and other good vibes, as well as information on polar bear blubber and vitamin poisoning. Thanks also for offers of assistance. We've got things covered, but truly appreciate it.
Magnet news:
On the 19th the team at West Boca sent an endoscope down his gullet, under general anaesthesia. They found an ulcer with a shadowy something in the middle that was likely the magnets. That something in the middle couldn't be gotten out. Surgery required.
The surgery occurred yesterday, the 20th, and took longer than anticipated—it was about 2 hours. rhonan had it right—one magnet had gone down the small intestine, and the other had stayed in the stomach, and they attracted. The ensuing irritation and vomiting episode caused them to begin tunneling toward each other, scar tissue forming a tube. Surgeons had to disconnect the tube, remove the magnets, and fix the holes. They were thrilled at how fixable it was, though - no ancillary damage like necrosis or sepsis or any of that. It could have been far worse.
Afterward, straight into ICU and morphine. It's not fully effective against the pain, though. He got out of ICU today and is still convalescing, no food or liquid yet. The innards have to heal first.
My weekend is starting after tonight's shift, and I'll most likely be spending all of it at the hospital. I'll certainly be there after shift tonight. Will continue to post phone-to-email-to-LJ updates as necessary and able.
Spring Dew: A quick game of ketchup
Not much more to add to this (except I don't have a phone-to-email gateway to update this site).
Are you tired of network tarpits yet?
You would not believe how hard it was to write a binary search that returned the correct index for a missing record in an array.
Some notes on a binary search implementation
A week later, and I finally have it working.
One technique used to debug a program is to have another program that does
the same thing, but implemented using a different method or language or both.
And I did. I ran the Perl program I had
over the 1.1G log file, then ran
ltpstat
over the same log file and got two different
results.
Not good.
ltpstat
returned 2% more connections than the Perl script.
Getting a dump from the currently running version on the LaBrea system and cleaning
the output showed a 2% difference again.
So I spent the past week trying to track down the problem. It was obvious
that ltpstat
was storing duplicate records, but why was
a different matter. My testing sample of about 1,100 connections is
apparently too small to completely test the program, so I had to test using
the 1.1G log file which has approximately
230,000 connections.
To help debug this problem, I wrote a linear search and would call it as well as the binary search. If both agreed, then I would return the information, otherwise, I would log the discrepency, do the search again, then exit. The reason for doing the search a second time? So I could set a breakpoint there, and let the program run for a couple of hours until it triggered. Then I could step through both searches to see where the problem was.
Yup, each run took several hours to trigger the bug.
I ended up testing four different binary search routines (including the original one I thought worked, plus one I modified from The Standard C Library, plus two other versions I wrote) before sitting down and working through things on paper.
And I still missed corner cases.
But finally, I tested my final version it against the Perl script and only had 122 discrepencies out of some 230,000 records (or 5% of 5%—too small for me to worry about after spending a week on this).
I took a snapshot of the currently running version (which had been running for a bit over three days now), cleansed the output of duplicates, and the final tally was 416,230 connections from 12,911 unique IPs. Again, nothing surprising about the ports being attacked:
Port # | Port description | # connections |
---|---|---|
Port # | Port description | # connections |
139 | NetBIOS Session Service | 160,799 |
135 | Microsoft-RPC service | 108,958 |
445 | Microsoft-DS Service | 67,506 |
80 | Hypertext Transfer Protocol | 23,921 |
4899 | Remote Administration | 9,225 |
22 | Secure Shell Login | 7,253 |
1433 | Microsoft SQL Server | 6,503 |
8080 | Hypertext Transfer Protocol—typical alternative port | 3,717 |
3128 | Squid HTTP Proxy | 3,329 |
1080 | W32.Mydoom.F@mm worm | 3,150 |
And again, the Microsoft specific ports account for 81% of the scans. I'll need to discuss with Smirk about blocking those ports in the core router. If nothing else, LaBrea is giving me an indication of which ports to block.
Sunday, January 22, 2006
For once this year, a post that has nothing to do with tarpits or hospitals, except for this self-referential title mentioning that this post has nothing to do with tarpits or hospitals.
It's weird. I have my email address sean@conman.org plastered all over my website here, and yet, I'm receiving a bazillion spams for a type of sexual aid at my private email address. But at my oh so public email address?
Nothing what so ever about this particular sexual aid.
How odd.
Monday, January 23, 2006
“Waaaaah waaaaah waaaaah”
I remember having this weird dream about ten years ago or so. In this dream, my boss (it wasn't you Smirk) walked up to me, opened his mouth and said “Waaaaah waaaaah waaaaah waaaaah waaaaah.”
“What?” I said.
“Waaaaah waaaaah waaaaah waaaaah,” said my boss. He then went around the office continuously saying “waaaaah waaaah waaaah,” even if he wasn't speaking to anyone in particular.
I do recall in my dream that a bunch of us were trying various methods to shut him up, since we all found his “waaaaah waaaah waaaaah” very annoying. After what seemed like hours, it dawned on me that this wasn't the normal behavior of our boss and that something else might be to blame. It was then I realized I was dreaming, and woke up. To my alarm clock, which was going “waaaah waaaaah waaaah waaaaah waaaaah” and had been for the past 40 minutes.
Yes, I'm a heavy sleeper.
I also found out (about the same number of years ago) that my alarm clock would simply give up after three hours of me hitting the snooze button (not only did I hit the snooze button for three hours, the alarm clock itself was on the other side of the room—yes, for three hours, every nine minutes, I would roll out of bed, take two steps across the room, hit the snooze button, two steps back and roll back into bed, without disturbing my sleep!)
I no longer use that alarm clock—oh, it still works, it's just that the hour button has broken off (personally, I would have expected the minute button to break off) so it's rather difficult to set. I've been using my cell phone as an alarm clock. No snooze mode, so I have to be semi-conscience to manually set the alarm, but the alarm itself only sounds for a minute, which may be too short a duration.
I only mention this to explain to Smirk was I was nearly three hours late to work today.
Tuesday, January 24, 2006
Update for those playing at home
The Younger started in on liquids today, and it looks like he'll be eating solid food starting tomorrow. That, and he's flirting with the nurses, so he's healing right up. I'm expecting he'll be in The Hospital for the next few days or so.
A thought process …
I have to wonder the thought process of a script kiddie that runs the
SSH22 Scanner
(which tries to log into servers via
ssh
using a series of precanned userids and passwords) as fast
as it can. Doing so only brings attention to the fact that it's sucking up
network bandwidth and loading down the box, gaining the attention of the
sysadmin, thus negating the purpose of running such a program in the first
place.
I also have to wonder at the thought process of a script kiddie that logs
back into a box after loosing the connection and not bothering to see if
anyone is logged in as root
before starting the SSH22
Scanner
back up.
Wednesday, January 25, 2006
So when does 2 not equal 2?
If someone was to drop a bomb on this building, it would wipe out 50 percent of the Lisp community. That would probably be a good thing. It would allow Lisp to start over.
That wasn't exactly what John said, but it was pretty close. Again, the main thing here was somebody with impeccable Lisp credentials saying that Lisp was far from perfect and in fact needed to do better than Common Lisp. The fact that Patrick, Henry, and John all argued essentially the same thing was a wake-up call, I think.
Finding Lisp: ILC 2005, Wednesday report (late)
Now, I have this love-hate relationship with Lisp. I love the
idea of Lisp, and it is a cool language, but I hate the
implementations of it (it doesn't play well with other languages), lack of
syntax (which expresses itself in Lisp as a metric buttload of
parenthesises), the nasty low level details one has to keep in mind (really!
In Common
Lisp, 2 (this particular instance of the number) is not necessarily
equal to 2 (this other instance of the number)—I kid you not!) and the
rather archaic way it still operates (CAR
and CDR
are remnents from the very first implementation where one took the “contents
of the address register” for the current element in a list, and “contents of
the decrement register” for the rest of the list—even though you now have
FIRST
and
REST
LAST
(oh, it's called
LAST
even though it returns the rest of the list)REST
(apparently, I had it right the first time, and LAST
is used
for something else in Lisp—sigh) nobody ever uses those).
Oh, and to get any real speed out of Lisp you have to forego using lists (which is what Lisp stands for—“LISt Processing”) and use more specific data structures (never mind the fact that Lisp is still taught as if it only has lists).
The “John” mentioned above is John McCarthy, inventor of Lisp, which is one of the two oldest programming langauges (the other being FORTRAN), and if he, of all people, doesn't have much positive to say about Lisp, then what chance does it have of gaining wide spread acceptance in the industry (Greenspun's Tenth Rule notwithstanding)?
Thursday, January 26, 2006
He's Out!
The Younger has left the hospital!
Thank God that's over.
And if The Younger ever eats another magnet again, I'll kill him! Sheesh!
(/= 2 2)
My recent post on Lisp generated some heat. First, from Zach:
From: Zach XXXXX <XXXXXXXXXXXXX>
To: Sean Conner <sean@conman.org>
Subject: (/= 2 2)
Date: Thu, 26 Jan 2006 10:56:38 -0500I came across your Lisp post today, and I wanted to comment on a couple things.
FIRST
andLAST
are not analagous toCAR
andCDR
;LAST
returns some part of the tail of a list (depending on its optional second argument). The list-oriented analogue ofCDR
isREST
. (You had that, but crossed it out.)As for 2 not necessarily being equal to 2, there are many equality predicates in Common Lisp. For example, comparing 2 and 2 with the
EQL
,EQUAL
,EQUALP
, and=
functions will all return true (or there is a bug in the implementation). It is only the object “same, identical” predicateEQ
that will sometimes fail on two apparently identical numbers, and even then it's only likely when the number enters the bignum range. Fortunately, most documentation and tutorials give the good advice to useEQL
(notEQ
) as a default “sameness” test. A novice would have to take a fairly peculiar learning path to get burned by usingEQ
on numeric values.
We had a bit of an exchange and I ended up writing:
From: Sean Conner <sean@conman.org>
To: Zach Beane <XXXXXXXXXXXXX>
Subject: Re: (/= 2 2)
Date: Thu, 26 Jan 2006 15:18:07 -0500 (EST)Hmmm.
Odd thought just cross my mind. It may not be fully applicable to Lisp and it's half formed, but I'll try to get it out anyway.
Lisp is a dynamically typed language, and a lot of programmers like dynamic typing since it frees them from the burden of having to declare the type of a variable [1] (often times the excuse is “I'm prototyping—I don't know what types I'll end up using.” [2]). Having to declare the type of a variable is seen as pointless pedantry and something the compiler can keep track of [3].
I feel the same way about the equality functions in Lisp—pointless pedantry and something the compiler should keep track of.
- Yes, in Lisp, values have types, not variables.
- A weak excuse in my opinion. Strikes me as being muddle headed about what the program is supposed to do. I should mention that I actually liked Ada, and tend to prefer strict type checking, if only to make sure I don't make bone headed mistakes.
- Fair enough argument, but I'd prefer to state my intent to the compiler anyway. The more information it has, the better off it is.
Just how much of a pointless pedantry is Lisp's equality functions? Well, there are five defined in Common Lisp (quoted from Paul Graham's ANSI Common Lisp):
(eq object1 object2)
- Returns true iff object1 and object2 are identical. [It doesn't say what it means by “identical” but given that
(eq 2 2)
can be false in a conforming implementation of Common Lisp, I'm guessing “identical” means “same address”](eql object1 object2)
- Returns true iff object1 and object2 are
eq
, or the same character, or numbers that would look the same when printed. [So even here,(eql 2 2.0)
is still false, at least, according to Rodney A. Brooks Programming in Common Lisp.](equal object1 object2)
- Returns true iff object1 and object2 are
eql
; or are conses [one element of a list] whose cars [first element] and cdrs [rest of the list] areequal
; or are strings or bit-vectors of the same length (observing fill pointers) whose elements areeql
; or are pathnames whose components are equivalent. May not terminate for circular arguments. [So, I guess this means that(equal 2 2.0)
may be false as well](equalp object1 object2)
- Returns true iff object1 and object2 are
equal
,char-equal
[um … I guess there are more than just the five equality functions then], or=
; or are conses whose cars and cdrs areequalp
; or are arrays with the same dimentions whose active elements areequalp
; or are structures of the same type whose elements areequalp
; or are hash tables with the same test function and number of entries whose keys (as determined by the test function) are all associated withequalp
values. Reasonable to assume that it may not terminate for circular arguments. [So, if you want to compare arrays, you can't useequal
, and still(equalq 2 2.0)
is false.](= n1 &rest ns)
- Returns true iff the difference between each pair of arguments is zero. [Finally! We have an equality test where
(= 2 2.0)
may be true!]
Why so many? I don't even want to hazzard a guess (perhaps Lisp programmers like micromanaging equality whereas C programmers like micromanaging variable type information—oh darn, I hazzared a guess).
My friend Andrew wrote in:
From: Andrew XXXXXXX <XXXXXXXXXXXXXXXXXXXX>
To: Sean Conner <sean@conman.org>
Subject: An apology for Lisp
Date: Thu, 26 Jan 2006 14:58:44 -0500 (EST)If you still think Lisp is wrong, then consider this: is 2.0 equal to 2? Why or why not? I think reasonable people could disagree, and the designers of Common Lisp did too; that's why there are multiple equality predicates with different semantics.
As for avoiding lists to get “real speed”—you say that like nobody uses O(N) complexity data structures to do real work in C! Lisp lists are what they are; for many applications they're fast enough. Otherwise, Common Lisp has arrays, hash tables, and more.
Okay, but if there are languages out there that can do type inferences on data, then certainly languages can also do equality inferences with data as well. Or perhaps I've been corrupted with Perl and it's very lax approach to equality …
And to Lisp's other data structures? Most introductions to Lisp I'm familiar with start with lists, and rarely, if ever, mention any past that. For instance, Rodney Brooks' Programming in Common Lisp, a 12 chapter book and arrays are first mentioned in Chapter 12! (“Other Features of Common Lisp”). And even though Paul Graham mentions arrays, trees and hash tables in Chapter 4 of his ANSI Common Lisp, the chapter title doesn't do them any service in my opinion (“Specialized Data Structures”—I really didn't know arrays were a specialized data strcture). Also, the way lists are described ad-naseum in Lisp books, which special pains to point out the structure of the “cons cells” that it seems like Lisp is still stuck with a 60s implementation.
Oh, it's still useful, just not as useful as I expected
By using ltpstat
I've been able to see that the LaBrea tarpit isn't quite
as effective as I first thought. Yes, it does slow down scans, but not
quite as much as one thinks. I'm guessing that scanning software now
includes the “timeout” concept—if a connection takes too long, drop the
connection and move on.
A few days ago I added a feature to ltpstat
to remove
entries that have not seen any activity for over an hour (default setting).
After running the tarpit for over a day, I see the following stats:
Jan 27 01:57:08 ltp ltp-report: Start: Wed Jan 25 17:27:55 2006 End: Fri Jan 27 01:57:08 2006 Running time: 1d 8h 29m 13s Jan 27 01:57:08 ltp ltp-report: Pool-max: 1048576 Jan 27 01:57:08 ltp ltp-report: Pool-num: 107287 Jan 27 01:57:08 ltp ltp-report: Rec-max: 1048576 Jan 27 01:57:08 ltp ltp-report: Rec-num: 107287 Jan 27 01:57:08 ltp ltp-report: UIP-max: 1048576 Jan 27 01:57:08 ltp ltp-report: UIP-num: 2558 Jan 27 01:57:08 ltp ltp-report: Reported-bandwidth: 32 (Kb/sec)
Okay, I've “captured” 107,287 connections. But how many of those are still active?
Jan 27 01:58:32 ltp ltp-report: Removing records with no activity for the past 1h Jan 27 01:58:32 ltp ltp-report: ... keeping 11180 records with activity since Fri Jan 27 00:58:31 2006
Well then. Over 96,000 connections were no longer “active” and of the 2,558 machines doing the scanning, some 2,200 had moved on.
So it looks like the LaBrea tar pit is really only useful to see what's
being attacked, and which machines on the Internet are really doing the
attacking (so far, 24.73.129.197
seems to be quite tenacious in
scanning).
And the ports being scanned? Again, it's the Microsoft specific ports as usual. No use making a chart this time.
Friday, January 27, 2006
Rabbit Hole Day
This article is intended to present a set of basic rules for writers to follow when creating “Conspiracy” or “Fringe Theory” texts. I hope that no- one is tempted to use them for “real world” applications—except for spotting the faults in existing material. However, real “fringe theory” buffs don't need me to teach them how to think. (But I've done so anyway, using my personal orbital mind control laser.)
Today is The Second Annual LiveJournal Rabbit Hole Day—a day where LiveJournal entries were supposed to be fun, wierd, unexpected, different, in honor of Charles Lutwidge Dodgson's 174th birthday (although he's not around to enjoy it). I thought it sounded like a neat idea, so I thought I'd try it here, even though this isn't LiveJournal.
Only you can see that I didn't quite get a round toit (those seem to be hard to find lately) and that's due to not having an idea of what to write about until too late, and then running out of time.
An idea I was trying to come up dealt with one Wolfgang Amadeus Mozart, seeing how it's his 250th birthday (and he too, is not around to enjoy it), but I had no good ideas. Nothing jelled until I found out that it was also the birthday of Captain Edward John Smith, R.N.R. (who too, isn't around to enjoy the celebrations, mainly because he was obligated to go down with his ship). Once I saw that, then it hit me—a conspiracy theory tying all three together.
I didn't get a chance to work on this at work, so I thought I might be able to at the weekly D&D game (since I rarely pay attention to the game anyway, and Bob, the DM is fine with that) but this week, of all weeks, his computer took a dive and it took several hours to help him debug the issue (his copy of Panda Network Antivirus updated itself and the update hosed his computer hard).
So there went that idea.
So no Rabbit Hole Day for me this year.
Maybe next year.
Saturday, January 28, 2006
Tunnelling through the Internet
Some of the networking we're doing at The Office is beginning to scare me, and not because it's complex, but because … well … it's complex and it seems like a kludge.
Basically, we're providing a backup Internet connection for one of our customers via DSL; they have an existing Internet connection with another provider. We are also giving them a block of IP addresses to use even when their primary Internet connection is working! So we need to route a portion of our network through some other provider to our customer.
And yes, it can be done, but the way it's being done leaves a … bad taste in my mouth. Basically, traffic for the customer's network arrives in our network, hits our router, the router will then encapsulate that network traffic within another IP connection back out out network and through the customer's existing provider to a router (which we set up) at the customer's office.
Yes, we're tunnelling IP packets within IP packets.
There's nothing inherently wrong with that, but it does have all sorts of fun implications for the customer's network traffic (for instance, they also have a wireless router at their office. They could only bring up Google. Any other site would fail to load, or only load a partial page. Turns out that since we're tunnelling IP packets within IP packets, the maximum packet size we can transmit is a bit smaller than normal, only the wireless router couldn't figure this out—I had to manually set the packet size to be about 3% smaller than normal on their wireless router. Why could they get to Google? Probably because Google configured their routers to send small packets).
(Oh, another for instance: it's almost impossible to traceroute to the customer's network, so now it's harder to trackdown networking problems. Oh, and another one I just remember: it adds an additional 11 hops that packets have to travel to reach their final destination.)
Sunday, January 29, 2006
Might as well get free phone service if you're being tapped …
“Here's one more tidbit on wire-taps: They get you free phone service! The feds tapped the phone of the Sisters of Mercy in Washington D.C. because of some anti-war stance or something they took in the 1980s. The good sisters noticed some kind of clicking on the phone at times, and finally decided that someone must have tapped into their phone. Their solution: Don't pay the bill so the phone company will have to shut off the phone. The phone never went dead, and they quit sending them bills! The Feds wouldn't let Ma Bell shut them down, and probably began paying the bills. The sisters talked long and free with their friends across the country!”
I, Cringley—The Falafel Connection
Well … that's one way of getting free phone service.
The article does present a different view of the recent NSA wire taps—that they're less listening in on all the calls and more a “connect-the-dots” by looking at who called who. Defintely an interesting read.
Oops.
I must apologize to all that read my site via RSS, especially to those that read the LiveJournal feed (for essentially spamming your frends list) last night.
I upgraded mod_blog
last
night—a huge update. Months ago, I did a near complete rewrite, but was
relunctant to put it into production despite testing and using the software
on a few other blogs (that aren't mine—they're someone elses). But a few
days ago I got an idea for a new feature for my blog, and of course I'm
going to want to work with the latest codebase of mod_blog
, so
…
Only … well … I forgot I made some changes to the RSS templates processing in the program, but forgot to upgrade the actual RSS templates used by my blog. There was also a bug in generating the permanent links (I got a bit overzealous with the formatting), and those two issues caused LiveJournal to have a fit with my feed for a few hours.
It's been ironed out (well, there is still an issue where I have to manually nudge the program whenever I make a new entry) and the RSS feed is working properly now.
Again, I'm sorry.
Monday, January 30, 2006
Calling out the the Lazy Web
Since jwz (and his site is definitely not safe for work often asks the LazyWeb for advice and while I don't think any of my readers may be able to answer this, I might as well give it a shot (before I end up calling tech support and being put on hold indefintely).
I've got a Cisco 2611 with the latest software. It needs to be connected to our office via DSL (on the interface ATM0/0) to our office (which is already set up for DSL, but to require the other side to authenticate). What magical Cisco incantations do I need to get this to work? (and if know how to integrate it with tunnelling, that would be great)
Thanks.
Tuesday, January 31, 2006
I wonder if Bush will wear a gorilla suit for the State of the Union Address …
MacGyver, with the obligatory paper clip
I fixed a communication problem with some scissors, a bit of wire and a paper clip.
No, really.
I got two pieces of equipment communicating by using stuff I was able to find around the office (because that's all I had to work with).
Yesterday, Smirk drops a BayTech DS9 with one DS72 network interface and eight DS30 modules (which control 12 electrical outlets). I was told to test it and configure the device.
So I plug the device into the office network, started up the DHCP server, and powered the DS9. Sure enough, I see the DHCP request come through, it gets an IP address and I try connecting to it.
Nothing.
I scan the device with nmap
and yes, it is
studiously ignoring all attempts to connect to port TCP port 23 (standard
telnet
) and UDP port 161 (SNMP). Since it's studiously ignoring the network, my
only recourse is to get at it through the serial port.
It's an RJ-45 connector on the DS9, and I have a DE-9 (commonly referred to as DB-9 but DE-9 is the proper designation for the connector) on my workstation. I just so happen to have an RJ-45 to DE-9 adaptor (for the Cisco routers). I may or may not need the NULL modem adaptor, but that's easy enough to figure out.
Only it doesn't work. Straight through, or with the NULL modem adaptor.
I have to reach for the heavy guns. I hunker down and read the manual. It's a good manual, and it gives the exact layout for the cabling between the RJ-45 and the DE-9 that I need to communicate with the DS9. So I go hunting for a continuity tester to see if the Cisco-specific adaptor I have is wired such that I can use it.
Only we don't have a continuity tester. A $5,000 network cable analyzer sure. But something to test the continuity of a single wire? Nada. Not even Dan the Network Engineer has one.
So I carefully break open the Cisco-specific RJ-45 to DE-9 adaptor and lo—it's not wired according to the DS72 manual, so it's useless. So are the other RJ-45 to DE-9 adaptors we have lying around.
So I have to make one.
Only we don't have any spare DE-9 connectors (male or female) lying around, but plenty of RJ-45 connectors. Hmmm …
I grab about a foot length of network cable, crimp an RJ-45 connector on one end, and (carefully) use scisors (since we lacked actual wire strippers) to strip the insulation off the wires at the other end. I then carefully wrapped the exposed wire around a paper clip to make a small coil, and then slipped that coil around the pins of a male-to-male DE-9 adaptor, and plugged that into the serial cable at the back of my workstation. The RJ-45 end of the jury-rigged cable went into the DS9.
The fragile kludge made from scisors, wires and a paper clip (oh, and a crimping tool) worked beautifully and I was able to configure the DS9 (and I kept cursing myself for not having my camera around to document such an engineering feat).
I did find out one thing about the device, and I don't think it's mentioned in the manual (although I was more concerned about the serial connection than anything else) but the device simply drops packets unless they come from a configured SNMP trap host. Good thing to know.
Oops, Part II.
Again, I must apologize for spamming the LiveJournal feed of this blog. I swear, I have the bugs under control this time! I even found why I had to manually nudge the program whenever I made a new entry (because it thought said entry was still in the future, albeit just a second or two in the future, and the software doesn't display entries before its time).
Again, I'm sorry.