Monday, May 15, 2006
It's just another manic Monday
If Mondays keep being like this I may just not bother getting out of bed on this particular day of the week.
It started out innocently enough when M, who has a few co-located servers here, came over for some help with a Linux server that wasn't booting. It would start to boot then freeze hard at a certain point when running the start up scripts. Even booting into single user mode with a backup kernel would fail at about the same point.
It was then that I received a call from Dan the Network Engineer that our entire network was melting down.
Sure enough, our network was hosed.
Thus began several tortuous hours trying to track down the target of the attack. Much madly rushing about, unplugging network connections, probing the routers, the switches and various servers trying to track down the target of the attack, or even the source (or sources) of the attack.
Turns out it was M's machine that had problems booting. It was the target for the denial of service attack (well, no wonder it couldn't boot—it probably crashed when the network interface came up and it was faced with half a millions packets per second).
We found the attacking IP address and Smirk called the people responsible for that address. He ended up talking to some “just hired this week” junior network admin who knew next to nothing about networking; all I got was Smirk's side of the conversation, but it was amusing in a schaudenfraude kind of way.
Once we identified the source and target, we were able to stop the attack rather quickly.
And I got to install a new server to replace the one that was slagged by an onslaught of hostile network activity.
Joy.
