Sunday, January 04, 2004
Why I did what I did during a DDoS attack
While looking for referer spam I found that my entry about fending off a DDoS attack has been quite popular, but upon rereading I found it rather terse. So I figure I might as well clear up some details of what exactly I did.
netstat -an
to check the state of all network connections,
and given that there were an enormous number of connections in the
SYN_RECV
state is an indication that a SYN
flood
(where hundreds of connections are initiated but not completed, thus
flooding out legitimate traffic) is underway.
Normal TCP/IP networking is open to an attack known as “
SYN
flooding”. This denial-of-service attack prevents legitimate remote users from being able to connect to your computer during an ongoing attack and requires very little work from the attacker, who can operate from anywhere on the Internet.
SYN
cookies provide protection against this type of attack. If you sayY
here, the TCP/IP stack will use a cryptographic challenge protocol known as “SYN
cookies” to enable legitimate users to continue to connect, even when your machine is under attack. There is no need for the legitimate users to change their TCP/IP software;SYN
cookies work transparently to them. For technical information aboutSYN
cookies, check out <http://cr.yp.to/syncookies.html>.If you are
SYN
flooded, the source address reported by the kernel is likely to have been forged by the attacker; it is only reported as an aid in tracing the packets to their actual source and should not be taken as absolute truth.Help from the Linux Kernel 2.4 Configuration screen
sysctl -w net.ipv4.tcp_syncookies=1
is the command used to
enable SYN
cookies in the Linux kernel. This helps some with
the type of attack we were experiencing.
sysctl -w net.ipv4.tcp_max_syn_backlog=2048
increases the
number of incomming connections the kernel can keep track of. Increasing
this value is a bit of a double edged sword in such an attack—on the one
hand, we allow more connections, thus hopefully allowing legitimate
connections through, but on the other hand, we allow more connections, thus
allowing more machines to SYN
flood the machine. Given some of
the other steps I took, this was probably a good idea overall.
sysctl -w net.ipv4.tcp_syn_retries=2
(which I forgot to
mention in the original entry) decreases the amount of time the kernel
spends trying to establish a TCP/IP connection (from a default
value of 5 attempts to two) which helps to flush the bad connections from
the system quicker.
route add -host <ip-addr> reject
which causes the
kernel to ignore packets from the given IP address, and also flushes current connections from
said IP addresses from the
system. This was the thing I was doing that kept the system up and
running during the attack. I ended up writing a script to continuously
check the connections, then once a certain threshhold of bad connections was
exceeded, ban all the addresses.
The site was eventually taken down dispite all the attempt I made to keep it up since the network traffic to the site in question was swamping the rest of the network the machine was on (it was the colocation facility that said enough is enough and shut the site down). Other than that, I was fairly successful in keeping the website accessible.