The Boston Diaries

The ongoing saga of a programmer who doesn't live in Boston, nor does he even like Boston, but yet named his weblog/journal “The Boston Diaries.”

Go figure.

Tuesday, July 03, 2007

The dreaded “Early Morning Call”

Ah, nothing like getting a call at the ungodly hour of 10:30 to fix a munged firewall. For reasons I'm still trying to figure out it apparently crashed last night and the fix was a power cycle.

But in looking over the log files, I kept seeing:

Jul  2 08:48:07 localhost kernel: SPOOF-SRC-172 IN=eth0 OUT=eth1
	SRC=172.21.159.36 DST=XXX.XXX.XXX.XXX LEN=40  TOS=0x00 
	PREC=0x00 TTL=238 ID=0 DF PROTO=TCP SPT=11100 DPT=1516 
	WINDOW=0 RES=0x00 RST URGP=0

I had programmed the firewall to reject any private IP address (since you're not supposed to see private IP addresses on the Internet at large) yet here I was, seeing a private, albeit a less known private, IP address. Only the source address kept changing for each packet logged, but always within the 172.21.0.0/24 block.

This meant that the address was originating somewhere on our network, but where?

Pinging one of the private IP addresses from the router directly connected to the firewall didn't show anything. It seems that the ping packets were sent to the main core router, which just dumps private IP addresses to the bit bucket.

But why was that router throwing anything from 172.21.0.0 to the firewall?

Now, the rule I have for filtering these packets in iptables looks like:

iptables --table filter --append FORWARD --source 172.16.0.0/12
	--jump LOG --log-prefix "SPOOF-SRC-172 "

iptables --table filter --append FORWARD  --source 172.16.0.0/12
	--jump REJECT

iptables --table filter --append FORWARD --destination 172.16.0.0/12
	--jump LOG --log-prefix "SPOOF-DST-172 "

iptables --table filter --append FORWARD --destination 172.16.0.0/12
	--jump REJECT

which doesn't record the MAC address (which uniquely identifies an Ethernet interface on a network). However, by adding:

iptables --table mangle --append PREROUTING --source 172.16.0.0/12 
	--jump LOG --log-prefix "WHAT "

the MAC address is logged.

Cool!

Jul  3 17:48:24 localhost kernel: WHAT IN=eth0 
	OUT=MAC=00:10:e0:02:09:dd:00:02:85:12:a7:c0:08:00 
	SRC=172.21.117.49 DST=XXX.XXX.XXX.XXX LEN=40 
	TOS=0x00 PREC=0x00 TTL=239 ID=0 DF PROTO=TCP 
	SPT=11112 DPT=3716 WINDOW=0 RES=0x00 RST URGP=0  

What's recorded is the MAC address of the interface (the first six octets), followed by the MAC address the packet came from (the next six octets) followed by the protocol (the final two octets, and in this case, it's an IP packet).

Now, the first three octets of a MAC address are the vendor code—who makes that particular Ethernet device. Easy enough to look up and—

A Riverstone?

It's the router?

There's nothing on our network that uses the 172.16.0.0/12 private IP space. And yet, here's one of our routers, sprewing forth garbage packets.

Sigh.

Time to get with Dan the Network Engineer on this …

Update on Thursday, July 5th, 2007

Between Dan the Network Engineer and I, the problem was solved.

Obligatory Picture

[It's the most wonderful time of the year!]

Obligatory Links

Obligatory Miscellaneous

You have my permission to link freely to any entry here. Go ahead, I won't bite. I promise.

The dates are the permanent links to that day's entries (or entry, if there is only one entry). The titles are the permanent links to that entry only. The format for the links are simple: Start with the base link for this site: http://boston.conman.org/, then add the date you are interested in, say 2000/08/01, so that would make the final URL:

http://boston.conman.org/2000/08/01

You can also specify the entire month by leaving off the day portion. You can even select an arbitrary portion of time.

You may also note subtle shading of the links and that's intentional: the “closer” the link is (relative to the page) the “brighter” it appears. It's an experiment in using color shading to denote the distance a link is from here. If you don't notice it, don't worry; it's not all that important.

It is assumed that every brand name, slogan, corporate name, symbol, design element, et cetera mentioned in these pages is a protected and/or trademarked entity, the sole property of its owner(s), and acknowledgement of this status is implied.

Copyright © 1999-2019 by Sean Conner. All Rights Reserved.