Network Monitor : Filtering

Broadcasts

PaulELong — Wed, 20 Aug 2008 17:31:00 GMT

We can categorize traffic into two general types; directed and broadcast traffic. In the most general sense, a broadcast is sent to anybody that wants to listen. What I’d like to talk about is how broadcasts work and what they are used for.

How broadcasts work

In general a broadcast is a special address. Different layers, in particular the hardware and network layers have specific addresses defined as broadcasts. So let’s look at each layer specifically.

Hardware Layer:

This layer is normally controlled completely by hardware. It defines how the electrical signals go across the wire and how to decide what traffic you network adapter should listen to. In reality network traffic is always broadcasted to everybody on the same segment. Switches and Routers will further determine how segments are defined and may block or allow this type of traffic.

Your network adapter is setup to “listen” for a hardware address that is assigned to it. This 6 byte address is usually hard coded by the network card manufacturer. A manufacturer is assigned a group of addresses and they make sure they are all unique when shipped. When your network adapter sees an electrical signal, it decodes the Ethernet header information and checks if it’s one of the addresses it’s listening for. If it matches, then it sends the packet to the OS for more processing.

Here’s an example of the Ethernet portion of a packet.

    - Ethernet: Etype = Internet IP (IPv4),DestinationAddress:[FF-FF-FF-FF-FF-FF],SourceAddress:[00-06-5B-61-2E-7A]
     + MacAddress DestinationAddress: *BROADCAST [FF-FF-FF-FF-FF-FF]
     + MacAddress SourceAddress: Dell Computer Corp. 612E7A [00-06-5B-61-2E-7A]
       UINT16 EthernetType: Internet IP (IPv4), 2048(0x800)

You can see that the Source Address is comprised of 6 bytes, 00-06-5B-61-2E-7A. The first 3 bytes, 00-06-5B, indentify it as the Dell Computer Corp. The final 3 bytes are unique and set by the manufacturer.

In wireless, all traffic is inherently broadcast based. The air is the medium, ok not really air, but the idea is that everybody sees that traffic. But again once your wireless adapter sees the traffic; it will inspect the address and if it matches, it knows to send to the OS.

Broadcasts at this layer are special addresses that are industry defined. Each NIC is configured to listen to its personal address and any broadcast related traffic. For Ethernet, the broadcast is simply an Ethernet address of FF-FF-FF-FF-FF-FF.

So for instance, if we look at an ARP packet with NM3.2, you can see that the destination address is FF-FF-FF-FF-FF-FF, and there for while be listened to by any machine that receives the packet.

    Ethernet: Etype = ARP,DestinationAddress:[FF-FF-FF-FF-FF-FF],SourceAddress:[00-07-B3-29-F8-00]

Network Layer:

The network layer in the case of IPv4 and IPv6 also does the same thing. The only difference here is that it may also include information on how to route traffic. For instance an IP address will determine if traffic is local or not for outgoing traffic.

[See http://blogs.technet.com/netmon/archive/2007/07/20/intro-to-name-resolution.aspx for more info about routing.]

When incoming traffic arrives at the IP layer, it again checks the address to see if it’s something it should be listening for. IP addresses however are not hardcoded. They are assigned and a NIC can listen on multiple IP addresses if it wants.

At this layer the broadcast again is defined as a specific IP address. For IPv4, this could be 255.255.255.255. This can also be restricted to the current sub net, so for instance 192.168.1.255 for a class C network. (For more info on IPv4 classes see http://en.wikipedia.org/wiki/Classful_network).

Why have Broadcasts?

You may have already figured this out but broadcasts are used to send information that every machine on the same segment. For instance, when IPv4 needs to see if an address is already taken it will broadcast an ARP packet and ask if an address is available. In fact, you may have already seen that ARPs tend to always be broadcasts.

    Frame: Number = 41, Captured Frame Length = 60, MediaType = ETHERNET 
    + Ethernet: Etype = ARP,DestinationAddress:[FF-FF-FF-FF-FF-FF],SourceAddress:[00-07-B3-29-F8-00]
    + Arp: Request, 192.168.100.253 asks for 192.168.100.13

Broadcasts are also used to announce general things like their names to everybody. When NETBIOS starts up, it sends out an announcement to anybody who’s listening to see if this name has been used before. When the name is heard machines called browse masters listen to these names and record them. This allows you to see all the machines on your local network. Your machine may ask the browser for a complete list.

    + Ethernet: Etype = Internet IP (IPv4),DestinationAddress:[FF-FF-FF-FF-FF-FF],SourceAddress:[00-1D-09-AB-5D-0A]
    + Ipv4: Src = 192.168.1.5, Dest = 192.168.1.255, Next Protocol = UDP, Packet ID = 3092, Total IP Length = 78
    + Udp: SrcPort = NETBIOS Name Service(137), DstPort = NETBIOS Name Service(137), Length = 58
    + Nbtns: Query Request for MachineX <0x00> Workstation Service

When a computer boots, often broadcasts are used to ask the DCHP server for an address. In fact each time you boot a machine without an IP address, it must use a broadcast to find and communicate with the DHCP server. The message below is discover request that is sent when a machine is looking to find its first IP address.

    + Ethernet: Etype = Internet IP (IPv4),DestinationAddress:[FF-FF-FF-FF-FF-FF],SourceAddress:[00-60-08-01-D3-03]
    + Ipv4: Src = 0.0.0.0, Dest = 255.255.255.255, Next Protocol = UDP, Packet ID = 288, Total IP Length = 328
    + Udp: SrcPort = BOOTP client(68), DstPort = BOOTP server(67), Length = 308
    + Dhcp: Request, MsgType = DISCOVER, TransactionID = 0x83484743

Another application, and perhaps the first thing you had in mind when you heard the word broadcast, is sending audio or video data to minimize the network utilization. Rather than sending a separate feed to everybody, you can broadcast to a general address, and NICs can then be told to listen to these special broadcast addresses. This form of broadcast is also called multicasts.

What is P-Mode?

Your NIC has a special mode called Promiscuous. This mode allows your NIC to see all traffic regardless if it’s meant for your machine or not. However, P-Mode is not necessary to see broadcast traffic as your NIC is already listening for these. By default this mode is disabled for each NIC. Even if you don’t enable P-Mode you will still see traffic where your IP or Ethernet address is not involved. This is because of the broadcast traffic. So this is why to see traffic when you click on the “Other Traffic” node in the conversation tree.

Conclusion

Now hopefully you’ll be able to identify broadcast traffic and understand some reasons why this kind of traffic exist. There are many other reasons broadcasts are used and IPv6 uses something similar to IPv4. Perhaps with this knowledge you can inspect traffic and make sure broadcasts are going to the intended network segments as sometimes ill configured routers/switches can make for a noisier than necessary network.

Understanding HTTP Flow with Netmon 3 - By Yuri Diogenes

PaulELong — Fri, 21 Dec 2007 18:37:00 GMT

1. Introduction

One of the most common protocols that we need to deal with these days is the HTTP Protocol. This is not only a privilege of Internet users, there are a lot of Intranet users that also use this protocol for internal transactions.

This post will show how to use Network Monitor 3 to better understand HTTP traffic and also to help you troubleshooting HTTP traffic.

2. HTTP Components

On HTTP we pretty much have two messages: HTTP Request and HTTP Response. The picture below shows an example of these messages:

Figure 1 – HTTP Messages

Here is a brief explanation of the main components of a message:

HTTP Version: the http version in use by the message (ex.: HTTP 1.0 or HTTP 1.1).
Method: this is about the action that the client is requesting to the server (ex.: GET and POST).
Status Code: this code describes what happened on that transaction (ex.: 200, 301 and 407).
Reason: complement to the status code (ex.: OK, NOT OK).
Headers: the content of the header will depend on the version of the HTTP. For instance, HTTP/1.1. has some headers that needs to be present for the method in use (request or response).
Body: some messages will contain the body, which is the data itself. Some other messages will have a blank line.

Based on this brief explanation about the main components of a message, let’s see how NetMon 3 can help us tracking down a HTTP conversation.

3. Understanding HTTP Messages using Netmon3

On this example the server is trying to access the website www.sysinternals.com. This server (Windows Server 2003) is behind a Proxy (ISA Server 2004) and using Integrated Authentication. All the traffic was captured from this server while was accessing this web site.

To help understand the HTTP conversations add the columns “HTTP is Request” and “HTTP is Response”. Those columns will have a number 1 in the column if this sentence is TRUE. This will help to identify what HTTP message was in use at that time.

Figure 2 – Choosing Columns.

For this example is quiet easy to identify the traffic, however on a real world scenario it might be difficult to locate the packet that has the URL request that you want. You might say, “Well, let’s create a filter for this request.” The thing is, if you create a filter for this request you will see only one packet requesting for this URL and this is not what we want here.

There is one cool feature on Netmon3 that allows you to use a filter to find a packet. To use this feature you need to click on the menu Frames than click in Find (or click Ctrl+F). The following window will appear:

Figure 3 – Find Packet based on a filter.

In this case I want to find a packet that matches with the following criteria:

Contains(http.request.URI,”sysinternals.com”)

After typing this and clicking Find the packet that matches with this request is selected as the current frame.

To make even easier to read the trace we can also change the color for the HTTP packets. This will allow you to quickly identify the HTTP traffic. For this example we will set the HTTP Request in Red and the HTTP Response in blue. Follow the steps below to configure that:

Click in Filter.
Click in Color Filters.
Click in Add and type the query specified on the figure below:

Figure 4 – Color Filter feature.

After type this query click, the color (red) and the Bold style click OK.
Click in Add again and now type the following query: protocol.HTTP.response
Choose the color blue and leave it bold
Click OK again.

Here an example of how it will look after you apply the color filter:

Figure 5 – Frame summary after applies the filter.

Now we can close the Find Dialog window and look at the packet. Here is the HTTP part of the packet:

- Http: Request, GET http://www.sysinternals.com/ 
- Request: 
  Command: GET
  - URI: http://www.sysinternals.com/
    + Uri: 
    ProtocolVersion: HTTP/1.0
    Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
    Accept-Language: en-us
    UA-CPU: x86
    Proxy-Connection: Keep-Alive
    UserAgent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4322)
    Host: www.sysinternals.com
    HeaderEnd: CRLF

As you can see, this is a HTTP Request message and some of the components of the messages previously explained appear on this packet. Let’s check the answer for this packet:

- Http: Response, HTTP/1.1, Status Code = 301
  - Response: 
    ProtocolVersion: HTTP/1.1
    StatusCode: 301, Moved permanently
    Reason: Moved Permanently
    Via: 1.1 SRVISA
    Connection: Keep-Alive
    Proxy-Connection: Keep-Alive
    ContentLength: 31
    Date: Sun, 26 Aug 2007 15:05:10 GMT
    Location: http://www.microsoft.com/technet/sysinternals
    ContentType: text/html
    Server: Microsoft-IIS/6.0
    XPoweredBy: ASP.NET
    Set-Cookie: ASPSESSIONIDCCRASDTB=OKKIMCCDOMFAEPIPJCLNPEBN; path=/
    Cache-control: private
    HeaderEnd: CRLF
    + payload: HttpContentType = text/html

Note: it is important to mention that in my lab there were no multiple streams involved. Which make it easier to track it down the answer, since it is the next packet in the sequence.

This HTTP Response message is really important to emphasize one particular point, which is the Status Code.

The status code on this answer is 301. This number itself already says what is going on in this answer. It is important to know at least the meaning of status code based on the number range. The ranges are:

Status Code	Means
200 – 299	Success
300 – 399	Redirection
400 – 499	Error on the client side
500 – 599	Error on the server side

The netmon3 parser for HTTP has the main codes already defined. If you click on the Parser Tab, click on Protocols and HTTP, you will see on the right panel those definitions.

Figure 6 – Netmon3 HTTP Parser.

You also can view this code on the Table object on the Parser tab, as showed below:

Figure 7 – Table View.

Since this is a redirection answer, the field “location” has the place where the page is now located. This is presented to the client (requester) that based on that will send another HTTP Request for this URL.

4. HTTP with Netmon3 Conversation

The conversation feature on netmon3 allows you to view the frames aggregated on the same conversation. For this next example, let’s see the frames aggregated for the HTTP request for the URL www.microsoft.com:

Figure 8 – Filtering by conversation.

Clicking on the conversation tree filters out packets based on the HTTP traffic automatically. This can help to understand the whole conversation that client and server are having during this access. Another way to customize this filter is right clicking on the conversation and chooses the option Copy Conversation Filter to Clipboard” as showed in figure 8. Remember that all filters are applied in combination with the current node that is selected in the Conversation Tree. Be sure to click on the root of the tree if you don’t want the frames to be qualified further by the conversation tree.

Looking at this conversation we can see another status code that means there was an error on the client side:

- Http: Response, HTTP/1.1, Status Code = 407
- Response: 
  ProtocolVersion: HTTP/1.1
  StatusCode: 407, Proxy authentication required
  Reason: Proxy Authentication Required ( The ISA Server requires authorization to fulfill the request. Access to the Web Proxy filter is denied. )
  Via: 1.1 SRVISA
  - ProxyAuthenticate: Negotiate
  WhiteSpace:
  AuthenticateData: Negotiate
  - ProxyAuthenticate: Kerberos
    WhiteSpace:
    AuthenticateData: Kerberos
  - ProxyAuthenticate: NTLM
    WhiteSpace:
    AuthenticateData: NTLM
  Connection: Keep-Alive
  Proxy-Connection: Keep-Alive
  Pragma: no-cache
  Cache-Control: no-cache
  ContentType: text/html
  ContentLength: 4106
  HeaderEnd: CRLF
  + payload: HttpContentType = text/html

The reason why this request was considered an error on a client side is because the ISA requires authentication and the Internet Explorer on the first attempt to access the web site did not send the user credentials. After the response from the server, and depending on the browser and on the configuration, the client will use either NTLM or Kerberos to send another packet with the credentials.

5. General Information

There are many commands that you can use to obtain more information about your HTTP traffic using netmon 3. Let’s see some of them:

Filter	Explanation
contains(Http.Response.StatusCode,"301")	Show all HTTP packets where the Status Code is 301
Property.HttpIsRequest	Show all HTTP Request packets
Property.HttpPragma	Show all HTTP messages that cannot be cached. More information about the Pragma Field, see the HTTP Field definition.

Yuri Diogenes

Security Support Engineer – ISA Server Team

Color Filtering Error Messages

PaulELong — Thu, 28 Jun 2007 23:32:00 GMT

Color Filters in Network Monitor are a simple way to make frames stick out in a trace. Dealing with large traces often makes it difficult to see important information. The sea of data represented by network traffic becomes a difficult backdrop to catch errors that occur. This blog will focus on creating color filters to make these types of errors stick out.

The Protocols

For this blog, I concentrate on the protocols above the transport layer: Kerberos, LDAP, SMB and HTTP. I could have dove into TCP or ICMP as well, but those types of errors are in a different class. For instance TCP resets, don’t always indicate a problem. But this should give you a good background to understand how to create color filters to flag errors for other protocols you work with.

Kerberos

We’ll start with the simplest filter. When we flag an error in Kerberos, we use a structure called “KrbError”. So we’ll simply filter on any frame which has this structure created. We can do this by using the name of the structure as our filter.

KrbError

LDAP

For LDAP, we need to look at frames where the LDAPResult is not zero. But due to an engine quirk, we can’t just search for frames where the Result code is not zero. Instead we’ll search for frames that have a ResultCode, and where the description string does not have success in it.

(!LDAPResult.ToString.contains("Success") && LDAPResult.ResultCode)

I also want to flag Abandon Request for LDAP, since these may also be an indication that something went awry. The following filter catches these.

(LDAPAbandonRequest)

HTTP

HTTP return’s a status code that’s 400 or larger when an error occurs. But one problem is this value is a string. For this filter, we will use the StringToNumber plug-in and convert to a number first so we can use our mathematical operators.

http.Response.StatusCode.StringToNumber >= 400

SMB

SMB has an NTStatus code that is set when an error occurs. The only modification we are going to do here is ignore one specific error. This is because SMB will return an error STATUS_MORE_PROCESSING_REQUIRED (22) when SMB expects more frames with the rest of the data. This isn’t exactly an error, so my filter ignores that specific value.

smb.NTStatus.Code != 0 AND smb.NTStatus.Code != 22

Creating the Color Filter

Now that we’ve determined the various things we want to flag, now it’s time to create the color filter. Just go to the Filter menu and open the Color Filter dialog. Simply click on Add and paste the following.

(KrbError )

(smb.NTStatus.Code != 0 AND smb.NTStatus.Code != 22)

(!LDAPResult.ToString.contains("Success") && LDAPResult.ResultCode)

(LDAPAbandonRequest)

(http.Response.StatusCode.StringToNumber >= 400)

Then choose an appropriate color, I chose red, and exit. Now any problem frames that match our filter will show up as red. Color filters are global to NM3.1, so any new instance of NM3 or any new traces you open will use this new color filter automatically.

Expand to your favorite Protocols

You could continue to do this for every protocol you work with. Sometime trying to find the proper filter is the trick, so hopefully these examples will help you understand different ways of doing this.

Wireless Capturing With Network Monitor 3.1

PaulELong — Fri, 15 Jun 2007 17:22:00 GMT

One of the exciting new features in NM3.1 is the ability to capture wireless network data and management packets on Vista. This new feature provides Network Monitor a useful tool for trouble shooting wireless problems.

What do you mean, wireless Management packets?

With the introduction of NDIS6, we now have the ability to query the OS in a standard way to receive information regarding data that is specific to wireless transmission. The first piece of info we see on wireless frames is stuff like signal strength and data rate. This is available for any wireless card that supports Native WiFi; more on that later. We now append a WiFi structure which contains the 802.11 MAC frame plus Metadata such as Signal strength.

But even more exciting than that, (ok I’m a geek, but I’m guessing you may be one too! J), we can now sniff management packets. These are the cool packets that need to occur in order to find a WiFi Access Point (AP) and that the AP can send out in order to announce itself. Now you can find out what’s going on when your WiFi signal disappears. Or you can see what other AP’s are broadcasting in your area.

Supported Hardware

In this section I will list the current hardware with MS drivers which support Native WiFi, and thus sniffing of management packets. This list is sure to change and be updated as drivers are updated, new adapters are added, or new hardware appears. There is more hardware out there that uses the same chipsets. (We do not have the time to test every single adapter on the market). I will attempt to keep this section up to date, though contacting your vendor may be the most reliable way to get accurate information.

Warning: OEMs (Original Equipment Manufacturer) may change the chipset without modifying the product name or in some instances the version number.

Chipset	Driver	OEM Retail Model
RTL8185	6.1099.312.2007	Xterasys2526g Belkin F5D7010v7 Belkin F5D7000v7 Netgear JWAG511 CompUSA 54Mbps Wireless G PC Card
Ralink RT73	3.0.2.0	Dlink WUA-1340
Ralink RT61	2.0.3.0	Hawking HWPG1
Marvell Libertas (USB)	1.0.0.49	Dlink DWL G122d1
Marvell Libertas (PCCard)	1.0.0.49 1.0.0.52	Trendnet TEW-421PCH/W:B1 Netgear WG511v2 Netgear WG511U
Atheros 5002..5005	7.3.1.42	Dlink DWL G650 Dlink DWI G520 Dlink DWA-642 Netgear WG511U D link DWA-556 Dlink DWA-643 Dlink DWA-552 Dlink DWA-542 Dlink DWA-645

Last updated 6/28/2007 2:10 pm PST

NOTE: That the Windows Logo Kit 1.0c has released. Please verify with your manufacture that your NIC has passed this certification to determine if NM3.1 supports wireless sniffing. The list above will no longer be updated now that the certification is complete.

Manufactures may provide their own drivers which may also support Monitor mode, but you’ll have to contact them directly to see if that is the case. Some information may be altered or omitted by the NIC upon reception of a data or management packet and thus not be correctly presented to NM3. An example is the CRC of packets.

Important Note: Switching into this mode with a driver that has not been verified, may cause your system to hang or blue screen. Be careful and save you data before using NM3.1 on a system with a wireless card.

Wireless Meta Data

As I mentioned above, each wireless packet will have a header. Like Ethernet, this contains the hardware address info, but this may also contain information about the transmission. While this metadata may differ for each vendor, there are some common fields which we return from the driver and display in the frame details. Some of the more interesting fields are listed below:

PhyType – Shows you the physical media type for this packet, for example, 802.11b.

Channel – The physical WiFi channel for this packet. This is usually a number, but the range of which depends on the PhyType and manufacturer. Normally channels for 802.11b range from 1-11. Now you can see if you are using the same channel as your neighbor, and change your AP base channel to improve your connection.

lRSSI – Receive Signal Strength Indicator is a measurement of RF Energy as detected by the hardware. This value does not measure signal quality, only its strength. It is possible to have high strength but not high quality. But you can use this to get an idea of the power of the signal at a given location.

Rate – The current transfer rate. Wireless will change the transfer rate based on the quality of the signal. While you may think you are getting 11 Mbps or 54 Mbps, you may only be getting 1 Mbps!

Cool Wireless Tricks

Now you can track down the dead spots at your location and see if there’s a way to affect your signal strength. For instance you could continually ping your router as you walk around the house. Then setup color filters to signal packets with low or marginal signal strength and/or data Rate. A sample color filter could be set as follows:

WiFi.MetaData.lRSSI < 20 OR WiFi.MetaData.Rate < 10

It’s important to note, that the RSSI value is based on your adapters definition of a max. For instance some cards return a value between 0 and 60, and others between 0 and 100. You’ll have to check with your manufacture for details, but you can probably get a good idea of the max by getting close to your Wireless AP, and using that to approximate your max.

So with your continuous ping going, walk around to places where you normally sit with your laptop and look for any RED frames, or whatever color you chose. You can also experiment with the orientation of your wireless router. You may find you get a better signal strength when you face it a different direction, or even when you turn it on its side.

Working with WiFi Monitor Mode

By default when you start a trace with a wireless adapter, you are normally already connected to a wireless AP. In this mode, you only see traffic to and from your machine and various types of broadcast traffic. But before you have already connected to an AP, the wireless NIC is sending network traffic in order to find an AP to connect to. NM3.1 can put your wireless NIC into monitor mode to see this type of traffic.

Important Note: When you place you WiFi NIC in monitor mode, you will disconnect your current wireless network connection! You will not be able to access the internet or your local network in this mode.

So with a NIC that supports the NWifi standard, NM3.1 can now place your NIC in monitor mode and do some interesting things. With NM3.1 you can perform two types of scanning modes. In the first mode, you select a specific PhyType and Channel to sniff on, and you’ll see all traffic only on that Channel.

In the dialog above, we choose the radio button for “Select a layer and channel”, and then we have the ability to choose one of the PhyTypes (802.11a, 802.11b etc…). And with each PhyType, you get another drop down with all of the available channels for that PhyType.

Once you hit the Apply button, your NIC will disconnect from the AP (you’ll lose your network connection), and set the NIC to monitor traffic on the selected channel. If NM3.1 is currently capturing, the traffic will start capturing this channel only. Also, while in this mode, you must keep this dialog box open. It is actually a separate EXE which will bring up the LUA dialog and ask for permissions when you click on the Properties for a Wireless NIC. Once you close this dialog box, the NIC will return to normal operation and reconnect to the AP as if the machine was trying to connect for the first time. If NM3.1 is capturing, you will see traffic that occurs after the AP negotiation is complete.

You can also put the NIC in a scanning mode. This briefly scans each Channel in each PhyType you have checked and captures traffic. Once the timeout is reached, it moves on to the next selected channel.

If focus is on this dialog, you can see which channel is currently being scanned. This information is updated in the status bar at the bottom of the dialog.

This gives you the ability to capture a swath of data from each channel and determine stuff like, how many APs are available in reach of my machine and what strengths? Or what channels are not being used at all? This could allow you to pick a channel that’s not so crowded and thus increase your wireless throughput. You can also use this to t-shoot why you can’t get connected at all, given you have two wireless NICs or two machines, one to capture and the other to attempt to connect.

A Brand New Sniffing Experience

NM3.1’s new WiFi Features give you a new experience and present new ways to t-shoot problems that were not easy to figure out before. Determining wireless signal strengths and channel usages are just a few of the ways you can improve your wireless experience.

Follow the Stream

PaulELong — Thu, 05 Apr 2007 17:11:00 GMT

This is not some existential trip or search for life's meaning. Rather this refers to a feature protocol analyzers use to narrow down traffic in a network trace. I mentioned this briefly in my Cable Talk Blog back on November 15^th, 2006. In fact you should refresh your memory as we'll refer to topics discussed in that article.

http://blogs.technet.com/netmon/archive/2006/11/15/conversations-in-network-monitor-3-0-cable-talk.aspx

To follow a stream in a general sense means to narrow down the network traffic to a specific conversation. This could mean between two machines, between two specific TCP ports, or even two applications. You can also see how these conversations could be hierarchical; where a conversation between two machines could be further broken into conversations between specific TCP ports.

What's so good about that?

Why is this useful, you may ask? Given that a network trace could have thousands of connections occurring, you'll need a way to find that needle in the haystack. Also, you may be looking at a problem frame and want to look at the rest of the related traffic. Being able to quickly narrow down the packets for related network traffic can help speed up your analysis time on complex traces.

Turn On Conversations

By default we have disabled conversation tracking to save memory and provide better performance out of the box. But in order to benefit more fully from this protocol analyzer, conversations should be enabled before you start a new capture or open a preexisting capture.

Cat Skinning: Method 1

Using the conversation tree is one way to skim through the various conversations that NM3 has exposed. Remember that the parsers define what a conversation is to NM3 and the conversation tree is simply a visual representation of this conversation data. By clicking on a node in the tree, you see the trace summary information reflect only those frames associated with this conversation as well as every conversation built below it.

This is a really nice way to step through each conversation and get an overview of what's happening. In fact, by opening the tree for an IPv4 level conversation with sub-conversations, you can tell how many connections a server has and get some idea of the types of things that are happening on this machine. This is a fast way to differentiate a busy machine from an idle one. A virus that scans ports would be evident because it would create a series of TCP conversations.

Cat Skinning: Method 2

Often, you'll already have a frame in focus and now you'd like to see the rest of the conversation for this frame. Now as hinted to before there may be multiple streams which include this same frame. So you must first decide what context you want to relate the data in.

In NM3, all possible streams are described by the Conv Id Column. This lists each conversation with the protocol and a number which defines the unique number assigned for that conversation. NM3 simply increments a counter for each new conversation, but you can't guarantee the numbers will be the same on two different machines if the parsers are different. Here's an example of that column:

{HTTP:81, TCP:80, ESP:78, IPv4:77}

So using this information you can create a filter to view any of the conversations that this frame is listed a part of. For TCP, a filter of "Conversation.TCP.ID==80" will return all frames that have the same port pair and IP address pairs as this particular frame. What makes this really flexible is that you can show two streams at the same time by combining two separate conversation filters. You might find this helpful when looking at a trace where the server acts as an intermediary (like a proxy server or front end for a SQL DB or Exchange Server). So for instance the following filter will show both TCP conversations 80 and 81.

Conversation.TCP.ID==80 OR Conversation.TCP.ID==81

One missing feature is the ability to track back to the conversation tree from the currently selected frame. While making it a click away would be more ideal, you can make some NPL changes as mentioned in the Cable Talk Blog.

http://blogs.technet.com/netmon/archive/2006/11/15/conversations-in-network-monitor-3-0-cable-talk.aspx

So where's the Data?

While looking at your specific TCP stream, you may want to see the payload data to give you even a better idea of what is going on. For protocols we don't have parsers for, you could use the information in my "Using Columns and Properties" blog:

(http://blogs.technet.com/netmon/archive/2007/03/01/using-columns-and-properties.aspx)

We mention in this article that you can add the TCPPayload column. This payload data described as Text may help to discover what is going on. For other streams, you could use the techniques in the previously mentioned blog to create your own property to display info you need to see. More than likely, however, for any public parsers the summary information should be of great help already.

When you're lost in the jungle…

They often tell you to follow a stream as this will lead you to civilization. Somehow the analogy seems to have similar results in that you can derive some order from the chaos by following a conversation stream. Once you become more accustomed to look at data in this light, you can tame the most unruly of network traces.

Part 2: TCP Performance Expert and General Trouble Shooting

PaulELong — Fri, 26 Jan 2007 21:58:00 GMT

Performance issues are one of the more difficult problems to trouble shoot. Without a baseline, it's often hard to determine if something is really slower. But TCP does contain some built-in behavioral patterns that can be used as a signal to tell you something may be wrong with your network.

So the purpose of this article is to provide some indicators in TCP you can look at to investigate and a way to provide a simple graph of TCP traffic that can help you determine if there is a problem and if so what kind.

TCP Clues

TCP is the layer that is in charge of making sure your packet gets delivered. It tags each packet with a sequence number and when something is missing, the client informs the sender. Below I've listed some general things you can filter on in NM3 to give you clues to see if your network is working properly.

TCP Retransmits:

When dissecting a TCP trace, one of the more obvious problems you can spot are TCP retransmits. A retransmit occurs when a client detects a missing packet. From the sender's perspective, he now has sent the packet twice, so the second packet is called a retransmit. While a certain number of retransmits may occur without causing problem, excessive retransmits may be an indication that your network is sick. In NM3, you can search for retransmitted frames by using the following filter.

Property.TCPRetransmit

One you apply the filter, all frames that have been retransmitted will be displayed. Also each displayed frame will tell you the original frame that it is a retransmit of in the TCP Frame Summary. Some level of retransmits may be acceptable, but varies based on your network topology. Here's an example of how it displays in NM3:

Frame	Time Offs	Source	Destination	Description
457	16.375976	Sndr	Rcvr	TCP: [Continuation to #402]Flags=....A..., SrcPort=1236, DstPort=Microsoft-DS(445), Len=1460, Seq=658111387 - 658112847, Ack=2995420839, Win=65484 (scale factor 0) = 65484
464	16.577148	Sndr	Rcvr	TCP: [ReTransmit #457][Continuation to #402]Flags=....A..., SrcPort=1236, DstPort=Microsoft-DS(445), Len=1460, Seq=658111387 - 658112847, Ack=2995420839, Win=65484 (scale factor 0) = 65484

TCP Fast Retransmits

In some cases you may see multiple ACKs one after another in quick succession. The receiver can send these ACKs to the sender to indicate it is missing a TCP sequence range. Normally, a timeout would occur for the acknowledgement of a particular sequence before a retransmit occurs. However, if the TCP sender supports fast retransmit then it will occur after receiving these multiple ACKs. A retransmission generated by fast retransmit also changes the back-off algorithm used. If a retransmit occurs due to a timeout, then the sender reverts back to "slow start." However, if a retransmit occurs because of a fast retransmit then the sender goes into "congestion avoidance." [See RFC 2581 for more information on Congestion Avoidance and Slow Start.]

The response is called a "Fast Retransmit". This can be the behavior you see from one side when packets get lost in another segment of your network. In NM3 you can search for Fast Retransmits with this filter.

Property.TCPFastRetransmits

An example of a Fast retransmit looks like this:

TCP: [Request Fast-Retransmits #370]Flags=....A..., SrcPort=1268, DstPort=LDAP(389), Len=0, Seq=2021124596, Ack=1458852541, Win=64240 (scale factor 0) = 0

TCP SACK option

The SACK option (selective acknowledgments) is like an ACK, but the difference is that it can keep track of multiple sections of missing data. A normal ACK acknowledges the last consecutive sequence number that it received. In contrast a SACK can keep track of multiple missing segments. The SACK option contains multiple segments relating which pieces it has acknowledged and which are now lost. The number of segments is constrained to the amount of space available for TCP options. You can filter on these as well to see if your network is losing packets.

tcp.TCPOptions.Option.SACK

TCP Resets

Resets aren't always a sign that something is wrong. You should try to look at the traffic around a reset to determine if it looks normal or not. Resets can occur when an application shuts down, or if a router is configure to block a port. But they also occur when a problem occurs in a TCP session. So these can sometimes be an indication that something is wrong. The filter to find resets is:

tcp.Flags.Reset

Where's Waldo – Spotting a bad TCP Connection

Being able to take a trace and visualize it can be a useful way to look for performance issues. Most humans are better at looking at a picture and finding differences than analyzing data. Especially the large sequence numbers involved with TCP traffic.

Limitations with this Expert

Unfortunately, there are limitations to that amount of data this expert can deal with. The specific limitations will be due to the memory/CPU power of the machine you are work on. Excel, which this expert relies on, isn't build to plot 1000's of points. So you may want to limit that amount of data you try to analyze. For the following examples, a 500 MB file was transferred from the client to the server.

Using the TCP Expert

If you are familiar with the Part 1, (http://blogs.technet.com/netmon/archive/2006/11/30/part-1-poor-man-s-expert-using-excel-top-users.aspx), expert that locates the Top Users, then the procedure is much the same. Once you have the Excel Sheet Ready and the NM3 columns aligned, it's just a matter of copying the data in to the clipboard and hitting a button which launches a macro in Excel and creates the graphs we will examine later on.

Setting Up Your Excel Spreadsheet

Basically you just create a new spreadsheet, create a new macro (TCPPerf) and edit it, then paste the code at the bottom of this article. Once you complete this step, you can then take one of the sheets, (I delete all but one of them), and give it a default name like TCPPerf. And finally, create a button on this sheet and attach the TCPPerf macro to it.

Grabbing the data to Analyze from Network Monitor

This section describes how to prepare NM3 so we get the necessary columns for our calculations to appear. In NM3 we can add columns for any property that is exposed by the parsers. Actually you can also add any properties you want as well, so basically any piece of data is fair game. In the case of TCP, we want to add in the Seq/Ack Numbers as well as the Window Size and Payload length. So specifically add in columns for TCP Seq Number, TCP Ack Number, Windows Size, and TCPPayload Length in that order. Place them right after Time Offset. The resulting layout should look as follows.

After opening an existing trace or taking a new one, the next step is to filter down the specific data you want to analyze. In my case, I copied a file from my machine to a server. Since Window Size is negotiated in the TCP 3-way handshake, I made sure to disconnect to my client from the server so that may trace contains the entire conversation. I did this from the server in Computer Management under Shared Folders, Sessions. You can simply right click the session in question and select Close Session. Obviously you don't want to do this if you have something important occurring between this session and the server.

In NM3 it's easy to filter the conversations down, by using the Conversation Tree on the left. [Note: This requires that conversations are enabled when the trace is opened or a capture is started. You can set this option from the Start Page.]

Once you find the IP address pair, you can select each TCP conversation below, and look for the traffic relating to your copy. This traffic will appear as a bunch of SMB Reads (or SMB Writes if you copy to the server), intermixed with a bunch of TCP Continuation traffic.

SMB: R; Write Andx, FID = 0x4004 (\t1_up.tst@#24), 61440 bytes

SMB: C; Write Andx, FID = 0x4004 (\t1_up.tst@#24), 61440 bytes at Offset 184320

TCP: [Continuation to #256]Flags=....A..., SrcPort=1236, DstPort=Microsoft-DS(445), Len=1460, Seq=657940191 - 657941651, Ack=2995420737, Win=64118 (scale factor 0) = 64118

TCP: [Continuation to #256]Flags=....A..., SrcPort=1236, DstPort=Microsoft-DS(445), Len=1460, Seq=657941651 - 657943111, Ack=2995420737, Win=64118 (scale factor 0) = 64118

Once you have located and selected the appropriate TCP Conversation, highlight all the frames involved and copy them to the clipboard. I often use keystrokes, so I did this by selecting the first frame, hit Shift+Ctrl+End, and then right mouse click and copied them to the clipboard.

Now you simply open the Excel Spread sheet you created earlier and click the button. This will results in 4 sheets. The first sheet, PerfData, is just a copy of the data from NM3, plus some new columns that are based off this data. The second sheet, TCPPerf_PerfData, is data copied from the first sheet so that we can sort based off the data we calculated. The final two sheets contain the charts for each side of the transfer. The chart is named based on the machine adress that is sending the data.

Interpreting the Data

Since the resulting data we'll examine is a graph, it's useful to examine different graphs that are a result of different issues so we can refer to these examples as an indicator to the type of problem we may have. In my tests, I have duplicated the following situations: Increased Round Trip Time, Packet Loss, Small Window Size, and Bandwidth Reduction. All traces are taken from the sender (client) which is sending data to the server.

The data is graphed so that the left side axis contains the sequence numbers scaled for Length and UnACKed data. The right side axis is the scaled for the Window Size. Two different scales are used because often the Window size is much larger than the data being sent. UnACKed data is data which have been sent by the sender but has not been acknowledged by the receiver after the TCP timeout expires.

The Graphs

Each test was a copy of around a 500 Meg file from my machine to a server. I used the CMD prompt to do the copy to avoid unwanted Explorer traffic. So we'll start by looking at a base line capture. In this case the Round-Trip-Delay is less than a millisecond, and there are no packets lost. The bandwidth is around 100 MB.

Baseline - < 1 Millisecond RTT, 100 MB, No Packet Loss

In many of the examples I change the horizontal axis min/max so that the data isn't so compressed. You may find it useful to adjust both the max/min scales on the X-axis depending on how much extra trace traffic you took in your original trace.

The pink line represents the length of the data we are sending. For the most part it's a full packet size of 1480. The only reason it dips any lower than that is because SMB sends 61440 bytes of info (0xF000), so the dip at the end is the remainder.

Let's look at the UnACKed data (yellow line). The client continues to send data as fast as it can until it reaches the advertised window size (blue line). Note that since the scales for Window Size and Packet size can be vastly different, window size is put on the right vertical Axis. In this case, however, they happen to line up. This is helpful, because it shows that we keep sending data until the UnACKed data reaches the max advertised window size. Once we get to this point, we have to wait for an ACK from the server before we send more data. So we get in a state where we have to wait for an ACK before sending more data. Then once this particular SMB command completes, another delay occurs while waiting for the SMB response which for the next WRITE command.

Since our RTT is very low, we tend to see little effect of this delay. But note that if we had a larger window size we would have been able to even send more data before waiting for an ACK and thus improve the transfer time.

100 Millisecond RTT, 100 MB

In this sample, I purposely set the Round Trip Time to 100 Milliseconds. The basic effect is that anything that requires a response before it can continue, will incur a 100 millisecond delay. So this will certainly affect the time required to transfer the 500 MB file. As you can see here, a total of about 4 seconds is needed now. Another side affect is that since it's slower, the server is able to keep up with the request so the amount of UnACKed data is generally lower.

100 Millisecond RTT DownStream, 100 MB, 5376 Window Size

In this example we've cut the Windows size down to 5376 bytes. I've zoomed into a small segment of the entire transfer to show the details. The main thing to note is that we cannot fill the pipe up more than two segments. This means the number of segments we have to wait for an ACK for goes up, thus the total time to transfer also goes up. You can see here, for instance that there are many times we have to wait the return ACK before we can send a new packet (about 20 as compared to 5 in the baseline capture).

Down Stream 100 Millisecond RTT, %5 Packet Loss

Now we'll add in some packet loss. In this case the loss is on the Down Stream side, which means that responses from the server never make it to the client. Remember that the client is where the capture is being taken, so the resulting trace won't show any retransmits. Since the server's ACKs are being dropped, the client is affected because he cannot move his sliding window due to the fact that he thinks there is more unACKed data than there truly is. Thus the client has a difficult time sending data at a rate that fills the server's receive window. This is shown in the graph by the slow approach of the UnACKed data to the servers advertised window size.

Down Stream 100 Millisecond RTT, Up Stream %5 Packet Loss

The difference between this graph and the one above it is that in this case the packet loss is on the Up Stream side. So now the client does have to send retransmits, because now the server is complaining that data is missing to the client, instead of the other way around. So now the graph shows up inverted because data to the client can never fill up the window. Instead the client has to resend old data that has been lost. So this resets the UnACKed data to something lower than it was before.

56K Bandwidth Up and Down Stream

The final example is to show the difference when bandwidth is reduced. This is similar to the baseline test, but the main difference is the time line below is much longer. For each SMB Write it takes around 10 seconds. Another difference is that since there is a RTT delay inherent in this connection, you don't see much overlap once we hit the window size. Basically every request has enough of a delay so the we see a acknowledgement before sending the next TCP segment. So we tend to stay right at the Window edge rather than fluctuating, like we do in the baseline test.

TCPPerf Macro Code

You should be able to copy this into Excel's Macro Editor. The one problem I can foresee is that if formatting in your browser causes the text to wrap, Excel will complain. So keep this in mind.

A general overview of this code is that it copies the text from the clipboard and creates some columns to calculate UnACKed data and Advertised Window. We must also recalculate the Seq/Ack columns as there are not represented as numbers (they are numbers with hex in parens). Then we copy all the formulated data to another sheet so we can sort the resulting data. And finally we create a chart for the both sides of the conversation.

Sub TCPPerf()
' TCPPerf Macro

Application.ScreenUpdating = False
Application.Calculation = xlManual

' You can name your sheet different for multiple traces. The resulting
' sheet is created based on this name.
CurSheet = ActiveSheet.Name
' Populate the column headers
[A1].Value = "Frame"
[B1].Value = "Time"
[C1].Value = "TCPSeqData"
[D1].Value = "TCPAckData"
[E1].Value = "Window"
[F1].Value = "Len"
[G1].Value = "ConvID"
[H1].Value = "Source"
[I1].Value = "Dest"
[J1].Value = "Prot"
[K1].Value = "Desc"
[L1].Value = "Seq"
[M1].Value = "Ack"
[N1].Value = "Unack"
[O1].Value = "SrcData"
[P1].Value = "DstData"
  [Q1].Value = "SrcWindow"
[R1].Value = "DstWindow"
  [S1].Value = "AdvertisedWindow"

' Paste In Data from clipboard
Range("A2").Select
ActiveSheet.Paste

' Find Last Row in Data and save in LastRow
Range("A2").Select

Selection.End(xlDown).Select
Dim LastRow As Integer
LastRow = ActiveCell.Row

' Next take the text version of Seq/Ack from NM and convert to a number
Call SeqAckForm(LastRow)

' Create a column for UnAck'd data populate with calculation
Call UnAckData(LastRow)

  ' Create a column that gets the Src/Dest Seq and Advertised Window data
  ' depending on the sender.
Call SrcDestData(LastRow)

' Since we diabled calculations, cause one to occur now before we
' copy data around.
Calculate

' Define the name of the sheet we'll use to build our chart from
DataSheetName = "TCPPerf_" + CurSheet

' Transfer the data and calculated data to a new chart so we can sort.
Call TransferData(LastRow, CurSheet, DataSheetName)

' Build two charts, one for each client/server
Call BuildCharts(LastRow, DataSheetName)
Application.Calculation = xlAutomatic
Application.ScreenUpdating = True

End Sub

Sub SeqAckForm(LastRow)
'
' SeqAckForm Macro
'
' Just cut off the number at the first space and convert to a Number.
[L2].Value = "=VALUE(MID(RC[-9], 1, FIND("" "", RC[-9])))"
Range("L2").Select
Selection.AutoFill Destination:=Range("L2:L" & LastRow), _
Type:=xlFillDefault

[M2].Value = "=VALUE(MID(RC[-9], 1, FIND("" "", RC[-9])))"
Range("M2").Select
Selection.AutoFill Destination:=Range("M2:M" & LastRow), _
Type:=xlFillDefault
End Sub

Sub SrcDestData(LastRow)
' If the Source matches the first source, then take this seq, otherwise
' take the last value found.
[O2].Value = "=IF(H2=H$2, O1, M2)"
Range("O2").Select
Selection.AutoFill Destination:=Range("O2:O" & LastRow), _
Type:=xlFillDefault

' If the Source matches the first dest, then take this seq, otherwise
' take the last value we found.
[P2].Value = "=IF(H2=I$2, IF(P1<>0, P1, L2), M2)"
Range("P2").Select
Selection.AutoFill Destination:=Range("P2:P" & LastRow), _
    Type:=xlFillDefault

' If the Source matches the first source, then take window to be the
' src window, otherwise take the last value we found.
[Q2].Value = "=IF(H2=H$2, Q1, E2)"
Range("Q2").Select
Selection.AutoFill Destination:=Range("Q2:Q" & LastRow), _
    Type:=xlFillDefault

' If the Source matches the first dest, then take window to be the
' src window, otherwise take the last value we found.
[R2].Value = "=IF(H2=I$2, R1, E2)"
Range("R2").Select
Selection.AutoFill Destination:=Range("R2:R" & LastRow), _
Type:=xlFillDefault

' Now get the advertised window size from the other side based on which
' is the source address
[S2].Value = "=IF(H2=H$2, Q2, R2)"
Range("S2").Select
Selection.AutoFill Destination:=Range("S2:S" & LastRow), _
    Type:=xlFillDefault
End Sub

Sub UnAckData(LastRow)
' Calculate the Unack'd data by looking at the seq/ack columns we created
' before and the len field.
[N2].Value = _
    "=IF(IF(H2=H$2,L2+F2-O2,L2+F2-P2)<0,-1,IF(H2=H$2,L2+F2-O2,L2+F2-P2))"
Range("N2").Select
Selection.AutoFill Destination:=Range("N2:N" & LastRow), _
    Type:=xlFillDefault
End Sub

Sub TransferData(LastRow, OriginalSheetName, DataSheetName)
' TransferData Macro
'
' Add a new sheet for new data and charts.
Sheets.Add
ActiveSheet.Name = DataSheetName

' Copy all of the various columns we need from the original sheet.
' This is done so that when we resort the data, we reference the
' formula results.

Sheets(OriginalSheetName).Select
Range("B1:B" & LastRow).Copy
Sheets(DataSheetName).Select
Range("A1").Select
ActiveSheet.Paste

Sheets(OriginalSheetName).Select
Range("F1:F" & LastRow).Copy
Sheets(DataSheetName).Select
Range("C1").Select
ActiveSheet.Paste

Sheets(OriginalSheetName).Select
  Range("N1:N" & LastRow).Copy
Sheets(DataSheetName).Select
Range("D1").Select
Selection.PasteSpecial Paste:=xlPasteValues, Operation:=xlNone, _
    SkipBlanks:=False, Transpose:=False

Sheets(OriginalSheetName).Select
Range("H1:H" & LastRow).Copy
Sheets(DataSheetName).Select
Range("E1").Select
ActiveSheet.Paste

Sheets(OriginalSheetName).Select
Range("L1:M" & LastRow).Copy
Sheets(DataSheetName).Select
Range("F1").Select
Selection.PasteSpecial Paste:=xlPasteValues, Operation:=xlNone, _
    SkipBlanks:=False, Transpose:=False

Sheets(OriginalSheetName).Select
Range("A1:A" & LastRow).Copy
Sheets(DataSheetName).Select
Range("H1").Select
ActiveSheet.Paste

Sheets(OriginalSheetName).Select
Range("S1:S" & LastRow).Copy
Sheets(DataSheetName).Select
Range("B1").Select
Selection.PasteSpecial Paste:=xlPasteValues, Operation:=xlNone, _
    SkipBlanks:=False, Transpose:=False

' Now sort this data by Source so we can split and create 2 charts.
Range("A1:I" & LastRow).Select
Selection.Sort Key1:=Range("E2"), Order1:=xlAscending, Header:=xlGuess, _
    OrderCustom:=1, MatchCase:=False, Orientation:=xlTopToBottom, _
DataOption1:=xlSortNormal
End Sub

Sub BuildCharts(LastRow, MainSheetName)
' Find where the source changes to destination.
Range("A1:G1").Copy

For Each Cell In Range("E2:E" & LastRow)
    If Cell.Value <> [E2] Then
      Exit For
    End If

    Cell.Select
Next

' Paste in the original column headers
CurCellRow = Selection.Row + 1
Range(CurCellRow & ":" & CurCellRow).Insert Shift:=xlDown

' Create two names for each chart based on the the senders name.
ChartName1 = Range("E" & (CurCellRow - 1)).Value
ChartName2 = Range("E" & (CurCellRow + 2)).Value
Range("A1:D" & CurCellRow).Select

' Now create a chart for each address.
Call Chart(1, CurCellRow - 1, ChartName1, MainSheetName)
Call Chart(CurCellRow, LastRow + 1, ChartName2, MainSheetName)
End Sub

Sub Chart(FirstRow, LastRow, Name, MainSheetName)
' Set the range of the data we will use for our chart.
MinScale = Range(MainSheetName & "!A" & FirstRow + 1).Value
Sheets.Add

' Create a Name for this chart and set teh active sheet to that name.
TopUsersChart1Sheet = "Chart_" + Name + CurSheet
ActiveSheet.Name = TopUsersChart1Sheet

' Add in a new chart.
Charts.Add
ActiveChart.Location Where:=xlLocationAsObject, Name:=TopUsersChart1Sheet
ActiveChart.ChartType = xlXYScatterLines
ActiveChart.SetSourceData Source:= _
    Range(MainSheetName & "!$A" & FirstRow & ":$D" & LastRow)

' Create a secondary access
ActiveChart.SeriesCollection(1).AxisGroup = 2

' Set the min/max scale based on the time in the first/last row.
ActiveChart.Axes(xlCategory).MinimumScale = _
    Range(MainSheetName & "!A" & FirstRow + 1).Value
ActiveChart.Axes(xlCategory).MaximumScale = _
    Range(MainSheetName & "!A" & LastRow).Value

' Make the min Y scale -1
ActiveChart.Axes(xlValue).MinimumScale = -1

' Make the long time lables slant a bit.
ActiveChart.Axes(xlCategory).TickLabels.Orientation = -25
End Sub

And Much More…

The general idea here is that you can take the TCP data and look at it in a graphic form to help you to see at a high level if there's a problem. There are so many other things that one could add. Resets could be added in to visually indicate those in the graph. You could also create other graphs to represent the seq/ack responses that would give you another view of the data. But hopefully this will give you a simple tool and some specific filters to spot check performance issues on your network.

Conversations in Network Monitor 3.0 – Cable Talk

PaulELong — Wed, 15 Nov 2006 20:48:00 GMT

Conversations for network protocols have been around for ages. But protocol analyzers have not been built to take advantage of them in the way NM3 does. Using the conversation tree and filters that reference conversations, make NM3 a powerful ally when trying to narrow down traffic.

It's important to note that conversations are disabled by default in NM3. This is because conversations tend to eat up memory and we wanted to be able to capture for long periods of time. You can turn on conversations from the start page or in options.

What is a Conversation with regards to networking?

In layman terms, a conversation is simply a communication stream, normally between two machines. When you call and talk to your friend on the phone, the stuff you talk about is your conversation. With networking protocols, it is much the same thing, but as with everything else in computers there are more levels of complexity.

When two machines talk to each other with a specific protocol, there is often a set of parameters that each frame has in common. Normally there is a value or values that are associated with that connection. When you look at that protocol in any frame, you should be able to determine what other frames are related by looking for the same values.

The most familiar type of conversation is the one TCP sets up when it creates its communication stream. When you computer talks to another using TCP, the first thing it does is negotiate which ports to use. This source/destination port pair defines what the conversation is for these two machines.

Conversations are Hierarchical!

Now one catch is that the source/dest port pair for TCP is specific to the pair of machines that are talking. So you may find other frames with the same source/dest port pairs, but these are not part of the same TCP Conversation. So it's important to understand that conversations can be hierarchical. The TCP source/dest pair is also dependant on the IP source/dest addresses.

So for a pair of machines, defined by their source/dest IP address, there may be many TCP conversations. And then on top of TCP, there may be multiple NBT, SMB, HTTP or other types of conversations.

Conversations not limited to connection oriented protocols

Certain protocols like UDP or ARP are called connectionless. They don't guarantee delivery of a packet and have no sense of sequence numbers. But this doesn't mean that they can't be associated in a conversation. For UDP we can use the port numbers to group those frames together. For ARP we associated the sender/target address and combine those frames into a single conversation. The only thing we need for a conversation is some basic set of parameters that we can match to say one frame is related to another.

Mommy, where do conversations come from?

When a parser is written, you have the option to define conversations for that parser. In fact you could define multiple conversations for the same parser. I won't go into all the details, but basically you tell the engine what parameters to use to correlate like packets. You can build the conversation based on the parent, or base them solely on parameters in the current frame. When you build them on the parent, you get the hierarchical nature I'm talking about above.

How can I use conversations to troubleshoot problems?

In Network Monitor 3.0, when you click on a node in the conversation tree, it automatically filters and shows you only traffic pertaining to that conversation. So for instance, if you click on an IPv4 node that is equivalent to Conversation ID 5, a filter of "Conversation.IPv4.Id == 5" is enabled. This doesn't change the display filter physically, but implicitly applies this filter so you only see that traffic displayed.

You are of course free to filter further by applying a display filter. If you right click the node in the tree and select "Copy Conversation Filter to Clipboard", you can see what filter is being applied. This will require you paste the clipboard text somewhere so you can see it. This is an easy way to add this permanently to your display filter.

This is basically part of the same kind of functionality Wireshark users know as "follow the stream". The difference though is that you can follow TCP, IPv4 and many other types of streams.

Since the conversation tree is hierarchical you can quickly see the traffic flowing between two IPv4 nodes. So if you know the machine you are interested in, you can see all traffic by clicking on the node. Further more you can drill down into each TCP conversation below it.

By the way, you'll notice the ConvID in my display that you may not have in yours. I'll explain more on this later.

So you can continue to drill down until you find the traffic you'd like to example more closely. Different protocols have different levels of information. For example you can see SMB broken down by file name followed by the SMB File ID.

In this example, you can see that HTTP traffic will list the initial HTTP command and URL. This makes it easy to find web traffic in question.

Certain parser features depend on conversations

Conversations have a purpose other than grouping frames. Today they are also used to hold state information that may be useful to other frames in the conversation. For instance, TCP keeps track of sequence numbers so it can detect retransmits and lost segments. This is done in the conversation so that you don't get collisions with duplicate sequence numbers on different ports.

Info on the ConvID = lines in My Display

Until we get some easy way to associate displayed frames with the conversation tree, I added some NPL code to display this info. This is fairly simple to add if you want the same functionality. Here's an example of how I did it for IPv4.NPL. The same thing applies to any conversation.

In NPL we create the description by calling ConversationDescription. So below you can see I simple add in the property ConvID when the conversation description for IPv4 is created.

Post.Conversation.ConversationDescription = FormatString("(%s - %s) ConvID = %d", Source, Destination,ConvID)

The Future of Conversations

Conversations are new and over time will evolve and change. There are already requests to associate the current frame with the node in the tree. Others have suggested we provide a way to filter the tree to only show relevant information. We are also hoping to someday to show specific processes and break out the traffic specific to each process.

Into to Filtering with Network Monitor 3.0

PaulELong — Tue, 17 Oct 2006 19:22:00 GMT

Challenges of Filtering

One of the biggest changes between NM2.x and NM3.0 is the way you do filtering. Old NM2.x hacks may be challenged by the loss of the UI wizard to build filters. On the other hand, Ethereal users may be pleased and further encouraged by the built in intellisense, but more on this later. Hopefully I'll be able to ease the transition and provide some tips and tricks for filtering with NM3.0.

What exactly does filtering do?

When you type in a filter and hit apply, each frame is evaluated against the filter. If the result of the filter is TRUE, then the frame is displayed (or captured in the case of a capture filter). So to start simple, if you want to see every frame where the TCP source port is 80, you could type:

Tcp.SrcPort == 80

In this example, the parser engine looks at each frame and evaluates this expression. If the current frames port is set to 80, then it includes that frame in your view.

Where do I start?

Intellisense is technology in many MS products that shows you a list of possibilities given some starting place. In the context of filtering, it allows you to see the available data items for a given protocol or structure. So if you type "TCP." in any of the filter windows you will see a list of available data fields. If you further type in "TCP.Flags." you then see a list of the names for each of the TCP flags. Now unfortunately in the Beta2 version of NM3.0 we only show you two levels deep. So when you type "TCP.Flags." you won't actually see anything more. But in the released version of NM3.0, we should have this implemented fully.

So one little known trick is that you can start off with a "." (period) and you will see intellisense for all the top level items.

In this list you can see stuff like Protocol, which will show you a list of all protocols underneath. So when you are not sure where to start, this is as good a place as any.

Built-in Filters (Standard Filters)

We've actually included a bunch of built in filters with the product. These are common filters that do mostly general things, like target a specific protocol, or narrow traffic down to a specific machine. Once you choose a filter the filter text box is populated with the filter, and you can then apply it. You may have to change some part of the filter, as often they reference placeholder for IP Addresses or Ports. These filters provide a bunch of examples that may help you understand how filters work.

You can also save your own filters and bring them up in other sessions. This lets you access your most used filters easily.

Applying a filter

In order to apply a filter in NM3.0, you can either press the button (paper with a pin on it), or you can hit Ctrl+Enter. One advantage to using the key stroke, Ctrl+Enter, is that it always applies the current filter. When you use the button, you may have to turn off the current filter if you have one applied already. This UI glitch will probably be something we address in the future. Note that we use Ctrl+Enter because our filters can be multiple lines. Having multiple lines helps readability and allows you to add comments as well.

What do I want to filter on?

I suppose it depends on what you are looking for. In general you have some problem you are investigating, or perhaps you simply want to get rid of all uninteresting traffic. Filters allow you to narrow down the traffic and see only the data you want to focus on. You'll often start with something and then further narrow down what you are looking for by adding expressions separated by ORs and ANDs.

Filters can reference anything in the NPL. So this includes protocol data fields like we mentioned above. But this also includes Properties, which are derived by NPL and usually based on the data.

For example, the value of the TCP Window size is a combination of the Windows Scale and the TCP Window Size. So we could create a property to hold the real window size. Each column, with a few exceptions, is also just property value. So this means you can search any column for data in a filter as well. We'll show some examples of this.

Show me all frames where X exists…

So let's start with a simple filter to find all the ARP packets.

ARP

OK, almost too easy. A simple principle to remember is that by simply typing a protocol, structure or property you filter for the existence. So when you type ARP, you are looking for any frame which parses as an ARP protocol. Similarly if you wanted to get rid of all ARP frames, you just say

NOT ARP

Also if you like C like terminology you can type

!ARP

So let's apply this same principle to a structure. Say we want to find all TCP traffic where SACKs are used (Selective Acknowledgments). You don't care what the values of the SACK are, you just want to show only those frames where they exist. So your filter would be:

Tcp.TCPOptions.Option.SACK

And finally, let's apply this to a property. In TCP we have a property called "TCPRetransmit" that gets set whenever a retransmit is found. By the way, this property requires that conversations are enabled, which isn't the default in NM3. So to find all retransmit frames, just type:

Property.TCPRetransmit

Actually the Property portion is only needed if the term was defined as something other than a property as well. Preceding it with the Property suffix makes sure you are referencing it correctly in case it is defined as a structure or protocol too. It never hurts to begin you terms with Property, Struct, or Protocol as called for.

Using equivalence and comparative operators

Another type of filtering that is often required is to look for a specific value in a trace or a value within a certain range. For example, say you wanted to look for traffic on a certain port. In this case you would type a filter like:

Tcp.Port == 5555

This would return any frame with a port (either source or destination) is 5555. What's interesting here is that Port is defined as a "pair". This simply means that we pair up the source and destination port so you don't have to. If you were to explicitly type this out using source and destination ports you would have to type:

Tcp.SrcPort == 5555 OR Tcp.DstPort == 5555

But the Port "Pair" property takes care of this for you. This makes creating filters with paired properties much easier. Since NPL defines these pairs, this could be established for any pair of terms that act this way. Pairs have also been created for Ethernet and IP addresses. So to filter on frames that involve at least one IP address containing 192.168.1.1, you would type:

IPv4.Address == 192.168.1.1

And similarly for Ethernet addresses:

Ethernet.Address==0x1185AE4E95

It's also important to note that when you use this in the negative case, != (not equals), the expansion is different.

Tcp.Port != 5555

Expands to:

Tcp.SrcPort != 5555 AND Tcp.DstPort != 5555

This should be no surprise to you Boolean math heads, but for the commoner this may seem incorrect at first. The typical reaction is to use OR instead of AND. But this will show you only frames where either port is not 5555.

You can also use all the comparative operators like >, <, >=, <=. So if you wanted to search for instances where the window size started getting small, you could so something like.

Tcp.Window < 1200

Now as I mentioned before, if windows scaling is enabled, this may not be the real size. So a better way to do this in NM3.0 is to use the property that calculates the real window size. So this filter would be:

Property.WindowSize < 100

Using Contains to search strings

A common task is to be able to search ASCII and Unicode strings. You can use the Contains plug-in to do this. It searches the associated string and ignores case. This can be used in two different ways, with the same results. Use which ever method you feel more comfortable with.

Contains(property.Description, "error")

You can also use as an operator on a string object.

Property.Description.Contains("error")

As I mentioned before, you can search any property. And remember that most columns are just properties. The exceptions are FrameNumber, TimeOffset, TimeDelta, TimeOfDay, and ConvID. But the rest are fair game. So in this case we can search the description property text using the Contains plug in. Note that using this to search binary data doesn't work. This is something that will probably be addressed in future versions, as it is useful to search binary data.

Generating compound statements by using AND's and OR's:

When narrowing down frames, you'll often start by using one expression to filter the frames down and then you'll want to add other expressions to further restrict your search. So let's say I wanted to look for all frames on port 5555 that also have a window size less than 100 bytes.

Tcp.Port == 5555 AND Property.WindowSize < 100

You can also use the C like shorthand:

Tcp.Port == 5555 && Property.WindowSize < 100

Or to search for traffic between two machines, you could type

IPv4.Address == 192.168.1.1 && IPv4.Address == 10.0.0.1

You have to be careful when using AND's and OR's. The English language tends be more ambiguous when using these terms. Asking a car dealer to see all the Red and Green cars may make sense to both you and him, but the same query to NM3.0 would result in a potentially ugly car that maybe only Santa would appreciate.

Here's another example for filtering on subnets.

(IPv4.Address >= 10.53.0.0) && (IPv4.Address <= 10.53.255.255)

In this example, it's important to understand that the address is really just 32 bits of data. So checking if an address is between the lower and upper bounds of the networks range, is really the same thing as checking if it's in the same subnet.

Using math operators in filters

It's also possible to use math operators, like +, -,*, /,&, and | in filters. The last two being "bitwise and" (&) and "bitwise or" (|). As an example, we can use this to filter out on a subnet, but this time using the "bitwise and". This simulates what subnet mask does.

((ipv4.SourceAddress & 255.255.0.0) == 10.53.0.0)

((ipv4.DestinationAddress & 255.255.0.0) == 10.53.0.0)

Comments in the filter window

It's also possible to add comments in the filter window. This helpful if you want to document how a filter works. This also allows you to comment out a section temporarily so you don't have to remove that portion of the filter completely. Comments can be used with either // for a single line comment and /* */ if you want to comment more than one line.

Extra Credit, modifying NPL to filter on a new property IPTTL

We talked before about searching on properties. There may be special cases where you want to create a property yourself so that you can search on it. For this example, we'll create an IPTTL property that can reference both the IPv4 and IPv6 hop count values.

So the first step is to modify both IPv4.NPL and IPv6.NPL to add our property. For IPv4, we'll add the property to the TimeToLive data field and for IPv6 the HopLimit data field. In NPL properties are placed in square brackets "[ ]" before the data field definition. The bracket section can contain multiple lines separated by commas but we won't have to worry about this for our example.

In IPv4.NPL, we see that the TTL parameter is defined as follows.

…

};

UINT8 TimeToLive;

[NextProtocol]

UINT8 NextProtocol = FormatString("%s, %d(%#x)",

ProtocolTypeTable(this), this, this);

So what we do is add in property called IPTTL as follows:

[IPTTL]

UINT8 TimeToLive;

The property automatically attaches itself to the data that follows it. So now let's modify the IPv6 parser. Here's how it looks in its original form.

[NextHeader]

UINT8 NextProtocol = FormatString("%s, %d(%#x)",

ProtocolTypeTable(this), this, this);

UINT8 HopLimit;

So we'll make the same change here:

[IPTTL]

UINT8 HopLimit;

The properties are named the exact same, so we now can reference the same property name. If IPv4 exists it references TimeToLive, and if IPv6 exists we reference HopLimit.

You must now reload the parsers. This is similar to a recompile as we want to save time each time you run NM3.0. This can be done from the parsers tab. You can hit the button from the tool bar, select Tools Reload Parsers, or type Ctrl+Alt+B.

Once the parsers are reloaded, you can then use this in a filter as follows:

Property.IPTTL == 0

This will return all frames where the TimeToLive or HopLimit is set to zero.

And that's your filtering introduction

While it takes a while to get good at filtering the discussion above should give you a good premier. Hopefully this will help you understand the basics of how to use filtering to find the data you want with Network Monitor 3.0.