Network Monitor 3.2 has arrived!

I’m so excited about this release I had to commandeer Paul’s blog for the day and write about it. My name is Tawanda Sibanda and I am the lead program manager for Network Monitor. The team put a lot of effort into this version, adding many of the requests we heard from our customers, while simultaneously fixing bugs and stabilizing the product. We hope you enjoy this release! Please feel free to leave feedback on Paul’s blog or participate in our Network Monitor survey

Now, let’s deep dive into some of the details of Network Monitor 3.2

Where are the bits?

Network Monitor 3.2 is available on Microsoft.com. The link is: http://www.microsoft.com/downloads/details.aspx?FamilyID=f4db40af-1e08-4a21-a26b-ec2f4dc4190d&DisplayLang=en.

Network Monitor 3.2 will also be offered as an optional feature package in the next few weeks via Microsoft Update if you have a previous version of Network Monitor 3.x installed. To check for updates, click on Help>Check for Updates from the product menu (versions 3.1 and higher) or visit the site http://update.microsoft.com/Microsoftupdate.

So What’s New in Network Monitor 3.2?

• Process Tracking: Now you can identify rogue applications sending network data! View all the processes on your machine generating network traffic (process name and PID).  Use the conversation tree to view frames associated with each process.

• Capture engine re-architecture to improve capture rate in high-speed networks.   Network Monitor 3.2 drops significantly fewer frames that Network Monitor 3.1.
• Find conversations:  You asked for this. Many of our users found conversation tracking to be difficult to use as the view grew hard to manage, and it was difficult to correlate the frames they were seeing with the conversation nodes in the tree. Now, you can quickly isolate frames in the same network conversation.  Just right-click on a frame and select a conversation to track, and you will see all the frames in that conversation. View TCP streams, HTTP flows etc.

• Extensive parser set:  Parsers for over 300 protocols!  As before, the parsers are fully customizable.
• Better parser management:  By default only a subset of parsers are loaded.  You can load the full parser set by going to Tools>Options>Parser and choose Full vs. Stub implementations.
• CodePlex Ready:  In the upcoming months, we plan to place all our Windows parsers on the Microsoft open-source CodePlex site and allow the community to modify and contribute parsers.  You can find out more information on this here. This version of Network Monitor seamlessly integrates new parser packages.
• Network Monitor API: Create your own applications that capture, parse and analyze network traffic!
• More extensive documentation of the API and NPL.  Access the documentation from Help > NPL and API Documentation.
• IA64 builds.
• PCAP capture file support*.
• ContainsBin Plug-in:  Search frames for arbitrary byte sequences or strings.
• … and more.  See our Release Notes in the Help directory of the installation folder for a complete list of new features and known issues.

Enjoy!

*This feature includes software developed by the University of California, Berkeley and its contributors.
  This feature includes software developed by the Kungliga Tekniska Hogskolan and its contributors.
  This feature includes software developed by Yen Yen Lim and North Dakota State University.

Open Source Network Monitor Parsers

With the release of Network Monitor 3.2 we plan to do all of our parser development as an open source project on http://www.codeplex.com/NMParsers. We will be releasing parser packages for Microsoft Protocols on a regular schedule. All parser issues will be tracked, assigned, and fixed on CodePlex first. Then on a regular schedule, we will post a new installer package which will update your NM3.2 parsers.

Open Documentation = Open Parsers

In the beginning of this year we released the documentation of all the Windows Protocols on http://msdn.microsoft.com/en-us/library/cc216517.aspx. With NM3.2, we have also created parsers for most of the released protocol documentation. And as time passes, we will continue to update the released parsers as well as introduce new parsers.

3^rd Party Development Welcome

So now it’s your turn to contribute. If you see a problem with a parser or decide that there’s a better way to describe the data, you can file an Issue Report against the parser and track it through CodePlex. If the change is implemented, the next install of the Microsoft Parsers will contain your change. You will also have access to the latest version of the NPL if you want to integrate it manually.

Also, if there’s a parser you want to share with the community, you can share your parsers here as well. We will be providing information on how to build a parser package as well as providing guidance on how to test your parsers. And the main project page will be updated to list all available packages.

Stay Tuned

Since the parsers will release with NM3.2 as well, the first release of a parser only package will happen sometime soon after. We are currently shooting for a monthly release cycle. You should see the first parser package posted in October 2008. Stay tuned to that site for updates and announcements.

Broadcasts

We can categorize traffic into two general types; directed and broadcast traffic. In the most general sense, a broadcast is sent to anybody that wants to listen. What I’d like to talk about is how broadcasts work and what they are used for.

How broadcasts work

In general a broadcast is a special address. Different layers, in particular the hardware and network layers have specific addresses defined as broadcasts. So let’s look at each layer specifically.

Hardware Layer:

This layer is normally controlled completely by hardware. It defines how the electrical signals go across the wire and how to decide what traffic you network adapter should listen to. In reality network traffic is always broadcasted to everybody on the same segment. Switches and Routers will further determine how segments are defined and may block or allow this type of traffic.

Your network adapter is setup to “listen” for a hardware address that is assigned to it. This 6 byte address is usually hard coded by the network card manufacturer. A manufacturer is assigned a group of addresses and they make sure they are all unique when shipped. When your network adapter sees an electrical signal, it decodes the Ethernet header information and checks if it’s one of the addresses it’s listening for. If it matches, then it sends the packet to the OS for more processing.

Here’s an example of the Ethernet portion of a packet.

    - Ethernet: Etype = Internet IP (IPv4),DestinationAddress:[FF-FF-FF-FF-FF-FF],SourceAddress:[00-06-5B-61-2E-7A]
     + MacAddress DestinationAddress: *BROADCAST [FF-FF-FF-FF-FF-FF]
     + MacAddress SourceAddress: Dell Computer Corp. 612E7A [00-06-5B-61-2E-7A]
       UINT16 EthernetType: Internet IP (IPv4), 2048(0x800)

You can see that the Source Address is comprised of 6 bytes, 00-06-5B-61-2E-7A. The first 3 bytes, 00-06-5B, indentify it as the Dell Computer Corp. The final 3 bytes are unique and set by the manufacturer.

In wireless, all traffic is inherently broadcast based. The air is the medium, ok not really air, but the idea is that everybody sees that traffic. But again once your wireless adapter sees the traffic; it will inspect the address and if it matches, it knows to send to the OS.

Broadcasts at this layer are special addresses that are industry defined. Each NIC is configured to listen to its personal address and any broadcast related traffic. For Ethernet, the broadcast is simply an Ethernet address of FF-FF-FF-FF-FF-FF.

So for instance, if we look at an ARP packet with NM3.2, you can see that the destination address is FF-FF-FF-FF-FF-FF, and there for while be listened to by any machine that receives the packet.

    Ethernet: Etype = ARP,DestinationAddress:[FF-FF-FF-FF-FF-FF],SourceAddress:[00-07-B3-29-F8-00]

Network Layer:

The network layer in the case of IPv4 and IPv6 also does the same thing. The only difference here is that it may also include information on how to route traffic. For instance an IP address will determine if traffic is local or not for outgoing traffic.

[See http://blogs.technet.com/netmon/archive/2007/07/20/intro-to-name-resolution.aspx for more info about routing.]

When incoming traffic arrives at the IP layer, it again checks the address to see if it’s something it should be listening for. IP addresses however are not hardcoded. They are assigned and a NIC can listen on multiple IP addresses if it wants.

At this layer the broadcast again is defined as a specific IP address. For IPv4, this could be 255.255.255.255. This can also be restricted to the current sub net, so for instance 192.168.1.255 for a class C network. (For more info on IPv4 classes see http://en.wikipedia.org/wiki/Classful_network).

Why have Broadcasts?

You may have already figured this out but broadcasts are used to send information that every machine on the same segment. For instance, when IPv4 needs to see if an address is already taken it will broadcast an ARP packet and ask if an address is available. In fact, you may have already seen that ARPs tend to always be broadcasts.

    Frame: Number = 41, Captured Frame Length = 60, MediaType = ETHERNET 
    + Ethernet: Etype = ARP,DestinationAddress:[FF-FF-FF-FF-FF-FF],SourceAddress:[00-07-B3-29-F8-00]
    + Arp: Request, 192.168.100.253 asks for 192.168.100.13
    

Broadcasts are also used to announce general things like their names to everybody. When NETBIOS starts up, it sends out an announcement to anybody who’s listening to see if this name has been used before. When the name is heard machines called browse masters listen to these names and record them. This allows you to see all the machines on your local network. Your machine may ask the browser for a complete list.

    + Ethernet: Etype = Internet IP (IPv4),DestinationAddress:[FF-FF-FF-FF-FF-FF],SourceAddress:[00-1D-09-AB-5D-0A]
    + Ipv4: Src = 192.168.1.5, Dest = 192.168.1.255, Next Protocol = UDP, Packet ID = 3092, Total IP Length = 78
    + Udp: SrcPort = NETBIOS Name Service(137), DstPort = NETBIOS Name Service(137), Length = 58
    + Nbtns: Query Request for MachineX <0x00> Workstation Service

When a computer boots, often broadcasts are used to ask the DCHP server for an address. In fact each time you boot a machine without an IP address, it must use a broadcast to find and communicate with the DHCP server. The message below is discover request that is sent when a machine is looking to find its first IP address.

    + Ethernet: Etype = Internet IP (IPv4),DestinationAddress:[FF-FF-FF-FF-FF-FF],SourceAddress:[00-60-08-01-D3-03]
    + Ipv4: Src = 0.0.0.0, Dest = 255.255.255.255, Next Protocol = UDP, Packet ID = 288, Total IP Length = 328
    + Udp: SrcPort = BOOTP client(68), DstPort = BOOTP server(67), Length = 308
    + Dhcp: Request, MsgType = DISCOVER, TransactionID = 0x83484743

Another application, and perhaps the first thing you had in mind when you heard the word broadcast, is sending audio or video data to minimize the network utilization. Rather than sending a separate feed to everybody, you can broadcast to a general address, and NICs can then be told to listen to these special broadcast addresses. This form of broadcast is also called multicasts.

What is P-Mode?

Your NIC has a special mode called Promiscuous. This mode allows your NIC to see all traffic regardless if it’s meant for your machine or not. However, P-Mode is not necessary to see broadcast traffic as your NIC is already listening for these. By default this mode is disabled for each NIC. Even if you don’t enable P-Mode you will still see traffic where your IP or Ethernet address is not involved. This is because of the broadcast traffic. So this is why to see traffic when you click on the “Other Traffic” node in the conversation tree.

Conclusion

Now hopefully you’ll be able to identify broadcast traffic and understand some reasons why this kind of traffic exist. There are many other reasons broadcasts are used and IPv6 uses something similar to IPv4. Perhaps with this knowledge you can inspect traffic and make sure broadcasts are going to the intended network segments as sometimes ill configured routers/switches can make for a noisier than necessary network.

NM3 TV – Video Help for using NM3

If a picture is worth a thousand words, than a video must be worth, …30fps * 1000 * …uh…ok…, like a whole lot. So this blog is going to be devoted to providing short help videos to help understand how to use the features of NM3.

My original intention was to provide a cool Silverlight interface with the video right here on this page. But given the crawl, walk, run mantra for developing NM3, I decided to take that same approach here.

Here’s the list of videos, though not all are available now. I will continue to update the page as I complete each video.

• NM3 Overview
• UI Overview Part 1 – Start Page and Parser Tab (Alt Links: YouTube)
• UI Overview Part 2 – Capture Tab (Alt Links: YouTube)
• Tool Options
• Capturing with the UI (Alt Links: YouTube)
• Capturing using NMCap (Alt Links: YouTube)
• Advanced NMCap usage
• Filtering (Alt Links: YouTube)
• Advanced Filtering
• Conversation Tree (Alt Links: YouTube)
• Reassembly (Alt Links: YouTube)
• Parsers Tab
• Simple NPL Modification
• Creating a new Parser (Alt Links: YouTube)
• NM API Overview
• Building an API Example

Hopefully this increases your effectiveness with NM3. Feel free to provide feedback about other topics you’d like to see covered.

The Swimming Pool and the Drain Pipe

With the introduction of Network Monitor 3.2, we have introduced a new feature that allows us to buffer frames so that packets are not dropped. While this is a wonderful improvement this new feature introduces some side effects that you may be interested in.

First there was just a Drain Pipe

Before NM3.2, we had only a Drain Pipe. The problem was that if you tried to cram enough network data through that pipe, it could overflow and drop packets. This was a problem we wanted to fix for NM3.2, so we built up a swimming pool around our drain pipe. As long as the pipe can drain the information quickly enough the swimming pool will never fill up. But if a burst of data comes in and the speed of the information overwhelms the drain pipe, the swimming pool can hold the data until the drain has time to process that information.

Why not a Liquid Plumber Instead?

Data could not be processed in the first place due to two halves of the same resource problem: the CPU and the Network Speed. The CPU speed amounts to the size of the drain pipe, and the Network Speed to the rate at which you are dumping in water. If your CPU can’t keep up, then it may not be able to process the frames quickly enough. And on the other side of the seesaw, if the rate of network data is too fast then you may have the same problem regardless of how fast your CPU is. So there can always be a bottle neck given a fast enough network data rate, or an over utilized CPU.

How Big is this Swimming Pool?

This is actually a very good question because as you may be asking, what if my whole disk fills up? Currently we’ve limited the size of the capture buffer to 20 Gigabytes. You can also change this value via the registry if you need to. That registry key is a DWORD and it located here:

HKCU\Software\Microsoft\Netmon3\Settings\Capture\RootChainCaptureSizeLimit

The value of the registry key is the maximum size of the capture buffer in MB. We will also honor the MinDiskQuotaPercentage which by default is set to stop when only 2% of the free disk space is left. When either limit it reached, the new incoming frames will be dropped.

Things that Effect Draining Speed

In general if you are capturing data with no filters the speed at which we can drain the swimming pool is the fastest. As soon as you apply a capture filter, we must inspect each packet and determine if it passes. The more complex the filter, the more time it takes to process each packet. Therefore this will affect the rate at which the pool can fill up. When you start capturing, you can look at the pending frame count and determine if it is rising. If it is, then you may be in a situation where your swimming pool will continue to fill up over time. Of course traffic is not always predictable, but you should be able to get confidence about your particular capturing scenario.

Capturing and Applying Filters

Another thing you may notice is that one you stop a capture, any pending data in the buffer will continue to be processed until its emptied. With NMCap we will continue to process frames until all pending frames are done. The same holds true when taking a capture with the API. In fact we have created two options to determine how you want StopCapture to react. You can now stop immediately or stop when the pending data is cleared out. When applying a capture filter we actually keep track of the packet as it enters the buffer, so that the filter is applied correctly in regards to when you applied it as opposed to when the packet is actually inspected.

Swimming Season is Open

The best thing about this new feature is that you can be confident that you have gathered all the frames from a network trace. In the rare case there are dropped frames, then the number of dropped frames will be displayed for you in the UI and when using NMCap.

A Great New Way to Give Us Feedback for Network Monitor 3.2

Scenario Voting is a web-based system that will provide you with a list of scenarios to try and then easily "vote" on your experience. Each vote reflects information such as your satisfaction with the scenario, its importance to you and any additional comments you might have. Additional information may be requested on a per scenario basis. This information will help the Network Monitor Product Team assess the current state of the scenarios we are supporting in the product. Benefits you will receive include learning about new scenarios supported by Network Monitor 3.2 and helping you structure your evaluation and testing.

A unique aspect of this survey is that you can choose scenarios you want to send us feedback about. If you are unfamiliar with a scenario, in some cases, there are hints to help you understand how to complete the scenario. This may also help you learn how to do things in Network Monitor that you may not be familiar with. And you can always return and complete or update scenarios at any time. So please take a minute to send us feedback.

http://input.microsoft.com

This survey requires that you’ve already joined our connection at http://connect.microsoft.com. But of course the NM3.2 beta also requires this, so this should not be a problem for anybody using NM3.2 already.

Network Monitor 3.2 Beta has released!!!

I’m excited to announce that the Network Monitor 3.2 Beta has released. There are some exiting new features listed below of which many I have already spoken about in this blog:

http://blogs.technet.com/netmon/archive/2007/12/28/santa-s-bag-was-full.aspx

One feature I didn’t mention is our new capture buffer to avoid dropping frames. After adding our drop frames counter, we found out under certain busy situations it would go above zero and we just couldn’t have that. We now buffer the frames before parsing and displaying them. While this does add some complexities to capturing, it insures that packets are more reliably captured which is obviously very important. I will blog about this feature specifically, but I wanted to call it out here.

Where are the bits?

As usual we are hosting our beta at http://connect.microsoft.com under the Network Monitor 3 Project. If you are already signed up, you should have it listed on the main page. If not, look for the project and join up with us to help us find bugs. There is also a web page forum front end to our newsgroup if you need to get support or ask general questions.

What’s New since Netmon 3.1

• Process Tracking: View all the processes on your machine generating network traffic (process name and PID).  Use the conversation tree to view frames associated with each process.

• Find conversations:  Quickly isolate frames in the same network conversation.  Isolate TCP streams, HTTP flows etc.

• PCAP capture file support
• Capture engine re-architecture to improve capture rate in high-speed networks.   Network Monitor 3.2 drops significantly fewer frames that NM3.1
• Extensive parser set:  Parsers for over 300 protocols!  Parsers for the protocols covered by the Windows Open Protocol Specifications (see http://msdn.microsoft.com/en-us/library/cc216517.aspx).
• NM API: Create your own applications that capture, parser and analyze network traffic!
• Better parser management:  By default only a subset of parsers are loaded.  You can load the full parser set by changing the parser search order in Tools>Options>Parser
• Support for frame truncation.  Go to Tools>Options and limit the number of bytes captured per frame to improve performance.
• More extensive documentation of the NPL which includes documentation on the new NMAPI.  Access the documentation from Help > NPL and API Documentation
• Enhanced filtering on items within NPL while loops or arrays.  You can specify an index into the array or while loop to filter on
• IA64 version now available.
• ContainsBin Plug-in:  Search frames for arbitrary byte sequences or strings. For example, ContainsBin(FrameData, ASCII, “msn”).
• More UI indications of conversation status, dropped frames and the number of frames in the capture buffer.

• … and more.  See our Release Notes in the NM3.2 installation directory for a complete list of new features and known issues with the Beta.

Enjoy!

The Network Monitor Team

An Evangelist in Las Vegas

A few weeks ago I visited the glitzy town of Las Vegas for the Interop Conference 2008. While I’m not much of a gambler, the lure of technology was more than enough to get me excited about the trip. Pack thousands of tech-heads in a luxurious hotel, present information about new technologies, and now you’ve got a hot time in the dessert.

Buzz Words: Unified Communication and Virtualization

I don’t know if they do word counts on submitted Power Point presentations, but if I had to guess, both UC (Unified Communications) and Virtualization would be at the top of the list. And both technologies have a direct impact on the future of Network Monitor.

In terms of Virtualization, there are many challenges. The host machine can now house dozens to hundreds of virtual servers which means the backbone connection to the host machine requires a quick link. This means a 1 gig or even 10 gig network connection may be the norm for these configurations.  Making sure we can support these environments from the host machine and hosted VMs is going to be an important scenario to understand and support properly. 

For Unified Communications, the range of devices expands as your phone/pc/video conference merges.  And as wireless environments with remote users becomes the norm, trouble-shooting connections for voice and video become incredibly important in this domain.  And finally, interoperability between multiple devices and infrastructures requires a tool that can easily determine where interoperability problems may exist.

Roaming the Exhibition Floor Jungle

The Interop Exhibition Floor is a microcosm of the hosting city, Las Vegas. Sounds from various booths ring out like slot machines and shows start on the hour as vendors attempt to garner your attention for longer periods of time. Booth babes and circus-like performances try to steal your eye away, in hopes of attracting you to their line of products. For me, however, I was curious about products that compete with or compliment Network Monitor.

What I found is that in general the protocol analyzer type tool that seems to be popular is more a of an aggregate tool that tells you where your network is sick. It may not tell you what the exact problem is, but it helps you monitor your network as a whole. I suppose that is where our tool would come in. Network Monitor is more targeted for solving specific problems and diving deep. Our conversation tree and soon to release process tracking creates a unique way to take a lot of data and filter down to a specific problem.

Soaking up the Technology

As I introspect over my trip I try to find connections between our product’s future and the current state of technology. I realize that as things continue to accelerate, the need for a way to drill down from the very large to the very specific is of vast importance. And while we can continue to do general things, it will be more and more important to provide technology specific tools to help troubleshoot the dizzying array of protocols, devices and network mediums. As we continue to reach forward in this regard, we should be able to use our community to understand and provide tools to help us all solve difficult networking problems.

We’re very excited about showing off the beta of NM 3.2. Watch for it on our site, (http://connect.microsoft.com), in about a month! If you register into our community we’ll send you a note when it’s ready.

SMB2 Parser for NM3.1

We have decided to release an SMB2 parser for Network Monitor 3.1 (released July 07) to hold people over untill the beta for Network Monitor 3.2 releases in early June.

Where can I get the SMB2 parser?

You can download SMB2.NPL parser, along with SPARSER.NPL, CER.NPL, FCCS.NPL, SCNA.NPL and SMB.NPL (all supporting parsers) on http://connect.microsoft.com under the Network Monitor 3 project. If you’ve already signed up you’ll see it as one of your active projects. If you need to sign up you will need to create a passport account and join our project. Once you are in on the Network Monitor 3 project, click on the Downloads link on the left. You will see SMB2 Parser as one of the selections.

How do I use the new SMB2 parser?

Look at the article on using the SSL parser (http://blogs.technet.com/netmon/archive/2007/10/23/new-ssl-public-parser-available-how-to-deal-with-new-parsers.aspx) in the sections “Where do I stick it?” and “Working with NPL Parser path”. The instructions for installing the SMB2 parsers are the same.

Happy SMB2 parsing!

Reassembly with NM3

Ever wonder how a network works? Maybe it just seemed so easy, and in your mind sending a file was just putting each byte on the wire to the receiving machine. That’s not too far from the truth but you’d be very selfish to think that the network was there for your bidding only. Since a network has to be shared between many users, strategies have been created to chop your file up and send it in chunks.

Like Packing for a Move

We’ve all done it before. We take all of the bits and pieces of junk we’ve collected over the years and are tasked with moving them from one location to another. We invite friends and relatives, and like ants, they arduously move each piece from house A to house B. If your house=file, and boxes=packets, then we’ve created an analogy for reassembly. If you are organized, you’ll label each box, 1 of 20, 2 of 20, and so on. When you get to the new location, you can put all your things back together using the labels you created and verify you received everything.

The architects of our Internet are not any cleverer. They’ve borrowed these same techniques to determine how a chopped up file can reach its destination and be reconstituted. As long as the data is sequenced, the other side should be able to put the pieces back together.

The Transport Layer

In networking we like to talk about layers. You may have heard terms like “Layer 3 switch” or the “Network Layer”. Well, the architects of our Internet are just like everybody else. They get easily confused if too much is going on. Transferring network data from one place to another is a difficult problem. And a common strategy used to solve difficult problems is to divide and concur. Each layer is responsible for different things, and the transport layer is the one we use to define how files get chopped up and rebuilt.

We’ll narrow our focus down to TCP for a bit. This is the workhorse transporter for most networking applications. There are others, such as SPX/IPX, but TCP is by far the most popular. And it turns out RCP, SMB and HTTP can also fragment data. SMB might chop your file up into 4K chunks, and then each of these chunks could be further fragmented by TCP.

Labeling Your Boxes

So in our analogy, labeling=sequencing. We could label each packet 1 of 20 and so on, and some protocols do use this strategy. But for TCP, we go a bit further and describe the number of bytes that are sent. We’ll say, for instance, that this packet contains Sequence 1000-2000. What we actually send is the first sequence number and the size, but this range can be derived from that data. When the other side gets the data, even if it’s out of order, it knows how to put the puzzle back together. Also, the receiving TCP will keep the sender up to date by acknowledging what it’s received so far. In the simplest scenario, the receiver ACKs the latest segment it received. If the sender gets an acknowledgment for sequence 2000, then that confirms the receiver has seen all data up to that point.

Using NM3 to Reassemble the Data

When we capture data from the network, we are capturing it before the data has been put back together. But this can make it difficult to read, as you might imagine. With NM3 we’ve created a way to reassemble the data, so that you can see the data as it is seen by the application layer (there’s one of those darn layers again, see http://en.wikipedia.org/wiki/OSI_model). So when you click on a web page and get a trace, you can see the entire packet as sent by the browser, rather than a bunch of fragments after TCP has gotten to them.

Data Before You Reassemble

So if you start a network trace and view a web page you’ll notice that the traffic that gets created shows HTTP and TCP. The HTTP packets are the headers sent/received by your web browser. But the TCP that is in between is also your browser traffic. The original HTTP packet was larger than would fit in single packet, so TCP has chopped it up for you.

In this example, you can see that the server has responded with a page. Frame 6 is a continuation of frame 5 and is where TCP has chunked up the data. Frame 7 is an acknowledgment so the server knows we are receiving data. And frame 8 is the final frame in the fragmented data.

Data After You reassemble

To reassemble with NM3.1, you go to the Frames menu and select “Reassemble All Frames”. In NM3.2 we’ve created a more prominent button on the tool bar so this should be easier to find. Once the reassembly is complete a new window opens and contains all the original frames PLUS new frames for each reassembled piece.

So frame 9 is the new frame we inserted. While the Description doesn’t look much different, the devil’s in the details. Quite literally we have to look at the frame details to see the difference.

Frame: Number: 9, Captured Frame Length = 4449, MediaType= PayloadHeader
+ PayloadHeader: Re-assembled Payload
+ Http: Response, HTTP/1.1, Status Code = 502, URL: http://239.255.255.250/

The major difference is that we don’t see Ethernet, IP, or TCP anymore. We’ve replaced the original network header with our own called PayloadHeader. This header contains info about the protocols that have been reassembled, as well as information we might need from those layers. We don’t show the original frame numbers, but you can enable some debug NPL to get this information if you need. Just look at Payload.NPL for more info.

If you were to open up HTTP Response in the details, you would also see that all the HTML and header information is in this one packet. This reassembling of the data makes is very easy to understand that data from the HTTP level.

Also realize that in cases where both HTTP and TCP are fragmenting data, there can be multiple levels of reassembly. In those cases you may see a PayloadHeader for each TCP fragmentation, and then another for the HTTP fragmentation.

Reassembly FAQ

Reassembly doesn’t seem to be working for me, why?

The reassembly feature does depend on two things. One, conversations have to be enabled. Make sure this option is enabled when you open a trace. Currently the option can be found on the start page. Two, you can only reassemble a saved trace. If you just started a trace, you’ll need to save and reopen it.

Why do all the original frames show up?

When NM3 does reassembly, new frames are added in. Rather than remove the original frames, we’ve decided to leave them in as they still provide information about the original TCP data. Also, if there’s a problem with reassembly, perhaps due to missing packets, you could use the data to figure out why it didn’t reassemble correctly.

Can I only see reassembled frames?

Well yes, you can filter on only those things that have been reassembled. As we mentioned above, we add a PayloadHeader, which is just a protocol we’ve devised. So you can filter out all reassembled data, but applying a filter of “PayloadHeader”. I actually create a color filter for this so they stick out.

What you can’t do, however, is see only data that has been reassembled and the data that was never originally fragmented. While we realized this would be useful, we haven’t exposed a way to make this possible. However, with the API and NM32, it would be possible to do this programmatically.

Why doesn’t the Reassembly window have a conversation tree?

When we originally designed reassembly, the conversation window wasn’t really integral to the product yet. The simple work around is to save the capture file and reopen.

Putting it All Together

Networks have devised ways to break apart data and rebuild them. But with protocol analyzers that capture this raw data, it’s sometimes difficult to follow. With the reassembly option in NM3 your network traffic is put back together making it easier to read. Add this ability with the new Process Tracking feature in the upcoming NM3.2 and finding the only the data you need will be easier than ever.

Santa’s Bag was Full

The Elves are working furiously on the next version of Network Monitor 3.2. But it seems that NM3.2 won’t be under the tree for Christmas for this season. In a few more months, however, we will have a new Beta available for public consumption. And you can’t blame Santa, his bag already stuffed with Xboxes and such. Room in his bag should be reserved for children’s gifts you know. So you will have something under the tree for Christmas we will give you a sneak preview of the new things we are working on.

Stocking Stuffers

The first thing you’ll notice with NM3.2 is that the UI has been tweaked a bit. We moved around some items to make them more visible like Reassembly and Find. We’ve, err I mean the Elves, created a Most Recently Used list on the Start Page to make opening previous captures a quick operation. We uncluttered the tool bar to get rid of some of the lesser used items. And we are still envision more changes for NM3.2, like single line filter boxes to go along with our multi-line filters, as well as a way to get rid of windows you don’t use as often. This streamed line UI should give you more real-estate and make discoverability of some lesser used, but powerful, features more prominent.

All I Want is to Open LibPCap/WinPCap Files

And now you’ll be able to with NM3.2. One of the advantages of the NM3 capture file format is that we can store data from multiple network interfaces in the same file. But that makes it impossible to save as a LibPCap format, with out splitting the traffic. But why should we limit you from opening LibPCap files? Now when you open a LibPCap file, we convert it in memory to our format and display it just like any other capture file. I’m sure this will please more than a few folks. At this point you can save it as a NM3 capture file if you wish.

Finding a Conversation in the Tree

No, not some lame version of the "12 Days of Christmas".  And I'm not referencing the Christmas tree, but rather the NM3 conversation tree. The conversation tree is very helpful for quickly narrowing down traffic, but I’m sure you have had more than one occasion to locate the current frame in the tree. A exciting new feature we now have is the ability to right click a frame and highlight that conversation in the tree. This in turn filters the traffic based on that conversation. The cool thing this is this is not limited to TCP conversations. You can filter on any conversation as displayed by the Conv ID column, like all traffic for that machine-to-machine conversation or just HTTP level traffic.

The Big Gifts

I’ve noticed that some kids will often save the biggest gifts for last. I guess the assumption is that the larger the gift the grander the surprise. Of course, you occasionally get the big box stuffed with socks and underwear from Grandma. No such disappointment here.

NM Application Interface

We get asked about Experts all the time. The availability of the API is a giant step in that direction. While it’s much more than just a way to create experts, it allows a simple programming interface to query and access data fields using our parsing engine. This means you can create applications where only your imagination is the limit. Not only can you look into the packet data, you can also control the NM capturing engine. So you can design programs to stop/start captures based on whatever events you have.

Let me give you an example of the applications I’ve already tinkered with. Perhaps at some point when the API is released, we can also release some of these examples.

TopUsers – Lists traffic based on IPv4, IPv6, MacAddresses, and show columns for bytes sent, frames sent, etc. Simply click on a column header to sort and find out who is the chattiest user on your network.

EventCap - Lets you capture data until a certain Event error appears in the Event Log. This gives you a simple way to capture traffic leading up to a failure logged in the Event Log.  Basically a one step version of the blog I wrote on this subject: (http://blogs.technet.com/netmon/archive/2007/02/22/eventmon-stopping-a-capture-based-on-an-eventlog-event.aspx)

FiltCap – Lets you query capture data using a simple SQL like command and export that data as a CSV (Comma Delimited File). No longer are you trapped by the limitations of what the UI can do. You can use your favorite tool to graph and analyze trace data.

Process Tracking

I’m often trouble shooting traffic from a specific application. I want to know why Internet Explorer isn’t working properly. Process tracking is a way to show data based on the application that is sending it. Our final design will create new Nodes in the conversation tree that show each process that is running and then allow you to filter on only that traffic when you click on that Node.

We’ll even save that data from the capturing machine so that given you’ve taken the trace with NM3.2, you’ll have all the data necessary to see the processes that were running at the time the capture was taken. Imagine how much faster you’ll be able to narrow down a specific problem with this new feature!

Now How Much Would You Pay…

While we are not done adding features, these are the major ones we have so far. We plan to release a Beta in the early part of next year (we think April time frame). And while we haven’t been able to add every feature people have asked for, and there’s been a bunch of them, we are hoping this next version has a little of everything for everyone.

Happy Holidays from the Network Monitor Team

Understanding HTTP Flow with Netmon 3 - By Yuri Diogenes

1. Introduction

One of the most common protocols that we need to deal with these days is the HTTP Protocol. This is not only a privilege of Internet users, there are a lot of Intranet users that also use this protocol for internal transactions.

This post will show how to use Network Monitor 3 to better understand HTTP traffic and also to help you troubleshooting HTTP traffic.

2. HTTP Components

On HTTP we pretty much have two messages: HTTP Request and HTTP Response. The picture below shows an example of these messages:

Figure 1 – HTTP Messages

Here is a brief explanation of the main components of a message:

• HTTP Version: the http version in use by the message (ex.: HTTP 1.0 or HTTP 1.1).
• Method: this is about the action that the client is requesting to the server (ex.: GET and POST).
• Status Code: this code describes what happened on that transaction (ex.: 200, 301 and 407).
• Reason: complement to the status code (ex.: OK, NOT OK).
• Headers: the content of the header will depend on the version of the HTTP. For instance, HTTP/1.1. has some headers that needs to be present for the method in use (request or response).
• Body: some messages will contain the body, which is the data itself. Some other messages will have a blank line.

Based on this brief explanation about the main components of a message, let’s see how NetMon 3 can help us tracking down a HTTP conversation.

3. Understanding HTTP Messages using Netmon3

On this example the server is trying to access the website www.sysinternals.com. This server (Windows Server 2003) is behind a Proxy (ISA Server 2004) and using Integrated Authentication. All the traffic was captured from this server while was accessing this web site.

To help understand the HTTP conversations add the columns “HTTP is Request” and “HTTP is Response”. Those columns will have a number 1 in the column if this sentence is TRUE. This will help to identify what HTTP message was in use at that time.

Figure 2 – Choosing Columns.

For this example is quiet easy to identify the traffic, however on a real world scenario it might be difficult to locate the packet that has the URL request that you want. You might say, “Well, let’s create a filter for this request.” The thing is, if you create a filter for this request you will see only one packet requesting for this URL and this is not what we want here.

There is one cool feature on Netmon3 that allows you to use a filter to find a packet. To use this feature you need to click on the menu Frames than click in Find (or click Ctrl+F). The following window will appear:

Figure 3 – Find Packet based on a filter.

In this case I want to find a packet that matches with the following criteria:

Contains(http.request.URI,”sysinternals.com”)

After typing this and clicking Find the packet that matches with this request is selected as the current frame.

To make even easier to read the trace we can also change the color for the HTTP packets. This will allow you to quickly identify the HTTP traffic. For this example we will set the HTTP Request in Red and the HTTP Response in blue. Follow the steps below to configure that:

• Click in Filter.
• Click in Color Filters.
• Click in Add and type the query specified on the figure below:

Figure 4 – Color Filter feature.

• After type this query click, the color (red) and the Bold style click OK.
• Click in Add again and now type the following query: protocol.HTTP.response
• Choose the color blue and leave it bold
• Click OK again.

Here an example of how it will look after you apply the color filter:

Figure 5 – Frame summary after applies the filter.

Now we can close the Find Dialog window and look at the packet. Here is the HTTP part of the packet:

- Http: Request, GET http://www.sysinternals.com/ 
- Request: 
  Command: GET
  - URI: http://www.sysinternals.com/
    + Uri: 
    ProtocolVersion: HTTP/1.0
    Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
    Accept-Language: en-us
    UA-CPU: x86
    Proxy-Connection: Keep-Alive
    UserAgent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4322)
    Host: www.sysinternals.com
    HeaderEnd: CRLF

As you can see, this is a HTTP Request message and some of the components of the messages previously explained appear on this packet. Let’s check the answer for this packet:

- Http: Response, HTTP/1.1, Status Code = 301
  - Response: 
    ProtocolVersion: HTTP/1.1
    StatusCode: 301, Moved permanently
    Reason: Moved Permanently
    Via: 1.1 SRVISA
    Connection: Keep-Alive
    Proxy-Connection: Keep-Alive
    ContentLength: 31
    Date: Sun, 26 Aug 2007 15:05:10 GMT
    Location: http://www.microsoft.com/technet/sysinternals
    ContentType: text/html
    Server: Microsoft-IIS/6.0
    XPoweredBy: ASP.NET
    Set-Cookie: ASPSESSIONIDCCRASDTB=OKKIMCCDOMFAEPIPJCLNPEBN; path=/
    Cache-control: private
    HeaderEnd: CRLF
    + payload: HttpContentType = text/html

Note: it is important to mention that in my lab there were no multiple streams involved. Which make it easier to track it down the answer, since it is the next packet in the sequence.

This HTTP Response message is really important to emphasize one particular point, which is the Status Code.

The status code on this answer is 301. This number itself already says what is going on in this answer. It is important to know at least the meaning of status code based on the number range. The ranges are:

The netmon3 parser for HTTP has the main codes already defined. If you click on the Parser Tab, click on Protocols and HTTP, you will see on the right panel those definitions.

Figure 6 – Netmon3 HTTP Parser.

You also can view this code on the Table object on the Parser tab, as showed below:

Figure 7 – Table View.

Since this is a redirection answer, the field “location” has the place where the page is now located. This is presented to the client (requester) that based on that will send another HTTP Request for this URL.

4. HTTP with Netmon3 Conversation

The conversation feature on netmon3 allows you to view the frames aggregated on the same conversation. For this next example, let’s see the frames aggregated for the HTTP request for the URL www.microsoft.com:

Figure 8 – Filtering by conversation.

Clicking on the conversation tree filters out packets based on the HTTP traffic automatically. This can help to understand the whole conversation that client and server are having during this access. Another way to customize this filter is right clicking on the conversation and chooses the option Copy Conversation Filter to Clipboard” as showed in figure 8. Remember that all filters are applied in combination with the current node that is selected in the Conversation Tree. Be sure to click on the root of the tree if you don’t want the frames to be qualified further by the conversation tree.

Looking at this conversation we can see another status code that means there was an error on the client side:

- Http: Response, HTTP/1.1, Status Code = 407
- Response: 
  ProtocolVersion: HTTP/1.1
  StatusCode: 407, Proxy authentication required
  Reason: Proxy Authentication Required ( The ISA Server requires authorization to fulfill the request. Access to the Web Proxy filter is denied. )
  Via: 1.1 SRVISA
  - ProxyAuthenticate: Negotiate
  WhiteSpace:
  AuthenticateData: Negotiate
  - ProxyAuthenticate: Kerberos
    WhiteSpace:
    AuthenticateData: Kerberos
  - ProxyAuthenticate: NTLM
    WhiteSpace:
    AuthenticateData: NTLM
  Connection: Keep-Alive
  Proxy-Connection: Keep-Alive
  Pragma: no-cache
  Cache-Control: no-cache
  ContentType: text/html
  ContentLength: 4106
  HeaderEnd: CRLF
  + payload: HttpContentType = text/html

The reason why this request was considered an error on a client side is because the ISA requires authentication and the Internet Explorer on the first attempt to access the web site did not send the user credentials. After the response from the server, and depending on the browser and on the configuration, the client will use either NTLM or Kerberos to send another packet with the credentials.

5. General Information

There are many commands that you can use to obtain more information about your HTTP traffic using netmon 3. Let’s see some of them:

Yuri Diogenes

Security Support Engineer – ISA Server Team

NM3 One Click Capture Tool - BETA

Wish you could get a network capture from a computer with no capture software installed? Wish you could have a network traffic "flight recorder" on a USB key? Here's your answer!"

The motivation behind creating this tool was to provide an extremely simple way to get a trace from an end user. And the name says it all, though maybe a small bit of false advertisement as it may take a bit more than just one click :). Once you have it downloaded to your machine, you can launch the executable and it will lead you through some simple dialogs that install the tool and start a trace. Once the trace is complete, which means it was manually stopped by the user or the default time of 2 hours have passed, a window will be opened to the location for the capture, called OneClick.cap.

Two Versions for Two Scenarios

One Click Autorun: The main scenario is a customer who has network access and can simply click on the EXE from a share or get the tool locally using the internet. Running this version will install NM 3.1 on your machine (if you do not already have a previous version of NM 3) and begin capturing. The capture will terminate after 2 hours, or if you press the 'x' key on your keyboard. If you did not have NM3 on your machine previously, NM 3.1 will subsequently be uninstalled.

Extract Only: For this scenario, the user doesn’t have network access at all. This version is tailor made for use with a USB drive to be inserted into a problem machine. If you place the files on the root of a flash drive, One Click will run automatically when you insert the drive.  You can also run One Click by double clicking the file "OneClick.cmd" in the destination folder. The resulting capture will be copied back onto the USB device.

Detailed Tour of a One Click Capture

First thing I should mention is if you are running on Vista and don’t have NM3.1 previously installed or you are not a member of the Netmon User’s group, then you will need to run the EXE with elevated rights. Just right click the EXE and select “Run as Administrator”.

When you launch the One Click tool, it will prompt you with a EULA dialog. Assuming you accept the terms, then click YES. Remember that we do install a driver for Vista in cases where NM3 is not already installed. Also one caveat here is that if NM3.0 is installed, we use that driver rather than installing a new one. The disadvantage here is that NM3.0 can’t do wireless monitor mode or RAS capturing.

Once you accept the EULA, a CMD prompt dialog with a red background and white text shows up. We leverage NMCap for this job which is a command line utility, hence the CMD window. The window contains instructions as to where the capture file will be put by default. If you want to change this location, you can type a new one now.

If you choose the default be pressing enter, the capture will be started. At this point typing X on the keyboard will stop the capture once you are done. If you don’t press the X key, the capture will complete after 2 hours.

When the capture completes, an explorer window will open to the location where the capture file was stored. This allows the user to simply send the file to you in email or upload it to a location where it can be accessed by the original requestor.

If multiple captures are taken with the tool, the current OneClick.cap file will be renamed and appended with an incrementing number on the end.

Obtaining the Tool

At this point we are beta testing the tool. So the only means of getting it is on the Network Monitor project on http://connect.microsoft.com. When you join the project, you will see both the Auto-run and Extract Only packages in the download section. Once we release the tool, we will make it available on the Microsoft Download site as this is more convenient, which is our goal for this tool.

New SSL Public Parser Available: How to deal with new parsers

You’ve made some changes to your parsers and want to keep things organized. With the release of SSL.NPL, which we will also talk about today, it makes sense to understand how to work with parsers when you want to update or modify them. So I’d like to go over some strategies for keeping out of NPL H-E-double hockey sticks. If you are not familiar with this term, don’t worry, it is just a synonym for a type of file management night mare. It’s happened before, so we have learned some lessons and will attempt to help, but sometimes…some things…are in the user’s hands.

Where do I get it?

The SSL.NPL parser is available on our connect.microsoft.com site. http://Connect.microsoft.com is where we host our Netmon community. You can keep up to date, file bugs, and ask questions on our new forum (it’s still a newsgroup, but there’s now a web interface). From the connect.microsoft.com site, you should be able to get to the Network Monitor 3 project if you’ve already signed up. If not, just create a new account and then join the Network Monitor 3 project. Once you are on our site, you can click on the Downloads link on the left side. This will show you a list of files, one of which is the SSL.NPL parser.

Where do I stick it?

You could just plop the SSL.NPL file right on top of the old one in C:\program files\Microsoft Network Monitor 3\NPL, and things will work just fine potentially for the rest of you and your computer’s life. But there is a potential problem lurking.

In this case it’s the next update that may cause a problem. Not so much if we send out a new SSL.NPL, but more if you decide to change a parser yourself. It’s called forward integration, and it makes things complex. In this situation you have to figure out the difference you made and integrate into the new SSL parser, or vice versa if you prefer. Either way it’s an issue that not easy to resolve automatically and does require some user intervention.

But even in the replacing the SSL with new one, there are problems. On Vista, we should not be writing to the Program Files directory. So saving a file there using NM3 could end up shadowing it to some other directory (AppData\Local\VirtualStore\Program Files\Microsoft Network Monitor 3\NPL). So if you ever have a situation where you think you are updating the correct version of SSL.NPL in the NM3 editor, you may not be touching the invisible one. This has also caused problem in the past when trying to update from NM3.0. In any case we hope to address this for NM3.2.

Working with NPL Parser path

The solution is to store your NPLs in your own personal NPL directory using the Tools, Options dialog on the parsers tab. In NM3, you have the ability create a parser load order. This is a list of directories that are searched in order for each parser we encounter. When an NPL file includes another, we search each path in order and load the first NPL we encounter that matches. Even if the file is local to that NPL file, we still use the search order path.

We do create a personal NPL directory (documents\NM3\NPL), but it doesn’t automatically get added to your parser path. So it will search your local directory first and then each path listed in the options dialog in order (top to bottom). Add your local version first and then add the directory for versions of MS supplied files and finally add in the “C:\program files\Microsoft Network Monitor 3\NPL” directory so it searches that directory last. It won’t search the Program Files directory automatically once you add at least one directory in parser search path.

Once you’ve modified your path and added the new version of SSL.NPL, you’ll either need to restart NM3.1, or rebuild the parsers from the parser tab. Since we’ve already plugged a stub version of SSL into public build, no other modifications are need in the parsers to call the new SSL parser. In the case of a new parser, however, you would have to add this to sparser.npl.

This technique helps solve the problem by making sure you don’t clobber the original files. If we update a file, you can also check your local changes and forward integrate our changes in to your version. And if it’s Vista, we can make sure the versions of MS updates end up in an obvious place. Forward integration is a whole other topic, but there are difference tools which display file differences and make it easy to take changes from one file into another. But for simple changes this can be done by hand.

Improvements in NM3.2

There are things we can do in the future to make this more transparent to the user. And working more cooperatively with Vista is one of our goals for the next version. Fortunately this is not something most of us do all the time.

Trouble Shooting Name Resolution with NM3

Using the previous blog on “Intro to Name Resolution” we should have a basic idea of what is supposed to happen when a name needs to be resolved.  Now let’s discuss how you’d use Network Monitor to determine where the problem is when connecting to your network resource.  But let’s first list out some general breaking scenarios and the suggested course of action.

1.       DNS address not found – This could be due to the fact the machine you are trying to reach hasn’t registered itself properly or the machine name you are using is incorrect.  Resolution: verify the resource name; you should check to make sure it’s properly registered in DNS.  It may also be possible that the name hasn’t properly propagated to your DNS server.

2.       Machine not responding to NBTSS – Either the machine is not responding at all, or the NBTSS service is not started.  Resolution: check with owner of the machine to make sure NBTSS is running properly.  This may be indication of a larger problem.

3.       Machine not responding to ARP – In this case perhaps the machine is hung or not accepting network requests.  Resolution: find the owner of that machine.   It may require a reboot and this could be an indication of a larger problem as to why it’s getting in this state to begin with.

4.       Machine not responding to connect request – The machines name can be resolved, but the particular service you are trying to reach is not running properly.  The machine may also be in a hung or unavailable state or you may not have rights to connect in this fashion.  This could also indicate the IP Address you are receiving is old and inaccurate.   Resolution: Check with the owner of that machine to make sure the IP address is correct.  If it is correct, then check DNS to make sure that address is properly registered.  This might also indicate the machine is blocking traffic, perhaps due to a firewall.

So the first step in t-shooting any network problem is getting a trace.  So start an NM3.1 capture, attempt to connect to the resource in question, and then stop the trace.

The next thing we’ll want to do is filter out everything but name resolution type traffic for the resource we are connecting to.

nbtns.NbtNsQuestionSectionData.QuestionName.Name.contains("ResourceName")

OR

dns.QRecord.QuestionName.contains("ResourceName")

Checking DNS

Once you apply this filter you’ll see any name resolution traffic, either DNS or NBTNS that involves the machine you are trying to reach.  Depending on how your machine is setup, you may see both DNS and NBTNS or just one of the two.  If you see DNS traffic show up, then you can just look at the corresponding response to see if it was successful.  If it was not, then you have situation #1 described above.  Here’s an example of the successful summary info provided for DNS.

DNS: QueryId = 0xDF19, QUERY (Standard query), Query  for  ResourceName of type Host Addr on class Internet

DNS: QueryId = 0xDF19, QUERY (Standard query), Response - Success

And here’s an example of a failed DNS request.

QueryId = 0xA92E, QUERY (Standard query), Query  for  asdfg.northamerica.corp.microsoft.com of type Host Addr on class Internet

QueryId = 0xA92E, QUERY (Standard query), Response - Name Error

Checking NBTNS

If instead you see NBTNS traffic, then you’ll see the same kind of things, but only NBTNS will be the protocol.  So for instance you might see this for NBTNS.

NbtNs: Query Request for RESOURCENAME  <0x00> Workstation Service

NbtNs: Query Response, Success for RESOURCENAME  <0x00> Workstation Service, 192.168.1.100

If in your case the response does not come back at all OR it comes back negative for NBTNS and there is no successful DNS traffic to resolve your name, then the resource you are looking for is not registered properly.  If the machine that responded to the NbtNS request is not the owner of the Name then you may also have a WINS problem.  In that case you may have to check the WINS server to make sure it’s data is replicating properly.  This is situation #2 above.

Checking ARP

Once you’ve resolved the name to an IP address, you may have to further find the hardware address.  This only occurs if the traffic is local to your subnet.  We haven’t talked about IP Subnets, but the basic idea is you can determine if the address is local or not by using your subnet mask.  You could actually apply the following tricky filter to determine this.  This is similar to what your computer does when determining if traffic is local or not.

(ipconfig.LocalIpv4Address & 255.255.252.0) == (192.168.1.100 & 255.255.252.0)

In this example the subnet mask is 255.255.252.0.  You can determine this from the NetworkInfoEx frame if you started the capture in NM3.   This is also available from the command line when you run IPCONFIG.  The 192.168.1.100 is the destination address that is returned from DNS or NBTNS.

If applying the filter returns no frames, then the machine in question IS NOT on the same subnet.  If it returns all frames, then it IS on the same subnet.  The filter simply returns 0 or 1, which when applied as a filter shows all frames or none.

If the IP address is remote, then your machine sends the packet to your preferred gateway and lets your router determine where to route the packet.  In this case you’ll never know the hardware address, so you can continue to the next section.

If it is a local address, then you should see an ARP request for the packet, assuming it’s hasn’t been resolved already.  To avoid extra calls, ARP requests are cached.  In fact you can see this cache by typing “ARP –a” from a command prompt.  You can also clear the cache of the entry, “ARP –d 192.168.1.100”.

So assuming that is has not been resolved already, you should see ARP traffic.  At this point I would recommend a filter to just show the ARP traffic for that IP address.  So if the machine you were looking for is 192.168.1.100, your filter would be as follows.

Arp.TargetIp4Address == 192.168.1.100

OR

Arp.SendersIp4Address == 192.168.1.100

We choose the same address for the target and sender so we can see both the request and response.  You should see at least one of each, again assuming it hasn’t been cached.  If you see only ARP requests, then the machine may not be responding.  This is situation #3 above.

ARP Successful Response:

ARP: Request, 65.53.5.59 asks for 65.53.4.240

ARP: Response, 65.53.4.240 at 00-11-43-03-E5-A4

A failure is indicated by the lack of a Response.   While the lack of a response doesn’t always indicate a problem, it could mean the machine is not responsive.

Name Resolves but No Response

At this point the name resolution portion seems to be working.  But the resource in question may be there and not responding.  The next step is to take a look at the traffic going to that machine.  You can do this with the following filter.

IPv4.Address == 192.168.1.100

This will show only traffic where this machine is involved.  Since you took the trace with traffic only leaving your machine, this should show you traffic to and frame your machine to the destination resource, 192.168.1.100.

At this point you’ll usually see one of two things occurring.  You might see an attempt to connect to the machine from TCP.  When you look at the summary for TCP it looks like this.

TCP: Flags=.S......, SrcPort=2932, DstPort=HTTPS(443), Len=0, Seq=2697306090, Ack=0, Win=65535 (scale factor not found)

The S (synchronize) in the Flags indicates that TCP is attempting to connect to the port, in this case 443.  If you don’t see a response from this machine going back to your IP, then the service or machine you are connecting to is in some kind of hung state.  It’s also possible in this case that the port is being blocked by a firewall somewhere in your network.  The next thing I’d recommend is trying to PING the machine by the IP Address you obtained above.  If this succeeds, the problem is specific to the server or a firewall.  If it doesn’t respond then again it might be the machine, though it’s possible a firewall may again be blocking PINGS as well.

If you narrow it down to just one port and PINGs work fine, this indicates that the program or service you are talking to may be hung or not working properly.  Or it still may be a firewall that is just blocking that one port. 

Wrapping Things Up

Unfortunately there are other things that could go wrong.  And while IPv6 has a different scheme for resolving names, it’s still an important step in the process of making a connection.  The aim of this blog was just to help you understand one major component of getting connected, Name Resolution.  Perhaps next time you run into a situation where you can’t reach a resource, you’ll fire up Network Monitor and see if Name Resolution is at the root of the problem.