Network sniffing is a major part of my life -- I've probably pored over, on average, a trace a day every day for the past seven years. This is an area where having the right tool is of the upmost importance as a good tool can cut hours (or even days) off of your work while a bad tool can send you around in circles.
I divide tools into two categories -- tools for capturing data (sniffers) and tools for analyzing data. I'll run through the tools I use for both; however, I will throw out the caveat that most of my work is done remotely on behalf of other customers. The tools I use are more suited to the work I do than to the work of most administrators.
Sniffers are divided, roughly, into hardware and software sniffers.
Hardware sniffers are specialized boxes that are self-contained appliances. On the upside, they are very capable & require no changes on the systems being monitored; however, on the downside, they are fairly expensive. In some situations (wireless, ATM, fiber), a hardware sniffer may be the only feasible option.
For me, though, access to a hardware sniffer is somewhat of a luxury. If the customer has one, if it is available, and if the staff that is trained to use it are available, then we might be able to get the traces we are looking for.
On the other hand, software sniffers run on existing machines, plugging into low-level interfaces to gather network traffic. This has several advantages -- it's cheap, it's easy, and it's readily available; however, it also has two significant disadvantages. First of all, it may mean installing software on production servers, which can be painful in an environment with proper change management (although you may be able to get around it, more on this in a minute). Second, a software sniffer will not see anything that the network card filters out in hardware (such as runt packets), so it isn't much help in diagnosing layer 1 issues. In most cases, this is not an issue as I troubleshoot far more application layer or network layer issues and, if a hardware problem is the root cause, then there's probably reporting on the switch or other hardware that will help troubleshoot.
(There is a third pseudo-category that encompasses switches, routers, and other network devices. Some of these have the ability to run network captures on their own interfaces, allowing administrators lucky enough to own such equipment to gather sniffs at intermediary devices.)
The tool I use most frequently is netcap.exe. This is a command-line interface to the Network Monitor Agent, which is the portion of Network Monitor that does the actual capturing. Netcap ships in the support tools for Windows XP and Windows Server 2003 -- these tools can be installed from the \support directory on the CD-ROM. Once you've installed the tools, you can copy netcap.exe out of the support tools directory and run it on any Windows 2000, Windows XP, or Windows Server 2003 machine.
To use netcap to gather some data:
I like netcap because it is small (<60KB zipped), runs on almost all of the Win32 systems I deal with daily, and requires no installation. The last item is particularly important as, in some cases, it will get you around change control in a production environment.
The only downside of netcap is that it does not support promiscuous mode (p-mode). P-mode is a way of bypassing the normal tendency of network cards to only pass along traffic destined for the machine the card is installed on. By default, a network card will only pass a frame to the operating system if it is destined to the node's MAC address, multicast address, or a broadcast address; however, when p-mode is turned on, the network card passes all frames received. P-mode was very useful when most infrastructures were made up of hubs as, in shared media, all nodes on the hub receive traffic for all other nodes; however, since almost all environments are now switched, p-mode is of limited usefulness. The only time that I have required a p-mode sniffer in the last few years was in using a spanning port on a switch to gather traffic destined for a different endpoint.
In cases where p-mode is necessary or where netcap cannot be used, there are a few other options. The full version of Network Monitor (which ships with SMS) captures traffic in p-mode; however, the 'lite' version that is included in NT4 Server, Windows 2000 Server, and Windows Server 2003 does not. Ethereal (with libpcap) allows for p-mode captures on a variety of platforms including Win32 & Linux. On most Unix systems, tcpdump can be used.
In network analyzers, my primary concern is the depth & accuracy of the parsers in the analyzer. Parsers take raw network data & turn it into analyzed output, recognizing & decoding various protocols. The lack of a parser for a protocol that you are working with will seriously limit productivity as you either have to decode the raw hex data by hand, make guesses based on plain-text information in the frames, or, if you are so inclined, write your own parser.
My primary analysis tool for a long time has been Microsoft Network Monitor. There are a lot of reasons for this; however, primarily, it is because it's a good all-purpose tool and...well...it's freely available to me.
There are three versions of Netmon, two of which are readily available. The full version, which ships with Systems Management Server (SMS), includes a number of bells & whistles that are not included in the 'lite' version included in NT4 Server, Windows 2000 Server, and Windows Server 2003. Most notably, the 'lite' version lacks promiscuous mode (p-mode), experts (which analyze a trace for specific information), and the ability to remotely capture sniffs.
The 'PSS' version, which is a time-bombed build that Microsoft Product Support Services uses to diagnose customer issues, supports p-mode but, otherwise, has the same feature set as the 'lite' version.
As network analyzers go, Netmon's interface is one of the easier to master. Its parsers are also relatively complete, although there are some important ones that are missing or incomplete.
In the last year, Ethereal has been competing with Netmon to be my favored analyzer. Ethereal's interface isn't as easy to grasp as Netmon's; however, after getting used to it, I find Ethereal's text-based display filtering to be much quicker & more powerful than Netmon's. Additionally, Ethereal has a parsers that Netmon does not (although it, too, is occasionally missing something I need). Finally, Ethereal has a powerful set of statistical analysis & graphing tools that I've found allow me to pull out additional information about network traffic.
Is there any way to sniff traffic between two devices where device b does not respond to a ping request or any other request? Keep in mind device b is a embedded board with no OS. I'd like to see what device b is doing during that time. Any help would be greatly appreciated considering I have been troubleshooting this issue for 4 days now.
The usual approach there is either to do one of several things:
1. Some switches have tcpdump or something similar on them, so you can run the trace on the switch that device B is connected to.
2. If not, the switch will hopefully support a mirroring/port spanning mode which allows you to mirror the port that device B is on & run your sniffer on a machine connected to that mirrored port.
3. Plug device B and a your sniffer into a hub, plug the hub into the switch.
Let me know if that doesn't make sense.
One other thing...to use this approach, you'll have to use a sniffer that supports promiscuous mode. Neither netcap.exe nor the lite version of Netmon that's included with Windows 2000 Server/Windows Server 2003 will work. You can use the full version of Netmon from SMS or Ethereal with Winpcap.