Microsoft Enterprise Platforms Support: Windows Server Core Team
EPS Team Blogs
Product Team Blogs
Hi, My name is Kim and I’m a support engineer in the Core Performance team. I recently joined this group about six months ago and noticed that there are a variety of ways to gather the same data. I’d thought I share some of the tricks I’ve learned on gathering memory dumps.
You could say I'm bad at fractions. My last job I liked to explain as:
1. Half technical.
2. Half process.
3. Half babysitting (no one mentions the third half. Fight Club Rules I guess)
I recently tried to explain my new role as:
1. One-third "knowing how things Should work"
2. One-third "knowing what data to capture when its Not working"
3. One-third trying to figure out the difference.
4. (Fight club rules apply for the fourth-third)
Knowing how things should work depends on the issue at hand. Typically you can be as generic or granular as necessitated by the situation. For example: All I need to know about my car is that I turn the key and it starts. That’s how it should work. When it doesn’t start I would need to know more; Is it out of gas? Bad spark plug? Alternator? Battery? Bad key-chip? The more you know the more you can eliminate until you pin-point an area to dig into.
Once you have an area to focus on we typically need to run capture data. Capturing data can be broken down into two areas; A snapshot of a single moment and a collection of snapshots over time. The best example of single snapshot is a memory dump.
The concept is basic: whatever the computer is doing at any one moment in time, freeze it, and put all that info into a file. In practice there are several ways to take that picture.
The most common, hands on way to force a memory dump is to configure the server to dump on a specific keystroke combination. Specifically by hitting the right CTRL key and pressing the SCROLL LOCK key two times.
Name : CrashOnCtrlScroll Data Type : REG_DWORD Value : 1
When we don’t have the option to connect a keyboard directly was can configure the system to reboot remotely via Non Maskable Interface (NMI).
Name : NMICrashDump Data Type : REG_DWORD Value : 1
There's a tool called NotMyFault that will crash the box on demand.
Locate and right-click Command Prompt
Select Run as administrator.
Type NotMyfault.exe /crash
NotMyFault.exe can be triggered automatically when an event is recorded in the event log. For example, if you're getting
Event ID: 2019 Source: Srv Description: The server was unable to allocate from the system nonpaged pool because the pool was empty.
intermittently, when we get around to forcing the dump the issue may not be present. In that case we can setup a trigger to call NotMyFault as soon as 2019 pops its pretty little head.
Setting up event triggering on 2003
‘eventtriggers /create /tr "Non Paged Pool Event" /eid 2019 /so SRV /tk \\server\share\NMF.bat’
Setting up event triggering on 2008
Event ID: 2019
The server can be setup to reboot automatically when an Event 333 is triggered on Windows 2003 servers via hotfix and a registry setting:
Location: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager Name: RegistryFlushErrorSubside Type: REG_DWORD Value: 2
When running a Hyper-V server with a problematic Hyper-V guest machine we can do the following to generate a memory dump from that guest:
When running a VMWare server with a problematic Vmware guest machine we can do the following to generate a memory dump from that guest:
All the above is dependent on having the machine configured correctly to save the memory.dmp file. The server may reboot when triggered but if it does not write a memory.dmp file then something is preventing the data from being transferred from memory to be written to disk. This includes but is not limited to:
A great tool to use is dumpconfigurator which automates much of the above.
Are there other ways to trigger the server to crash and save a memory dump? Sure! But hopefully everything listed above would cover most situations as to make the other options unnecessary. In the last few months of ramping up in Perf, I've used each of the above at least once to get the data needed to fix the issues at hand :)
Senior Support Escalation Engineer
Microsoft Customer Services and Support
this is a useful blog as one-stop shop to dump generation ways.
hope the author will add other useful ways to generate a dump for VM on Hyper-V.
-livekd has a feature to capture a VM dump.
C:\livekd>LiveKd -hv Srv08R2 -p -o C:\Dumps\Srv08R2.dmp
Hi memorydumpguy - the link to the vm2dmp information is under the third bullet for HyperV :)
Windows XP does not support CrashOnCtrlScroll for USB keyboards.