Thoughts from the EPS Windows Server Performance Team
Useful Microsoft Blogs
We have briefly discussed the MmSt pool tag before, but only tangentially. This post is to go much further in depth to the use of this tag and common issues we see related to it.
The MmSt pool tag is used by the Memory Manager when reserving memory for section prototype PTEs. A prototype PTE is a structure that maps the physical location of a page or set of pages to a memory mapped file. The MmSt allocations reside in Paged Pool memory. MmSt allocations are most commonly seen when the system is under heavy file I/O and has many open files mapped into the memory cache. The Cache Manager should reduce its MmSt allocations when paged pool memory pressure occurs, but this may not always happen quickly enough to prevent some issues.
The symptoms for issues dealing with high MmSt paged pool usage are common to most other paged pool depletion issues including:
You will need to use a tool such as Poolmon, Process Explorer, a memory dump, or a LiveKD dump to first confirm that you have a paged pool issue, and second to determine if MmSt is the tag consuming most of the paged pool memory. If it is in fact the MmSt tag causing the server to run out of paged pool memory then you will typically want to collect data to confirm this and find root cause.
Poolmon will be one of the primary tools you use to determine which pool tag is consuming most of the memory. A poolmon log indicates how much pool memory is being used by all paged and nonpaged pool memory tags. The poolmon log should run until the symptoms occur, and should end up looking something like this:
Tag Allocs Used Allocs Used
MmSt 0 0 8684 293147416 Mm section object prototype ptes , Binary: nt!mm
This indicates that the Paged pool tag MmSt is using approximately 293MB. Depending on the maximum paged pool limit on the machine, this could be too much and cause the system to fail additional pool allocations.
Process Explorer can be used live on the your machines to determine the maximum paged and nonpaged pool memory limits without needing to dump the machine. It is the least intrusive method of accomplishing this. Internet connectivity to the Public Microsoft Symbol Server is required for this to work successfully.
Collecting a memory dump file from the server while it is in a problematic state or consuming high paged pool memory will confirm if MmSt is the responsible pool tag. The debugger will indicate that there have been pool allocation failures if at some point the machine could not allocate pool memory.
Sample from a dump file:
3: kd> !poolused /t 10 4 Sorting by Paged Pool Consumed
NonPaged Paged Tag Allocs Used Allocs Used MmSt 0 0 3696056 334902032 Mm section object prototype ptes , Binary: nt!mm MmPd 2 525328 2470 225509872 Mm page table commitment bitmaps , Binary: nt!mm
The values above indicate that the MmSt Paged pool tag had 3696056 memory allocations and is consuming approximately 334MB of Paged Pool memory.
RamMap, a tool from Sysinternals, can be used to determine why high MmSt pool usage is occurring. RAMMap is an advanced physical memory usage analysis utility for Windows Vista and higher.
RAMMAP Screenshot showing Mapped File and Metafile (MFT):
RAMMAP screenshot showing memory mapped files.
Implementing the steps in the first two bullets below will resolve a majority of these issues:
In addition to providing the latest binaries and adjusting the registry keys for paged pool memory as stated above, data will need to be collected to determine root cause and additional troubleshooting steps if required.
Poolmon Overview: http://technet.microsoft.com/en-us/library/cc737099(WS.10).aspx
Finding Poolmon.exe: http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=18546
How to use Memory Pool Monitor (Poolmon.exe) to troubleshoot kernel mode memory leaks - http://support.microsoft.com/kb/177415
RAMMAP Askperf Blog: http://blogs.technet.com/b/askperf/archive/2010/08/13/introduction-to-the-new-sysinternals-tool-rammap.aspx
Process Explorer: http://technet.microsoft.com/en-us/sysinternals/bb896653
How to generate a kernel dump file or a complete memory dump file in Windows Server 2003: http://support.microsoft.com/kb/972110
thanks for the post. My high MmSt usage comes due to an Superfetch regression in Windows 7. Everytime a software uses FindFirstFile to query directories (TSVN and other tools) Superfetch heavily allocates MmSt memory. I sometimes have 3GB memory usage if all programs are closed. I have over 1GB paged pool usage (most by MmSt or File), 1GB in $MFT and the rest from the Windows services programs. This never happend in Vista only in Windows 7. In Windows 8 the paged pool usage (outstanding allocations) seems to be reduced.
For those on 2003, you can also use defrag -a to determine the size of the MFT.