Symptoms: Lack of Free System Page Table Entries (PTEs), system-wide delays (I/O request failures), and low on Paged Pool Memory and/or Non-paged Pool Memory on 32-bit Windows
Introduction
The purpose of this article is to provide prescriptive guidance on how to troubleshoot free system page table entries (PTEs) in regards to Windows performance analysis.
Start with the following performance counters to analyze free system PTE’s:
- \Memory\Free System Page Table Entries
A page table is the data structure used by the Windows Virtual Memory Manager (VMM) to store the mapping between virtual addresses and physical addresses in memory. The performance counter Free System Page Table Entries is the number of page table entries not currently used by the system.
From the process perspective, each element of virtual address conceptually refers to a byte of physical memory. It is the responsibility of the Virtual Memory Manager (VMM) in conjunction with processor memory management unit (MMU) to translate or map each virtual address into a corresponding physical address.
The VMM performs the mapping by dividing the RAM into fixed-size page frames, creating system PTEs to store information about these page frames, and mapping them. System PTEs are small kernel-mode buffers of memory that are used to communicate with the disk I/O subsystem and the network. Each PTE represents a page frame and contains information necessary for the VMM to locate a page.
Note: Troubleshooting System PTE’s is explained in more detail at in the “Detection, Analysis, and Corrective Actions for Low Page Table Entry Issues” article mentioned in the References section below.
Applies to:
- 32-bit Windows Server 2003 (all editions) unless otherwise specified
- 32-bit Windows XP (all editions) unless otherwise specified
- 32-bit Windows Server 2000 (all editions) unless otherwise specified
Symptom Details:
- Lack of Free System Page Table Entries (PTEs): Use the “Memory\Free System Page Table Entries” performance counter for values under 5000. Alternatively, the !pte command in the kernel debugger can be used to examine PTEs.
- Periodic system-wide delays: System-wide delays or hangs that occur regularly or occur during elevated load on the system. This is measured by the user experience and I/O response times of the system.
- Low on Paged Pool Memory: Usage of more than 80% of the maximum size of kernel paged pool kernel memory. Use “!vm” command while in a kernel debug to determine the size of the kernel paged pool memory, then compare it to the “\Memory\Pool Paged Bytes” performance counter. A chart of estimated sizes is provided below in the More Information section of this article.
and/or
Low on Non-Paged Pool Memory: Usage of more than 80% of the maximum size of kernel non-paged pool kernel memory. Use “!vm” command while in a kernel debug to determine the size of the kernel non-paged pool memory, then compare it to the “\Memory\Pool Nonpaged Bytes” performance counter. A chart of estimated sizes is provided below in the More Information section of this article.
Possible Cause: Use of Physical Address Extension (PAE) Kernel
How to Diagnose
- To determine if the Windows 2003 server is booted with the PAE kernel, by checking for a value of 1 in the registry key, “HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\PhysicalAddressExtension”. Alternatively, the boot.ini file can be searched for the PAE boot switch.
Note: when execution protection is enabled on 32-bit Windows, the system automatically boots in PAE mode (automatically selecting the PAE kernel). - The PAE switch allows the Windows operating system to address more than 4GB’s of physical memory and in certain environments the PAE switch is automatically enabled. On x86 systems, the page table index is 10 bits wide (9 on PAE), allowing you to reference up to 1024 4-byte PTEs. Therefore, the /PAE switch causes PTE entries to use twice the normal allotted virtual address space. For more information, see “Microsoft Windows Internals” by Mark E. Russinovich and David Soloman.
Possible Solutions and/or Recommendations
- Migrate to a 64-bit Windows: 64-bit Windows has much higher amount of memory available System PTE’s. Specifically 64-bit has a maximum memory size of 128GBs for PTE’s while 32-bit Windows has a 660MB maximum for PTE’s. For more information, see https://support.microsoft.com/kb/294418.
- Migrate to Windows Server 2008 or Windows Vista: Windows Vista and Windows Server 2008 use dynamically memory pools to better manage system PTE memory. For more information, see https://support.microsoft.com/kb/294418.
- Remove the /PAE switch: If not needed, then remove the /PAE switch from the boot.ini file.
Possible Cause: Use of the /3GB switch
How to Diagnose
- The /3GB switch significantly reduces the number of free system PTE’s. To check if the /3GB switch is enabled use one of the following methods:
- Open the boot.ini file and look for the /3GB switch on the most recently used boot option.
- Check the value of the registry key: “HKLM\SYSTEM\CurrentControlSet\Control\SystemStartOptions”
- Check the value of the WMI property root\cimv2\Win32_OperatingSystem.MaxProcessMemorySize for a value close to 3,145,536 (3GBs) and greater than 2,097,024 (2GBs).
- Refer to the “Estimated Kernel Resources Chart” in the More Information section for the starting amount of free PTE’s in different operating system configurations. Note the different in the amount of free PTE’s when the /3GB switch is enabled.
Possible Solutions and/or Recommendations
- Use the USERVA switch: Use the /USERVA switch on Windows Server 2003 to give back some of the memory to the kernel specifically for system PTE’s. For more information on the USERVA switch, go to: “How to use the /userva switch with the /3GB switch to tune the User-mode space to a value between 2 GB and 3 GB” https://support.microsoft.com/kb/316739.
Note: For a Windows 2000 server in a /3GB scenario the SystemPages registry setting is used to accomplish the same effect as the /userva switch. For more information, go to https://technet2.microsoft.com/windowsserver/en/library/c5ccbaec-f552-4f61-a488-8ee3330d1eeb1033.mspx. - Remove the /3GB switch: If not needed, then remove the /3GB switch from the boot.ini file.
- Reduce the amount of physical memory: See “To much physical memory on 32-bit Windows” below.
- Migrate to a 64-bit Windows: 64-bit Windows has much higher amount of memory available System PTE’s. Specifically 64-bit has a maximum memory size of 128GBs for PTE’s while 32-bit Windows has a 660MB maximum for PTE’s. For more information, see https://support.microsoft.com/kb/294418.
- Migrate to Windows Server 2008 or Windows Vista: Windows Vista and Windows Server 2008 use dynamically memory pools to bettermanage system PTE memory. For more information, see https://support.microsoft.com/kb/294418.
Possible Cause: Too much physical memory
How to Diagnose
Refer to the “Estimated Kernel Resources Chart” in the More Information section for the starting amount of free PTE’s in different operating system configurations. Note the how increasing the amount of physical memory on the server reduces the number of free PTE’s on the server.
Possible Solutions and/or Recommendations
- Reduce the amount of physical memory: Reduce the amount of physical memory to free up more virtual memory for system PTE’s.
- Migrate to a 64-bit Windows: 64-bit Windows has much higher amount of memory available System PTE’s. Specifically 64-bit has a maximum memory size of 128GBs for PTE’s while 32-bit Windows has a 660MB maximum for PTE’s. For more information, see https://support.microsoft.com/kb/294418.
- Migrate to Windows Server 2008 or Windows Vista: Windows Vista and Windows Server 2008 use dynamically memory pools to better manage system PTE memory. For more information, see https://support.microsoft.com/kb/294418.
Possible Cause: High resource consumption and/or poorly written device drivers
How to Diagnose
Possible Solutions and/or Recommendations
- Remove any unnecessary third-party drivers and drivers not listed on the Hardware Compatibility List (HCL): Drivers that are not on the Hardware Compatibility List (HCL) have a higher likelihood of causing system problems and/or misuse of system resources. For more information about the Microsoft Hardware Compatibility List (HCL) go to, https://winqual.microsoft.com/hcl/.
- Use /BASEVIDEO: Use the /BASEVIDEO boot switch in the boot.ini file or a generic video driver to free up system page table entries. Video boards use the system page table entries to map their buffers in kernel space. This usage competes with the need for system page table entries by Microsoft Exchange.
- Use the USERVA switch: Use the /USERVA switch on Windows Server 2003 to give back some of the memory to the kernel specifically for system PTE’s. For more information on the USERVA switch, go to: “How to use the /userva switch with the /3GB switch to tune the User-mode space to a value between 2 GB and 3 GB” https://support.microsoft.com/kb/316739. For a Windows 2000 server in a /3GB scenario the SystemPages registry setting is used to accomplish the same effect as the /userva switch.
- Remove the /3GB switch: If not needed, then remove the /3GB switch from the boot.ini file.
- Reduce the amount of physical memory: See “To much physical memory on 32-bit Windows” below.
- Migrate to a 64-bit Windows: 64-bit Windows has much higher amount of memory available System PTE’s. Specifically 64-bit has a maximum memory size of 128GBs for PTE’s while 32-bit Windows has a 660MB maximum for PTE’s. For more information, see https://support.microsoft.com/kb/294418.
- Migrate to Windows Server 2008 or Windows Vista: Windows Vista and Windows Server 2008 use dynamically memory pools to better manage system PTE memory. For more information, go to https://msdn2.microsoft.com/en-us/library/bb870880(VS.85).aspx.
Estimated Kernel Resources Chart
This chart shows the estimated, initial maximum size of kernel resources on 32-bit Windows Server 2003 computers when using PAE, /3GB, with booted different physical memory sizes. Use the “!vm” command when doing a kernel debug to find the actual kernel resources.
|
References
- Detection, Analysis, and Corrective Actions for Low Page Table Entry Issues
https://www.microsoft.com/downloads/details.aspx?FamilyID=ed0e8084-abf7-4c00-ba6a-7d658cdb052a&DisplayLang=en - “Windows Internals Microsoft Windows Server 2003, Windows XP, and Windows 2000” by Mark E. Russinovich and David Soloman.
- Comparison of 32-bit and 64-bit memory architecture for 64-bit editions of Windows XP and Windows Server 2003
https://support.microsoft.com/kb/294418 - System Pages
https://technet2.microsoft.com/windowsserver/en/library/c5ccbaec-f552-4f61-a488-8ee3330d1eeb1033.mspx?mfr=true - How to use the /userva switch with the /3GB switch to tune the User-mode space to a value between 2 GB and 3 GB
https://support.microsoft.com/kb/316739 - Active Directory Integration Tuning
https://blogs.technet.com/cotw/archive/2008/04/07/symptoms-lack-of-free-system-page-table-entries-ptes-system-wide-delays-i-o-request-failures-and-low-on-paged-pool-memory-and-or-non-paged-pool-memory-on-32-bit-windows.aspx
Contributors:
Clint Huffman, Shane Creamer, Rick Anderson, Maximilian Silva, Matthew Walker, Pavel Lebedynskiy, John Rodriguez, Mike Lagase, Yong Rhee.