If you enable /3GB in the boot.ini of a Windows Server 2003 x86 server, you risk running out of address space for the kernel.
You can tweak this by adding the switch /USERVA=wxyz where wxyz is the number of megabytes that should be allocated to the user mode processes. This will give more address space back to the kernel.
But how should you choose the correct value for /USERVA?
Here's the easiest way. This doesn't involve pool monitoring applications, debugging tools or enabling Free System Page Table Entries (FSPTEs) tracking registry keys (trackPTEs).
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Microsoft Windows Server 2003, Enterprise" /fastdetect /NoExecute=OptOut /3GB
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Microsoft Windows Server 2003, Enterprise without 3GB" /fastdetect /NoExecute=OptOut
How to calculate the right value for /USERVA
Take your value for FSPTEs you got when you rebooted with only /3GB defined in the boot.ini. In our example we'll use 6,400 for the number of FSPTEs.
6,400 is a counter of the number of 4KB memory "blocks" are available to the kernel. So the kernel has 6,400 * 4KB = 25,600 KB = 25MB of free address space to use.
(We need to reboot the system, as we don't know how many PTEs are used by the system load all the required kernel resources when /3GB is specified as this changes the sizes of NPP and PP memory)
FSPTEs must be greater than 10,000 at all times. So let's just make that 15,000 to ensure that future changes to required kernel resources (e.g. new video or network drivers) will not fail.
15,000 * 4KB = 59MB
In this example we have 25MB of free kernel memory and need 59MB, so we must add 59-25 = 34MB
So we need to set USERVA to be 3072MB – 34MB = 3038
So we add the switch /USERVA=3038 to boot.ini
This will give every user-mode application 3038MB of address space, and give the kernel the 59MB of free address space it needs to be happy.
This means that USERVA and FSPTEs are directly related to each other by a factor of 4KB/4*1024 = 256
But will there be any other negative impact to any other critical kernel resource (like Pool of Non-Paged memory or Pool of Paged Memory)?
In my tests, there will be no change in these resources, and this whitepaper confirms that adding or tuning /USERVA will only have an impact on the count of FSPTEs:
With the "/USERVA" boot.ini switch, you can customize how the memory is allocated when you use the /3GB switch. The number following /Userva= is the amount of virtual memory address space in megabytes (MB) that will be allocated to each user process. If you set /3gb /Userva=3030 in Boot.ini, 3,030 MB of memory is reserved to the process space, as compared to 3,072 MB when you use the /3GB switch alone. The 42 MB that is saved when you set /Userva=3030 is used to increase the kernel memory space and free system page table entries (PTEs). The PTE memory pool is increased by the difference between 3 GB (specified by the /3GB switch) and the value that is assigned to the /Userva switch. There is no reduction in any other kernel resource as a result of this switch.
Here's the results I found when changing the USERVA on a Windows Server 2003 SP2 x86 server with 3582MB RAM (4GB inserted, but the video card claimed 400MB at power-on):
Free System Page Table Entries
Free Kernel Memory
(1) = This is the lowest value for FSPTEs on this system, which is higher than 15,000, so no action is needed for this server.
(2) = Exchange mailbox servers should never have a /USERVA lower than 2800 as this will cause problems for store.exe
Below: Vertical Axis = Free System Page Table Entries, Horizontal Axis = Value for USERVA