Troubleshooting SPSite/SPWeb leaks in WSS v3 and MOSS 2007
In an earlier article I have discussed that all SPSite and SPWeb (and potentially also PublishingWeb) objects need to be properly disposed to avoid memory pressure situations. Roger Lamb has a nice summary of coding patterns which will actually cause such leaking objects and also provides information about how such code would have to be adjusted to avoid such leaks.
In this article I will provide some details about how a sharepoint administrator is able to isolate if the site contains custom code which does not properly dispose these objects and also how to isolate the components that are leaking these objects.
Overview
As discussed in my earlier article each SPWeb and SPSite object holds a reference to an SPRequest object which holds a reference to a SharePoint COM object that is responsible to communicate with the backend SQL server. The dispose of the SPWeb and SPSite object is necessary to ensure that the SPRequest object will release the COM object and also all the memory allocated by this COM object is released.
In SharePoint v2 the only way to identify such leaking objects was to analyze a memory dump - which usually required that a support case with Microsoft has to be opened to identify the code that is not correctly disposing the SPWeb or SPSite objects.
SharePoint v3 on the other hand contains code which monitors correct disposal of SPRequest objects (and therefore correct disposal of SPWeb and SPSite objects). This code has two different features:
- It monitors how many SPRequest objects are open in parallel on all threads.
- It checks if SPRequest objects still exist at the end of the thread that created it
I will now discuss how these build in features can be used to identify how expensive (in regards of SPRequest allocations) your site design is and also if the code contains any controls which don not properly dispose SPWeb and SPSite objects.
How many SPRequest objects are allocated by a single request to my site?
Usually you would think that if you don't add any custom code to your page that only one SPSite and one SPWeb object should be required (the site collection and the site the page lives in) but this is not correct. The reason is that all the master pages contain three navigation controls: The Top navigation, the Breadcrumb control and the Left Navigation control.
Per default these navigation controls are bound to one of the Site Map Providers shipped with SharePoint. To show the elements in the navigation control the control queries the site map provider which will now enumerate the sites in the sharepoint database to retrieve the title and the URL to be displayed on the navigation control. Each of the enumerated sites will be instantiated during the enumeration to retrieve these properties. That means if you have a navigation control configured to enumerate 3 levels of sites and on each level you have 20 sites that you will end up with more than 400 SPRequest objects. So you need to be very careful with how to configure your navigation controls and also how many sites to create on each level!
A valid question now could be: why does the site map provider not dispose each of the objects directly after it retrieved the URL and the Title? That should reduce the number of parallel existing SPRequest objects heavily. Indeed that would be correct. But remember: you have usually multiple navigation controls on a single page! Disposing these objects directly after the properties have been read would mean that for each navigation control on the page the same objects have to be retrieved from the database again and again - which would impact the performance of the site. To avoid this the items retrieved by the navigation controls are cached to ensure that other navigation controls needing the same items can leverage them without causing additional requests to the backend database. The caveat is that the overall memory consumption of the site will go up through this implementation.
As indicated in the overview section SharePoint v3 monitors how many SPRequest objects exist in parallel on the same thread. As soon as SharePoint identifies that with a new allocation the number of SPRequest objects exceeds a configurable threshold SharePoint will create the following entry into the ULS log:
12/12/2007 12:29:54.05 w3wp.exe (0x1380) 0x126C Windows SharePoint Services General 0 Medium Potentially excessive number of SPRequest objects (14) currently unreleased on thread 5. Ensure that this object or its parent (such as an SPWeb or SPSite) is being properly disposed. Allocation Id for this object: {2DF92EAD-4959-4A21-8BA0-FBD91E8FDD48} Stack trace of current allocation: at Microsoft.SharePoint.SPRequestManager.Add(SPRequest request, Boolean shareable)
The default threshold to get the above warning is 8 - which is actually far to small for real live scenarios as we have seen that with navigation controls on your master pages you will for sure exceed this number. As you can see in the sample above the warning outlines that 14 SPRequest objects live in parallel on the same thread. As we have seen before with navigation controls a larger number of SPRequest objects is absolutly normal and as expected. So you can expect the ULS log to be filled up with many similar warnings.
You should inspect the ULS logs of your site and verify what are the highest numbers in these warnings to get an idea about how expensive your current site design currently is in regards to memory consumption through allocated SPRequest objects. If the number is higher than 150 you might need to consider to change the design of your site or the number of levels to show in your navigation controls to reduce the memory footprint.
Afterwards you can fine tune the threshold to avoid unnecessary warnings. This can be done by adjusting the threshold through the following a registry key (e.g. to 50):
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Shared Tools\Web Server Extensions\HeapSettings
LocalSPRequestWarnCount = 50
The actual number to configure here depends on your site structure. After setting this registry key you need to restart the sharepoint services and the application pools (or restart the server) to ensure that the settings get effective.
Also be aware that you will only see the above warning if you have configured diagnostic logging for the General category to at least Medium level - which is actually the default. So if you haven't changed the configuration of diagnostic logging you will see these warnings.
Do I have code on my site which does not correctly dispose SPSite/SPWeb objects?
One method (not completely reliable but a good indicator) is to monitor the ULS log message mentioned in the last section. If the number for some threads gets higher and higher during the livetime of a thread, without going down to the configured threshold again the chance is high that there is some code which does not properly dispose SPWeb and SPSite objects.
Be aware that you cannot just look at the callstack listed in these messages to identify the bad code as this warning will be written during allocation of the objects! This message does NOT indicate that the object is not properly disposed! So you can only use it as an indicator that some code might be wrong but not to identify the module causing the problem!
To identify the bad code we need to use a different approach.
As mentioned in the Overview section SharePoint not only monitors the number of SPRequest objects allocated by each thread but also checks if SPRequest objects still exist at the end of each thread and create the following message into the ULS log.
05/01/2007 12:58:47.31 w3wp.exe (0x105C) 0x09A8 Windows SharePoint Services General 8l1n High An SPRequest object was not disposed before the end of this thread. To avoid wasting system resources, dispose of this object or its parent (such as an SPSite or SPWeb) as soon as you are done using it. This object will now be disposed. Allocation Id: {5BFFCA4B-3B91-45BF-98CD-0BB508BE30EE} To determine where this object was allocated, create a registry key at HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Shared Tools\Web Server Extensions\HeapSettings. Then create a new DWORD named SPRequestStackTrace with the value 1 under this key.
Be aware that the livetime of a thread can be much longer as a single request. As SharePoint only checks if undisposed SPRequest objects exist when the thread ends and not at the end of each request the number of entries of this type can be much smaller than you might expect. So you might have to search in multiple ULS logs and not just the latest for this warning (you can actually search for the string 8l1n as this is the unique identifier for this message).
In addition: if the object has been disposed by the garbage collector then you will see a slightly different message stating that the garbage collector reclaimed the object:
05/01/2007 14:48:43.32 w3wp.exe (0x105C) 0x09A8 Windows SharePoint Services General 8l1n High An SPRequest object was reclaimed by the garbage collector instead of being explicitly freed. To avoid wasting system resources, dispose of this object or its parent (such as an SPSite or SPWeb) as soon as you are done using it. Allocation Id: {098092FF-E4D9-4BA3-B2B8-5C6CE9B96554} To determine where this object was allocated, create a registry key at HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Shared Tools\Web Server Extensions\HeapSettings. Then create a new DWORD named SPRequestStackTrace with the value 1 under this key.
To identify which code is responsible for the problem after you identified the problem you have two options:
-
You can search the earlier ULS logs for the allocation ID in the warning (if the allocation was done beyond the threshold you will find it)
-
You can follow the hints in the warning and set the following registry key as this will ensure that the stack trace of the allocation of the SPRequest object is preserved in memory and added to the 8l1n ULS log entry:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Shared Tools\Web Server Extensions\HeapSettings
SPRequestStackTrace = 1
The second option is more reliable as it will ensure that you get a callstack for each of the 8l1n messages. Be aware that after setting this registry key you need to restart the sharepoint services and the application pools (or restart the server) to ensure that the settings get effective.
By analyzing the emitted callstack you should then be able to identify the component causing the problem.
