Today I analyzed a memory dump which showed an interesting pattern.
Based on the dump several hundred SPRequest objects had been allocated on a single thread for a single request.
Each SPRequest object has a pretty large unmanaged COM object behind which controls the connection to the SQL database. So each SPRequest object adds a signification amount of memory to the overall memory consumption of the worker process.
This is a pretty common problem which is usually caused by either incorrect navigation settings (deep enumeration of the site) or custom controls which enumerate a large amount sites in the sharepoint site hierarchy.
In this specific dump it was different as the callstack showed that the SPRequest objects were leaked from standard SharePoint code before the actual page processing started.
That mean the objects were allocated in the code which retrieves the page from the SQL database and invokes the jit compilation. Navigation and custom controls are not involved at this point.
The callstacks looked like this:
at Microsoft.SharePoint.Library.SPRequest..ctor() at Microsoft.SharePoint.SPGlobal.CreateSPRequestAndSetIdentity(…) at Microsoft.SharePoint.SPWeb.InitializeSPRequest() at Microsoft.SharePoint.SPWeb.EnsureSPRequest() at Microsoft.SharePoint.SPWeb.get_Request() at Microsoft.SharePoint.SPWeb.GetWebPartPageContent(Uri pageUrl, …) at Microsoft.SharePoint.ApplicationRuntime.SPRequestModuleData.FetchWebPartPageInformationForInit(…) at Microsoft.SharePoint.ApplicationRuntime.SPRequestModuleData.FetchWebPartPageInformation(…) at Microsoft.SharePoint.ApplicationRuntime.SPRequestModuleData.GetWebPartPageData(…) at Microsoft.SharePoint.ApplicationRuntime.SPVirtualFile.GetFile(String virtualPath, …) at Microsoft.SharePoint.ApplicationRuntime.SPVirtualFile.GetFile(String virtualPath) at Microsoft.SharePoint.ApplicationRuntime.SPVirtualPathProvider.GetFile(String virtualPath) at Microsoft.SharePoint.ApplicationRuntime.SPVirtualDirectory.get_Files() at System.Web.Compilation.WebDirectoryBatchCompiler.AddBuildProviders(…) at System.Web.Compilation.WebDirectoryBatchCompiler.Process() at System.Web.Compilation.BuildManager.BatchCompileWebDirectoryInternal(VirtualDirectory vdir, …) at System.Web.Compilation.BuildManager.CompileWebFile(VirtualPath virtualPath) at System.Web.Compilation.BuildManager.GetVPathBuildResultInternal(VirtualPath virtualPath, …) at System.Web.Compilation.BuildManager.GetVPathBuildResultWithNoAssert(…) at System.Web.Compilation.BuildManager.GetVirtualPathObjectFactory(…) at System.Web.Compilation.BuildManager.CreateInstanceFromVirtualPath(…) at System.Web.UI.PageHandlerFactory.GetHandlerHelper(…) at System.Web.HttpApplication.MapHttpHandler(HttpContext context, …) at System.Web.HttpApplication.MapHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() at System.Web.HttpApplication.ExecuteStep(…) at System.Web.HttpApplication.ApplicationStepManager.ResumeSteps(…) at System.Web.HttpApplication.System.Web.IHttpAsyncHandler.BeginProcessRequest(…) at System.Web.HttpRuntime.ProcessRequestInternal(HttpWorkerRequest wr) at System.Web.HttpRuntime.ProcessRequestNoDemand(HttpWorkerRequest wr) at System.Web.Hosting.ISAPIRuntime.ProcessRequest(IntPtr ecb, Int32 iWRType)
We can see that the SharePoint virtual path provider is called from the standard ASP.NET System.Web.Compilation classes. The interesting piece is the one marked in Red: BatchCompileWebDirectoryInternal.
That indicated that ASP.NET is trying to compile not only a single page but all pages in the same virtual directory. In SharePoint terms that means all pages in the same pages library.
Batch compilation is controlled through a setting in the web.config. web.config files created by SharePoint have batch compilation disabled:
<compilation batch="false" debug="false">
For the problematic web application this setting had been changed to
<compilation batch="true" debug="false">
The problem is that SharePoint does not support batch compilation of pages in SharePoint document libraries (see KB 953459).
As batch compilation is unsupported and unexpected SharePoint does not include special code to handle this situation. Usually exactly one SPRequest object is created for a single http request at this point. With Batch compilation a new SPRequest object is created for each single page in the document library and these were leaked.
To resolve this problem it is required to change the compilation settings in the web.config back to the supported setting batch="false".
Nice article but how to know that the problem is because of batch compilation without going into the config file because people mostly look for other reasons such as memory leak because of object or some other coding issues.
by reviewing the ULS log after enabling callstacks.
Callstacks can be enabled using the information from these articles:
Then check if the callstack is similar to the one I listed here in the article.
btw: the easiest check is to verify that the setting in the web.config is correct. ;-)