The goal of this post is to show how DebugDiag 1.2 can assist you identifying a potential source of bottleneck on a scenario where TMG user mode process (wspsrv.exe) is consuming high amount of CPU.
First part is to make sure you collect the user mode dump while the issue is happening. To do that, use the approach that I explain in the following post:
Once you have the data you can use DebugDiag to analyze the dump. Follow the steps below in order to perform this analysis:
1. After installing Debug Diag (64 bits edition in this case), launch it and cancel the first window.
2. Click Advanced Analysis tab.
3. Click Add Data Files button and choose the dump file that was previously collected.
4. Choose the scenario that applies to this issue in the top pane. In this case the scenario is Crash/Hang Analyzers as shown below:
5. Click Start Analysis.
6. Wait until the report is generated.
Reviewing the Report
Don’t go too far on the report before reviewing the first part of it, which is the Analysis Summary. Here it is the example for this scenario:
In this case the warning message says:
Detected a possible critical section related problem in wspsrv.dmp Lock at 0x015e7c70 is Unlocked Impact analysis 0.67% of threads blocked (Threads 78) The following functions are involved in the root cause GapaEngine_1cc44e8_bace5e90+10e22
The thread number has a hyperlink on it, when you click on this hyperlink you will see the stack that it is referring to:
ntdll!ZwWaitForSingleObject+a ntdll!RtlpWaitOnCriticalSection+e8 ntdll!RtlEnterCriticalSection+d1 GapaEngine_1cc44e8_bace5e90+10e22 0x454b64d8 0x0300e000 0x015ccbe8 0x4b80e418 0x015ccbe8 GapaEngine_1cc44e8_bace5e90+ff44 0x00004441`014dd475 0x00000010
The recommendation that DebugDiag gives is:
The following vendors were identified for follow up based on root cause analysis Unknown vendor for module C:\Program Files\Microsoft Forefront Threat Management Gateway\IPS\GapaEngine_1cc44e8_bace5e90.dll Please follow up with the vendors identified above
In other words, it is telling me to investigate further this module. Now what? Well, now you have an initial path to follow, you know that GAPA Engine is involved, which means that you can start doing some tests, such as:
It is important to remember that troubleshooting performance issue can be a long process and DebugDiag can assist you to find the root cause. However, sometimes finding the culprit doesn’t fix the issue, just show who is causing the problem, in this case further investigation is needed to find out how to really fix the issue.