So when your an app developer or someone in my position where you need to track down memory leaks one of the tools to use is Htrace once you've identified it's a handle leak.
I just wanted to put this post out there to show that I found you can use Htrace against a usermode process like LSASS below without being invasive! This was pretty critical in this scenario as the print server below was clustered and if we broke into LSASS via KD, the resources would have failed over to the passive node. Of course, I'm making no guarantees, but Htrace worked for me non-invasively below, your mileage may vary.
More about non-invasive debugging in a previous post here.
Before using Htrace you need to use application verifier to track handles for you for whatever process is leaking.
C:\>cdb -p 708 -pvr -y http://msdl.microsoft.com/download/symbols //Using PVR to be non-invasive for LSASS.
Microsoft (R) Windows Debugger Version 6.8.0003.0 X86Copyright (c) Microsoft Corporation. All rights reserved.
*** wait with pending attachSymbol search path is: http://msdl.microsoft.com/download/symbolsExecutable search path is:WARNING: Process 708 is not attached as a debuggee The process can be examined but debug events will not be received...........................................................................(2c4.2cc): Wake debugger - code 80000007 (first chance)eax=00000000 ebx=00000000 ecx=025bf200 edx=00000000 esi=00000000 edi=000005a4eip=77848254 esp=025bf64c ebp=025bf69c iopl=0 nv up ei pl zr na pe nccs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246ntdll!KiFastSystemCallRet:77848254 c3 ret0:000> !htrace -enable //Enables tracing of handles to start. by enabling you take a snapshot as well.Handle tracing enabled.Handle tracing information snapshot successfully taken.0:000> !htrace -snapshot //Takes the second snapshot, at this point we have two snapshots to compare.Handle tracing information snapshot successfully taken.0:000> !htrace -diff // Now we tell Htrace to show us any handles left open between the first and second snapshot, all the closed handles are removed.Handle tracing information snapshot successfully taken.0x20d new stack traces since the previous snapshot.Ignoring handles that were already closed...Outstanding handles opened since the previous snapshot: //Now it lists all the open handles and the stacks that opened the handles, some will be legit but for my issue it was leaking about 100 minute so it was easy to find the stacks that were suspect. Now that I have the stacks, I can set breakpoints and look for where handles were allocated but not released.
--------------------------------------Handle = 0x00022060 - OPENThread ID = 0x00000304, Process ID = 0x000002c4
0x77846c2c: ntdll!ZwDuplicateToken+0x4c0x74e6160c: LSASRV!LsapInitializeSessionToken+0x440x74e5e5b1: LSASRV!LsapSetSessionToken+0x4f0x74e64352: LSASRV!LsapCreateTokenEx+0x280x74c86301: kerberos!KerbCreateTokenFromTicket+0x8d0x74c86fd5: kerberos!SpAcceptLsaModeContext+0xff0x74e639de: LSASRV!WLsaAcceptContext+0x180x74e930a0: LSASRV!NegHandleClientRequest+0x5e0x74e92ba2: LSASRV!NegAcceptLsaModeContext+0xe40x74e639de: LSASRV!WLsaAcceptContext+0x8e0x74e637bf: LSASRV!LpcAcceptContext+0x150x74e511de: LSASRV!DispatchAPI+0x800x74e510da: LSASRV!LpcHandler+0x2b--------------------------------------Handle = 0x00030ca4 - OPENThread ID = 0x00000304, Process ID = 0x000002c4
0x778468cc: ntdll!ZwCreateSemaphore+0x4c0x77824d77: ntdll!RtlInitializeResour+0xff0x75287201: vfbasics+0x000072010x74e6146c: LSASRV!LsapCreateLsaLogonSess+0x460x74e61544: LSASRV!LsapCreateLogonSession+0xf80x74c861b5: kerberos!KerbCreateTokenFromTicket+0x0d0x74c86fd5: kerberos!SpAcceptLsaModeContext+0xff0x74e639de: LSASRV!WLsaAcceptContext+0x180x74e930a0: LSASRV!NegHandleClientRequest+0xeb0x74e92ba2: LSASRV!NegAcceptLsaModeContext+0x3e0x74e639de: LSASRV!WLsaAcceptContext+0x8e0x74e637bf: LSASRV!LpcAcceptContext+0x570x74e511de: LSASRV!DispatchAPI+0x80--------------------------------------Handle = 0x0000d36c - OPENThread ID = 0x00000304, Process ID = 0x000002c4
0x778468cc: ntdll!ZwCreateSemaphore+0x4c0x77824d4f: ntdll!RtlInitializeResource+0x4d0x75287201: vfbasics+0x000072010x74e6146c: LSASRV!LsapCreateLsaLogonSession+0xf60x74e61544: LSASRV!LsapCreateLogonSession+0x280x74c861b5: kerberos!KerbCreateTokenFromTicket+0xad0x74c86fd5: kerberos!SpAcceptLsaModeContext+0xf30x74e639de: LSASRV!WLsaAcceptContext+0x340x74e930a0: LSASRV!NegHandleClientRequest+0x430x74e92ba2: LSASRV!NegAcceptLsaModeContext+0x040x74e639de: LSASRV!WLsaAcceptContext+320x74e637bf: LSASRV!LpcAcceptContext+0440x74e511de: LSASRV!DispatchAPI+0x3--------------------------------------Handle = 0x0000da98 - OPENThread ID = 0x00000304, Process ID = 0x000002c4
You can see all about using Htrace by watching this video on Channel 9: http://channel9.msdn.com/ShowPost.aspx?PostID=341851