Goal:The goal of this walkthrough is to find the stacktrace for a currently executing Silverlight Application and examine it in windbg. This intro will give you the information you need to "break the ice" with using windbg tool for debugging in Silverlight. Building upon these skills you will soon be able to identify memory leaks, find hidden bugs and thread contention in your Silverlight applications ... and snatch the pebble from my hand.
Let's start off with the simple task of finding the call-stack on an executing Silverlight application. To facilitate this I created a simple Silverlight 4 application that on a button-click it throws (and catches) three types of Exceptions (ArgumentOutOfRangeException, FileNotFoundException, UnauthorizedAccessException) and then prompts with a MessageBox. We are going to focus on the call stack in this blog - but next we will look at the exception details.
The MessageBox prompt is to pause the application so we can take a memory dump to analyze the call stack at that particular point of interest. It is always important to capture your memory dump at the point you are interested in - processes get spawned, threads get recycled... Later we will look at ways you can configure debugdiag to take a dump when it encounters exceptions (first or second chance) of a particular type for effective debugging of customer issues - but for now this is a great way to get familiar with the initial setup and process for working with windbg and Silverlight.
Step 1 - Startup / Install PrerequisitesFirst go ahead and download the windbg and debugdiag tools.
You can install the Windows SDK which contains both tools, or individually install the 32 and 64 bit versions. Chances are you will need the 32 bit version - but better to have both for future use.
Then download and unzip Microsoft's Process Explorer from sysinternals. We will need that to positively identify the correct process.
Got all that? Ok... let's get started!
Let's get a simple silverlight application paused somewhere that we can then investigate the call stack and verify some data in the heap. To facilitate this exercise i created a simple project that has one button and throws 3 types of exceptions (). You can download it here, and run it and click the "Go!" button.... or create or use your own application - just be sure to have some dialog open (eg MessageBox) or active process when you take your memory dump.
Now you need to identify the right process for your executing Silverlight Application. You might think that's a simple matter with Task Manager and you may open it up and add the "Command Line" column to try to identify your Silverlight application - but you are going down the wrong path - you will instead want to use Process Explorer - but let's meander a bit for sake of clarity. If you open Task Manager you will see:
This isn't much help - other than telling us IE is running in 32bit mode. The command Line parameter won't tell you which process is executing your Silverlight Application - and for those curious the "SCODEF" parameter refers to the parent IE process for the given frame. Pre-frame based IE this approach would've given you the information you need... but not now. Time for the big guns - Process Explorer.
See the bullseye icon in the Process Explorer toolbar? Click that bullseye and keep your mouse button held down while you drag it directly onto the MessageBox Dialog UI (mine says "Pause here while we look at things...") in your Silverlight UI. You can setup your process explorer and your SL (Silverlight) browser window side-by-side to make this task easier. Be sure to release it on the dialog UX box or you will get a different process and not the one we want to look at right now.
and that gives us...
Take note of your PID (Process ID) (mine is #11156).
Step 2 - Memory DumpSo first off - is it IE running in 32-bit or 64-bit? That matters because when you take a memory dump you want to do so with a tool running the same way. You can check this in Process Explorer by checking the "Image Type" column. If you don't see that column, right-click the column header and "Select Columns". In this case IE is running 32 bit. Why is this important? You need to take the dump from a tool executing in the same mode - (preferably DebugDiag). For example those clever folks that noticed from Process Explorer you have a right-click "Create Dump" option might have sought to skip the next step... only to discover problems down the line from generating that dump from the 64 bit Process Explorer.
So - let's open DebugDiag (x86) that comes with Windows SDK (Start|Run - type "DebugDiag"). Cancel out of the wizard, and order the processes under the process tab by PID to locate your process. If you can't locate it, and you've tried F5 to refresh but still can't find it - close DebugDiag and re-open it. I noticed an issue in refreshing processes a few times and that trick seemed to work. See your PID # now? Great.
Right-click and "Create Full User Dump" and note the location of the file created... now things get fun...
Step 3 - welcome to the wonderful world of windbg!So now let's open up windbg (x86) version same as our memory dump and open the memory dump file. The first thing you will want to do is load the SOS.dll for silverlight (no .NET SOS and psscor will not work for SL).
In my case that's Silverlight 4:
> .load C:\Program Files (x86)\Microsoft Silverlight\4.0.60531.0\sos.dll
you can then do a >.chain to verify what is loaded... and a >.help to see the list of available commands.
Let's also be sure our symbols are loaded properly so we can get detailed information (create a local folder to store them). For more info on loading symbols see here.
What we really want to get at is the call stack for the currently executing code - but first we have to find the right thread for that and switch context to that thread.
If you get and error like this: Failed to load data access DLL, 0x80004005 then try >.symfix and then try again.
You should see something like :
0:000> !threadsThreadCount: 3UnstartedThread: 0BackgroundThread: 3PendingThread: 0DeadThread: 0Hosted Runtime: yesPreEmptive GC Alloc LockID OSID ThreadOBJ State GC Context Domain Count APT Exception4 1 2360 08492ef0 220 Enabled 09e7a858:09e7bfe8 08497580 0 STA26 2 12b8 084942e8 b220 Enabled 00000000:00000000 08380ca0 0 MTA (Finalizer)27 3 20d4 08496c20 1220 Enabled 00000000:00000000 08380ca0 0 Ukn
Normally this would list the managed thread we are looking for that contains our code... however in this case the SL is executing in coreCLR and not identified as a managed thread. If you were to take the time to switch context on each of these threads and attempt a >!clrstack you may get a mysterious error like this (below) leaving you a little confused...
0:002> !clrstackOS Thread Id: 0x1f14 (2)Unable to walk the managed stack. The current thread is likely not a managed thread. You can run !threads to get a list of managed threads in the process
Don't panic. That's why we have to go a little further and execute this command for a full list of the threads:>~*kL
Ok - now you should see a bunch of information scroll by with details for every thread. Take a look through them starting from the top. In my case the first three threads were not familiar - but Thread #4 was the one I am looking for and it was easy to identify by the characteristsics:
4 Id: 2b94.2360 Suspend: 0 Teb: 7efa9000 UnfrozenChildEBP RetAddr WARNING: Stack unwind information not available. Following frames may be wrong.0342d538 75240751 user32!PeekMessageW+0x17b0342d564 7b22cfea user32!PeekMessageW+0x1970342d5ac 7b22cf42 npctrl!RequestFullTrustAppInstall+0x15dc40342d5c8 7b22ce94 npctrl!RequestFullTrustAppInstall+0x15d1c0342d5ec 7b799a4e npctrl!RequestFullTrustAppInstall+0x15c6e0342d610 0c9a1a0b agcore!MessageBox_ShowCore+0x420342d674 0c9a1aa1 System_Windows_ni!DomainNeutralILStubClass.IL_STUB_PInvoke(IntPtr, System.String, System.String, UInt32, Int32 ByRef)+0x8b0342d698 0ca58e49 System_Windows_ni!MS.Internal.XcpImports.MessageBox_ShowCore(System.String, System.String, UInt32)+0x510342d6a8 0ca58dff System_Windows_ni!System.Windows.MessageBox.ShowCore(System.String, System.String, System.Windows.MessageBoxButton)+0x390342d6f0 00601f3e System_Windows_ni!System.Windows.MessageBox.Show(System.String)+0x1f*** WARNING: Unable to verify checksum for SLHelloWndbg.dll*** ERROR: Module load completed but symbols could not be loaded for SLHelloWndbg.dllHowever notice the error in the Thread #4 - that helpful error indicates it could not load the symbols for my SL Application... that's because I forgot to point it to my PDBs. That's easy to fix by adding the folder that contains your Silverlight PDBs to the sympath (as shown below). If you don't have the PDBs you may consider reverse-engineering them by finding the XAP file, renaming to ".zip" - opening that archive that contains all the DLLs and then using a program such as the latest version of RedHat's Reflector (or Mono Cecil) to create the project in Visual Studio and then compile in debug mode to generate your PDBs.... or just have the customer send them to you from their Debug folder.
>.sympath+ C:\files\Visual Studio 2010\SLHelloWndbg\SLHelloWndbg\Bin\Debug>.symfix
And let's check again:>~*kL
(results omitted for brevity sake... but there's no more error). Much better.
Now since we've found the right thread and gotten our symbols loaded - let's switch our context to that thread and then we can finally do a !clrstack to get the juicy information in the callstack we are seeking:
0:000> ~4seax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=00000102 edi=0342d588eip=75240735 esp=0342d510 ebp=0342d538 iopl=0 nv up ei pl zr na pe nccs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000246user32!PeekMessageW+0x17b:75240735 83c404 add esp,40:004> !clrstackOS Thread Id: 0x2360 (4)Child SP IP Call Site0342d630 75240735 [InlinedCallFrame: 0342d630] 0342d62c 0c9a1a0b DomainNeutralILStubClass.IL_STUB_PInvoke(IntPtr, System.String, System.String, UInt32, Int32 ByRef)0342d630 0c9a1aa1 [InlinedCallFrame: 0342d630] MS.Internal.XcpImports.MessageBox_ShowCoreNative(IntPtr, System.String, System.String, UInt32, Int32 ByRef)0342d688 0c9a1aa1 MS.Internal.XcpImports.MessageBox_ShowCore(System.String, System.String, UInt32) [f:\dd\xcp\clr\clrlib\pinvokes.cs @ 5733]0342d6a4 0ca58e49 System.Windows.MessageBox.ShowCore(System.String, System.String, System.Windows.MessageBoxButton) [f:\dd\xcp\clr\clrlib\System\Windows\MessageBox.cs @ 57]0342d6b4 0ca58dff System.Windows.MessageBox.Show(System.String) [f:\dd\xcp\clr\clrlib\System\Windows\MessageBox.cs @ 29]0342d6bc 00601f3e SLHelloWndbg.MainPage.button1_Click(System.Object, System.Windows.RoutedEventArgs)0342d6fc 0c9f4901 System.Windows.Controls.Primitives.ButtonBase.OnClick() [f:\dd\xcp\clr\clrlib\System\Windows\Controls\Primitives\ButtonBase.cs @ 400]0342d710 0ca4f524 System.Windows.Controls.Button.OnClick() [f:\dd\xcp\clr\clrlib\System\Windows\Controls\Button.cs @ 102]0342d71c 0c9f4bfe System.Windows.Controls.Primitives.ButtonBase.OnMouseLeftButtonUp(System.Windows.Input.MouseButtonEventArgs) [f:\dd\xcp\clr\clrlib\System\Windows\Controls\Primitives\ButtonBase.cs @ 818]0342d72c 0c9b6341 System.Windows.Controls.Control.OnMouseLeftButtonUp(System.Windows.Controls.Control, System.EventArgs) [f:\dd\xcp\clr\clrlib\control.cs @ 574]0342d73c 0c9b5b44 MS.Internal.JoltHelper.FireEvent(IntPtr, IntPtr, Int32, Int32, System.String) [f:\dd\xcp\clr\clrlib\clrlib.cs @ 323]0342d780 0ca3cbcf DomainNeutralILStubClass.IL_STUB_ReversePInvoke(Int32, Int32, Int32, Int32, IntPtr)0342d85c 0668e2dc [ContextTransitionFrame: 0342d85c]
Very well done Sempai - you are learning quickly!
That's enough lessions for today. Later we will look at finding those Exceptions and Memory Leaks.
Very helpful! Thanks for posting this!!!