Attaching a debugger to a hung process or one that is running that we can cause to crash through an action is one thing… but sometimes a process will not even reach a running state and simply throw an error at the very start.
If you’re lucky the error is caused by I/O and Process Monitor is able to give you a clue as to what was missing – however if the problem is due to data being bad rather than missing, we may not see anything obvious with this tool (note that this symptom is not specific to process startup).
One of the tricks we can use is to launch the program through a debugger – in WinDbg there is the option File > Open Executable, from where you can browse to a program and optionally add arguments and a folder to use for “start in” when the process starts.
This is good for interactive user-mode processes, but services are a bit special as they run in session 0 which is now not interactive, so we need another plan…
Under the following key in the registry, Windows will look for a key that matches the name of the executable being started: HKLM\Software\Microsoft\Windows NT\Image File Execution Options
If there is a key that matches, then Windows will look for a string value named Debugger – if there is one present then the command line specified here is started and the original command line is passed as the last argument.
To work through an example that is safe, let’s first make a copy of NOTEPAD.EXE and name is PAULTEST.EXE.
The Debugging Tools for Windows are needed, so let’s install those – I am going to assume the 64-bit version is installed, so if you are testing on 32-bit you will need to amend the command lines accordingly.
Using Registry Editor, drill down to the key: HKLM\Software\Microsoft\Windows NT\Image File Execution Options
Create a new key named PAULTEST.EXE, then drill down into it.
Create a new string value named Debugger and set it to the following data (all on one line), including the quotation marks: ”C:\Program Files\Debugging Tools for Windows (x64)\ntsd.exe” -server tcp:port=12345 -noio
Open an elevated command prompt and enter the following commands: cd /d “C:\Program Files\Debugging Tools for Windows (x64)” paultest.exe windbg –remote tcp:port=12345
What should happen now is that WinDbg starts and attaches to the PAULTEST.EXE process which has not started to load its modules yet as we have hit an initial breakpoint.
This will happen for every instance of the process whose executable is named PAULTEST.EXE
Even though the file on disk and executable name started is PAULTEST.EXE, the module internally is still “notepad” for the purposes of debugging
As it is not possible to have more than 1 process listening on a TCP port, you cannot have more than 1 instance of PAULTEST.EXE running at any given time as the subsequent attempts to start NTSD to set up the debug server will fail
The WinDbg line above specifies no server name for the “remote” connection, so localhost is assumed. If PAULTEST.EXE is running on server FOO, then you can run WinDbg on another machine with the following command line: windbg –remote tcp:server=FOO,port=12345 (Alternatively the IP address of FOO can be used for the server argument.)
As the debugger is in control of the process startup, you can either step through it (p), allow it to run (g), set some breakpoints (bp, ba, etc.) or break on debugger events such as module loads (sxe ld).
Sometimes you’ll want to step through the module loads until one is loaded in which you wish to set a breakpoint – otherwise you’ll have to use “break unresolved” (bu) to allow it to be deferred until the module loads.
Another trick is to search for a function name containing “main” in the executable (notepad!*main*), set a breakpoint on it (bp notepad!WinMain) and let it continue (g) so that all the modules are loaded and we’ve still not passed control over to the program to have it do something.
The debug server method above is perfect for being able to debug the startup of services, but there are a few things to take into account:
1. If debugging a service running in a shared svchost.exe instance, you should first isolate it – and bear in mind that you cannot leave the Image File Execution Options for svchost.exe when rebooting or every hosted service will attempt to use it, and fail to start (Isolating service svcname running under svchost.exe is easy via the command: SC CONFIG svcname TYPE= OWN)
2. Services running under Local Service do not have privileges to access the network, so the ntsd.exe process spawned is unable to attach to the listening TCP port even though its state is reported as LISTENING by netstat
3. The Service Control Manager (SCM) expects a response to its start control request within 30 seconds by default, and if it doesn’t get one then it terminates the process it was starting – this can be controlled by specifying the timeout in milliseconds through a registry value: Path: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control Name: ServicesPipeTimout Type: REG_DWORD Data: (timeout in ms, default is 30000)