Mark's Blog

The Case of the Notepad that Wouldn't Run

Dave Solomon was on campus a couple of weeks ago presenting a Windows internals seminar to Microsoft developers. Before I joined Microsoft I taught the classes here at Microsoft with him, but now with my other responsibilities here I step into the class and guest present a module or two if my schedule permits. This time I presented the security module, which describes logon (authentication) and the access check (authorization) model. It also includes a separate section on Vista’s User Account Control (UAC) feature, which consists of several technologies including virtualization and a new Mandatory Integrity Control (MIC) security model that’s layered on top of the existing Discretionary Access Control model that Windows NT introduced in its first release.

UAC allows for users, even administrators, to run as standard users most of the time, while giving them the ability to run executables with administrator rights when necessary. There are several mechanisms by which executables can trigger a request for administrator rights:

1. If the executable image includes a Vista manifest file that specifies a desire or need for administrator rights (this would be added by the developer who creates the image).
2. If the executable is in Vista’s application compatibility database as a legacy application that Microsoft has identified as requiring administrator rights to run correctly.
3. If the user explicitly requests an elevation using Explorer’s “Run as administrator” menu item in the context menu for executables (also can be set as an advanced shortcut property). Note that this does not run the executable under the Administrator account, but rather under the account of the logged in user, but with the Administrator group enabled in the process security token.
4. If the executable is determined to be a setup or installer program (for example, if the word “setup” or “update” is in the image’s name).

Perhaps the most common need for administrator rights comes from setup programs, which generally can’t install properly without write access to HKLM\Software and \Program Files, two locations that only administrators can modify. As an ad-hoc demonstration of the last request method, during the presentation I copied \Windows\Notepad.exe to my account’s profile directory, renaming it to Notepad-setup.exe in the process. Then I launched it, expecting to see a Consent dialog like the one below ask me to grant the renamed Notepad administrative rights:

To my consternation, no such dialog appeared. In fact, nothing happened. I reran it and got the same result. I was thoroughly confused, but didn’t have time to investigate in front of the class, so I moved on.

When I later got a chance to investigate what had happened, I started Notepad-setup.exe using Windbg (part of the free Debugging Tools for Windows) by clicking “File->Open Executable” followed by “Debug->Go” (or you can press F5). I then stepped through the initial instructions of Notepad’s entry point, Winmain. I saw it call an initialization function named NPInit that invokes LoadAccelerators to load Notepad’s keyboard accelerators. Strangely, LoadAccelerators was failing, causing NPInit to return an error to Winmain and Notepad to silently exit. But why would Notepad fail to load its accelerators, which should be included in the Notepad image itself?

My next step was to see if the file’s name was somehow causing the different behavior so I tried running a copy of Notepad.exe with the original name from my user directory, but got the same behavior (or lack thereof). It was time to watch what was happening with Filemon.

This scenario called for logging the operation of Notepad’s successful execution and comparing that to the log of the failing execution. I started Filemon, set the Include filter to Notepad.exe and the Exclude filter to list the processes that reference Notepad’s image when Notepad launches, including Svchost (where the prefetcher runs) and Explorer (which I was using to launch Notepad):

I collected both traces, but before I could compare them I had to remove the columns that are always different in different execution traces: Sequence, Timestamp, and Process. To do this I loaded the traces into Excel, selected the data in the first three columns, deleted it, and saved the traces back out as tab-demitted text. You can get the two trace files here.

There are a number of text comparison tools available, but one that’s both free and that serves the needs of this type of comparison is Microsoft’s Windiff. Simply open both files and red and yellow lines highlight differences.

The first few lines that Windiff flags are Notepad reading its prefetch file, which has a different name in each trace because the name encodes the full path of the Notepad image it is associated with in a hash number:

The next set of differences are operations present only in the successful run of Notepad, and appear to be queries of some kind of global Windows resource cache that’s new to Windows Vista:

It wasn't clear to me why one run references the cache and the other doesn’t, so I continued to scan through the differences. The next group of differences are at lines 47-51 and are simply due to the different paths of the two Notepad copies:

Finally, at line 121 I came across something that looked like it might be the source of the problem:

The execution of \Windows\Notepad.exe successfully reads a file named Notepad.exe.mui from the \Windows\En-us subdirectory. Further, at line 172 in the trace comparison the failed launch of Notepad tries to read a file of the same name from an En-us subdirectory, but fails because the subdirectory doesn’t exist:

I knew that .mui files store language-dependent resources like strings and accelerators, so I was pretty certain that Notepad’s failure to load its accelerators was due to its inability to find the appropriate resource file for my local, US English (En-us). To verify this I made an En-us subdirectory in my profile directory and copied Notepad.exe.mui into it, reran Notepad from my directory, and it worked.

Previous versions of Windows used .mui files to separate language-specific data from executables, but didn’t know that in Windows Vista this capability is exposed for applications to use. The nice thing about the .mui support is that resource-related functions like LoadAccelerators and FindResourceEx do the magic of the language-specific resource files so application developers don’t need to do anything special coding to take advantage of it.

Now that I had Notepad working outside of the Windows directory I turned my attention to why I hadn’t been presented with a UAC Consent dialog asking me to give it permission to run with administrator rights. What I discovered empirically and then confirmed later in the Understanding and Configuring User Account Control in Windows Vista article on Microsoft.com, is that heuristic setup detection only applies to files that don’t have an embedded manifest that specifies a security TrustLevel. Notepad, like all the Windows executables in Windows Vista, does include a manifest. You can see it when you do a dump of Notepad’s strings with the Sysinternals Strings utility:

So, thanks to Filemon, the case of the Notepad that wouldn’t run was closed!