The Case of the Notepad that Wouldn't Run

The Case of the Notepad that Wouldn't Run

  • Comments 34
  • Likes

Dave Solomon was on campus a couple of weeks ago presenting a Windows internals seminar to Microsoft developers. Before I joined Microsoft I taught the classes here at Microsoft with him, but now with my other responsibilities here I step into the class and guest present a module or two if my schedule permits. This time I presented the security module, which describes logon (authentication) and the access check (authorization) model. It also includes a separate section on Vista’s User Account Control (UAC) feature, which consists of several technologies including virtualization and a new Mandatory Integrity Control (MIC) security model that’s layered on top of the existing Discretionary Access Control model that Windows NT introduced in its first release.

UAC allows for users, even administrators, to run as standard users most of the time, while giving them the ability to run executables with administrator rights when necessary. There are several mechanisms by which executables can trigger a request for administrator rights:

  1. If the executable image includes a Vista manifest file that specifies a desire or need for administrator rights (this would be added by the developer who creates the image).
  2. If the executable is in Vista’s application compatibility database as a legacy application that Microsoft has identified as requiring administrator rights to run correctly.
  3. If the user explicitly requests an elevation using Explorer’s “Run as administrator” menu item in the context menu for executables (also can be set as an advanced shortcut property). Note that this does not run the executable under the Administrator account, but rather under the account of the logged in user, but with the Administrator group enabled in the process security token.
  4. If the executable is determined to be a setup or installer program (for example, if the word “setup” or “update” is in the image’s name).

Perhaps the most common need for administrator rights comes from setup programs, which generally can’t install properly without write access to HKLM\Software and \Program Files, two locations that only administrators can modify. As an ad-hoc demonstration of the last request method, during the presentation I copied \Windows\Notepad.exe to my account’s profile directory, renaming it to Notepad-setup.exe in the process. Then I launched it, expecting to see a Consent dialog like the one below ask me to grant the renamed Notepad administrative rights:

To my consternation, no such dialog appeared. In fact, nothing happened. I reran it and got the same result. I was thoroughly confused, but didn’t have time to investigate in front of the class, so I moved on.

When I later got a chance to investigate what had happened, I started Notepad-setup.exe using Windbg (part of the free Debugging Tools for Windows) by clicking “File->Open Executable” followed by “Debug->Go” (or you can press F5). I then stepped through the initial instructions of Notepad’s entry point, Winmain. I saw it call an initialization function named NPInit that invokes LoadAccelerators to load Notepad’s keyboard accelerators. Strangely, LoadAccelerators was failing, causing NPInit to return an error to Winmain and Notepad to silently exit. But why would Notepad fail to load its accelerators, which should be included in the Notepad image itself?

My next step was to see if the file’s name was somehow causing the different behavior so I tried running a copy of Notepad.exe with the original name from my user directory, but got the same behavior (or lack thereof). It was time to watch what was happening with Filemon.

This scenario called for logging the operation of Notepad’s successful execution and comparing that to the log of the failing execution. I started Filemon, set the Include filter to Notepad.exe and the Exclude filter to list the processes that reference Notepad’s image when Notepad launches, including Svchost (where the prefetcher runs) and Explorer (which I was using to launch Notepad):

I collected both traces, but before I could compare them I had to remove the columns that are always different in different execution traces: Sequence, Timestamp, and Process. To do this I loaded the traces into Excel, selected the data in the first three columns, deleted it, and saved the traces back out as tab-demitted text. You can get the two trace files here.

There are a number of text comparison tools available, but one that’s both free and that serves the needs of this type of comparison is Microsoft’s Windiff. Simply open both files and red and yellow lines highlight differences.

The first few lines that Windiff flags are Notepad reading its prefetch file, which has a different name in each trace because the name encodes the full path of the Notepad image it is associated with in a hash number:

The next set of differences are operations present only in the successful run of Notepad, and appear to be queries of some kind of global Windows resource cache that’s new to Windows Vista:

It wasn't clear to me why one run references the cache and the other doesn’t, so I continued to scan through the differences. The next group of differences are at lines 47-51 and are simply due to the different paths of the two Notepad copies:

Finally, at line 121 I came across something that looked like it might be the source of the problem:

The execution of \Windows\Notepad.exe successfully reads a file named Notepad.exe.mui from the \Windows\En-us subdirectory. Further, at line 172 in the trace comparison the failed launch of Notepad tries to read a file of the same name from an En-us subdirectory, but fails because the subdirectory doesn’t exist:

I knew that .mui files store language-dependent resources like strings and accelerators, so I was pretty certain that Notepad’s failure to load its accelerators was due to its inability to find the appropriate resource file for my local, US English (En-us). To verify this I made an En-us subdirectory in my profile directory and copied Notepad.exe.mui into it, reran Notepad from my directory, and it worked.

Previous versions of Windows used .mui files to separate language-specific data from executables, but didn’t know that in Windows Vista this capability is exposed for applications to use. The nice thing about the .mui support is that resource-related functions like LoadAccelerators and FindResourceEx do the magic of the language-specific resource files so application developers don’t need to do anything special coding to take advantage of it.

Now that I had Notepad working outside of the Windows directory I turned my attention to why I hadn’t been presented with a UAC Consent dialog asking me to give it permission to run with administrator rights. What I discovered empirically and then confirmed later in the Understanding and Configuring User Account Control in Windows Vista article on Microsoft.com, is that heuristic setup detection only applies to files that don’t have an embedded manifest that specifies a security TrustLevel. Notepad, like all the Windows executables in Windows Vista, does include a manifest. You can see it when you do a dump of Notepad’s strings with the Sysinternals Strings utility:

So, thanks to Filemon, the case of the Notepad that wouldn’t run was closed!

Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment
  • Hi Mark,

    As usual, a very nice writeup that's so helpful to us who sometimes have to investigate why Windows does what it does. I just wanted to warn you that there are two editor's notes (in brackets) still embedded in your entry.

    Thanks, and keep up the good work!

  • Thanks a lot for your article, which lives up to the usual standard (ie. the best windows insight ever, period).

    There are a couple of things that I found really interesting here, which are:

    - a small, simple piece of software as notepad refusing to run without any kind of error message because its does not find language-specific locales. I hope it is an RC-XX kind of problem, but I have huge fears it will not be as easy as that: I have just been confronted by a (poorly coded?) .aspx app that refused to load and gave strange error messages on users that had their IE preference language set up as "en-securid" (most likely some braindead bsecurity sw was to blame here for th e change, but what about the warped mind of the api designers that let web apps be designed without a fallback language?)

    - the doc about uac was illuminating, in its description of the mechanism used for detecting when an executable is an installer and when it is not. I really cannot find any way to define it other than 'laughable': poor design that will lead to security headaches for years to come. Can't MS just stick to common security methodologies that have been in the field for decades? eg. let the user run whatever frickin' executable he wants, and make sure that user-level executables cannot do system-level damage, ever. Was it too hard asking for admin credentials at the very moment when the app tries to access critical section of the system, intead to guestimating 'what it might look likely to try to do'?

    Just my 2c.

  • > Then I launched it, expecting to see a
    > Consent dialog like the one below ask me to
    > grant the renamed Notepad administrative
    > rights:

    Yikes.  I thought that dialog was only asking for consent to run the program.

    For safety that dialog ought to be modified to provide three possibilities:
    Run the program with administrative rights.
    Run the program with ordinary rights.
    Don't run the program.

    For a lesser amount of safety, just change the wording to say what it will do in each case:
    Run the program with administrative rights.
    Don't run the program.

    The reason the latter change is less safe is that users who want to run the program will surely choose to run it as administrator rather than not running it.

  • Interesting, but I did not catch how new Vista technology would provide better security? Is it somehow restricted to run all these things (to become an admin for some particular file) in scripts.

  • PingBack from http://dancmorgan.wordpress.com/2006/10/02/the-case-of-the-notepad-which-wouldnt-run/

  • Thanks again for another great post. It does bring up a question. Why is there an assumption that all setup programs require admin access? Since there is no limitations on running programs from your profile (although that would be interesting) as long as the program doesn't enhanced access why single out installers? There may be a few situations where you want to install a program just on one user or test out a program in the relative safety of limited user. I also wonder that since limited user only has full write access to their profile, where should an program should be install into by default? Create a %USERPROFILE%\Program Files\... ? Thinking about any nt based windows, not just vista. And not just .msi files.

  • Gaetano: how would you stop an application constantly trying to perform administrative functions and spamming the user with consent dialogs? How would you handle programs which work fine on Windows XP as a limited user, which correctly handle the error return values when attempting to perform some administrative task? UAC creates a consent dialog at the start of a program, and that's it. Ilya: UAC only kicks in when a program is run from Explorer. No consent dialogs appear when running a program from a command prompt, from within a script, or programmatically. They run with the permissions of the parent process (typically CMD.EXE). To perform administrative tasks from a command window, right-click a Command Prompt shortcut and select Run As Administrator.

  • Monday, October 02, 2006 5:31 PM by Mike Dimmick > Gaetano: how would you stop an application > constantly trying to perform administrative > functions and spamming the user with consent > dialogs? Not being Gaetano, my answer might be different. When an application tries to perform an administrative function, Windows will ask the user. If the user approves, the application gets the privileges. If the user disapproves, the application gets the denial. If the user asks to be asked again later, then Windows will make some decision about this time but will ask the user again the next time the program tries to perform an administrative task. There is precedent for this. When a program tries to access the network, Windows Firewall asks the user. The user can say to approve, disapprove, or be asked again later. If the user chooses to be asked again later then Windows decides whether or not to grant the access this particular time. Of course the precedent also shows an improvement that could be made: the user could be given a chance to deny this time but be asked again later.

  • Mike: I would imagine that a sort of granted-security-permissions cache is present, so that after an app. has triggered a 'need more perms' dialog once, subsequent provileged calls will not trigger it anymore (afaict both sudo and macosx do it). I also strongly agree with catfish: why should a security dialog pop up if the software just wants to be installed in the user folders / reg hive, and needs no special admin privileges? Besides, I can only imagine that the 'bad guys' will immediately make sure their self-installing code has the correct name and resource strings to escape this basic security anlysis, so there seems to be little security added in doing the 'guestimate'. Also, I do not see how asking beforehand for admin privileges would change with apps "which correctly handle the error return values when attempting to perform some administrative task". I am not too versed in windows development, but I can imagine keeping backwards compatibility with 'bad' apps can be a gret hinderance. Isn't the introduction of a new OS the best (only?) opportunity to deprecate old habits and force third party developers to straighten up their act? ps: Note to self: always count to a hundred before posting flames.

  • So the next piece of malware is going to come as email message with 'windows_update.exe' attachment. User gives the consent with UAC, and puff, there's a cloud of orange smoke.

  • I have the same concern that many others have--if users weren't stopped from clicking through warnings on running unknown executables on XP SP2, why should we expect them not to click through the dialogs to allow an install of a piece of malware with the filename "setup.exe"?

  • > I have the same concern that many others have--if users weren't stopped from clicking through > warnings on running unknown executables on XP SP2, why should we expect them not to click through > the dialogs to allow an install of a piece of malware with the filename "setup.exe"? There are only so many hoops that you can make people jump though. Internet Explorer (and I assume Outlook Express?) run in Protected mode which provides more, well, protection, than running a program from windows explorer. But there's still nothing stopping a user from download a malware application, saving it to their desktop, double-clicking it, and clicking on "Allow". As Larry Osterman says, if people want to see the dancing bunnies, they damnit, they'll SEE the dancing bunnies. > I am not too versed in windows development, but I can imagine > keeping backwards compatibility with 'bad' apps can be a gret hinderance It's actually a GOOD app that would correctly handle an "access denied" error and degrade gracefully. It's the bad apps that would require the prompt as soon as they tried to do something admin-y. Good apps that ARE admin-y should be asked up-front (to provide a better experience). Good apps that do not need to be admin-y should not get asked at all. Now the better experience I mentioned was that if you launch an admin-y app, and if the admin-consent dialog only popped up on your first admin-y thing, then each time you run it, the admin-consent dialog would pop up in seemingly random places (depending on what admin-y thing you did first). Popping it up at the start provides a consistent experience.

  • I like your article Mark, very informative. But in next articles can you turn off some strong colors like you do in this article. They look pretty much like flowers.

  • "Popping it up at the start provides a consistent experience." Agree, but it looks like UAC itself is not providing _any_ new protection in such case. You only need to give the consent once at program startup, through dialog confirmation - just like you do today. Given the extremely intrusive nature of UAC, the question is: was it worth implementing in the first place?

  • "Popping it up at the start provides a consistent experience." It probably does. I have not downloaded and tested vista myself (yet), but there are many, many blogs that complain about how the UAC dialogs always pop up at unexpected times / are very hard to correlate to the specific permission that is being asked to the user. Otoh I have tested ubuntu and macOsX, and every single time the root pwd is asked, it easy very easy to tell why (of course, I'm an it guy, not avg. joe). I think this reflects a very common ui strategy on the part of microsoft: every time something is deemed too hard to understand by the user, "hide it". Note that is is very different from "make it simpler" or "remove it alogether". This is sadly a great burden for everybody, because users are never pushed towards learning the way their computer works / why things happen, but instead get accustomed to use a magical-mysterious-tool that works almost ok 50% of the time and the other 50% is just plain crazy. A golden standard of this was removal of filename extension from the default view in explorer: the extension part is still there, and it is what is actually being used by the os to take decisions, but it is not shown to the user anymore. Sense is lost forever. No new user will ever learn that renaming a file changing the extension will change its icon and make it open with a different app. I still am convinced that in this case "sense" is lost, too: uac dialogs are there to augment security. renaming an app to setup.exe has nothing to do with security. uac dialogs should not be triggered. as easy as aristotle would have had it...