I must visit and meet 1000’s of different customers over the course of a calendar year. I spend alot of that time discussing App-V and in these discussions around app-v i have a number of discussions around moving/migrating operating systems. On top of these discussions I end up talking about the Microsoft application compatibility toolkit and shims plus why app-v and the appcompat toolkit are such complimentary products.
Luckily as i was scanning over technet i came around this brilliant chris jackson article around the use of the appcompat toolkit and app-v. Chris wrote this for the March 2010 Technet and you can read this and many more articles here.
Its funny as I always meant to blog around this topic, and as many of my customers know (as we use the below approach or very similar) its a great way to get your apps working including some of your app-v apps sequenced on a legacy platform and moved up to a newer OS.
I thought the below article was very apt around the appcompat discussion and app-v and I hope it helps people take notice of the great tool sets that are released that can be user to help move your applications over to windows 7 easier.
Now over to Mr App Compat Chris Jackson to bring this discussion to the masses!
Microsoft Application Virtualization (App-V) 4.6, boasting full support for Windows 7, is right around the corner; many customers who are planning a Windows 7 deployment are including App-V among the components of their desktop transformation project. (An OS deployment is frequently combined with an overhaul of both applications and infrastructure in a “modern desktop” or “next-generation desktop” initiative.)
When IT professionals think about pairing investments in App-V and Windows 7, the following questions almost always come up as part of the conversation:
Let’s explore each of these questions.
Microsoft App-V is, first and foremost, an application management and deployment solution that can convey significant benefit to the enterprise—reducing packaging costs, increasing system stability, and supporting today’s highly mobile workforce with dynamic access to software assets. But as part of the marketing messaging, the overloaded term application compatibility grew to be misinterpreted over time: that App-V could help with compatibility problems between the application and the OS. For the most part, it can’t. (The exceptions today exist mostly for historical reasons, and are not something to depend on, so I won’t go into the details here.)
The resulting customer confusion has led to some clarification in the messaging; we now forego use of the term “application compatibility” and instead speak directly to the underlying benefits: that you reduce application-to-application conflicts (note the strategic omission of the word compatibility), and that, as a result, you significantly reduce regression testing.
The official stance of the product team on application-to-OS compatibility is this:
As noted in prior discussions, App-V is not a general purpose application-to-OS compatibility solution; however, if an application compatibility shim allows an app to work on a given Windows version natively (non-virtualized), it will in most cases, and for most shims, work with App-V when the shimmed app is virtualized. Thus, as a general rule, App-V will support app use with shims that are provided as part of Microsoft’s App Compat tools as long as the shimmed application can run natively on the targeted OS version.
So, it’s pretty clear that App-V isn’t intended to be an application-to-OS compatibility solution. (We will discuss combining shims with App-V later in this article.) Let’s look at the other impacts that application virtualization has on OS compatibility.
When we talk about application compatibility, it’s fair to separate package compatibility from runtime compatibility. In fact, our recommended process for application-compatibility testing (shown in Figure 3 of my June 2009 article on Planning Your Application Compatibility Project) separates install testing from runtime testing. Let’s start with the official product team guidance:
It is often possible to sequence on one OS and run the virtualized app on a different OS; however, this scenario is both app- and OS-dependent and is not guaranteed to work for all app/OS combinations because App-V is not a general purpose OS compatibility solution. If problems are encountered, the customer may be required to sequence on the same OS environment as the App-V client is running in order to resolve those problems.
OK, so this doesn’t sound so promising—the official stance is, basically, “it depends.” But when you’re thinking about measuring risk, compare the three primary setup technologies in use today:
Setup.exe: This runs arbitrary code, and thus the arbitrary code is subject to the same potential runtime problems that the arbitrary code of the application itself is, so it will have to be thoroughly tested.
Windows Installer: This also runs code to execute an install, but much of this code is structured, declarative code. Only the imperative code of custom actions is arbitrary, so outside of quantifiable changes in rules that affect the processing of code running against the database and the arbitrary code in custom actions, you expect better (though still not perfect) compatibility and less testing.
Microsoft App-V: This doesn’t have to run code at all. It just copies over a blob of data that is the virtual file system, and then says “OK, your virtual file system is here.” So, your primary concern isn’t whether the application will install (copying one blob of data is, after all, really easy), it’s the runtime compatibility of the application. So, you expect install testing to be shortest of all three technologies. That’s goodness!
So, while nobody is officially going to say that packages will all work, it seems the primary reason for the messaging is that many people conflate install issues and runtime issues. In general, you can look to reduce your projected costs of application installation testing significantly if you have already invested in Microsoft App-V.
Remember the tantalizing bit from the support statement? “App-V will support app use with shims that are provided as part of Microsoft’s App Compat tools….” How do you actually implement that? Are the two fundamentally compatible?
The answer is, fortunately, yes. In fact, you have a couple of different options for doing so.
For those of you who are unfamiliar with shims, a shim is one of the very few four-letter words in use by Microsoft that isn’t an acronym of some sort. It’s a metaphor based on the English language word shim, which is an engineering term used to describe a piece of wood or metal that is inserted between two objects to make them fit together better. In this particular case, the two objects are the application program and Windows, and the shim material is additional code that causes the two to behave better together, as shown in Figures 1 and 2.
Figure 1 Before the shim is applied, the application interacts directly with Windows.
Figure 2 After the shim is applied, the application interacts with Windows indirectly; the shim code is injected and can modify the request to Windows, the response from Windows, or both.
A shim works using API interception. The Windows API is implemented using a collection of DLLs. Each application built for Windows imports these DLLs, and maintains a table of the address of each of these functions in memory. Because the address of the Windows functionality is sitting in a table, it is straightforward for the shim engine to replace this address with the address of the shim DLL instead. The application is generally unaware that the request is going to a shim DLL instead of to Windows itself, and Windows is unaware that the request is coming from a source other than the application (because the shim DLL is just another DLL inside the application’s process).
For example, a very commonly used shim is a version-lie shim. To implement this shim, we intercept several APIs that are used to determine which version of Windows the application is running on. Normally, this information is passed on to Windows itself, and it answers truthfully. With the shim applied, however, these APIs are intercepted. Instead of passing on the request to Windows, a different version of Windows is returned (for example, Windows XP instead of Windows 7). If the application is programmed to run only on Windows XP, this is a way to trick the application into believing it’s running on the correct OS. (Frequently this is all that is necessary to resolve an application compatibility problem!)
There are a huge number of tricks you can play with shims. For example:
Leveraging shims as part of your application compatibility approach can lead to significant cost savings and accelerate a deployment of Windows 7 dramatically.
In my white paper on Managing Shims in the Enterprise from November 2007, I outline the two primary options most people choose between when deciding how to manage shims as an application-compatibility solution in their organization. To briefly recap, they are:
Single, Centrally Managed Shim Database With this option, you deploy a single, centrally managed shim database that contains entries for every application that requires a shim. You can then push out updates to the central database as you discover additional applications that require fixes. This is the approach that Microsoft uses. We shipped Windows 7 with a system shim database fixing thousands of applications, and as we find more we ship updates to the database via Windows Update. You can do the same thing, baking a shim database into your master image and updating via your systems management software (such as System Center Configuration Manager).
Per-Application Shim Databases Alternatively, you can deploy application fixes directly with the application. Some customers choose to do this when they only have one or two applications that require fixes, as it can be less expensive in a small-scale environment than setting up a process for managing a central database. A significant disadvantage of per-application shim databases is that it becomes difficult to update the shim database if there are any problems. However, App-V removes this as a drawback, as now you can update the per-application database and have the updated version streamed out to your users. (Though, as you will see, this introduces another, far more significant drawback.)
The approach that Microsoft App-V uses to deploy software broadens the choices you have for deploying application fixes across the enterprise, and you can choose whichever approach best fits the process model currently in place in your organization.
So, from a tactical standpoint, what do you have to do in order to deploy a single, centrally managed shim database and have it picked up by an application sequenced using App-V? It’s easy—just install it as you normally would! An application installed using App-V is launched in much the same way that the application normally would be, and uses the same loader mechanisms where shims are applied. They’re just launched by a surrogate process. Specifically, sfttray.exe is what does the work to launch the new process, rather than explorer. The process tree, as a result, is shown in Figure 3.
Figure 3 A Surrogate Process Tree
When the application is launched, it proceeds through the loader as any other application would. The Microsoft Application Virtualization Client Interface Layer (sftintf.dll) makes a call into CreateProcessW, which calls into the internal API CreateProcessInternalW. It is in the CreateProcessInternalW API that the shim engine is invoked and shims are wired up to the process.
OK, so that’s pretty easy. Are there any catches? Well, yes, there is one. It doesn’t handle elevation well. You can’t simply ask for elevation of an application (using a RunAsAdmin shim) or repair problems with applications that require elevation using ElevateCreateProcess, for example. Why? Because of the bubble.
For example, let’s take an application that’s trying to launch an automatic update of itself (unfortunately, an all-too-frequent task). Running natively, say it had a problem in that it used the CreateProcess API, which is unable to invoke elevation. It would then return error -1073740756 – STATUS_ELEVATION_REQUIRED. The ElevateCreateProcess shim would catch this return value and then invoke the Application Information Service to provide the elevation. But this service cannot find the application to elevate, because the service lives outside of the bubble!
So, as long as your application does not require elevation, deploying a single shim database solution is trivially easy if you already have a process worked out—you simply continue doing the same thing.
The alternative for deploying shims to resolve application compatibility issues is to deploy the shims with the application. In the MSI world, you would typically include the .sdb file in the installer, and then incorporate a custom action that called sdbinst against the .sdb file it dropped. This would install the custom shim database as part of the installer. (If you subsequently need to update the shims for that application, you need to locate all of your installed clients and run a script to install an update, which complicates things somewhat.)
With App-V, you can do much the same thing. You can include the shim database (.sdb) file into the sequence itself, so the .sdb file lives inside of the bubble. First, you sequence the application that requires shims in order to function properly. Once you’ve done that, I like to drag the custom shim database into system32.
(While normally I advise people to never, ever put anything into system32 unless your paycheck is signed by Steve Ballmer and your reporting chain includes Steven Sinofsky, in this particular instance I choose to do so because it simplifies scripting—system32 will be in your path, and you’re not actually leaving anything in system32 on a production system, just in what looks like system32 to the virtual file system).
Once I’ve done this, I edit the .sprj file. I navigate to the Virtual File System tab and click on View | Virtual File System | Add. Then, I browse to the location where I dropped the .sdb file (c:\windows\system32\appshim.sdb) and click on OK. Now, this .sdb file will be present inside the bubble and it will be installed when I install the softgrid package. Since the file is part of the container, it will be carried around as part of the application (and can be updated when the application is updated).
Of course, simply having the .sdb file on the file system isn’t sufficient to apply the application fixes—they have to be installed. However, the shim database must be installed outside of the App-V bubble. If you install the shim database inside of the bubble, then the virtualized .sdb file won’t be discoverable until after the process has been created, when it’s too late for the shim engine to find it and take action on it. You need to have the shim database accessible to the loader outside of the bubble as the process is being created.
Fortunately, App-V contains a mechanism to run scripts as part of the application launch process. If you take a look at the OSD file for an application using your favorite text editor, you’ll see that it is just XML that can be edited. The <SOFTPGK> element contains an <IMPLEMENTATION> element, a <DEPENDENCY> element, a <PACKAGE> element, an <ABSTRACT> element, a <MGMT_SHORTCUTLIST> element and a <MGMT_FILEASSOCIATIONS> element. The <DEPENDENCY> element is the one you want to edit—you are going to add child <SCRIPT> elements to execute the installation of the custom shim database. Here is an example of a script that can install the custom shim database when the application is launched, and remove it again when the application exits (so it is a JIT installation that cleans up behind itself, an important goal of App-V):
<DEPENDENCY> <CLIENTVERSION VERSION="18.104.22.168"/> <SCRIPT EVENT="LAUNCH" PROTECT="TRUE" TIMING="PRE" WAIT="TRUE" EXTERN="TRUE">
<SCRIPTBODY LANGUAGE="Batch">sdbinst /q appshims.sdb</SCRIPTBODY>
<SCRIPT EVENT="SHUTDOWN" PROTECT="TRUE" TIMING="POST" WAIT="TRUE" EXTERN="TRUE">
<SCRIPTBODY LANGUAGE="Batch">sdbinst /u appshims.sdb</SCRIPTBODY>
With this script, you will be calling the shim database installer (sdbinst.exe) from outside of the App-V bubble, but using an argument of an .sdb that lives inside of the App-V bubble and is consequently deployed as part of the application deployment sequence. Thus, it is easy to deploy and update the application fixes as if they were any other part of the application.
Installing custom shim databases is a process that requires administrator rights; is it a problem that we’re installing this at runtime? You don’t want to require admin rights for your users, and your users don’t want to have to click on UAC dialogs every time they launch their application. Fortunately, you don’t need to do that; the call to sdbinst.exe is automatically elevated for you. The process tree for installing the shims is shown in Figure 4.
Figure 4 A Shim Installation Process Tree
A command shell is created, which then makes a call to sdbinst.exe to install the custom shim database. This fails twice with STATUS_ELEVATION_REQUIRED before a call is made to the Application Information Service, which is the service that provides elevation. This service calls into consent.exe, which checks the policy to determine whether the application should be elevated. The policy (in the virtual environment) returns yes without having to prompt the user; consent.exe approves the elevation, and an elevated instance of sdbinst.exe is created that installs the custom shim database. But it’s important to note that a policy to elevate without prompting only works silently if you are able to elevate. If you are a standard user, every launch of the application will result in a prompt for administrator credentials, as will every time you close the application. Because this will likely be seen as extremely undesirable in a standard user environment, this approach only works if you have deployed local administrator users. We hope that the percentage of users this applies to is extremely small!
Microsoft Application Virtualization is a powerful application management and deployment tool. When working it in to your plans for a Windows 7 migration, you can benefit from the potential for reduced cost during installation testing (though this cost is not promised to be zero).
You can also continue to leverage most of the same processes for resolving application compatibility issues using shims. If you’re deploying a single, organization-wide shim database, the transition will be very minimal—you need only be careful to avoid elevation shims (in much the same way you should avoid elevation in general in an App-V environment). If you are deploying per-application shims, there are some elegant ways to package and deploy the shim database with the application, but you have the very significant drawback of being unable to reasonably run the application as a non-administrator user. As such, my strong recommendation (as it is with MSI-based application deployments) is to strive to deploy shims as a single, centrally managed database, which you can deploy using your existing systems management software. This enables you to look to a future where a larger and larger percentage of your users will run as non-administrators.
Chris Jackson (“The App Compat Guy”) is a principal consultant at Microsoft and the technical lead of the Windows Application Experience SWAT Team. Jackson is a widely recognized expert in the field of Windows application compatibility, creating the technical documentation, training, and service offerings used inside and outside of Microsoft based on years of real-world experience with enterprise customers and independent software vendors. Jackson can be reached via his popular blog.
Great Article Justin !!
Perfect clear and complete.
Excellent article. Thank you for this work which explains outstandingly App-v
I am a little confused.
At one point you said that :
Installing custom shim databases is a process that requires administrator rights; is it a problem that we’re installing this at runtime? You don’t want to require admin rights for your users, and your users don’t want to have to click on UAC dialogs every time they launch their application. Fortunately, you don’t need to do that; the call to sdbinst.exe is automatically elevated for you.
And again as your concluding remarks you said:
If you are deploying per-application shims, there are some elegant ways to package and deploy the shim database with the application, but you have the very significant drawback of being unable to reasonably run the application as a non-administrator user.
Could you please clarify these contradicting statements.