Thoughts from the EPS Windows Server Performance Team
One of the most common questions we ask when troubleshooting issues that only occur on certain systems is, “What’s different between the two machines?” In many cases, the response is usually, “nothing”. Given that in large enterprise environments, desktop deployment is usually performed through the use of some sort of imaging process, this is actually not a surprising answer. From a high-level perspective, imaging hundreds (or thousands) of PC’s on similar hardware should in theory produce clone machines. But the reality is that every machine in inherently unique whether or not is built from an image – and each machine’s uniqueness only increases over time. Let’s take a look …
To begin with, remember that when a master image is created, it is (normally) Sysprep’d. If you are unfamiliar with Sysprep, there is an excellent Sysprep reference on Technet called What is Sysprep. In a nutshell however, Sysprep configures a number of OS settings on the master image to ensure that every copy of the image is unique when deployed to a target system – in particular the System Security Identifier (SID). Were we to push an image out that had not been sysprep’d, the target computers would all share the same SID, which would present a security risk. Think of it in these terms – we’re all human, yet we are all unique on account of our DNA. OK, before we branch out too far into the metaphysical, let’s get back to the technology!
Setting aside the SID for a moment, each target system is similar from the standpoint of OS installation, base installed applications, configured system settings and so forth. Of course, once these systems are distributed to users, the similarities gradually diminish. We all customize our workstations to optimize our own experiences – everything from font sizes within applications, Internet Explorer favorites, folder structures within Microsoft Outlook and our Documents folders – we all place our own stamp on systems that we use. Beyond the cosmetic though, in environments where the desktop systems are not tightly managed, we’ve seen numerous instances where something as critical as system updates are not always applied consistently. Centralized patch management systems such as WSUS certainly make administrators’ lives easier, however they are not always 100% successful. Systems that are offline may not get patched when they are brought online, the patch installation itself may fail for different reasons – such as not having a prerequisite patch (or Service Pack) installed already.
From an application standpoint, we used to deal with a number of “mismatched” scenarios when working on Internet Explorer cases. IE Add-On components can dramatically alter the behavior of IE and web-based applications. Those of you that we’ve worked with in the past can probably remember that one of the first troubleshooting questions was, “what happens if we disable all the unnecessary add-ons”. The Add-On components that might be installed may vary wildly from one user to the next – different toolbar add-ons, different media player extensions, versions of common software add-ons such as Java and Flash – all of these represent a departure from the baseline image that represents the last time that the system was in any sort of a consistent state.
And that transitions nicely into a quick discussion regarding the fact that the user account under which certain problems are reproducible is also important. The most common occurrence of this sort of problem is when users have insufficient rights to perform certain actions. “It works fine for user X (who has local admin rights), but not for user Y (who doesn’t)” – in some cases, the customers with whom we were speaking were not aware that the users had different local rights, because the users were not supposed to have local admin privileges. Over time, as we all know, things change. User X might have been granted local admin privileges as part of their participation in the UAT process for a new internal software package. However, once the rights were granted, they were never revoked – a simple oversight but one that can make all the difference when troubleshooting issues.
Finally, we should talk about firmware updates. Although systems may have identical hardware configurations, and the same base images deployed to them, the firmware levels for various hardware components may vary from machine to machine depending on when the systems were built by the OEM. Differing firmware levels for components could certainly lead to inconsistent (and undesirable!) behaviors between machines. Abstracting from the client level for a moment, those of you who are responsible for systems that are connected to SAN’s have probably experienced odd issues when HBA firmware revision levels are different – this is normally where we see a lot of calls as they relate to firmware. At the client level, the obvious area where you’ll see differences is with BIOS levels. However, firmware levels on DVD drives, Network cards, video cards and so forth may all play a part in varying behaviors between systems.
OK, that brings us to the end of this post. Today’s post wasn’t overly technical – but I just wanted to share some of the common questions / answers that we see around inconsistent behaviors. Take care folks!
- Sumesh P.