What’s New in Windows Servicing: Reduction of Windows Footprint : Part 2

What’s New in Windows Servicing: Reduction of Windows Footprint : Part 2

  • Comments 12
  • Likes

My name is Aditya and I am a Sr. Support Escalation Engineer for Microsoft on the Windows Core Team. This blog is a continuation of the previous Servicing Part 1. So to understand this blog better, it is recommended that one reads the previous blog post.  As mentioned in the previous, this is a 4 part Blog series on Windows Servicing.

What’s New in Windows Servicing: Part 1
What’s New in Windows Servicing: Reduction of Windows Footprint : Part 2
What’s New in Windows Servicing: Service Stack Improvements: Part 3

Before we dive into Single Instancing and Delta Compression, I thought it would be a good idea, to talk about why this was introduced and how it worked in the previous Operating Systems. The reason for both Single Instancing and Delta Compression was to reduce the Windows (Windows 8.1 and Windows Server 2012 R2) footprint. Here is how and why:

Windows Footprint Reduction Features:The disk footprint of a Windows directly affects end-users, as it reduces the amount of available space for music, videos, pictures, and all other content. Even as we shift more user content to the cloud, factors such as high-resolution photos and videos, limited and costly bandwidth, and safety/security concerns over cloud storage mean that local storage requirements would remain constant for the next few years.

The disk footprint of Windows also directly affects our OEM partners. Available storage is one of the most important metrics, that an end-user looks at when purchasing a system, and OEMs are pushed to provide higher storage capacity. The current trend is that many of the OEM’s are shifting to SSD storage, due to its small footprint (enabling smaller, sleeker devices), low power consumption, low noise, and improved performance. Unfortunately, SSD storage can cost as much as 10x the price of conventional spindle based storage, which means that OEMs can only add limited storage to their systems before the cost becomes too great.

If Windows consumes less of the available disk footprint, while still providing a great end-user experience, this provides end-users with more disk space for their content, without requiring the OEM to spend more on storage, thus reducing the price of PCs.

For rollback purposes, the previous versions of Windows Components are sometimes kept in WinSxS store after installation of new updates through Windows Updates. The MUM Servicing feature which was introduced in Windows 7 and Windows Server 2008 R2 ensures that the disk space growth due to GDR installations can be reclaimed after a Service Pack (SP) installation by running Disk Cleanup Utility manually.

Windows strives to constrain servicing footprint growth, which are due to GDR installations either before or after a SP installation. The feature also focuses on enabling the servicing footprint reduction support at the Component Based Servicing technology level that targets for the following scenarios:

1. Consumers opt in for automatic updates on their Windows 8 devices, and notice that the WinSxS store footprint no longer grows significantly over time.

2. Consumers notice that the WinSxS store footprint has grown due to update installations over time, and then run Disk Cleanup Utility to reduce the WinSxS store footprint and reclaim disk space on their devices.

3. OEMs service their golden images in technician labs over time to keep them up-to-date and secured. Before the image is delivered to ODM for deployment at factory floor, they clean up the image by running DISM to scavenge away all the superseded components and recapture the smaller sized image.

4. Similarly, IT Admins service master images in their image libraries to keep them up-to-date and secured. Before the image is ready for deployment to Client machines, they clean up the image by running DISM to scavenge away all the superseded components and recapture the smaller sized image.

The feature to reduce the disk space used by Windows with the focus on Windows Components. Windows Update routinely installs patches on released Windows machines but does not always remove previous content that is replaced by the patches and which are not in use anymore. The purpose of this feature is to reduce disk footprint growth over time and also to provide a means by which power-users can reduce the original disk footprint of Windows.

This feature reduces disk footprint grown over time by uninstall and deletion of content that can be removed from the system and compression of unused content that may not be removable from the system

Reducing the footprint of Windows also improves deployment performance, which benefits consumers, Enterprise, and OEMs.

1. Single Instancing Catalogs: This feature contributes to the component store footprint reduction by single-instancing catalogs across the CATROOT, and Windows Servicing Stack stores. 

Term

Definition

Catroot

%windir%\system32\catroot

Servicing Stack Packages

%windir%\servicing\packages

Servicing Stack Catalogs

%windir%\winsxs\catalogs

 The redundant catalogs are single-instanced by hard-linking them across the three stores, nullifying the Windows Servicing Stack footprint overhead. To minimize impact to other catalog clients, changes were scoped to just those catalogs installed by the Servicing Stack.

More information on how hard-linking affects and works in the Windows Servicing Stack, one can refer to this TechNet article:

Manage the Component Store

2. Delta Compression of Superseded Components: This feature contributes to the component store footprint reduction by significantly reducing the size of files that have been superseded by later updates, yet they remain on the computer in case the user needs to uninstall a recent update. 

Term

Definition

Component

The smallest serviceable unit that includes files, registry data, meta-data, and etc., that describes how to service that set of files, and etc.

Installed component - winner

This is the ‘winning’ version of a component in a component family. This is the payload that is projected into System32 (or whichever location is specified in the component manifest).

Installed component - superseded

These components are installed, but are older versions than the winning component. The payload exists in the component store, but does not get projected to System32. If the winning component is uninstalled, the highest versioned remaining component becomes the new winner.

Latent component

These components are available for installation under the proper circumstances, but are not currently installed. The most common form of a latent component is a component that belongs to an optional feature that is currently disabled.

Superseded components are kept in the component storein case a user uninstalls the winning component (by uninstalling an update, for example). End-users infrequently uninstall updates, making those updates a prime target for reclaiming space. This feature uses a type of compression known as delta compression to dramatically reduce the size of superseded and latent components.

Delta compressionis a technology based on the differencing of two similar files. One version is used as a baseline and another versions is expressed as baseline + deltas.

The delta compression is performed against the winner component at the time of compression. This means the deltas for a specific component is different from machine to machine, depending on which winner was available at the time of compression.

Let me explain this by use of the following diagram Figure 1, in which V1, V2, and V3are all installed components prior to compression. During compression, V1 and V2 are compared against V3, the current winner, to create the necessary deltas.

clip_image002

Figure 1

In the next example, refer Figure 2 below, where V1 and V2 are installed, with V2 being the winner. After compression, V1 delta is created using V2 as the basis. Subsequently, V3 is installed. After the next compression, V2 delta is created using V3 as the basis.

Figure 2

Decompression or Rehydration:If the winning component is uninstalled, Windows Servicing Stack decompresses any components that are using the uninstalled version as their baseline, and makes the next highest versioned component the new winner. The uninstalled version is marked for deletion, and later when the Servicing Stack’s maintenance task runs, the uninstalled version is deleted, and any remaining superseded files are compressed against the new winner. For example refer to figure 3 below.

Figure 3

There may be cases where a file needs to be decompressed, but its basis file is also compressed. In these case, the Windows Servicing Stack would decompress the full chain of files necessary to decompress the final winning file.

Figure 4

At this point the big question that comes to mind is When Do We Delta Compress Components? The answer is pretty simple, Delta compression of superseded and latent content in the component store happens as part of the Servicing Stacks maintenance task. This process can be triggered either manually, or automatically.

Manual maintenance:Manually triggered by dism.exe.

Dism /online /cleanup-image /startcomponentcleanup

Automatic maintenance: Triggered by a scheduled maintenance task when the system is idle.

Task Scheduler Library  -->  Microsoft  -->  Windows  -->  Servicing

The automatic case is interruptible and resume-able. It automatically stops when the computer is no longer idle, and resumes when it becomes idle again.

For more detailed information, please refer to What’s New in Windows Servicing: Part 1.

Definitions:

Term

Definition

Delta compression

Compressing a file by capturing a diff of the file against a basis file. Requires the basis file to decompress.

Backup directory

Directory containing copies of boot critical files that are used to repair corruption

Manifest

Files describing the contents of a component. Windows is essentially defined by component manifests, approximately 15,000 of them (on amd64).

I hope this blog would have helped in understanding the efforts, put in the background by all the Windows team, in order to reduce the size of WINSXS considerably in Windows 8.1 and Windows Server 2012 R2.

The next blog in the series we would be discussing about the Servicing Stack improvements in KB2821895for Windows 8, and why it will assist your upgrade to 8.1?? Till then happy reading….

Aditya
Senior Support Escalation Engineer
Microsoft Platforms Support

Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment
  • Hi Aditya. Thanks for the post! Questions: 1. If servicing uses delta-compression for superseded components, what components/files are NTFS-compressed and why? In the Part 1 you posted the log http://bit.ly/1bO3OO3 in which NTFS compression follows Delta compression.
    //////////////////////
    2. Do delta-compressed files have a "Compressed" attribute?

  • Looks like comments are totally broken in this blog :( Trying again. Thanks the post! Two questions:
    ///////////////////////////
    1. If servicing delta-compresses superseded components, what components/files are NTFS-compressed and why? In the Part 1 you posted the log http://bit.ly/1bO3OO3 in which NTFS compression follows Delta compression.
    ///////////////////////////
    2. Do delta-compressed files have a "Compressed" attribute?

  • Nice post

  • Hi Vadim,
    In Windows 8.1, the payload from superseded components are delta-compressed. For driver, however, only NTFS compression is enabled. The reason is that due to the existence of hard-links between component store and driver store, delta compression will end up creating another instance of the file and will increase the disk usage.
    Yes delta-compressed files have a "Compressed" attribute...

  • >>Delta compression is done for superseded components only - because they are no longer hard linked from WinSxS to other locations in the file system. //

    >>NTFS compression is used for drivers to reduce their space usage. Delta compression cant be used for these files as they may have been hard linked to other places in the file system. NTFS compression is same as the compression that you may enable by going into a file's Properties > Advanced > checking "Compress contents to save disk space."//

    >>Compression attribute is set for both cases-: Get-ChildItem | ft Name, Attributes//////
    >>The key difference is that after delta compression, the compressed file itself will only contain differences (deltas), there will always be a base file to which these differences will be combined with - to yield the actual file. The delta file hence will be much smaller in size.

  • Hi Aditya, Vimal. Thanks for your clarifications! :)

  • Amazing stuff. I feel primitive when I see how superior is CBS.

  • Hi guys,
    I hope you read late comments, as I have a couple more questions on delta-compression that this post doesn't cover :)

    In CBS.log, I'm seeing (some lines are skipped):

    The following actions will be performed:
    Regeneration Candidates: 46
    Removal Candidates: 564
    Superseded/LDR (Delta Compression) Candidates: 904
    NTFS Compression Candidates: 12
    Null-Delta Compression Candidates:
    Mutable file Candidates: 5
    Boot-Recovery Candidates: 43
    Deletion Candidates (Non Driver): 0
    Driver Deletion Candidates: 13

    2014-07-31 04:20:02, Info CSI 0000000e Regenerating Component for: [ml:280{140},l:196{98}]"x86_microsoft-windows-t..icesframework-msctf_31bf3856ad364e35_6.3.9600.16410_none_0aab24d30a29830b"
    ...
    2014-07-31 05:06:46, Info CSI 00000e7a Beginning Null-Delta compression of components with mutable files
    ...
    2014-07-31 05:06:47, Info CSI 00000e8f Null-Delta compression of mutable files complete. Total files compressed: 2, Total files untouched: 0, Total files skipped due to backing: 0, Total file size compressed:24576, Total delta file size: 8192, compression ratio: 66%
    2014-07-31 05:06:47, Info CSI 00000e90 Beginning Null-Delta compression of boot-recovery components
    2014-07-31 05:09:43, Info CSI 00000f3d Null-Delta compression of boot-recovery components complete. Total files compressed: 75 (0x0000004b), Total files untouched: 0, Total files skipped due to backing: 0, Total file size compressed:119103488, Total delta file size: 39927808, compression ratio: 66%

    My questions are:
    1. What is regeneration? What conditions trigger it? Is it "rehydration" using your terminology?
    2. What is Null-Delta compression and how is it different from Delta compression?
    3. What are examples of mutable files and boot-recovery components? It's not clear from the log. Why is specifically Null-Delta compression applied to them?

    Thanks,
    Vadim
    :)