WSUS Product Team Blog

WSUS Product Team thoughts, information, tips and tricks and beyond :-)

Problem Solved: The WSUS Export Bug

Problem Solved: The WSUS Export Bug

  • Comments 24
  • Likes

If you are managing updates for Windows Server 2012, you might have noticed that your WSUS server has been synchronizing a greater than usual number of updates these days. Windows Server 2012 is not only Microsoft’s most powerful, but also Microsoft’s most secure operating system to date. Since RTM, we’ve published dozens of updates that improve the security, reliability, and  functionality of Windows. Most notably, the Defender team has been publishing virus and malware definition updates 4 times a day, to ensure that your PC is never left unprotected from even the latest threats.

We’ve also added many new products to WSUS, including updates to Adobe Flash Player, Skype, Lync Server, and Office 2013. There are more WSUS updates than ever before.

All these updates certainly can take a toll on a WSUS server, especially on servers that have auto-approval rules. Many administrators were recently surprised to see that exporting updates from WSUS servers was failing, resulting in zero-size output files. It became clear to us rather quickly that the issue was due to a limitation in the CAB file format, which is an uncompressed file size limit of 2 GB.

I am pleased to announce that, as of today, a fix for this issue is available from Microsoft. An article describing how to get this update is available at:

Many people have been asking on the forums and even through this blog for this fix for some time, and were wondering why it took more than 4 months to complete. To that end, I thought it would be interesting to share some of the engineering story as well.

One of the unique challenges of working on the WSUS team, compared to other server role teams in Windows Server, is that our role actually ships, “out-of box,” all the way back to Windows Server 2003 R2. We had to be very careful to design a solution that would work on all versions of Windows Server since then. To keep everything as simple as possible, we constrained ourselves on using compression algorithms that are already publicly available in the .NET framework or public Win32 API. Here is what we considered:

 

Framework

Max Size

Benefits

Drawbacks

GzipStream

.NET 4.0

Unlimited, 4 GB prior to .NET4

Built in CRC

Results in 2 files

No support for signtool

DeflateStream

.NET 4.0

Unlimited, not available until .NET4

Supports larger file sizes than CAB

Results in 2 files, harder to work with raw DEFLATE format vs. Gzip, no support for signtool

CAB

.NET 2.0

2 GB uncompressed file limit

Built in CRC

Same file format

Supported by signtool

Temp files needed (extra disk I/O), 2 GB limit per uncompressed file

Zip64

.NET 4.5

unlimited

Built in CRC

No CLR support for Windows Server 2003 R2; no support for signtool

We decided it wouldn't be a great customer experience to require people to install .NET 4.5 on their servers. Plus, we wanted to preserve compatibility with Windows Server 2003 R2 if at all possible. .NET 4.5 is not available on Windows Server 2003 R2. We could have also adapted the CAB format to produce multiple output files, but we preferred to produce only a single output file, especially since the resulting files are comparatively small.  On systems running the latest version of .NET 4.0, the maximum file size is limited only by NTFS file size. While Gzip does store the length of the compressed content using 32 bits, the structure stores only the lowest 32-bits thereby allowing for larger file sizes, limited only by NTFS file size (prior to .NET 4.0, the length of the compressed content couldn’t exceed 32 bits, or about 4 GB). This does mean that there’s a limit of 4 GB on systems using .NET 3.5 (i.e., WSUS 3.0 SP2 (3.2) ). Windows Server 2012 is on .NET 4.5 and therefore not affected by the limit. Similar to CAB, GzipStream also includes built-in CRC error detection, which makes the export and import processes highly reliable. Best of all, the compressed output is generated on-the-fly, without the need for temporary uncompressed output files. This greatly decreases disk I/O and we found it reduces the time needed for import and export significantly.

Using our current export algorithm, the export process results in 2 files, a metadata.txt file and an update.cab file, which must be kept together. The CAB format archives these two files together. However, since Gzip is not an archive format, the end user would have needed to keep these files together manually. We would have preferred to produce only a single export file to ease distribution. Therefore, we changed the WSUS exported data XML schema to allow for both metadata and update data in a single compressed file.

In parallel, we also worked with our test engineers to formulate a test plan for the hotfix. Our test matrix is large, as we need to ensure that every supported version of Windows Server will be able to install this update. For Windows Server 2003 R2, that matrix is made even bigger by our support for both x86 and x64 architectures. We also worked with Customer Support Services (CSS) to gather validation data with regard to cases that were currently opened about this issue. And of course, nothing works perfectly the first time, and we’ve gone through several revisions to make sure that the latest WSUS update is more robust than ever!

At this point, we have a hotfix that we feel really good about releasing: well-informed by customer needs, developed by our esteemed software engineers, and thoroughly tested and validated. I hope that it will improve the experience of the many WSUS admins around the world, and I look forward to hearing your feedback about this update.

Thanks for being a valued Windows Server and WSUS customer!

 

The WSUS Team

 

Comments
  • @Rufus, the *uncompressed* data must not be more than 4 GB on WSUS 3.2.

  • Thanks for  you comment.  But I thought the application of KB2828185 and Gzip circumvents this restriction.

  • Hi Rufus, with KB2828185 the limit, as stated in the table above, is "Unlimited, 4 GB prior to .NET4" -- WSUS 3.x uses the .NET framework 3.5. To get around the limit, you can call the WSUS API from a .NET 4.5-based application.

  • If you are using WSUS 6.x (on WS2012 with KB2828185, or WS2012 R2), you should not have any limitations on the exported data size.

  • I get it.  The above solution cannot assist me in resolving my issue (the metadata exceeds 4GB).  However, I was able to find a workaround.  Perhaps, someone with a similar issue will find some utility from the workaround.

    1. Create a standalone (do not add this machine to the domain) virtual machine on the Internet-side of your network.

    2. Install WSUS and the WSUS KB2720211 patch.

    3. Select updates that you need.

    4. Copy the WSUS machine from the Internet-side to the closed side of your network.

    5. Add the machine to the domain.

    6. Configure WSUS clients to report to WSUS server.

    7. If clients receive a 800B0001 error, apply the System Update Readiness Tool to possibly fix the error.

  • Hi -- thanks for this patch, it's been extremely helpful with advancing a disconnected WSUS environment. One thing that was important to note for me was that your WSUS environment may be working with remote SQL even if the registry is not configured correctly. We've got Remote SQL in place, but the "wYukonInstalled" value was still set to "1" (true) -- despite no Windows Internal Database being installed on the box anymore. With that flag set to True, the patch looks for the WID to update and fails otherwise. Toggling the value to "0" allowed the patch to be installed successfully. Thought this might be useful for people with remote SQL installations :)

  • Hi, I am having a problem since installing this update on my disconnected WSUS setup.

    Everything seems fine when doing the export, I get a xml.gz file which contains a 2.5gb XML file. when I try to import it I get the 'Fatal Error: unable to uncompress package'. When I check the logfile there is no more information than that. Right now I am trying to import just the xml after extracting it using 7-zip...

    Any other ideas in case this doesn't work?

  • Sadly that didn't work either I still get the same error message.

    Need some help with this!

Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment