If you are managing updates for Windows Server 2012, you might have noticed that your WSUS server has been synchronizing a greater than usual number of updates these days. Windows Server 2012 is not only Microsoft’s most powerful, but also Microsoft’s most secure operating system to date. Since RTM, we’ve published dozens of updates that improve the security, reliability, and functionality of Windows. Most notably, the Defender team has been publishing virus and malware definition updates 4 times a day, to ensure that your PC is never left unprotected from even the latest threats.
We’ve also added many new products to WSUS, including updates to Adobe Flash Player, Skype, Lync Server, and Office 2013. There are more WSUS updates than ever before.
All these updates certainly can take a toll on a WSUS server, especially on servers that have auto-approval rules. Many administrators were recently surprised to see that exporting updates from WSUS servers was failing, resulting in zero-size output files. It became clear to us rather quickly that the issue was due to a limitation in the CAB file format, which is an uncompressed file size limit of 2 GB.
I am pleased to announce that, as of today, a fix for this issue is available from Microsoft. An article describing how to get this update is available at:
Many people have been asking on the forums and even through this blog for this fix for some time, and were wondering why it took more than 4 months to complete. To that end, I thought it would be interesting to share some of the engineering story as well.
One of the unique challenges of working on the WSUS team, compared to other server role teams in Windows Server, is that our role actually ships, “out-of box,” all the way back to Windows Server 2003 R2. We had to be very careful to design a solution that would work on all versions of Windows Server since then. To keep everything as simple as possible, we constrained ourselves on using compression algorithms that are already publicly available in the .NET framework or public Win32 API. Here is what we considered:
Unlimited, 4 GB prior to .NET4
Built in CRC
Results in 2 files
No support for signtool
Unlimited, not available until .NET4
Supports larger file sizes than CAB
Results in 2 files, harder to work with raw DEFLATE format vs. Gzip, no support for signtool
2 GB uncompressed file limit
Same file format
Supported by signtool
Temp files needed (extra disk I/O), 2 GB limit per uncompressed file
No CLR support for Windows Server 2003 R2; no support for signtool
We decided it wouldn't be a great customer experience to require people to install .NET 4.5 on their servers. Plus, we wanted to preserve compatibility with Windows Server 2003 R2 if at all possible. .NET 4.5 is not available on Windows Server 2003 R2. We could have also adapted the CAB format to produce multiple output files, but we preferred to produce only a single output file, especially since the resulting files are comparatively small. On systems running the latest version of .NET 4.0, the maximum file size is limited only by NTFS file size. While Gzip does store the length of the compressed content using 32 bits, the structure stores only the lowest 32-bits thereby allowing for larger file sizes, limited only by NTFS file size (prior to .NET 4.0, the length of the compressed content couldn’t exceed 32 bits, or about 4 GB). This does mean that there’s a limit of 4 GB on systems using .NET 3.5 (i.e., WSUS 3.0 SP2 (3.2) ). Windows Server 2012 is on .NET 4.5 and therefore not affected by the limit. Similar to CAB, GzipStream also includes built-in CRC error detection, which makes the export and import processes highly reliable. Best of all, the compressed output is generated on-the-fly, without the need for temporary uncompressed output files. This greatly decreases disk I/O and we found it reduces the time needed for import and export significantly.
Using our current export algorithm, the export process results in 2 files, a metadata.txt file and an update.cab file, which must be kept together. The CAB format archives these two files together. However, since Gzip is not an archive format, the end user would have needed to keep these files together manually. We would have preferred to produce only a single export file to ease distribution. Therefore, we changed the WSUS exported data XML schema to allow for both metadata and update data in a single compressed file.
In parallel, we also worked with our test engineers to formulate a test plan for the hotfix. Our test matrix is large, as we need to ensure that every supported version of Windows Server will be able to install this update. For Windows Server 2003 R2, that matrix is made even bigger by our support for both x86 and x64 architectures. We also worked with Customer Support Services (CSS) to gather validation data with regard to cases that were currently opened about this issue. And of course, nothing works perfectly the first time, and we’ve gone through several revisions to make sure that the latest WSUS update is more robust than ever!
At this point, we have a hotfix that we feel really good about releasing: well-informed by customer needs, developed by our esteemed software engineers, and thoroughly tested and validated. I hope that it will improve the experience of the many WSUS admins around the world, and I look forward to hearing your feedback about this update.
Thanks for being a valued Windows Server and WSUS customer!
The WSUS Team
@Rufus, the *uncompressed* data must not be more than 4 GB on WSUS 3.2.
Thanks for you comment. But I thought the application of KB2828185 and Gzip circumvents this restriction.
Hi Rufus, with KB2828185 the limit, as stated in the table above, is "Unlimited, 4 GB prior to .NET4" -- WSUS 3.x uses the .NET framework 3.5. To get around the limit, you can call the WSUS API from a .NET 4.5-based application.
If you are using WSUS 6.x (on WS2012 with KB2828185, or WS2012 R2), you should not have any limitations on the exported data size.
I get it. The above solution cannot assist me in resolving my issue (the metadata exceeds 4GB). However, I was able to find a workaround. Perhaps, someone with a similar issue will find some utility from the workaround.
1. Create a standalone (do not add this machine to the domain) virtual machine on the Internet-side of your network.
2. Install WSUS and the WSUS KB2720211 patch.
3. Select updates that you need.
4. Copy the WSUS machine from the Internet-side to the closed side of your network.
5. Add the machine to the domain.
6. Configure WSUS clients to report to WSUS server.
7. If clients receive a 800B0001 error, apply the System Update Readiness Tool to possibly fix the error.
Hi -- thanks for this patch, it's been extremely helpful with advancing a disconnected WSUS environment. One thing that was important to note for me was that your WSUS environment may be working with remote SQL even if the registry is not configured correctly.
We've got Remote SQL in place, but the "wYukonInstalled" value was still set to "1" (true) -- despite no Windows Internal Database being installed on the box anymore. With that flag set to True, the patch looks for the WID to update and fails otherwise. Toggling
the value to "0" allowed the patch to be installed successfully. Thought this might be useful for people with remote SQL installations :)
Hi, I am having a problem since installing this update on my disconnected WSUS setup.
Everything seems fine when doing the export, I get a xml.gz file which contains a 2.5gb XML file. when I try to import it I get the 'Fatal Error: unable to uncompress package'. When I check the logfile there is no more information than that. Right now I am
trying to import just the xml after extracting it using 7-zip...
Any other ideas in case this doesn't work?
Sadly that didn't work either I still get the same error message.
Need some help with this!
I have Windows Server 2008 R2 update kb2828185 installed and .net 4.5.1. I'm still having issues with 4 GB. is the syntax the same? if not what is the new syntax?
I have done a few exports after applying the hotfix and tried to check the xml file. Seems like < is replaced by < and > is replaced by >. Wrote and application to replace the characters and the size of that xml reduced drastically. Is it possible to put that
brackets ( since the xmls would usually have <> angled braces and not the representations there of ).
Currently I am not able to export the file as it exceeds 4 GB and trying to install .NET 4 as suggested here. But still if we can replace those characters with braces it will surely take care of the size issue in my humble opinion.
thanks & regards,
Please read :
as Seems like < is replaced by "<" and > is replaced by ">" in my previous comment.
thanks & regards
Again, the actual characters replaced.
I meant that in the exported xml after the hotfix, the angled brackets are replaced by "&LESSTHAN (_<_)" and "&GREATERTHAN (_>_)" .Wrote and application to replace them and the xml size reduced drastically. If we can do something about that, lot of issues related
with size will be taken care of IMHO.
Sorry for too many comments back to back.
thanks & regards
rudresh seems no one replies back to my question and maybe you could help me. I have .net 4.5.1 and the hotfix installed but am still having issue with the 4 gb limitation. Is there a new syntax when doing the export of metadata?
Seems like it has something to do with your %TEMP% size also. If you can cleanup that directory , then exporting might work. Please let me know. Good luck.
thanks & regards,
Rudresh for replying. Tried cleaning using the wsus server cleanup wizard and the temp folder is clean.
I have 2008 r2 sp1 with 4.5.1 and the hotfix. Should I be able to export metadata > 4GB?