Leveraging On Premise Web Proxies to Reduce Internet Bandwidth Usage

Leveraging On Premise Web Proxies to Reduce Internet Bandwidth Usage

  • Comments 5
  • Likes

Some of you have asked about the possibility of leveraging existing on-premise infrastructure to reduce internet network bandwidth usage from PCs managed by Windows Intune. We are posting some information on this based on some internal testing that we have done in our labs.

If your network configuration includes a caching web proxy server, you can significantly reduce internet network bandwidth usage. Such caching web proxy servers can be configured to cache HTTP and updates binary download requests from Windows Intune to managed client computers. By avoiding duplicate downloads for content like Microsoft updates or endpoint protection signature updates, a caching web proxy server can reduce the consumed Internet bandwidth.

We tested the Windows Intune content caching efficacy of the following proxy servers in our labs:

  1. Microsoft Forefront Threat Management Gateway (TMG) server
  2. Microsoft Internet Security and Acceleration (ISA) server
  3. SQUID Proxy server

With the right configuration, we found all of them to be effective in caching Windows Intune content.

NOTE: We have included the prescriptive steps mentioned below for the proxy servers mentioned above, but if you happen to have a different proxy server in your company, the same basic caching configuration principles can be applied to achieve caching benefits. So, we encourage you to read the documentation below.

Below are the configurations that we used for our tests that you can use as guidelines to leverage and configure proxy servers for Intune content caching.

 

Squid proxy server

Squid is a caching proxy for the Web supporting HTTP, HTTPS, FTP, and more. It reduces bandwidth and improves response times by caching and reusing frequently-requested web pages. Squid has extensive access controls and makes a great server accelerator. It runs on most available operating systems, including Windows and is licensed under the GNU GPL.

Squid proxy server can be installed on a dedicated server machine inside the local area network and set up as a cache for the Windows Intune clients.

To leverage caching benefits from a SQUID proxy server, it is recommended that you tweak the cache file size (default is 4 MB) and cache object size (default is 100 MB) to much higher value to leverage high caching. Also, since the Windows Intune client leverages the Windows Background Intelligent Transfer Service (BITS) for issuing download range requests, you will need to tweak additional settings.

There are 3 main settings that need to be changed:

1. Increase the overall cache size to 5000 MB: cache_dir ufs /var/spool/squid 5000 16 256

2. Increase the size of a single file (object) that can be cached to 950 MB: maximum_object_size 950 MB

3. Make caching of BITS range requests more effective

For BITS range requests, by default the squid cache just forwards the request to the server. This results in very bad cache hits (The cache only gets hit for range requests if the file was requested without a range request and lives in the cache).

This setting tells the Squid cache to download the full file, regardless if there was a range request:

range_offset_limit 1 KB

Also, the Squid cache terminates downloading the full file if the client closes the connection without receiving the full file (which is almost always true for range requests).

This setting tells the Squid cache to keep downloading the file, regardless if the client got disconnected:

quick_abort_min -1 KB

For more information about these and more advanced settings the see http://www.squid-cache.org/

Microsoft Internet Security and Acceleration (ISA) Server/Forefront Threat Management Gateway (TMG)

ISA and TMG are Microsoft’s on premise caching products. Since Windows Intune clients leverage BITS to download content from internet, we want to also enable BITS in the caching rule. The ISA/TMG user interface only enables BITS for Microsoft update downloads. To meet Windows Intune download needs, we need to turn on caching for all BITS downloads. However this is not possible via the ISA/TMG user interface, so we wrote a script to create the rule programmatically with BITS caching enabled.

Another issue we saw in our testing was that “application/stream header” was being specified for some of the Intune content files. ISA and TMG consider content with this header as dynamic content and therefore don’t cache it by default. Therefore, this flag has to be explicitly set in the rule.

No specific time-to-live settings have to be set, since they are overridden by the content-age header. We set the content-age header to its maximum value – which is one year – when uploading the content.

Script to add Windows Intune Content Distribution Rule

Sub AddCacheRule()

' Define enumeration values.

Const fpcInclude = 0

Const fpcExclude = 1

Const fpcTimeInHours = 3

' Create the root object.

Dim root ' The FPCLib.FPC root object

Set root = CreateObject("FPC.Root")

'Declare the other objects needed.

Dim isaArray ' An FPCArray object

Dim rules ' An FPCCacheRules collection

Dim urlsets ' An FPCURLSets collection

Dim urlset ' An FPCURLSet object

Dim newRule ' An FPCCacheRule object

' Get references to the array object, the cache

' rules collection, and the URL sets collection.

Set isaArray = root.GetContainingArray()

Set rules = isaArray.Cache.CacheConfiguration.CacheRules

Set urlsets = isaArray.RuleElements.URLSets

On Error Resume Next

Set urlset = urlsets.Item("Windows Intune Content Distribution")

If Err.Number = 0 Then

urlsets.Remove "Windows Intune Content Distribution"

urlsets.Save

End If

Set urlset = urlsets.Add("Windows Intune Content Distribution")

urlset.Add "http://*.vo.msecnd.net/*"

urlsets.Save

' If a cache rule named "Windows Intune Caching" already exists, remove it.

On Error Resume Next

Set newRule = rules.Item("Windows Intune Caching")

If Err.Number = 0 Then

rules.Remove "Windows Intune Caching Rule"

rules.Save

End If

Set newRule = rules.Add("Windows Intune Caching")

' Set the descriptions of the new cache rules.

newRule.Description = "This rule caches content from Windows Intune Content Distribution."

' Add the URL set to the rule.

newRule.UrlSets.Add "Windows Intune Content Distribution", fpcInclude

' Enable caching BITS requests and dynamic content

newRule.CacheBITSContent = true

newRule.CacheDynamicContent = true

' Save the changes to the new cache rules.

rules.Save

WScript.Echo "Done creating Windows Intune Caching Rule."

End Sub

AddCacheRule

Setting up ISA/TMG caching for Windows Intune

After installing ISA on Windows Server 2003, run the above mentioned script to create the Windows Intune Caching rule for your ISA server system.

After the script completes, right-click on the navigation tree and click “refresh”. You should then be able to see the Windows Intune Caching rule under Configuration/Cache.

clip_image002

If you use Windows Server 2008 (and above) you have to install Forefront TMG. The same script can be applied on both versions. This is how the UI will look after a refresh:

clip_image004

Windows Intune Cache Rule

clip_image006

The script creates the rule “Windows Intune Cache Rule” on your system.

clip_image008

The rule is configured to cache Intune content

clip_image010

The rule defines that all the content will be cached. We store our software packages as .cab files, which are considered dynamic content. Therefore we have to explicitly also enable caching dynamic content.

clip_image012

The rule also sets no upper limit to the individual object size. Also it enables caching responses from Background Intelligent Transfer Service (BITS) requests.

After the rule has been created, caching has to be enabled on the system.

Enable caching on ISA

After this step you have to enable caching by reserving a part of your hard drive for the cached objects. For this, go to “Configuration/Cache” in the navigation tree. Go to the register tab “Cache Drives” and choose your preferred hard drive.

clip_image014

Reserve enough memory for the cache size, so your objects don’t get dropped after your cache gets filled up. 30 GB is a good number, depending though on the expected amount of packages you want to offer to your clients.

clip_image016

Finally you have to apply your changes to the system by clicking Apply and restarting the service:

clip_image018

Enable caching on Forefront TMG

Similar to ISA, Forefront TMG doesn’t cache by default. To enable caching, a cache file has to be created on a cache drive.

To do this, go to Web Access Rule/Configure (Related)/Web Caching in the navigation tree’s context menu. From there you get to the Cache Settings dialog.

clip_image020

You can define Cache drives on each hard drive available in the system. Reserve enough to be able to store the expected amount of data.

clip_image022

After you are done with your changes, make also sure you apply them and restart the service.

clip_image024

We hope this article will help you get the maximum benefit from your Windows Intune subscription, and we welcome your thoughts, comments and feedback.

Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment
  • What about using branch cache?

  • Branchcache cannot currently speed up the delivery of Windows Intune content.  There are a few technical challenges that keep Branchcache technology from working through the CDNs employed by Microsoft Update (which is the source for our update & end-point signature content).

    Branchcache integration is something that is on our radar for future releases.

  • Quick comment: This approach will also work for the new software distribution feature that is available in the recently announced Windows Intune beta.