Faster, Higher, Stronger – UAG Performance

edgeaccessblog — Sun, 02 Aug 2009 21:41:00 GMT

Everybody’s always talking about performance, but what does it really mean? In this post I’m going to describe some of the behind-the-scenes work we’ve been doing on UAG performance.

Performance improvements are a major area of focus for this release. First and foremost, in UAG we switched to 64-bit architecture, thus overcoming the address space limitation that existed in the 32-bit architecture. In addition, we invested in reducing CPU and memory consumption in major scenarios, establishing and verifying system settings for maximum capacity, and more. For example, one of the recent improvements is utilizing the port scalability feature of Windows in RPC over HTTP(s) scenarios.

Outlook clients tend to use relatively large number of connections with the server (an average of 15-20, and up to 30 connections per outlook client). Using the port scalability feature enables us to utilize several IPs between the UAG and the backend server thus significantly enlarging the number of available ports. This will enable concurrent publishing for significantly more Outlook clients.

Test methodology and testing tools

So how does the UAG product team validate system performance? Our performance testing environment is comprised of an end-2-end Microsoft environment, including physical UAG servers, Exchange backend, Active Directory/Domain Controller; and load generators that simulate end-user machines. The idea is to simulate the customer environment as closely as possible, including Web Farm Load Balancing (WFLB) towards the backend, array/load balancer configuration on UAG, etc.

Some of the tools we use for UAG performance improvements and testing:

· Exchange Load Generator (LoadGen) is a load testing tool developed by Microsoft Exchange team. Customers can use it to test UAG (and of course Exchange) performance in their environment prior to deployment. LoadGen supports various mail protocols; we use it to simulate Outlook Anywhere publishing.

· Avalanche appliance by Spirent Communications was used to simulate Web Publishing (e.g., OWA).

Avalanche console

Performance test example

We've been running performance and stability testing for quite a long time. As an example, here are some results of one of stability tests we ran in preparation for the UAG Beta release.

Test environment:

· 2xUAG machines in array/Windows Network Load Balancing configuration

· Exchange 2007 backend with 2 Exchange CAS servers (with WFLB load balancing), HUB, mailbox store

Test scenario: load on each UAG server:

· 1K concurrent Outlook Web Access (OWA) users (54 Mbps throughput)

· 2K concurrent Outlook Anywhere/RPC over HTTP(s) users

Test length: 72 hours

Typical test environment

The observed test results were as following:

End-user response times (collected by Avalanche, for OWA test):

Action	Response time (sec)
Get Logon Page	0.187
Login	0.706
Inbox	0.905

UAG server resources utilization (collected from UAG server performance monitor):

UAG performance statistics example

What’s next?

This is just a first glance on the UAG performance story. We’re working on further optimizing and verifying UAG performance and scalability towards our release candidate (RC) and the release itself, enhancing the performance tests with additional scenarios (e.g., ActiveSync), and more. Stay tuned!

Olga Levina

Program Manager, UAG Product Group

Contributors: Asaf Kariv, Dima Stopel, Oleg Ananiev

Performance Degradation in eGap 3.6 after apply Windows Server 2003 SP2

edgeaccessblog — Thu, 04 Sep 2008 16:47:00 GMT

1. Scenario

This post is based on a support call where customer was reporting a performance degradation issue that was noticed after applying Windows Server 2003 SP2 on eGap 3.6 appliances. The server started to slow down until a point where it was no longer answering external requests.

2. Gathering Data

The initial step here was to get performance monitor data from the OS and IIS perspective to understand what was happening. Fortunately the issue was really clear when we looked at the object Process and the counter %Processor Time. We found that the process whlerrsrvd.exe was constantly growing in memory consumption and not releasing resources. At that point the workaround implemented to drop the memory utilization of this process was to restart the Whale Log Server Service (whlerrsrvd).

3. Resolution

To fix this problem we had to install eGap 3.6 SP1. This update introduced a number of changes which contain a fix for memory management in whlerrsrvd.

4. Additional Information

Besides this issue that can potentially happen when you install Windows Server 2003 SP2 in a eGap 3.6 that doesn’t have SP1, you need also to be aware of other issue and our official statement about eGap/IAG with Windows Server 2003 SP2. For more information access the article below:

You may experience network-related problems after you install Windows Server 2003 SP2 on a computer that has Whale Communications IAG 3.6 or IAG 2007 installed

http://support.microsoft.com/kb/955090/en-us

Author

Yuri Diogenes

Security Support Engineer – ISA/IAG Team

Microsoft – TX

Technical Reviewer

Dan Herzog

Security Support Engineer – IAG Team

Microsoft – WA