The purpose of this article is to provide guidance for Forefront Protection 2010 for Exchange Server (FPE) capacity planning. This includes hardware information such as the number of processing cores and memory requirements. The guidance will provide information that will utilize the existing Forefront Security for Exchange Server (FSE) SP1 capacity planning tool for Exchange 2007 deployments and then provide enough detailed information to help with new Exchange 2010 deployments.
The goal is to provide guidance in-line with the documentation provided by Exchange. In all cases, this is the underlying data that was used to develop the guidance provided within this document.
A tool and additional updates to this document will be available in the future.
Note: All organizations are unique and they have requirements, policies, behaviors, and cultures that guide their requirements and inform hardware purchasing decisions. This document provides information about the additional load created by Forefront Protection 2010 for Exchange Server (FPE) on sample server environments and provides guidance to help with capacity planning decisions. This information should be combined with your experience, the Exchange capacity planning guidelines, and your general knowledge of your organization and IT landscape.
Reference Architectures and Server Roles
Enterprise Reference Architecture
There are two main reference architectures that are used for FPE deployments. The first is the Enterprise Reference Architecture show in Figure 1. This is a scalable unit that is intended to be deployed within a larger organization and is composed of an edge server, a hub server, multiple mailbox servers, an active directory server and a CAS server.
Figure 1. Enterprise Reference Architecture
Standard Reference Architecture
Figure 2 depicts the Standard Reference Architecture, which is intended to be a scalable unit, targeted at small to medium sized organizations. This architecture is composed of a dedicated edge server, one or more multi-role servers that encompass the hub server, mailbox server and CAS server roles and then a dedicated active directory server.
Figure 2. Standard Reference Architecture
Server Roles
Given the two reference architectures, the unique server roles are identified as follows:
FPE Edge Server
This is where the SMTP data stream comes into an organization and where message hygiene is performed. Message hygiene includes antispam as well as antivirus, antispyware and custom filtering. The FPE protection technology software resides on this server along with the Exchange Transport services. The messages that pass an enterprise’s message hygiene will be routed to the appropriate hub server for additional processing/routing. Messages that have undergone scanning as part of the message hygiene at the Edge do not have to be scanned again at the hub.
FPE Hub Server
This server accepts routing SMTP information from the FPE edge server, can perform additional message hygiene – based on configuration, and then routes the messages to their appropriate mailbox server or additional hub servers. The FPE protection technology software and the Exchange Transport services reside on this server.
FPE Mailbox Server
The FPE mailbox server accepts the incoming messages from the hub and can perform additional message hygiene – based on configuration. In addition, the mailbox server can perform scheduled scanning requests, and on-demand scanning requests. The FPE protection services in addition to the Exchange Mailbox services reside on this server.
FPE Multi-Role Server
This server contains the capabilities of the FPE hub server, FPE mailbox server, and Exchange CAS server in one server. Hence it provides the aggregate functionality.
FPE Server Performance Guidance for Exchange 2010 Deployments
Each of the FPE server roles are broken down and guidance is then provided.
FPE Edge Server
Factors Impacting Performance
On the FPE Edge Server it is critical to understand the load requirements for performing message hygiene. This can be broken down into two main components, SMTP traffic that results in messages delivered to a user in your organization and SMTP traffic that is detected as spam and/or malware. The FPE Edge Server can also be configured to send outgoing email from your internal users but the load from this activity is fairly minimal.
Another critical component for evaluating proper sizing on the FPE Edge Server is the configuration. There are several ways the FPE Edge Server can be configured, but only a few select ones are highlighted for sizing.
Note: Additional configuration information will be provided in an updated Capacity Planning Tool for FPE 11.0 RTM.
The three main areas of configuration are:
Engine Management
Engine management has two different components. The first is the Intelligent Engine Management (IEM) configuration which allows the user to have FPE automatically update and select the most appropriate engines to use based on a the default Engines and Performance setting of scanning with all available engines. This results in all engines scanning incoming mail when available. (Availability based on non-side-by-side engine updating.) This setting can be modified to manually select a certain number of engines.
The second component configures the number of engines used for each scan. Again, by default, the subset of available engines will be selected to perform message hygiene. The user can select a setting that ranges from always scanning with all the enabled/selected engines to only scanning with one engine
Enabling Premium Antispam
By default, the Premium Antispam functionality is enabled on all FPE Edge Servers. This can be disabled on the FPE Edge Server.
Number of Scanning Processes
By default, the number of scanning processes is set to 4. That means that there will be 4 processes utilized to perform malware message hygiene.
Performance Data
Figure 3 shows the CPU performance of a recommended FSE Edge configuration. This graph shows the relative performance of the previous product – FSE against the current version of the product, FPE 2010. There were samples taken and then a linear interpolation to see the projected CPU utilization against the incoming message rate.
Processors: 2x2 => 4 cores; Intel Core 2 Duo; 2.66 MHz
Memory: 4G RAM
Message Mix: Sample of Microsoft IT
Configuration: IEM (5 Engines), Scan with available engines => Bias: Prefer Certainty (No A/S)
Scanning Processes: 4
Figure 3. FPE 2010 CPU Utilization
Figure 4 shows the CPU performance relative to the number of scanning processes on a higher performance machine – 8 cores. This graph represents the scale up capacity using the default configuration verses a configuration that is tuned for a scanning process per core. The measurements represent the areas for normal throughput without the submission queue backing up.
Note: the submission queue starts to back up even though the CPU utilization never reaches more than an aggregate 23% of the server.
Processors: 2x4 => 8 cores; Dell R610; 2.93 GHz Xeon
Memory: 16G RAM
Message Mix: Sample of Microsoft IT
Configuration: IEM (5 Engines), Scan with available engines => Bias: Prefer Certainty (No A/S)
Scanning Processes: 4 and 8
Figure 4. Throughput Scalability on 8 Core Server
During these tests, the malware scanning detection was turned off with only the premium antispam capabilities enabled. The CPU utilization increased 3.3% on an incoming message rate of 100 messages per second. This was relative to the baseline numbers with FPE 2010 enabled on the Exchange Edge server w/o any message hygiene.
Memory measurements were also conducted on a variety of configurations and the recommendations are captured in http://technet.microsoft.com/en-us/library/cc482970.aspx. The overall memory footprint of the product is larger than that of its predecessor due to a new service. This increases the memory footprint approximately 650 MBs.
Capacity Planning Guidance
Processor
The recommendation is to use a 4 core server (not hyper-threaded) for the FPE Edge. Although the product does support 1 and 2 core server configurations, this will impact your overall throughput. You can also use more than 4 cores, however, in doing so, you should also change the configuration of the number of scanning processes to align with the number of cores on your server. The default configuration is 4 scanning processes and this will need to be increased to 8, for an 8 core server – for example.
Memory
The memory recommendation is outlined in http://technet.microsoft.com/en-us/library/cc482970.aspx. Overall, you should size your server configuration for at least 2GB over the recommended Exchange 2010 Edge Server guidance.
Scaling Up / Out
The default configuration, at about 75% utilization, will allow for approximately 45 incoming SMTP messages per second. This is on top of whatever incoming rate you experience for SPAM. The 45 incoming SMTP messages per second are those messages that are eventually routed to the Hub servers. This is with 5 scanning engines enabled.
If you desire to support a larger incoming rate, you can either (a) increase the number of cores on your server (b) decrease the number of engines used by each of the scanning processes, (c) or add another FPE Edge Server. The data on how this will impact scale will be available in a future technet article. However, there can be significant scale gain by going from 5 scanning engines to 4, or 3.
FPE Hub Server
The FPE Hub Server is a very similar configuration to that for the FPE Edge Server. There are two significant differences. First, the premium antispam functionality is not enabled by default on this particular role and, if an FPE Edge Server is forwarding mail to a FPE Hub Server, then message hygiene has already been accomplished except in cases where the configuration differs from the default. Hence, the message hygiene provided by the FPE Hub Server is for internally originating mail.
Factors Impacting Performance
The based paradigm that aligns with Exchange is a user that sends/receives a total of 50 messages per day that average 75 KBs/message. Out of the 50 messages per day, an estimated 20% are sent, and estimated 25% of the remaining 80% are received from the internet, and the remaining 75% of the 80% are received from internal recipients.
Using these calculations and the numbers from the FPE Edge Server, at peak times you will realize a message rate of ~8 messages per second to be scanned
Performance Data
The guidance from Exchange 2010 for a user with 50 messages per day is 1000 mailboxes per core. Using the default configuration, this is 4000 mailboxes that send/receive 50 messages per day on the recommended processor configuration. This results in a peak realization of approximately 8 messages per second that will require message hygiene from the Hub as it routes the messages.
Capacity Planning Guidance
Processor
The recommendation is to use a 4 core server (not hyper-threaded) for the FPE Hub Server. Although the product does support 1 and 2 core server configurations, this will impact your overall throughput and hub to mailbox fan-out. You can also use more than 4 cores, however, in doing so, you should also change the configuration of the number of scanning processes to align with the number of cores on your server. The default configuration is 4 scanning processes and this will need to be increased to 8, for an 8 core server – for example.
Memory
The memory recommendation is outlined in http://technet.microsoft.com/en-us/library/cc482970.aspx. Overall, you should size your server configuration for at least 2GB over the recommended Exchange 2010 Edge Server guidance.
Hub to Mailbox Fan-Out
Based on the recommended hardware, with 5 scanning engines and fully loaded mailbox servers being serviced by the hub, there is a 1:5 hub to mailbox relationship. In terms or cores, 1 core can support a downstream of 5000 users that send/receive 50 messages per day. You can scale this linearly, so if the representative users send/receive 100 messages per day, the hub can support 2500 per core; 200 sent/received per day yields 1250 downstream users per core.
Scaling Up / Out
If you desire to support a larger hub to mailbox ratio, you can either (a) increase the number of cores on your server, or (b) decrease the number of engines used by each of the scanning processes. The data on how this will impact scale will be available in a future technet article. However, there can be significant scale gain by going from 5 scanning engines to 4, or 3.
FPE Mailbox Server
Factors Impacting Performance
There are three factors that impact performance on the mailbox server: scanning of mailboxes via schedule and on-demand scan, the configuration of the server, and the load of the users that generate traffic that require message scanning.
The based paradigm that aligns with Exchange is a user that sends/receives a total of 50 messages per day that average 75 KBs/message. Out of the 50 messages per day, an estimated 20% are sent, and estimated 25% of the remaining 80% are received from the internet, and the remaining 75% of the 80% are received from internal recipients. Most of the activities that generate real-time scanning will be on-access scanning of the messages. This traffic can be modeled conservatively by assuming 20% of all access mails will be scanned out of two times the number of mails received on a daily basis. This is constant given the correlation between the Exchange server recommendations and the mail being received.
Two areas in configuration need special attention. First, the number of engines used in scanning on the FPE Mailbox Server role. As with all other roles, the number of engines will impact performance. Secondly, the user can enable rescanning of the messages – referred to as proactive scanning, if they enable scanning on signature updates.
Scheduled scanning is intended to run on a schedule and will scan all the mailboxes serviced by the Exchange Mailbox Server role. This should not be done during the day as it will significantly impact the overall performance of the mailbox server. On-demand scan is used to scan a small set of mailboxes at any given time. This will introduce a temporary performance impact on the mailbox server but is recoverable and dependent upon the number of mailboxes to be scanned and their size. Given the variety of the inputs for on-demand scan it cannot be adequately modeled to determine the overall impact on the mailbox server.
Performance Data
The guidance from Exchange 2010 for a user with 50 messages per second is 1000 mailboxes per core. Using the default configuration, this is 4000 mailboxes that send/receive 50 messages per second on the recommended processor configuration. Using the conservative approach of 20% of messages rescanned per user on twice the set of incoming messages, the impact of the scanning and overhead of the FPE protection software should not exceed 20%.
Capacity Planning Guidance
Processor
The recommendation is to use at least a 4 core server (not hyper-threaded) for the FPE Mailbox Server. Please consult the Exchange 2010 processor guidelines and make sure the minimum requirements are met. If using more than 4 cores, you must change the configuration of the number of scanning processes to align with the number of cores on your server. The default configuration is 4 scanning processes and this will need to be increased to 8, for an 8 core server – for example.
Memory
The memory recommendation is outlined in http://technet.microsoft.com/en-us/library/cc482970.aspx. Overall, you should size your server configuration for at least 2GB over the recommended Exchange 2010 Edge Server guidance.
Scaling Up / Out
In terms or cores, 1 core can support 20% less than the Exchange 2010 recommended users per core. For example, for a profile users receiving and sending 50 messages per day, 1000 users can be supported per core for an Exchange 2010 mailbox server. With FPE installed on the mailbox server, the value will be 20% lower, thus 800 users per core. Hence, you can up by sizing your mailbox server appropriately with more cores, or scale out by adding additional mailbox servers to support your profile number of users. Please refer to Microsoft Exchange Server Profile Analyzer for help in identifying our user requirements on the Mailbox server.
The 20% factor is with the default IEM and 5 engines. You can reduce this overall impact by changing the number of engines used to scan on the Mailbox server.
FPE Multi-Role Server
The FPE Multi-Role Server supports the FPE Hub functionality as well as the FPE Mailbox functionality. In addition, the CAS server may also be hosted on this machine. Hence, understanding the performance implications are a bit more complicated, especially when the FPE Hub server is really used as a message hygiene server w/o an associated FPE edge role somewhere in the organization.
Factors Impacting Performance
The main factor impacting performance is the overall responsibility of the server that directly relates to the expected load it must handle. In general, the Hub Server role is not meant to be the main message hygiene role in an organization and our assumptions are that it is not in this case.
In addition, the server has 8 total scanning processes associated with FPE protection technologies – 4 dedicated to the FPE Hub Server role and 4 dedicated to the FPE Mailbox Server role. This impacts the overall scanning throughput capabilities.
Performance Data
In lab measurements, the overall impact of the message scanning throughput is roughly a 25% impact. Exchange recommends no more than 500 users per core on a multi-server role configuration based on a user sending/receiving 50 messages per day. This leads to an aggregate impact of ~6.2 scanning messages per second on the server which correlates to 10% cpu impact. When factoring in the 25% scanning impact, this comes out to a 12.5% reduction in the number of users that can be supported on a multi-server role with FPE protection technologies installed.
Messages/Sec per 1000 users (Mailbox) : ((1000*40*2)/5) / (80*60*60/2) = 1.11
Messages/Sec per 1000 users (Hub) : (1000*(.75*.8*50)/(8*60*60/2)) = 2.083
Total impact => 3.19 messages/sec per 1000 users
Capacity Planning Guidance
Processor
The recommendation is to use at least a 4 core server (not hyper-threaded) for the FPE Multi-Role Server. Please consult the Exchange 2010 processor guidelines and make sure the minimum requirements are met. If using more than 4 cores, you must change the configuration of the number of scanning processes to align with the number of cores on your server. The default configuration is 4 scanning processes for Hub and 4 for Mailbox. Hence, you should increase the numbers proportionately on if using a server with more than 8 cores.
Memory
The memory recommendation is outlined in http://technet.microsoft.com/en-us/library/cc482970.aspx. Overall, you should size your server configuration for at least 3.5GB over the recommended Exchange 2010 Edge Server guidance.
Scaling Up / Out
The FPE Multi-Server role can support ~437 users per core that send/receive 50 messages per day. You can sale up by (a) increasing the number of cores on your multi-server role, (b) decreasing the numbers of engines used on the hub server role and mailbox server roles, or (c) migrating users to additional hardware.
FPE Server Performance Guidance for Exchange 2007 Deployments
Please reference the previous tool for FSE SP1: http://www.microsoft.com/downloads/details.aspx?FamilyID=522da65d-5263-4f5d-b929-8428a394b9af&displaylang=en
Please use this tool with the following guidance:
(1) If using the recommended hardware of a 4 core processor, you should be able to reduce CPU utilization by ~ 20% and increase the number of supported users by 25%.
(2) Please increase the amount of expected server memory by 500 MBs.
Frank Trujillo
Senior Program Manager