EXCHDXB

Understanding Basic Features in CCR

2012-05-20T13:45:00Z

Cluster continuous replication (CCR) is a high availability feature of Microsoft Exchange Server 2007 that combines the asynchronous log shipping and replay technology built into Exchange 2007 with the failover and management features provided by the Cluster service.

CCR is designed to provide high availability for Exchange 2007 Mailbox servers by providing a solution that:

Has no single point of failure.
Has no special hardware requirements.
Has no shared storage requirements.
Can be deployed in one or two datacenter configurations.
Can reduce full backup frequency, reduce total backed up data volume, and shorten the service level agreement (SLA) for recovery time from first failure.

CCR uses the database failure recovery functionality in Exchange 2007 to enable the continuous and asynchronous updating of a second copy of a database with the changes that have been made to the active copy of the database. During installation of the passive node in a CCR environment, each storage group and its database is copied from the active node to the passive node. This operation is called seeding, and it provides a baseline of the database for replication. After the initial seeding is performed, log copying and replay are performed continuously.

In a CCR environment, the replication capabilities are integrated with the Cluster service to deliver a high availability solution. In addition to providing data and service availability, CCR also provides for scheduled outages. When updates need to be installed or when maintenance needs to be performed, an administrator can move a clustered mailbox server (called an Exchange Virtual Server in previous versions of Exchange Server) manually to a passive node. After the move operation is complete, the administrator can then perform the needed maintenance.

Main Features in CCR:

Circular Logging with CCR
Transport Dumpster
File Share Witness
Replication Component
Replication Service Compenents
Transaction Log Replication And Replay
LLR and Auto Database Mount Dial

Circular Logging in CCR:

In CCR circular logging called continuous replication circular logging (CRCL), which is different from the ESE circular logging

Set-StorageGroup -Identity "First Storage Group" -CircularLoggingEnabled $true

In a CCR, LCR, or SCR environment, you should always use the following process to enable or disable circular logging:

Suspend-StorageGroupCopy
Enable or disable circular logging
Dismount and then mount the database
Resume-StorageGroupCopy

ESE circular logging is performed and managed by the Microsoft Exchange Information Store service, CRCL is performed and managed by the Microsoft Exchange Replication Service

Microsoft Exchange Replication Service manages CRCL so that log continuity is maintained, and logs are not deleted by the log deleter if they are still needed for replication. Therefore, enabling CRCL should not negatively affect replication.

Transport Dumpster:

Transport dumpster is a feature built into Exchange 2007 designed to minimize data loss by redelivering recently submitted messages back to the mailbox server after a lossy failure.

The Transport Dumpster is configured by default. You can view the transport dumpster settings by running get-TransportConfig

The transport dumpster is not enabled for SCR or (SCC).

The transport dumpster is enabled for CCR and (LCR).

In an LCR environment, the request for redelivery from all Hub Transport servers in the site occurs as part of the Restore-StorageGroupCopy task.

File share witness:

The file share witness uses a file share on a computer outside the cluster to act as a witness to the activities of the two nodes that are the cluster.
The file share for the file share witness can be hosted on any computer running Windows Server.
There is no requirement that the version of the Windows Server operating system hosting the file share match the operation system of the CCR environment. However, we recommend that you use a Hub Transport server in the Active Directory directory service
site containing the clustered mailbox server to host the file share, because this allows a messaging administrator to maintain control over the file share.
a single server can provide file shares for multiple CCR environments. However, each CCR environment should have its own dedicated folder and share on this server

Replication Compenents:

The two key services responsible for log generation, log shipping, and log replay activity are:

Microsoft Exchange Information Store service:

Responsible for servicing user and application requests, performing write ahead logging, and updating the database file via Extensible Storage Engine (ESE).
An operation occurs against the database (a client sends a new message), and the page that requires updating is read from the database file and placed into the ESE cache (assuming the page is not already in memory), while the log buffer is notified and it records the operation in memory.
The changes are recorded by the database engine but these changes are not immediately written to the database file on disk. Instead, these changes are held in the ESE cache and they are known as dirty pages because they have not been committed to the database file. The version store is used to keep track of these changes, thus ensuring that isolation and consistency are maintained.
As the database pages are changed, the log buffer is notified to commit the change, and the transaction is recorded in a transaction log file, which may or may not require closing the current Exx.log file and starting a new log generation. (Note that ESE is also responsible for closing a log file once it reaches its maximum file size (1 MB) and starting a new generation.)
Eventually the dirty database pages are written to the database file on disk.
The checkpoint is advanced.

Microsoft Exchange Replication Service

Responsible for log shipping and replay against the database copy.
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\MSExchange Repl\Diagnostics
When continuous replication is enabled, the Microsoft Exchange Replication Service is responsible for detecting when the current log file is closed by ESE, copying it, inspecting it, and replaying it into the database copy. This service is installed by default on all servers that have the Mailbox server role installed.
The executable for the Replication service is Microsoft.Exchange.Cluster.ReplayService.exe, which is located by default at <install path>\bin. The Replication service depends on the Microsoft Exchange Active Directory Topology Service.
The Replication service stores its diagnostic logging level setting in the Windows registry in the following location: HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\MSExchange Repl\Diagnostics

Replication Service Compenents:

v LogCopier

Responsible for copying closed log files from the source storage group to the target server that contains a copy of the storage group

v LogInspector

Responsible for verifying that the log files are valid. It checks the destination inspector directory on a regular basis. If a log file is found to be corrupted or unusable for replay, the Replication service will request a re-copy of the file.

v LogReplayer

Responsible for replaying inspected log files into the database copy.

v LogTruncater

Responsible for deleting log files that have been successfully replayed into the database copy.
When continuous replication is used, the LogTruncator only deletes log files that are not needed for recovery and replay.
Any log files on the active copy that have not been replicated and replayed into the database copy are not deleted by an online backup of the active copy.

Storage Group Structure in CCR:

v Share Name:

<Storage Group GUID>$

v Folder Path:

v Share Permissions:

<root domain>\Exchange Servers – Read

v Folder Permissions:

<rootdomain>\Exchange Servers – Read

<computer>\Administrators – Full Control

SYSTEM – Full Control

Transaction Log Replication And Replay:

Transaction log replication and replay is used to copy the changed data and update the passive copy's database.
The size of each generated log file is 1 MB
The transaction log file folder on the active node is shared using a standard Windows file share. (GUID) for the storage group is used for the share name, and ($) is added to the end of the share.
The Microsoft Exchange Replication service on the passive node connects to the share on active node using SMB protocol
The replication functionality copies the log files to the passive node as each log file is generated.
When the logs arrive at the passive node, they are checked for corruption and then replayed into the copy of the database that is stored on the passive node. The replay process makes the changes described in the change log to the passive node's database, which makes the passive node's database match the production database
The Microsoft Exchange Replication service on the passive node connects to the share on the active node and copies, or pulls, the log files using the Server Message Block (SMB) protocol. The passive node then verifies the log file and replays it into the copy of the database on the passive node.

LLR and Auto Database Mount Dial:

In the event that an unscheduled outage occurs that affects the active node, the passive node will bring the clustered mailbox server instance online and the Replication service on that node will attempt to copy the missing log files from the node that experienced the failure. If the copy process is successful (for example, because the server is online and the shares and necessary data are accessible), then the storage groups will mount and there will be zero data loss. If the copy process is unsuccessful, then the databases will be mounted based on the clustered mailbox server’s AutoDatabaseMountDial setting and how far behind it is in log replication per storage group. There are three possible values for the server setting AutoDatabaseMountDial.

Lossless Lossless is zero logs lost. When the attribute is set to Lossless, the system waits for the failed node to come back online before databases are mounted. Even then the failed system must return with all logs accessible and not corrupted. After the failure, the passive node is made active, and the Microsoft Exchange Information Store service is brought online. It checks to determine whether the databases can be mounted without any data loss. If possible, the databases are mounted. If they cannot be automatically mounted, the system periodically attempts to copy the logs. If the server returns with its logs intact, this attempt will eventually succeed, and the databases will mount. If the server returns without its logs intact, the remaining logs will not be available, and the affected databases will not mount automatically. In this event, administrative action is required to force the database to mount when logs are lost.

Good availability Good availability is three logs lost. Good availability provides fully automatic recovery when replication is operating normally and replicating logs at the rate they are being generated.

Best availability Best availability is six logs lost, which is the default setting. Best availability operates similarly to Good availability, but it allows automatic recovery when the replication experiences slightly more latency. Thus, the new active node might be slightly farther behind the state of the old active node after the failover, thereby increasing the likelihood that database divergence occurs, which requires a full reseed to correct.

Lost Log Resilience:

The order of write operations of Exchange data is always memory, log file, and then database file. LLR works by delaying writes to the database until the specified number of log generations have been created.

LLR = 10 (Exchange 2007 SP1, CCR)

LLR = 1 (Exchange 2007 SP1 , SCC or LCR or Standalone Mailbox)

LLR = AutoDatabaseMountDial + 1 (Exchange 2007 RTM)

In a lossy failure, there is at least one log file missing per storage group (Exx.log), but there also could be additional closed log files missing. This means that if the databases are brought online, data stored on NodeA (the failed node) is different than the data being generated on NodeB. This is referred to as divergence.

Divergence is when a storage group copy has information that is not in the active storage group. Divergence can be in the database or in the log files and can be caused by lossy failures,

Divergence is problematic because it means that data has been lost. In particular, database divergence is the worst case because it guarantees the need to reseed, which can be an expensive operation in terms of time and possibly bandwidth. Log file divergence also means data has been lost. However, log file divergence doesn’t necessarily cause database divergence because of LLR.

Remember that the order of write operations of Exchange data is always memory, log file, and then database file. LLR works by delaying writes to the database until the specified number of log generations have been created

LLR only runs on the active storage group copy. If you analyze the passive copy, you will see that its database is always up-to-date (in terms of the log files that exist on the passive node).

Troublshooting Autodiscover (Exchange 2007/2010)

2012-05-10T13:20:00Z

What is Autodiscover?

Exchange Autodiscover is a service which is run on Exchange Client Access Servers. It is one of the new features it included in exchange 2007.
The Autodiscover service makes it easier to configure Outlook 2007 or Outlook 2010 and some mobile phones. Autodiscover Service cannot be used with earlier versions of Outlook, including Outlook 2003. In earlier versions of Microsoft Exchange (Exchange 2003 SP2 or earlier) and Outlook (Outlook 2003 or earlier), you had to configure all user profiles manually to access Exchange.

The Autodiscover service uses a user's e-mail address and password to automatically configure a user's profile. Using the e-mail address, the Autodiscover service provides the following information to the client:

The user’s display name.
Separate connection settings for internal and external connectivity.
The location of the user’s Mailbox server.
The URLs for various Outlook features that manage functionality such as OOF, free/busy information, Unified Messaging, and the offline address book.
Outlook Anywhere server settings.

How the Autodiscover Service Works

When you install the Client Access server role on a computer running Exchange 2010/2007, a default virtual directory named Autodiscover is created under the default Web site in Internet Information Services (IIS). This virtual directory handles Autodiscover service requests from Outlook 2007 or Outlook 2010 clients and supported mobile phones under the following circumstances:

When a new user account is configured or updated.
When an Outlook client periodically checks for changes to the Exchange Web Services URLs.
When underlying network connection changes occur in your Exchange messaging environment.

Additionally, a new Active Directory object named the service connection point (SCP) is created on the server where you install the Client Access server role. And Autodiscover information is stored in it.

You can view this SCP using Active Directory Sites and Services after you have enabled the “View Services Node” option:

The SCP object is used by domain-connected clients to locate the Autodiscover service.
The SCP object contains two pieces of information:

- The serviceBindingInformation attribute.

- The keywords attribute

The serviceBindingInformation attribute has the Fully Qualified Domain Name (FQDN) of the Client Access server in the form of: https://cas01.contoso.com/autodiscover/autodiscover.xml, where cas01.contoso.com is the fully qualified domain name (FQDN) for the Client Access server.

The keywords attribute specifies the Active Directory sites to which this SCP record is associated. By default, this attribute specifies the Active Directory site to which the Client Access server belongs.

When a domain-connected client connects to the Active Directory directory service:
– The Exchange 2007 client authenticates to Active Directory and tries to locate the Autodiscover SCP objects that were created during Setup by using the user's credentials.

– In deployments that include multiple Client Access servers, an Autodiscover SCP record is created for each Client Access server. By using the user credentials, the Outlook 2007 client authenticates to Active Directory and searches for the autodiscover SCP objects.

– After the client obtains and enumerates the instances of the Autodiscover service, the client connects to the first Client Access server in the enumerated and sorted list and obtains the profile information in the form of XML data that is needed to connect to the user's mailbox and available Microsoft Exchange features.

An Outlook 2007/2010 client connects to the Autodiscover service as follows:

1- Outlook 2007/2010 sends a LDAP query to Active Directory looking for all available SCP objects.

2- Outlook 2007/2010 sorts and enumerates the returned results based on the client's Active Directory site by using the keyword attribute of the SCP record. One of two lists is created, an in-site list or an out-of-site list.

3. Outlook first tries to connect to each Autodiscover URL that it had previously generated from either an in-site list or an out-of-site list.

If that doesn't work, Outlook will try to connect to the predefined URLs (for example, https://autodiscover.contoso.com/autodiscover/autodiscover.xml) by using DNS.
If that fails also, Outlook will try the HTTP redirect method.
If that fails also, Outlook will try to use the SRV record lookup method.
If all lookup methods fail, Outlook will be unable to obtain Outlook Anywhere configuration and URL settings.

4. The Autodiscover service queries Active Directory to obtain the connection settings and URLs for the Exchange services that have been configured.

5. The Autodiscover service returns an HTTPS response with an XML file that includes the connection settings and URLs for the available Exchange services.

6. Outlook uses the appropriate configuration information and connection settings to connect to your Exchange messaging environment.

When Outlook 2007/2010 is started on a client that is not domain-connected, it first tries to locate the Autodiscover service by looking up the SCP object in Active Directory. Because the client is unable to contact Active Directory, it tries to locate the Autodiscover service by using Domain Name System (DNS). In this scenario, the client will determine right side of the user’s e-mail address, that is, contoso.com, and check DNS by using two predefined URLs. For example, if your SMTP domain is contoso.com, Outlook will try the following two URLs to try to connect to the Autodiscover service:

https ://contoso.com/autodiscover/autodiscover.xml
https ://autodiscover.contoso.com/autodiscover/autodiscover.xml

Another option related to DNS is made possible with an Outlook 2007 software update. When this software update is applied, Outlook 2007 clients will perform an additional check for a DNS SRV record to locate the Autodiscover service which does not require multiple Web sites and IP addresses.
For more information about this software update for Outlook 2007, see Microsoft Knowledge Base article 940881, A new feature is available that enables Outlook 2007 to use DNS Service Location (SRV) records to locate the Exchange Autodiscover service.

Problems with Autodiscover service or how it’s configured can causes issues such as:

Cannot view free/busy information.
Cannot download Offline Address Book (OAB) / receive error code: 0x8004010F.
The Out Of Office assistant is not working.
Prompt for a user name and password during the Autodiscover process.
Outlook anywhere stop working.

One of the most common issues I see has to do with certificates, so I will highlight few things about certificates.

Certificates are used to encrypt traffic between exchange servers and clients.

There are three things that need to be true in order for a certificate to be valid:

The name used to access the resource needs match the certificate exactly.
The Certificate date must be valid
The Certificate Authority which issued the certificate must be trusted by the client. (It needs to exist in the “Trusted Root Certificate Authorities)

Example: If I connect to OWA with mail.mydomain.com, then the certificate also needs to have mail.mydomain.com either on the subject or the subject alternative name field.

The components that are usually affected with certificate name mismatch are Autodiscover, Out of Office, Free Busy and Outlook Anywhere.

Reasons:

1- Not using a trusted certificate

Solution: use a 3rd party cert provider

2- The certificate name does not match the DNS name\s

Solution: create a new cert request containing all the names used to access the server. Minimum of

Autodiscover.domain.com

<ExternalName>.domain.com

<InternalName>.domain.local (if using for internal systems also)

3- Can’t resolve fully qualified domain names (FQDN)

Solution: make sure that the FQDNs for your external URLs as well as autodiscover have A records registered in DNS

Verify you can access the autodiscover XML file https://autodiscover.domain.com/autodiscover/autodiscover.xml

It should look like below

4- SCP Record does not contain the correct value.

Test from outlook:

- Hold CTRL and Click the outlook Icon in the system tray and select “Test Email Auto Configuration”

- Deselect the “Use Guessmart” and “Secure Guessmart Authentication” and click Test.

2. Check SCP value returned

- If you get info on the results tab then AutoDiscover is working

- If not, go to Log tab and look at the URL that is returned

- Test the URL (Type it into Internet explorer) if it is not working, change SCP to a valid URL, using Set-ClientAccessServer cmdlet.

- Set the SCP along with the internal URL: Set-ClientAccessServer CASServerName -AutoDiscoverServiceInternalUri https://mail.domain.local/Autodiscover/Autodiscover.xml

Troubleshooting Free/Busy Information for Outlook 2007/2010

The Availability service for Microsoft Exchange Server 2007/2010 provides calendar information for your users, which is known as free/busy information.
The Autodiscover service provides information for the Availability service by locating and providing the external and internal URLs for the Outlook 2007/2010 client.
There may be problem which involves a failure in either the Autodiscover service or the Availability service.
To determine whether the Autodiscover service is unable to provide information to clients by using Outlook 2007, you can enable outlook logging using the following steps:

In Outlook 2007, on the Tools menu, click Options, click the Other tab, and then click Advanced Options.
On the Advanced Options page, select Enable logging (troubleshooting), and then click OK.
Restart Outlook 2007, and then try to view free/busy information for another user.
In Microsoft Windows, click Start, click Run, and then type %temp%.
In Windows Explorer, open the olkdisc.log file and locate the files in the olkas directory.
The information that is contained in this directory can frequently provide information about which service is not functioning correctly.

If you are using outlook 2010, then do the below:

Launch Outlook 2010, Click File, Options, then Click Advanced.
Scroll down to Other, and check the box to enable troubleshooting logging.
Restart Outlook.

With logging enabled, a pop up will appear in the system tray reminding you that Outlook logging is enabled, and a header on the top of the outlook window

When Outlook launches a log file is generated and updated as the launch occurs. This file is located in your user’s TEMP folder. By default that is in the following location:

Windows XP: C:\Documents and Settings\<User Name>\Local Settings\Temp
Windows Vista and Windows 7: C:\Users\<User Name>\AppData\Local\Temp

You can also use Outlook 2007 to test the AutoConfiguration information that is provided by the Autodiscover service.

On the Test E-mail AutoConfiguration page, verify that the check box next to Use AutoDiscover is selected, and then click the Test button.

The following table provides a brief description of error codes that may occur when you test the AutoConfiguration for the Autodiscover service in Outlook 2007.

Exchange 2007 provides two ways for you to determine whether the Availability service is not functioning correctly:

1- Using the Event Log to Troubleshoot the Availability Service

2- Using the Test-OutlookWebServices Cmdlet to Troubleshoot the Availability Service as below:

Test-OutlookWebServices -id:user1@contoso.com -TargetAddress: user2@contoso.com

Troubleshooting Autodiscover service

Troubleshooting Autodiscover requires you to understand which part of autodiscover is not working. But generally, It is good practice to find out whether autodiscover working for internal clients or not. Internal clients who are belong to the same network of domain.
Following are some common procedures to check autodiscover configuration:

• Run Test-OutlookWebservices | fl
• Run Test-EmailAutoConfiguration on client to find how autodiscover is connecting and where exactly it's failing.
• Verify URL for autodiscover. Get-ClientAccessServer | fl
Check for AutodiscoverInternalServerUri attribute.
• You may also access the autodiscover url from IE and in response should get "600 invalid request".
• If autodiscover not working for external client verify authentication on Autodiscover virtual directory and if required you may recreate the virtual directory by running command:
Remove-AutodiscoverVirtualDirectory
New-AutodiscoverVirtualDirectory

Collecting Performance logs on Exchange server

2012-04-30T11:47:00Z

What is performance Monitor?

Performance Monitoring is one of the dark arts which administrators are rarely given the time to master. Performance Monitor on a Windows Server, in particular once Exchange Server is installed,
gives us the capability to collect enormous amounts of data; ensuring that any understanding of how our particular Exchange Server is performing is invariably lost beneath a sea of pretty but
unintelligible peaks and troughs. However it isn’t actually very difficult at all to configure Performance Monitor on your Exchange Server, collect some useful data and then interpret the results. It is
often then very easy to make changes to your servers which can have quite a dramatic effect on the performance of the server and the experience of Outlook clients alike.
There is now an enormous amount of information which can assist you in understanding what information to collect and how to interpret the results.

When to collect performance logs?

Performance monitor is one of the key tool you can use in many exchange issues such as,
Emails in the queue are moving slowly or Delay in receiving emails, specific process utilize a lot from memory, get a warning on Outlook “Outlook is retrieving data”, Backpressure issues, etc..

How to collect performance logs? And what tools you can use?

There are many tools that can help you to collect performance monitor data such as Experfwiz, Logman and Exchange Performance troubleshooter which is a build in tool under exchange management
console toolbox.

What is Experfwiz:

ExPerfWiz is a powershell based script to help automate the collection of performance data on Exchange 2007 and Exchange 2010 servers. It is no longer necessary to pick the correct counters
for the Exchange Server Roles that are installed as this script will automatically detect the roles installed and will add the appropriate counters for each role detected.

The default behavior of the script is to create a rolling blg file that will roll to a new log when the maximum size of the log has been reached up to a maximum of 8 hours. For Windows 2008 servers,
this is based on time as the -max parameter for logman.exe stops the data collection when the maximum log file size has been reached. There is logic in the script to prevent you from changing the
maximum size of the blg files on Windows 2008 servers.

How to use Experfwiz?

Download experfwiz from the link below (download tab)
http://archive.msdn.microsoft.com/ExPerfwiz

And then extract the file, and copy the extracted file and put it under scripts folder under c:\program files\Microsoft\exchange server\Scripts..

Note: I consider the binary files for exchange is installed on default location, which is under C as per Path above, so if it installed in different location, just past the file where the scripts folder locate.

Example:

Open the Exchange Management Shell

Type cd $exscripts

.\experfwiz.ps1 -duration 04:00:00 -interval 5 -filepath D:\Logs -full

NOTE 1: The example above will collect all counters on this server for four hours with interval of 5 seconds and save it under d:\logs.
NOTE 2: You can change the path where you have space instead of D:\.
NOTE 3: 04:00:00 means it will run for 4 hours and will stop automatically.
NOTE 4: Important, before running this script, you may do one or both of the following:

Set powershell's execution policy to RemoteSigned using (Set-ExecutionPolicy RemoteSigned)
Files downloaded from the internet using Internet Explorer are automatically blocked from running. Follow the below steps to Unblock this script from running:
Save the script file on your computer.
Click Start, click My Computer, and navigate to the saved script file.
Right-click the script file, and then click "Properties." Click "Unblock."

You can stop the tool anytime, by run .\experfwiz.ps1 –stop

What is Logman:

The LOGMAN.EXE utility can be used to create and manage Event Trace Session and Performance logs. Many functions of Performance Monitor are supported and
can be invoked using this command line utility. You can run it, if you want to run it against few specific counters.

Example: From a command prompt:

Logman.exe create counter PerfLog-Long -o "c:\perflogs\DCNAME.blg" -f bincirc -v mmddhhmm -max 250 -c "\LogicalDisk(*)\*" "\Memory\*" "\NTDS\*" "\Network Interface(*)\*" "\Paging File(*)\*" "\PhysicalDisk(*)\*"
"\Processor(*)\*" "\Process(*)\*" "\Redirector\*" "\Server\*" "\System\*" -si 00:00:30

You can change the highlighted path and intervals as required, add or remove any counters.

To start the log
Logman.exe start PerfLog-Long

To stop it after running for the amount of time needed:
Logman.exe stop PerfLog-Long

How to analyze the data and where to look?

1. You can use a built in Performance Monitor to open any .blg file by just double click on the file. Or from run, type Perfmon, And then check the threshold for counters based on predefined value as per

http://technet.microsoft.com/en-us/library/dd335215.aspx Exchange 2010
http://technet.microsoft.com/en-us/library/cc671175(v=exchg.80).aspx Exchange 2007

For example, if you think there is a disk issue, you can start to take a look into Avg. Disk sec/Read and write values.
The Default value for Avg. Disk sec/Read and write for “transaction logs and database files” drives is 20 MS and the maximum value should not be more than 50 MS.

2. Use PAL tool

The PAL (Performance Analysis of Logs) tool is a powerful tool that reads in a performance monitor counter log and analyzes it using known thresholds for most of the major Microsoft products
such as IIS, MOSS, SQL Server, BizTalk, Exchange, and Active Directory.

Steps on where to download and how to use it under:
http://pal.codeplex.com/

References:

http://technet.microsoft.com/en-us/library/cc722414.aspx
http://blogs.msdn.com/b/douggowans/archive/2007/01/02/a-very-quick-guide-to-monitoring-the-performance-of-your-exchange-server.aspx
http://503sayhelo.wordpress.com/2011/10/26/automating-collection-of-performance-monitor-counters-for-exchange-on-windows-2008/
http://blogs.technet.com/b/mikelag/archive/2009/01/23/getting-started-with-exchange-2007-performance-monitoring-and-windows-2008.aspx
http://technet.microsoft.com/en-us/library/bb201689(EXCHG.80).aspx
http://blogs.technet.com/b/askperf/archive/2008/05/13/two-minute-drill-logman-exe.aspx

Begin with Offline Address Book

2011-12-11T20:20:00Z

Brief Introduction –

An offline address book (OAB) is a copy of a collection of address lists that has been downloaded so that a Microsoft Outlook user can access
the information it contains while disconnected from the server. Microsoft Exchange generates the new OAB files, compresses the files, and then places the files on a local share.

An address list is a collection of recipient and other Active Directory objects. Each address list can contain one or more types
of objects (for example, users, contacts, groups, public folders, conferencing,and other resources). You can use address lists to organize recipients and resources, making it easier to find the recipients and resources you want.
Address lists are updated dynamically. Therefore, when new recipients are added to your organization, they're automatically added to the appropriate address lists.

As shown in the following figure, client applications, such
as Microsoft Outlook, display the available address lists that Exchange
provides.

Global address list as displayed in Microsoft Office Outlook 2007

So The offline address book is a snapshot of the Active Directory service information that is available in the global address list. Therefore,
some information is available in the global address list that is not available in the offline address book. The offline address book does not contain the following items that are available in the global address list:

Custom properties in Active Directory that an administrator has

added (for example, the Employee ID of each employee)

Organization hierarchy information
Group membership information

Outlook must be able to access the server to obtain this
information. Therefore, if you are in cached mode and if you are working online (that is, your connection status indicates“Connected”), Outlook uses both the offline address book and the global address list to provide a complete listing of user information. If you are in cached mode and if your connection status shows either “Disconnected” or “Offline,”you can see only the information that is available in the offline address book.

KB Articles Understanding Offline Address Books

http://technet.microsoft.com/en-us/library/bb232155.aspx

Administering the offline address book in Outlook 2003 and Outlook 2007

http://support.microsoft.com/kb/841273

Offline Address Book Generation:

Components Involved in Offline Address book Generation :

Each of the Exchange Server and Active Directory components involved in generating and maintaining offline address books is described as
follows:

Microsoft Exchange System Attendant

Responsible for generating the offline address book. The System Attendant calls Oabgen.dll. This is a MAPI application (referred to as
OABGen) that reads from Active Directory. On a server running Exchange Server 2003 SP2 or Later, the first time an offline address book is created, OABGen creates the OAB Version 2, OAB Version 3a, and OAB Version 4 folders.

Every time OABGen runs, it performs several tasks:

It creates the files for all offline address book versions.
It compresses the files.
It creates posts in the system folders for each version of the offline address that it is configured to generate.The offline address book files will be saved as attachments on the posts in the folders.It creates incremental offline address book files containing daily changes. The default setting is to generate an offline address book incremental file every morning at 05:00.

Microsoft Exchange Information Store

Responsible for storing the offline address book system folders listed directly under the OFFLINE ADDRESS BOOK. Each time a
new offline address book is created, the corresponding offline address book system folders are created, including the containers, such as
EX:/o=<Organization Name>/ou=<Administrative Group>, and /o=<OrganizationName>/cn=addrlists/cn=oabs/cn=<name of Offline Address List> during nightly online maintenance.

Directory servers

Active Directory servers are utilized in the offline address book generation. Offline address book configuration data is stored in the Configuration Naming Context partition in Active Directory, making it available to all domain controllers. In addition, global address list information is stored in global catalog servers.

Recipient Update Service (Exchange 2003 only)

although offline address book generation does not require the Recipient Update Service; address lists do require this component. The System Attendant calls the Rec4ipient Update Service, which in turn maintains the address list membership on recipients. If the address list is blank, the offline address book generation will fail.

KB Article Offline Address Book Technical Overview http://technet.microsoft.com/en/us/library/bb124614(EXCHG.65).aspx

OAB Generation process:

The Offline Address Book Generation (OABGEN.DLL) is a MAPI application that reads from the Active Directory and creates Offline
Address Lists. OABGen.dll is invoked by the Microsoft Exchange System Attendant service (MAD.exe).

When the OAB Generation process is invoked by the Microsoft Exchange System Attendant, it is responsible for connecting to the
active directory, read data from the active directory, compiling, sorting, creating and publishing the OAB files to the public folder information store.

These files contain a representation of mail enabled objects that are in the Active Directory. These files are downloaded by
Microsoft Outlook clients, and uncompressed into individual oab files.

Once these oab files have been uncompressed the Microsoft Outlook clients will have a representation of Active Directory
objects when they are not able to communicate with the Active Directory.

For E2k3 Offline address lists are stored in two
locations

1. The Active Directory

2. The Microsoft Exchange Public Folder Information Store.

For E2k7/E2k10 Offline address lists can be stored in three
locations

1. The Active Directory

2. The Microsoft Exchange Public Folder Information
Store.

3. File Structure \\MBX Server \ExchangeOAB\GUID

When an administrator creates an offline address list in the Active Directory via the ESM or EMC they are creating the placeholders objects in
the Active Directory. Once information store maintenance tasks run, the information store will create the respective folders

When an offline address list is generated the OAB Generation process, OABGen.dll will take that data and post it in to the OAB
system folders in the form of a message and attachments.

The versions of OAB that can be created by OABGen.dll
are as follows:

1. (OAB Version 2) ANSI Offline Address Book
This address book format that is used with both Microsoft Exchange 5.5 and
Microsoft Exchange 2000. Exchange 2003 also supports the ANSI Offline Address
Book. There are some known issues with the ANSI Offline Address Book,
especially with those Offline Address Books that have multiple sort locales.

2. (OAB Version 3a) Unicode Offline Address Book
This offline address book is new for Exchange 2003. This Offline Address Book
has additional information that helps Microsoft Outlook to reduce server Remote
Procedure Calls (RPC). Additionally, the Unicode Offline Address Book has new
features that are related to sorting rules for different language locales.
These features permit Outlook to use the correct sorting rule for the language
locale with the Offline Address Book.

3. (OAB Version 4) Unicode Offline Address
This offline address book started with Exchange 2003 with SP2 and later
versions . This Offline Address Book has additional functionality to help
reduce full OAB downloads.

Blog post : Overview of
the OABgen process

http://blogs.msdn.com/b/dgoldman/archive/2005/03/31/overview-of-the-oabgen-process.aspx

Offline Address Book Distribution Points

In previous versions of Microsoft Exchange the offline address books (OABs) were distributed by through the use public folders. Microsoft
Outlook clients can download the OAB from a system public folder by downloading the OAB message attachments. Because public folder servers will be optional component in Exchange 2007/2010 , a new distribution model has been introduced.
Below is a diagram that shows the interaction between the Active Directory, a Client Access Server, a Mailbox Server and the File Distribution Service.

As a replacement of using MAPI to download the offline address book files from a public folder information store, Outlook
clients will rely on Internet Information Services (IIS) and the BITS
(Background Intelligent Transfer Service) service for downloading the OAB
files.

Troubleshooting Offline Address Book:

When troubleshooting offline Address Book issues we should
isolate two things that are usually confused.

1. Offline Address Book
Distribution problems

2. Offline Address Book
Generation problems

First:

So let’s start with common issues that cause offline address book not to download
properly

1) if the distribution is through Public folders as in Exchange 2003 or if it is enabled in Exchange 2007/Exchange 2010.

a. Verify that OAB Container and files already exists on the Public folder database

b. Verify that the Public folder Database and offline address list properly assigned on the Mailbox store

2) if the distribution is Web Based as in
Exchange 2007/Exchange 2010

a. Verify that the Outlook Client is Able to retrieve OAB Url Properly through Autodiscover service .

This can easily be tested through
(Test-EmailAutoConfiguration) from Outlook Client .

b. Verify that you have the file structure and OAB files on the distribution point ( CAS Server )

c. Verify the Authentication methods on the OAB virtual directory

Second:

Now let’s focus now on how to troubleshoot
Offline Address Book Generation problems

Please review these two posts on troubleshooting OAB
Generation problems.

How to troubleshoot the OAB Generation process

http://blogs.msdn.com/b/dgoldman/archive/2005/07/16/how-to-troubleshoot-the-oab-generation-process.aspx

How to work with Exchange 2007 / Exchange 2010 OAB Event Logging

http://blogs.msdn.com/b/dgoldman/archive/2006/08/26/725860.aspx

Now let’s put all this together in one big scenario

Let’s assume that we have two sites London and Sao Paulo,
now how the OAB is generated and retrieved by users in each site.

For this scenario the OAB Generation server is one of the
E2k7/E2k10 MBX servers in London site.

For London site:

1) The OAB update process runs every day at 5:00 AM by default

OAB can be also be invoked by the Admin through the EMC or EMS through
the cmdlet Update-OfflineAddressList

2) The Generated OAB files ‘ll be placed in two locations on the MBX server since we have enabled the option
to distribute to both Public folders and Web-based

File System

Public folders

3) By Default MsExchange File Distribution service on CAS keeps polling every 480 minutes, once it finds a
new version of OAB files on the MBX it ‘ll copy it from MBX to CAS and keeps it ready for Outlook users to retrieve .

4) Now When an Outlook Client in London site start it‘ll issue a request to retrieve the OAB Url using Autodiscover .

5) Once the Url is retrieved,
the outlook‘ll connect to that Url and retrieve the OAB files using BITS .

For Sao Paulo Site :

1) Since the OAB Generation server is E2k7/E2k10,
Distribution mechanism selected should have
Public folders distribution in Addition to Web-Based to support Legacy
Exchange servers in the Environment .

2) After the OAB files are generated on the MBX server it is placed on the public folder database on that
server.

3) Public folder replication should be allowed between the servers to have the OAB files transferred to the
Legacy Exchange Public folder DB .

4) Now the Outlook Users in Sao Paulo site who have their Mailbox on Legacy Exchange server , should be
able to connect and retrieve the OAB files by connecting to their own PF stores