provtest AllAboutHMC.xml : Hosted Exchange

Hosted Messaging and Collaboration Knowledge Base Articles

kip.ng — Sat, 26 Sep 2009 01:58:00 GMT

This page used to provide just the HMC related updates but now it has been updated to provide the specific product service packs, rollup updates, hotfixes that have been tested by the HMC product group. With this, I will no longer provide any quarterly...(read more)

Yes, you should use web-based distribution OAB.

kip.ng — Fri, 17 Jul 2009 00:40:00 GMT

HMC has been in the market for a while and as it grows, we are beginning to see many HMC providers are expanding their infrastructure and having more users and companies subscribing to their environment.

I also observed that the growth is especially obvious recently with the current economy situation where it makes so much sense for many companies to move their email, their SharePoint and corporate IM to the hosted model because of low startup cost and a low and predictable maintenance cost. Not to mention, it is much more feasible to do so nowadays with the current bandwidth and technology.

I noticed that the growth is particularly strong in the MAPI space. Customers are demanding better and richer Outlook email, communication and collaboration experience which HMC offers today rather than those simple POP3 and IMAP4 email technology.

Now, because of the growth in MAPI organizations, I feel that it is probably important for me to briefly talk about the need to move towards web-based distribution OAB instead of continuing scaling out the number of OAB Public Folder server.

We all know that in order to host MAPI clients, having Outlook to access the address list offline, we need to make sure each organization has an Offline Address Book available to be downloaded. Prior to Exchange Server 2007, the storage of the OAB is only in Public Folder Server. In HMC, it is recommended to dedicate Public Folder servers to host OAB and each OAB PF server shouldn’t have anything more than 1000 OABs. If you have more than that, you should either scale out by creating a new OAB PF server to host the new OABs or you can consider increasing the MaxPageSize in Active Directory. The latter, in my opinion is slightly dangerous as it may potentially generate performance impact to your whole Active Directory infrastructure.

So, everything sounds logical here, what’s the concern here? Well, the primary concern is that each Public Folder OAB created consumes a Replication ID (ReplID) in the Public Folder Hierarchy. The Replication ID has a limitation of approximately 32k. What does that mean? It means if you have more than 32k OABs in the environment, you may have problem with your OAB generation and download if they are in Public Folder.

It should also be noted that ReplIDs do not get cleaned up. Once it is created, it stays in the Public Folder Hierarchy until you refresh the Public Folder Hierarchy (which can be a rather tedious exercise) even if you delete the OAB. So, it means, if you create a new OAB and then delete it, it will still take up 1 count.

How do you know how many ReplID count you have now? Easy, go to your Exchange Server, look at the perfmon counter, MSExchangeIS Public\ReplID Count, you should be able to find your ReplID Count there.

I know some of you are probably getting worried knowing that there is a limit there. Good news is, there is a default quota limit being set as documented in this article, you should also take a look at this article, http://technet.microsoft.com/en-us/library/bb851493.aspx. So, chances are, you probably know about this ReplID by now if you have exceeded the limit.

You can definitely increase the default quota limit to allow things to get going as long as it is not over 32k. The long term solution is to move to Web-based distribution which doesn’t have such limitation.

Obviously the caveats of moving towards a pure Web-based distribution OAB is that the environment needs to be upgraded to HMC 4.5 (if you are already running HMC 4.0, moving to 4.5 is a piece of cake but if you are running HMC 3.5, then it needs a bit of planning but not end of the world) and it requires the client to be Outlook 2007 and above. Older version of Outlook will still work for all their functionalities except when it comes to downloading Offline Address Book (OAB) and also any feature that may require OAB (such as selecting contact when you are offline).

Consider the fact that the process to reset the Replication ID should you exceed the limit may require you to reset the whole Public Folder Hierarchy; I will start thinking

1. Switching all my clients to Outlook 2007 only (after all, there is no reason not to do that as Outlook 2007 provides better OOF functionalities and also better free busy and availability data).

2. Move to HMC 4.5 if you haven’t already done so.

3. Then, enable Web-Based Distribution OAB and use that only. You may need to change your control panel to set it to only create Web-based Distribution OAB or something.

It should be noted that this is only applicable to OAB creation; it is not applicable to the normal Public Folder creation.

So, there you go. If you are not using Web-based Distribution OAB today in your HMC environment and you are anticipating rapid growth, you should.

HMC 4.5 and Exchange 2007 SP1 - Part #6 - Conclusion (for now)

kip.ng — Fri, 09 Jan 2009 01:42:00 GMT

Introduction

So far, we have discussed what and how we have tweaked Exchange 2007 SP1 for HMC 4.5 so that it can be catered for multi-tenants. However, I should highlight that there are still more to be discussed, for example Unified Messaging, Free/Busy and Availability Services and OWA segmentation. Here, I am going to conclude this series but before doing so, there are a few topics that I like to briefly cover.

There is more?

Yes, there is more. :) If you read through the HMC 4.5 Solutions help file, you may find some of the following there but I thought I should highlight this again (after all, who read Help file, right?) and it may make the blog more complete to cover some of those topics here.

(1) Resource Management

It is important to note that HMC does more than just multi-tenant enabling Exchange 2007. There are large Centralized Management and Provisioning pieces that I didn't cover which I think it is important to know. HMC also provides the mechanism to allow the service provider to properly manage the Exchange resources and plans. For example, it gives the service provider the capability to define different User Plans, such as properly defining the type of client feature such as Quota, MAPI enable, owaSignaturesEnabled and etc. Service provider can also specify the amount of space allocated to a specific tenant and how those spaces can be distributed to different mail stores and etc.

In another word, what we have covered here in this blog is purely on the Exchange customization part, which is only small part of the whole HMC solutions.

(2) Hub Transport or Edge?

When Microsoft introduces Exchange 2007, Edge Transport server role has been greatly discussed because for the first time, Exchange comes with a server role that is not part of the Windows domain and that Microsoft recommends putting that server role in the perimeters network. The Edge Transport server role was designed specifically to deploy into a perimeter network.

However, there is lag time for both newly created and updated accepted domains and accepted users on the Edge server because of built in EdgeSync delay replication and cache delay (see KB Article 936159). This can cause unwanted NDR messages for hosting organizations. For this reason, in HMC 4.5, we offer an alternative deployment scenario where the Microsoft Exchange 2007 SP1 Hub Transport server can be reached directly through the Internet. The Internet-facing Hub Transport server will offer antivirus and anti- spam protection similar to the security provided by the Edge server role.

It should be noted that when we released HMC 4.0, Edge was part of the deployment walkthrough but in HMC 4.5, we recommended the alternative deployment scenario instead.

(3) Exchange 2007 Rollups and Updates

This is another question that always comes up. How do I know if I can apply the latest Exchange 2007 Rollup? For Exchange to work, generally it is not an issue unless the rollups introduces some permission changes or etc.

However, HMC solution as a whole has to be tested because HMC makes specific cmdlets calls all the time to create mailbox, to modify calendar settings, quotas and etc. If some of the rollup changes the cmdlet, it will result in HMC to fail. For that reason, we recommend users to wait till the rollup has been full tested by the HMC team before deploying them.

(4) Is ExBPA useful for HMC?

Yes, it is. There are still many rules in ExBPA that are applicable as you can see that the number of customization that made by HMC aren't many. In fact, many of the recommended practices still apply.

Also, it is important to note that HMC will not violate the Exchange Product Group support stand. For example, Exchange PG does not recommend putting CAS in DMZ and do not recommend to have firewall between CAS and Mailbox Server roles, HMC will not recommend to do that as well.

(5) Disaster Recovery

I think this qualifies a separate blog but it is important to note that when you are troubleshooting an Exchange issue in HMC environment, you can't just treat it like a normal Exchange environment. For example, you should not simply move mailbox from one server to another from the EMC or EMS because that will create inconsistencies between the HMC Resource Management and the Exchange and AD.

Conclusion

So, there you go. While I know there are much to be covered but I think these few blogs should give everyone a good idea what are the things that HMC has changed in Exchange 2007.

HMC 4.5 and Exchange 2007 SP1 - Part #5 - Autodiscover in the Multi-tenancy environment

kip.ng — Fri, 02 Jan 2009 01:38:00 GMT

Introduction

So, we have covered quite a fair bit by now. In part #1, we talked about how we can properly partition the Active Directory so that each company or tenant can coexist in the same environment but yet remain isolated and segregated. In part #2, we talked about how we have tweaked the address list so that each tenant will only see what they are supposed to see. In part #3, we further explore how Offline Address Book needs to be tweaked in to work in a multi-tenant environment and in part #4, we discussed about how Out-of-Office doesn't work completely out of the box in a hosted Exchange environment and what HMC has introduced to make it work better.

So, what's next?

Shared Infrastructure in HMC?

Now that we have most of the things well setup at the service provider side, the HMC environment is ready to serve clients. For Outlook Web Access (OWA), it is easy; the users only need a web browser, the OWA URL, username and password. Most people can manage that without a problem.

For Outlook client, unfortunately it is a little bit more complicated than we like. Outlook has come a long way and it still carries what I will call requirements for some 'legacy' parameter that cause the configuration of Outlook more complicated than desired.

For example, to configure Outlook to access an Exchange mailbox, you have to create an Outlook profile and in that Outlook profile, you have to specify the Exchange Server name (which most users do not know), you have to specify username and password. That's not all, because almost all the users in the HMC environment comes from Internet, the Exchange Server Name is not resolvable and that we have to provide the configuration of RPC over HTTP so that it can utilize the RPC Proxy to access the Exchange server from Internet.

If that doesn't sound complicated enough, Outlook 2007 and Exchange 2007 enhanced some of the features by introducing querying and accessing web services for some of the features such as Availability Services, Out-of-Office service and Unified Messaging service. OAB web distribution was also introduced to replace the old OAB distribution method through Public Folder. All these features mean Outlook 2007 now has to know more URLs or configurations to access those services.

Now, this is only for Outlook, what about my Windows Mobile Devices? What about Entourage clients? What about devices that can utilize ActiveSync other than Windows Mobile, such as iPhone?

While the above sounded complicated, we do have some good news for you.

Firstly, those shared infrastructure such as Exchange Web Services for Availability Services, Out-of-Office, Unified Messaging services DO NOT need any isolation or segregation. So no tweaking is required. Out of the box, those services enforce authentication (making sure that only authorized users can access the shared infrastructure) and it knows what security context to use when retrieving information or setting information (making sure that users can make changes or retrieve information ONLY if they have been granted the permission).
Secondly, for all the complications introduced by Outlook 2007, we now have a feature called Autodiscover that hide those complications and make things easier.

The not good news is this, Autodiscover doesn't just work out of the box in HMC environment, it has to be configured for each domain introduced by the tenant and it may not work in some situations. We will explore that in this blog.

(1) Outlook 2003

Outlook has come a long way. However, if you look at Outlook profile creation and configurations, we haven't changed much. It looks almost the same since the birth of Outlook.

The Good: Once you have configured the Outlook client, it is pretty self-contained; all it needs is to access the mailbox server, it will then know where to locate the free/busy, set the OOF, and find the public folder and OABs. It doesn't need to access any other URLs (such as Availability Services, Web Distribution OABs and etc.).

The Bad: The client has no logic built in to automatically detect the client configuration needed to access the mailbox. In addition, it also does not have access to some of the new features introduced in Exchange 2007 like Internal and External OOF, UM and enjoy the real-time free/busy capability from Availability Services.

Solutions for easier profile configurations: Install the RPC over HTTP Client Configuration Web site. This site will allow the user to authenticate and then download their PRF file that has been populated with all the information they need to configure their Outlook.

(2) Outlook 2007

Outlook 2007 becomes slightly more complicated. The fact that it introduces the concept of Exchange Web services being accessed by Outlook directly means that now Outlook 2007 needs to know how to get to those web services. The answer to all these is a feature called Autodiscover.

Autodiscover has changed things quite a fair bit and for those are interested to know more, you can read this White Paper: Exchange 2007 Autodiscover Service as I don't intend to spend much time talking about the working which has been well covered by this white paper.

In a corporate environment, Microsoft Office Outlook 2007 clients locate an Autodiscover service running on a Client Access Server by directly querying the Active Directory directory service and locating relevant Service Connection Points (SCPs). However, as we know most of the users in the HMC environment aren't part of the HMC Windows domain and also they are accessing the service from the Internet.

For Internet users, Outlook 2007 will attempt to locate and connect to an Autodiscover service based on the e-mail domain of the user. For example, for the user johnc@alpineskihouse.com, Outlook 2007 will automatically try to connect to the following URLs in turn:

https://alpineskihouse.com/autodiscover/autodiscover.xml
https://autodiscover.alpineskihouse.com/autodiscover/autodiscover.xml
http://autodiscover.alpineskihouse.com/autodiscover/autodiscover.xml
And if all those failed, it will try to use DNS SRV records. However, this is only if the Outlook Clients have been updated with the appropriate rollup. A new feature is available that enables Outlook 2007 to use DNS Service Location (SRV) records to locate the Exchange Autodiscover service http://support.microsoft.com/kb/940881

The logic to form the URLs above is built into Outlook 2007, we can't really change that and it has to go according to that sequence. Now, obviously that can present a problem in a HMC environment. Why?

The first 2 URLs are SSL. When SSL is required, we must make sure that a valid security certificate is installed for the Web site that matches the site name. So, this can be a problem for HMC environment because it will be madness and totally impractical to have 10,000 unique valid security certificate for all the 10,000 domains hosted by your environment. Not to mention you may need many IP addresses and website just to cater for that.

This is where the 3rd URL, which is only a HTTP redirection, comes in. It redirects the Outlook to another HTTPS Autodiscover URL. That's the reason why you see in the Deployment Guide, we have to create an additional Web site in IIS just to do this redirection.

There are some customers who attempted to use CNAME rather than the redirect, meaning just point autodiscover.alpineskihouse.com to autodiscover.consolidatedmessenger.com, this will fail because the security certificate cannot be properly validated. A redirection is a must.

In order for Outlook to go to http://autodiscover.alpineskihouse.com/autodiscover/autodiscover.xml and get redirected, you must do 2 things,

DNS - Configure autodiscover.alpineskihouse to point to the HTTP redirect website or IP address.
Autodiscover.xml - your HTTP redirect website must be able to respond to /autodiscover/autodiscover.xml and then redirect them to actual HMC autodiscover HTTPS website.

Of course, you can also do the SRV record. The only concerns for this is that the Outlook 2007 has to be properly updated before it can enjoy this new features and also we have seen many customers do not do the SRV record correctly. I kind of like the idea that you put both the HTTP redirect as well as the DNS SRV record there so that in the event where the HTTP redirect website is down, Outlook 2007 could continue to function well.

One thing you should also know is that Autodiscover is not only used during the profile setup, it is also being used each time you query any of the web services, such as availability services or OOF.

(3) Other Clients

What about other clients? As you can see, the deployment of the Autodiscover functions very much depends on the client. If the client does not support, say HTTP redirect or the SRV record, then Autodiscover in HMC environment will fail. One example is iPhone. IPhone does not support HTTP redirect or the SRV record. Hence, the auto account configuration may not work in those clients until they have updated the clients.

Troubleshooting tips for Autodiscover in Multi-tenancy environment

I find that most of the Autodiscover problems in HMC can be categorized into the following,

Deployment problem - the Autodiscover not being configured correctly. It is extremely important that you set this correctly such as what the externalurl, the HTTP redirect and etc. Do run through the configurations again.
Certificates problem - it is important to understand the usage of SSL certificate in Autodiscover and also other Exchange web services. Currently, the best reference is this, White Paper: Exchange 2007 Autodiscover Service
DNS configuration issue - because Outlook depends on DNS to locate the Autodiscover service, DNS needs to be configured correctly.
Firewall or Network Connectivity Issues - Publishing the SSL, the Autodiscover service and etc. can be a challenge too sometimes.

Troubleshooting Tip: How to enable Exchange 2007 Provider debug trace?

kip.ng — Tue, 30 Dec 2008 06:33:00 GMT

What Exchange 2007 cmdlets are being executed when I submit a mailbox creation request or a create public folder request or create contact request to the MPS engine? Those who has been working with HMC before know that most of the time, there will be a series of cmdlets being called. So, the next question would be, what arguments are being submitted together with the cmdlet to Exchange servers?

I get asked about this all the time. So, here is how you can enable the Exchange 2007 Provider debug trace so that you can review the event log and find out what cmdlets are being executed and what are the arguments being submitted to the server.

Follow the following steps to enable the Exchange 2007 Provider trace.

Open up C:\Program Files\Microsoft Hosting\Provisioning\Exchange2007Provider.dll.config
Find a key that resembles the following,

<add key="LogLevel" value ="1">

Change it to,

<add key="LogLevel" value ="5">

Restart the provisioning engine.

Submit your request again, the trace will be logged in the Application event log. You should see a series of events that look like the following,

Event Type: Information
Event Source: Exchange 2007 Provider
Event Category: None
Event ID: 0
Date:  12/1/2008
Time:  4:41:17 PM
User:  N/A
Computer: MPS01
Description:
Procedure='PSCommand.LogCmdletAndParams'
Cmdlet='Set-Mailbox'
Parameters:
Identity='CN=johnc@AlpineSkiHouse.com,OU=AlpineSkiHouse,OU=ConsolidatedMessenger,OU=Hosting,DC=fabrikam,DC=Com'
DomainController='AD01.fabrikam.com'
Alias ='johnc'
EmailAddresses='SMTP:johnc@alpineskihouse.com'
WindowsEmailAddress='johnc@alpineskihouse.com'
EmailAddressPolicyEnabled='False'
HiddenFromAddressListsEnabled='True'
IssueWarningQuota='9000'
MaxReceiveSize='10000'
MaxSendSize='10000'
OfflineAddressBook='CN=AlpineSkiHouse OAL,CN=Offline Address Lists,CN=Address Lists Container,CN=FABRIKAM,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=fabrikam,DC=com'
ProhibitSendQuota='10000'
ProhibitSendReceiveQuota='20000'
RecipientLimits='unlimited'

Please ensure you reverse the changes after that, if not your event log will be flooeded.

This is applicable to HMC 4.x only.

HMC 4.5 and Exchange 2007 SP1 - Part #4 Internal and External Out of Office

kip.ng — Fri, 26 Dec 2008 23:35:00 GMT

Internal and External OOF in HMC Environment

So, the 4th main customization that HMC introduces to Exchange 2007 is the OOF Transport Agent. What is OOF? Here is the text book definition, "The Out-of-Office (OOF) feature is commonly used by end-users to let other people know when they are not available to respond to e-mail." Now, I don't intend to go into the working of OOF in Exchange 2007 because I think there are better articles out there that talk about it.

Here, I want to briefly explain how Exchange 2007 OOF, out of the box isn't really working complete in a HMC environment and how we make it work.

What's not working, really?

Exchange 2007 introduces the concept of Internal and External OOF. This is something that a lot of our customers have been waiting for. Now, we all know Exchange 2007 wasn't packaged with multi-tenancy enabled out of the box. The problem is this, the identification of what is considered internal and what is external.

If a mail is sent from one user to another (even though they are of different tenants) within the same Exchange Organization, it is considered internal. In another word, a hoster may hold both company A and company B, however, Exchange 2007 believes that both company A and company B are internal, and hence company A will never use External OOF to company B. This breaks the concept of internal and external OOF notification as in Exchange 2007 in HMC environment.

For example,

Hoster A has a HMC environment that hosts both company A and company B.
John from company A set an internal and external OOF. The internal OOF is only intended for users in company A.
Alice from company B sends an e-mail to John from company A. Instead of getting John's external OOF, she got his internal OOF.

How do we make it work then?

To support Out of Office (OOF) feature for tenants among different organizations within the same Windows domain, HMC introduces a customized OOF transport agent on all Exchange Hub Transport servers. As highlighted over here (http://technet.microsoft.com/en-us/library/cc545931.aspx), the OOF agent includes two utilities:

Categorizer Override Agent: A transport agent to override the categorizer behavior on Hub Transport servers. This agent sits in Submission Queue and to be more specific, OnSubmittedMessage in the Exchange 2007 Transport pipeline.

SMTP Domain Cache Task: A scheduled task to generate cache files for categorizer override agent to filter the OOF messages. Based on the deployment document, the interval recommended is 60 minutes.

So, how do these utilities work? It is quite simple as a matter of fact. Let's talk about the Categorizer Override Agent first. When a mail comes into the Hub Transport server, it hits the OnSubmittedMessage; it fires up the Categorizer Override Agent. The agent checks to see if the mail is sent using an external OOF template, if yes, then it expands the recipients to check and bypass suppression of the external OOF message for external and inter-company recipients.

Next it checks to see if it is an internal OOF template, if it is, it will then expand the recipients and suppress the internal OOF message for external and inter-company recipients.

For performance reason, that's where the SMTP Domain cache task comes in. The task read from the MPS SQL server for all the domains that have been provisioned and generate a .cache file with all the domains hosted by the environment periodically.

This file is used by the categorizer override agent to find out if it is part of this Exchange organization. Having it cached as a local text file, is obviously a big performance gain compared to going to Active Directory to make the query every time especially if the environment has large number of hosted domains.

So, there you go, with the above, it overrides the logic of Internal and External OOF when dealing with different companies in the same Exchange organization.

The OOF agent was first introduced in HMC 4.5 and Microsoft has also back ported this agent to HMC 4.0 environment, (http://blogs.technet.com/provtest/archive/2008/11/19/oof-agent-is-now-available-in-hmc-4-0.aspx)

What are some of the usual problems that you may face while installing this OOF agent?

The installation of the OOF agent is clearly documented here. However, there are a few areas that you may encounter issues,

The OOF agent included in the deployment ISO is a 32-bit agent. Hence, the default path, instead of the documented, C:\Program Files\Microsoft\Exchange Server\TransportRoles\Agents\CategorizerOverrideAgent, it will be C:\Program Files (x86)\Microsoft\Exchange Server\TransportRoles\Agents\CategorizerOverrideAgent instead.

Another thing you should know is that, the Microsoft Exchange Transport service runs as a NT_AUTHORITY\NetworkService account. Hence, you need to make sure whichever directory that you are installing this agent in, the NT_AUTHORITY\NetworkService must have permission to it.

Also, it is my recommendation to Install HMC 4.5 Rollup 1 instead as this rollup includes a 64-bit binaries.

Another common mistake that most people may have is not setting the path correctly in SmtpDomainCacheTask.exe.config file. A simple mistake there may, like removing a '&' may result in failure with strange error.

More about OOF?

In case you want to know more about the working of OOF, here are 2 articles that you may want to read,

Exchange Server 2007 Out of Office (OOF)
http://msexchangeteam.com/archive/2006/10/06/429115.aspx

Legacy client and Out of Office (OOF) interoperability
http://msexchangeteam.com/archive/2007/04/04/437544.aspx

HMC 4.5 and Exchange 2007 SP1 - Part #3 - Offline Address Book Generations

kip.ng — Fri, 19 Dec 2008 01:37:00 GMT

Introduction

In the last part, we discussed about Address List segregation, which is an important concept to understand in Hosted Exchange environment. It is my belief that if you understand that clearly, you have understood 70% of the Hosted Exchange concept in HMC. The rest will be just easy. :)

In this part, we will continue to discuss about Offline Address Book (OAB) Generations in a HMC environment. I don't intend to talk about the OAB generation process in Exchange 2007 because I think Dave Goldman's web log has covered most of them (http://blogs.msdn.com/dgoldman/default.aspx). Now, for those who are interested to know more, I think the following is a good starting point (although the content is mainly Exchange 2003, the whole OAB generation process hasn't changed much)

Overview of the OABgen process
http://blogs.msdn.com/dgoldman/archive/2005/03/31/Overview-of-the-OABgen-process.aspx

My primary objective in this blog is to discuss about the customization of OAB components in Exchange 2007 SP1 done by HMC.

What has been introduced by HMC?

HMC does not change the OAB component in Exchange 2007 SP1 much. It does make a few changes in terms of how it works to make sure that things will run better in a service provider environment. So, let's take a closer look,

(1) OAB Generation Process

This is probably one of the parts in HMC that confuses messaging engineers the most. Typically, when an Exchange engineer troubleshoots the OAB generation (OABgen) process, they will look at the event log and also look at the OAB generation schedule. What they will see is that in a HMC environment is that the schedules for the all the OALs have been set to nothing, meaning Never Run.

Instead of using the normal OABgen schedule which will scan through all OABs and update each one of them (which in a Hosted environment, you could be talking about 30-50k OABs, that can take a long time), HMC changes things a little bit here by making the process to be triggered by changes instead of by time. Meaning, if there is no changes made to users or contacts in the company, no offline address book update will be called or triggered for that company.

To do that, HMC does the following,

No Schedule - When creating an Offline Address List object, HMC will set the schedule of that Offline Address List object to Never Run.
HmOABUpdate - HMC introduces a separate components called Hosted E-mail Offline Address Book (OAB) Update Batch Application (HmOABUpdate), installed on one of the Microsoft Provisioning Servers (MPS).

This application will query the MPS SQL Server (which keeps track of all the creations, deletions, modifications and changes made to the environment through HMC) and based on those changes; it will call the Update-OfflineAddressBook only on the specific company that has been changed.

The application will also generate a set of logs to log any success of failures in a log folder in the same directory of the application, by default, which is <drive>:\Program Files\Microsoft Hosting\Provisioning\ Exchange OAB Update\log folder.

Now, for each Offline Address List object, there are a couple of interesting items to look at,

The Address Lists - Generally it should be pointing to the company address list (NOT the company global address list).
PublicFolderDistributionEnabled - Depending on the Organization Plan (<features/ outlookClient>) you use to create the company/tenant, this may be True or False.
PublicFolderDatabase - This will determine which Exchange server will be performing the OAB generation. If the distribution is primarily on Public Folder, then it is important to make sure that the number of OAB hosted by the server does not exceed the MaxPageSize specified in the Active Directory LDAP Policies. If it does, you may have OAB generation issues. I will cover this in the troubleshooting section later.

If all things are well, the generation process should be just like any other corporate Exchange environment, except it is being triggered from another application.

(2) OAB Limit and Distribution

There are a few unique challenges that a Hosted Exchange environment like HMC may face that are generally not faced by a corporate environment.

High number of OABs, when will you see a corporate environment have more than 1000 OABs? Almost never. However, this could be common in a HMC environment. When Microsoft released HMC 4.0, it provided a recommendation that each Microsoft Exchange Server is limited to supporting a maximum of 1000 Exchange Server Offline Address Lists (http://technet.microsoft.com/en-us/library/cc539031.aspx).

In HMC 4.5, we have raised it to 10,000 per server instead. The Knowledge Base article KB 945629 updates the sizing guidance for Exchange Offline Address Book (OAB) servers. This ten-fold increase compared to Hosted Messaging and Collaboration version 4.0 allows service providers to save on hardware, software licensing, and maintenance cost. However, it should be noted that after raising the limit in an environment, service providers should monitor Active Directory performance carefully to ensure the change has no negative impact. Service providers should also ensure the existing hardware for the OAB servers are capable of handling the increase to ensure that customers are still able to quickly download updates to their OABs, even during peak working hours.

Dedicated OAB Distribution Server, depending on what version of Outlook Client you intend to support. If you intend to support anything below Outlook 2007, then you have to have Public Folder as the distribution point and the recommendation is that, you should have dedicated OAB servers and each server should not have more than 10,000 OABs in it (as mentioned above).

If you have only Outlook 2007 Clients, you may be using primarily just the web distribution method, it doesn't mean you have no issue too because it means the load of download will be shifted to the CAS server and you should also monitor the performance of the CAS to see if there is a need to have dedicated CAS just to be a OAB distribution point (Deployment Walkthrough will walk you through on how to setup a dedicated CAS for Web Distribution).

Other behaviour that you should take note, OAB generation process actually has some interesting implication that is not generally encountered by a corporate environment.

OAB public folder expiration. This actually trace back to Exchange Server 2003, Update to modify the OAB public folder expiration in Exchange Server 2003 (http://support.microsoft.com/kb/832761).

Because HMC moved the generation process out to an application and only touches the changed OAB, you may potentially have an OAB that may not have any change for a while. OABgen sets the default expiration time of 30 days. So what's going to happen is that, when it expires, it will remove the content in those folders and the user will not be able to download the OAB. The trick is to assign a value of 0 (zero) to the "OAL Folder Lifetime" registry entry, messages in the public folder do not expire. If you set this registry entry to a value other than zero, messages in the public folder expire in the number of days that correspond to the value that you assign. After you create this registry value, every time that OABgen runs, OABgen reads the registry entry and stamps it on the public folders that it uses.

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeSA\Parameters

Value name: OAL Folder Lifetime (days)
Value type: REG_DWORD
Value data: 0

OABScan task is not required. This article should explain it. CPU usage by the Exchange System Attendant service on offline address list servers in a hosting environment increases to 50 percent, http://support.microsoft.com/?id=834315. While the article is written specifically for Exchange 2003, it is also applicable to Exchange 2007 if Public Folder OAB distribution is heavily used.

(3) OAB Creation and Maintenance

Even though HMC puts in tweaks here and there, creation of the Offline Address List is nothing more than calling the New-OfflineAddressBook cmdlet. Here, I just want to highlight one thing,

SkipPublicFolderInitialization parameter. This parameter was introduced in Exchange 2007 SP1 and the appropriate namespace and procedures have been updated to allow for this parameter. If you are creating an OAB that uses public folder distribution, use the SkipPublicFolderInitialization parameter to skip the immediate creation of the OAB public folders. The OAB will not be available for download until the next site folder maintenance cycle has completed. You do not have to specify a value with the SkipPublicFolderInitialization parameter. If you do not skip this initialization, it may cause the task to pause while it contacts the responsible public folder server to create the necessary public folders. If the server is presently unreachable, or is otherwise costly to contact, the pause could be significant.

So, it is a decision of the service providers whether they want to immediately create the OAB root folder or they want to wait till the next day.

(4) How does user know which OAB to use?

Now that we understand the OAB generation process is triggered from a separate HMC application and also that there are some tweaks required to make things work more suitable for a service provider environment, the next thing we need to sort out is that how Exchange knows which of those 10,000 OABs should it send the user to retrieve. The answer lies in one of the user object Active Directory attributes that we mentioned in part #1,

msExchUseOAB: DN of the Tenant specific OAL object, this attribute directs Outlook to the appropriate OAB to download for this user.

This attribute is populated when the mailbox user is created in HMC.

(5) A Web-based Offline Address Book (OAB) Server

By far, the discussions have been mostly around the Public Folder distribution for OAB. In HMC 4.0, the only method of distribution of OAB is through Public Folder even though Exchange 2007 has the capacity to do both Public Folder and Web Distribution.

HMC 4.5 has lifted that limitation, allowing service providers to deploy web distribution for OAB. Of course, introduction of this new feature requires some additional configurations.

In order to enable a specific Offline Address List to be distributed through web, each OAB has to be enabled for Web-based distribution and that it must have at least one OAB distribution virtual directory specified. Hence, it means from the MPS perspective, when MPS needs to create a new OAB, there must be some logic to assign the appropriate CAS server as the OAB distribution point. To do that, HMC introduces the concept of CAS Pool.

The concept is quite straightforward. What we really need to do is to create a pool of CAS servers that will be used as distribution server. The steps are documented in the Deployment Walkthrough. We must first create the pool by using CreateOABCASPool.xml. Next, we have to add each OAB Web Distribution server into the pool using AddOABCAS.xml.

Then when create the mailbox, we have to specify the CAS Pool that will be used by the company or tenant. I like to note that this is during the process of create mailbox rather than create organization or email plan subscription. The reason is that the creation of the OAB really happens on the first MAPI user creation. That’s why the specification of the CAS pool is on creating mailbox.

Everything sounds simple enough so far, now what will happen when we decided to add a new OAB web distribution server to the CAS pool? You may realize that only the new company may get all the web distribution servers listed in the OAL whereas the OAL created in the past may not have that updated. This is where HMC 4.5 also introduces a new tool called UpdateOABProvision.exe.

The UpdateOABProvision.exe tool is used to update all existing Offline Address Books (OAB's) when the list of Web-based OAB CAS servers that exist in an OAB CAS Pool has changed.

For example, if a new server is added to the OAB CAS Pool, the orchestration logic will need to update all OAB's associated with that pool so that they have an accurate list of distribution points, including the new server. Similarly, if an OAB CAS server is removed from the pool, all OAB's associated with that pool should be updated so that the removed server is no longer set as a distribution point.

The command-line syntax for running the UpdateOABProvision.exe tool is:

UpdateOABProvision.EXE -OABCASPool -Errorlog –MaxError

I also like to highlight one thing here which is in HMC 4.5, it really uses just the default CAS Pool. Each CAS Pool can cater for up to 128 servers, which is quite a big number, consider the fact that there is no 1,000 OAB limit like what we have when we are using Public Folder OAB distribution.

Troubleshooting OAB download Issues in HMC

Problem downloading OABs is one of the most common issues raised to us for newly setup HMC environment. Most of the time, the problems can be categorized into 2 types,

(1) OAB Generation Issues

This is probably the first thing you want to find out, that is if the OAB generation is successful. I will generally start with the following,

Confirm that the company OAB root folder exists. If not, you need to find out,

If you are looking at the right Public Folder Server, if you are not, please try to locate the right Public Folder Server
If you can find the root folder on some of the OAB Public Folder Server but not on others, then you may need to find out if you have a Public Folder Replication issue.
If you can't find the root folder anywhere, you may want to find out how the OAB was created in the first place.

If you can find the company OAB root folder, confirm that the OAB Version folders are there and that the contents are there. You can do that by using MFCMAPI, OWA or the Exchange Management Console.
HmOABUpdate application isn't working because it was not setup properly. This could be due to permission not properly setup, problem connecting to the SQL server and etc. I would first review the HMOBUpdate log to find out what's going on to get started.
Actual OAB Generation Issues, this could be due to various reasons. I will try some of the steps in this article to start with, How to troubleshoot the OAB Generation process, http://blogs.msdn.com/dgoldman/archive/2005/07/16/How-to-troubleshoot-the-OAB-Generation-process.aspx. Turning up the Diagnostic logging will usually lead you to the source of the problem.

If you can confirm that the OAB is there (that is it has been properly generated) and that the hierarchy has been properly replicated, then you problem will be more on locating the OABs.

(2) Locating OABs

I will start with this, Outlook 2003 and 2007 clients receive error 0x8004010f when downloading the Offline Address Book
http://blogs.msdn.com/dgoldman/archive/2007/04/26/outlook-clients-receive-error-0x8004010f-when-downloading-the-offline-address-book.aspx

In addition to this, I also like to take note that Outlook 2007 uses Autodiscover to find out where the OAB is. If the Autodiscover isn't working well, Outlook 2007 may get the same common 0x8004010f error as well. So, if you can't download OAB using Outlook 2007 but you can do it through Outlook 2003, I would start looking at Autodiscover.

I will also cover how Autodiscover work in the HMC environment in part #5.

HMC 4.5 and Exchange 2007 SP1 - Part #2 - Address Lists Segregation

kip.ng — Wed, 10 Dec 2008 20:33:00 GMT

Introduction

In part #1, I spent a fair bit of time explaining how HMC was introduced into Exchange as part of the multi-tenant enablement process for service provider. I also spent some time discussing about how the Active Directory was partitioned and what the primary attributes that are essential to the Exchange 2007 SP1 multi-tenant enablement.

So far, there wasn't anything complicated about it. Now, I am going to move on to the 2nd part of the customizations, which is probably one of the more important ones, that is the Address List Segregation.

Let's look at what address lists we need to segregate and why. Address list is about contact information which is very important because it makes managing and finding your contacts (internal as well as external) easier and hence makes communication easier. There are a few places where you will use Outlook to locate the contact information,

Contacts folder - this folder is created within the mailbox itself if you have an Exchange account. Think of this folder like a normal Inbox or Calendar folder in your mailbox. This is a private folder. If you don't use a Microsoft Exchange Server e-mail account, Outlook stores your Contacts with the rest of your Outlook data in a Personal Folders file that has a PST file extension.

Personal Address Book (PAB) - Outlook also supports Personal Address Books (PAB). Like Contacts, a Personal Address Book can store a contact's name, address, e-mail address, phone, and other information. Outlook stores the Personal Address Book in a file with a PAB file extension. The PAB is completely separate from your other Outlook data stored in your PST file (or in an Exchange Server store). You can add more than one PAB to an Outlook profile.

Server-side Address Lists - in Exchange 2007 environment, you have 2 types of address lists,

Global Address List (GAL) - The GAL contains information for all email users, distribution groups, and Exchange resources. Outlook needs this to work and Outlook can only see one GAL at a time.

Other Address Lists - Sometimes, Exchange Server administrator might create other address lists to organize Exchange users by department, surname, or other criteria. These additional address lists show up under the All Address Lists group in the Show Names from the drop-down list in the Outlook Address Book.

From the above, I think it is pretty obvious that Contacts folder is a private folder, like the Inbox folder in a mailbox, is only accessible to the mailbox owner. PAB is really an offline file that store contact information. There is really nothing required to be done on these contact stores for a multi-tenant environment because those are 'not shared' and are private by default.

Server-side address lists however are slightly different. The address lists are shared and because of how we designed Exchange and Outlook, the address lists in Exchange, out of the box is accessible by everyone in the environment. That obviously is a problem for multi-tenancy because it means users in company A will be able to see the users in company B. So, let's take a closer look at what HMC will change in order to segregate the address lists so that every company will only see what they are supposed to see.

Before that, I think it is important to highlight that address list in Exchange is really nothing more than an Active Directory object containing query logic or filter that specify what mail enabled objects in Active Directory should be included in that list. It is not an actual list or a table or store that consists of all the contact information. It is a filter or I like to call it query logic. That's not all, what is more important to understand is that this query logic is only essential when creating the mailbox, but not so much for the actual Outlook query. As we walk through the blog, the statement above will become clearer to you and you will why HMC does what it does.

What is HMC trying to do?

So, let's look at what HMC intends to achieve here,

Global Address List - By default when you install Exchange 2007, it assumes that it is only meant for one company; hence it comes with a Default Global Address List which lists of all email users, distribution groups, and Exchange resources in the whole environment. That is not something a multi-tenant environment would like to have. Therefore, the primary objective here is NOT to have one GAL that will display everyone in the environment but different GALs for different companies.

Other Address List - If there is any other address list being used by a company in the environment, those address lists have to be only accessible and visible to users in that company only.

So, with the above objectives in mind, let's take a closer look.

Global Address List Segregation

Outlook needs GAL to work and as we know, Outlook can only see one GAL at a time. In order to provide each company their own Global Address List, we need to create different GAL objects for different companies, having specific query logic that scope to just users, distribution groups and resources in the company.

Once we have different GALs for different companies, we need to address the following,

(1) What's the limitation of Global Address List?

The default configuration of the global address list (GAL) class object allows only 1000 address lists. This presents a clear problem because we do expect most hosters to have at least more than that. To make sure that the environment can support more than 1000, you need to use the MakeGalLinked tool to extend this limit.

This is documented in the deployment walkthrough. I am not going to spend time to cover that as I am quite sure you can find other blogs cover this better.

(2) How to make sure that Outlook clients access the correct global address list if there is more than one global address list?

If you refer to the following article, http://support.microsoft.com/kb/312287, you will realize that Outlook displays the global address list that meets the following criteria:

The user has permissions to access this global address list.
The user is a member of this global address list.
This global address list is the largest of all of the other global address lists.

The above Outlook logic means that the Default Global Address List will always be used because all the users created will always be a member of this GAL and it is obviously the largest GAL of all because it consists of everyone in the environment.

Because of the above Outlook logic, we need to either make sure the user is not a member of the Default GAL or we make sure that the user has no permission to it. HMC has selected the latter. We deny almost everyone except Exchange Servers and Domain Admins permission to Open Address List for Default GAL. You may notice that this isn't very different as compared to the steps documented in White Paper: Configuring Virtual Organizations and Address List Segregation in Exchange 2007.

With the default GAL not accessible to all the users, we need to now create different GAL for different companies so that Outlook will actually work.

What HMC did is this, when you create a company and the user with Outlook plan, it will automatically create a company GAL. HMC also ensures that only users (AllUsers@<company>) in that company/tenant will have the Open Address Lists permission their GAL. I am not going to list down all the permissions that we set in there but it shouldn't stop you from finding out yourself, right? If you want to know the exact permissions being granted, just run Get-ADPermission cmdlet, for example,

Get-ADPermission "AlpineSkiHouse GAL" | FL

(3) What query logic or filter should we use for the company GAL?

In White Paper: Configuring Virtual Organizations and Address List Segregation in Exchange 2007 , the article uses customattribute1 and they make sure that mailboxes or resources created must have that attribute, if not, it will not be included in that GAL.

This is where HMC 4.5 does things slightly differently. If you perform a Get-GlobalAddressList on one of the GALs created by HMC, you will find the query logic/filter, which is the RecipientFilter and LdapRecipientFilter attributes are empty.

These attributes are essential if you want to make sure a newly created mailbox will be included as a member of this GAL. Only if the user is a member of the GAL, it can access the GAL and that it can show up in the GAL. These attributes are being read during the creation of the mailboxes and if the mailbox falls under the query logic, the appropriate membership will be stamped.

So, why does HMC not have any value on those attributes then? The answers are,

HMC mechanism - HMC has its own mechanism to retrieve the company GAL and perform the membership stamping. How? Remember the otherWellKnownObjects attribute I talked about in part #1?
It is faster. Just imagine if the environment hosts 20,000 domains in the environment, how long do you think it will take to walk through those query each time I create a mailbox? Having HMC to accurately retrieve the correct GAL will be more effective.

You may ask, what about when I query this from Outlook? Does it use those query logic? The answer is no. When you query from Outlook, it retrieves the objects based on Address List Membership (showInAddressBook) attribute in each objects. In short, the query logic in the GAL itself is only used for stamping the membership during mailbox creation.

There you go, Global Address List segregation, not really a difficult concept but can be confusing sometimes.

Address List Segregation

Address List Segregation isn't very different as compared to the GAL Segregation. However, there are a few things worth mentioning here.

Firstly, out of the box, Exchange 2007 installation comes with a few default address lists, All Contacts, All Groups, All Rooms, All Users, and Public Folders. Those will be removed during HMC 4.5 deployment. So, a clean HMC deployment will have an empty All Address Lists container.

Next, like the GAL segregation, the permission of the All Address Lists container has been properly locked down. Now, in theory, we don't really need to be part of any Address Lists for Outlook to work. Being part of the GAL is more essential because if you are not, you can't even create an Outlook profile for that user.

However, some processes in Exchange, such as the Offline Address Book generation prefers using Address List than Global Address List when generating the OAB (I will cover the OAB generation process in my next part). For those reasons, we need to make sure each company will also have company Address List where every mail enabled objects in the company will be part of that address list (yes, there is no difference compared to GAL).

Like the GAL membership stamping process, the Address List membership stamping process is the same, it is done by HMC mechanism. Hence you will see the query logic/filter for all the Address Lists created by HMC are also empty.

Wait the minute, what about Outlook Web Access?

So far, we have been discussing about the Outlook experience of the Global Address List and Address Lists. I think it is important to highlight that things work a little bit differently in Outlook Web Access.

There is some permission that applies only to specific clients. For example, Outlook Web Access does not incorporate user permission sets when doing searches. The permissions set on OUs do not prevent search results from including other segregated group's recipients, due to how OWA works. This is where attribute msExchQueryBaseDN comes in.

In order for us to control or restrict the search results to include only the members of the appropriate address list in Outlook Web Access, we must set the attribute msExchQueryBaseDN on each user object to the distinguishedname (DN) of the OU or an address list containing the correct group of users. In the case of HMC, when a mailbox user is created, HMC will also populate this attribute with the OU of the company/tenant.

What about LDAP? I have clients needing this access.

If I have a group of users using other clients and needs LDAP access so that they can query the users of in their company/tenant, what happen then? Well, the way I look at it is this, as much as possible, don't.

However if you have to do it, make sure that the client has to at least go through some form of VPN or something because you really don't want to put your Active Directory server out and exposed them to the Internet just like that.

Also, if you are having HMC 4.0, you should look at this article, Hosted users can see other hosted users if they can access the HMC Active Directory by using LDAP tools in Microsoft Solution for Hosted Messaging and Collaboration version 4.0 (http://support.microsoft.com/kb/943864/en-us), to properly lock it down before allow your users to use it.

What are some of the usual problems that you may face with Address Lists?

(1) Incorrect Address List membership

The most common issue that I have seen with address lists issues are membership problem. Based on the above, you kind of understand how some of the things could have gone wrong. The fact that HMC does not use the RecipientFilter and LdapRecipientFilter means that you can't run Update-GlobalAddressList or Update-AddressList because, That two cmdlets will rebuild (there is no update process in Exchange 2007, only rebuild) the address list, which means when it finds that it is empty, it may reset those accounts that currently are supposed to be in the address list.

When an account lost the GAL or AL membership, you may notice that you may not be able to create an Outlook profile for that user, and the user may have other problems too.

To resolve that, you can either use ADSIEdit to manually slot those membership back or you can use the hosted Email 2007::ModifyMailbox to reset this attribute to its correct values without having to know the address list DNs. This procedure resets the showInAddressBook attribute on each call.

Here is a sample, assuming that you supply the original values for <emailAddresses> and <alias> the procedure will only reset the showInAddressBook value without modifying any other attributes.

<request>
<data>
    <preferredDomainController>AD02.Fabrikam.Com</preferredDomainController>
    <user>LDAP://CN=User1,OU=AlpineSkiHouse,OU=ConsolidatedMessenger,OU=Hosting,DC=fabrikam,DC=com </user>
   <alias>User1</alias>
    <emailAddresses>
        <value>User1@alpineskihouse.com </value>
    </emailAddresses>
    </data>
    <procedure>
      <execute namespace="Hosted Email 2007" procedure="ModifyMailbox" impersonate="1" >
           <before source="data" destination="executeData" mode="merge" />
           <after source="executeData" destination="data" mode="merge" />
        </execute>
    </procedure>
</request>

This functionality was added to account for the fact that the Exchange Cmdlets like to reset showInAddressBook value during most mailbox operations.

HMC 4.5 and Exchange 2007 SP1 - Part #1 - Overview and Active Directory

kip.ng — Thu, 04 Dec 2008 23:28:00 GMT

Introduction

One of the frequently asked questions about Hosted Messaging & Collaboration (HMC) is this, "What do you do to Exchange to make HMC work?" So, here I will attempt to demystify this.

HMC started long way back but the concept has not deviated much. I want to note that this specific blog focuses mainly on HMC 4.5 and Exchange 2007 SP1 and it should be noted that things discussed here probably is applicable to this specific version only unless mentioned otherwise.

There are quite a few customizations that we set in Exchange 2007 for HMC and it is probably too much to do it in one blog, so, I will split this into a few parts,

Part #1: Overview and Active Directory
Part #2: Address Lists Segregation
Part #3: Offline Address Book generations
Part #4: Internal and External Out of Office
Part #5: Autodiscover in the Multi-tenancy environment
Part #6: Conclusion

What does HMC mean to Exchange 2007?

Let me start by asking this, what exactly is HMC? Yes, yes, there are quite a fair bit of things in HMC such as tools for flexible business modeling for service providers, resource management so that service providers can offer a broad range of services that go from basic e-mail up to higher-value services like providing additional storage and hosting personal domains and shared calendars and contacts. However, if I have to use one to two words to explain it, I will say HMC is about Multi-tenant enablement.

Yes, I think there is no mystery that Exchange 2003 and Exchange 2007 were not really packaged or designed to cater for multi-tenant. Not that it is not capable to do so, just that it wasn't designed with that as the focus.

What is multi-tenancy? Multitenancy refers to a principle in software architecture where a single instance of the software runs on a software-as-a-service (SaaS) vendor's servers, serving multiple client organizations (tenants).

Think of it like a condominium with hundreds of different individual units living in it, that's what multi-tenant is. Instead of having only one company (note: I tend to use company and tenant interchangeably) to live in one Exchange environment (1:1), multi-tenant enablement for Exchange 2007 means we want to enable the same Exchange environment to cater for multiple individual and separate companies and like a unit in the condominium, each company should have their own space and privacy (m:1)

Let me continue with the condominium analogy. Now that we know we need to build a condominium, we also need to figure out what we need to build each unit in the condominium. We need to make sure that each floor will be partitioned nicely and that the tenants will have their own kitchen, doors, parking space and enjoy their home and privacy. Likewise with Exchange 2007, we need to make sure that each tenant will enjoy the full messaging features and capabilities but yet do not compromise privacy and security.

So what do we need to multi-tenant enable Exchange 2007? Here is what needed to be done,

Active Directory Partitioning and Permission - think of it like partitioning each floor to different unit. Every company will have their own separate 'space' and the users in that company should only have access to what they are allowed to. I will discuss more on this later on how we can achieve that.

Address List Segregation - access to the contact information in each company should only be limited to users in that company. Other tenants shouldn't know have access to that information.

Outlook Experience - Outlook experience should not be compromised. The users in each company should enjoy all the features available in Outlook, such as Outlook Anywhere with cached mode and Offline Address Book, Calendaring and resource management, using Free/busy feature, Internal and External Out of Office feature in Exchange 2007, Autodiscover, Outlook rules, mailbox delegation, mailbox quota and etc. Some of the above mentioned features will required to be tweaked for multi-tenancy and some do not. I will walk through them as we go.

Shared Components - as much as we try to make sure everything is segregated, there will be some information and shared components here. Just like living in a condominium, everyone will share the same garage and elevator.

Now that we have listed them down, it doesn't look that bad, does it? In fact, once we walk through how we do this in HMC, you may realize that we aren't doing that much customization to Exchange at all. HMC, however, is more than just doing these customizations, of course. It is also a tool that allows you to better manage your resource and also allow you to automate a number of things, but that discussion will be for a separate blog. I will briefly touch on this at the end of this series.

Those who has been dealing with Exchange 2007 probably realize that we have a white paper that talks about the similar thing, which is White Paper: Configuring Virtual Organizations and Address List Segregation in Exchange 2007 . I strongly recommend anyone who wants to know more about the Exchange 2007 HMC customization to also read this white paper and try setting it up in your test environment for your own learning.

The above white paper explained how you can come up with your homebrew version of hosted solution by making a whole list of manual tweaking. It also provides good explanations on some of those customizations executed. HMC, for most parts are doing the same thing but there are a few things that we did quite differently and also we have a Deployment Tool to help you to make those customization instead of having you manually go through those permission setting of various OU containers, removal of address lists and etc.

Active Directory Partitioning and Permission

Firstly, let me start with how we do the Active Directory partitioning and permission in HMC for Exchange. The process of deploying a HMC is quite straightforward. Everything is done through the Provisioning Deployment Tool that comes with the HMC Solution ISO file.

(1) List Object Mode (dsHeuristics)

This is the first thing that the Deployment Tool will do. This is about controlling the object visibility. By default in Active Directory, we control an object using List Contents permissions on the parent object. The concept is quite simple; it means an object that will only be visible to a user if the user has been granted List Contents permissions on the parent object. When a user has List Contents permission on a parent node, he or she can see and browse all objects that are children of that node without any further selectivity.

Obviously this can be a problem in a Hosted Environment, for example, if in my Active Directory, I have the following structure,

Fabrikam.com (Windows Domain)

Hosting (Hosting Container)

ConsolidatedMessenger (Reseller)

Company1

Company2

User1@ConsolidatedMessenger
User2@ConsolidatedMessenger

If I grant User1@Company1 List Contents permission say on Compmany1 OU, I will also have to do it on all the parent OUs above it. Without that, it stops someone looking at what lives under the object or container. However, granting it lets whoever has the permission the ability to go through the content, including Company2 OU, which is obviously a bad thing. This is where "List Object" mode comes in.

In this special mode, even if the user has not explicitly been granted or explicitly been denied List Contents permission on the parent object, the object will still be visible to the user if the user is granted List Object permission on both the parent object and the object itself.

The availability and setting of this mode is very important to multi-tenant enablement and this is deployed as part of Core Platform in the Deployment Tool. Essentially, this form the basis of security control and Active Directory partitioning in HMC.

(2) Active Directory Partitioning - Hosting Container

Now, I want to briefly walk through the structure that HMC will be creating in Active Directory. Here are a few things that you should know,

HMC does not support deployment in a child domain. While it is technically possible, it hasn't be well tested and hence it is not a supported model.
By default in HMC, it will create an OU called Hosting right under the Domain. This can be changed by setting the procedure parameter in Hosting Platform - Initialize Default Services - Initialize Active Directory for Hosting in the Deployment Tool.

That is only the creation of the master hosting container. You will then need to follow the Deployment Walkthrough in the Help File to create the Reseller container and also the Tenant container. From the Exchange perspective, you only need to know the following,

HMC Resellers - Each Reseller is fully contained within its own Organizational Unit (OU) in Active Directory. The administrator from this OU will have permission to manage the HMC Tenants in it. Example in the Deployment Walkthrough is ConsolidatedMessenger.com. The Reseller OU will usually contain the following objects

HMC Tenants
HMC Users
A _Private OU //you should for the most part ignore the contents of this OU its contents are primarily in place to support provisioning actions.
An Admins@<Reseller> group

HMC Tenants - Each Tenant is fully contain within its own Organizational Unit (OU) in Active Directory, this Organizational Unit is a child of an HMC Reseller. Example in the Deployment Walkthrough is AlpineSkiHouse.com. The Tenant OU will usually contain the following objects,

HMC Users
A _Private OU //you should for the most part ignore the contents of this OU its contents are primarily in place to support provisioning actions.
An Admins@<TenantOrg> group // This group contains all of the Tenant Admin
Custom or service specific groups (such as FolderAdmins, FolderUsers for Public Folder service)

HMC Users - Users objects within AD are the security principal.
HMC Mailboxes - An HMC mailbox is a user object that has been Exchange Mailbox Enabled.

Here is an example of the Active Directory Hosting Container structure in HMC

Fabrikam.com (Domain)

Hosting (Hosting OU)

_Private
ConsolidatedMessenger.com (HMC Reseller)

_Private
admins@ConsolidatedMessenger.com (Admins group for HMC Reseller)
administrator@ConsolidatedMessenger.com (Administrator for the HMC Reseller)
AlpineSkiHouse.com (HMC Tenant)

_Private
admins@alpineskihouse.com (Admins group for HMC Tenant)
johnc@alpineskihouse.com (HMC Users/Mailboxes)

Obviously, I have simplified quite a few things here, for example, in this whole Active Directory Partitioning process; the HMC Deployment Tool will also create a number of groups (such as AllUsers, AllCustomers, AllCustomers, AdminsGroups and etc.) and properly assign the appropriate membership and permissions in different containers.

Permission Inheritance is not blocked specifically. For example, admins@ConsolidatedMessenger.com group is granted with Write, Create and Delete Child Objects to ConsolidatedMessenger.com OU. And that permission is also inherited downward to AlpineSkiHouse.com. This allows users in the admins@ConsolidatedMessenger.com group the capability to manage all the HMC Tenants in the ConsolidatedMessenger.com OU.

For the purpose of this blog, I think that's all you need to know about the Active Directory partitioning in HMC for Exchange. If not, this blog may end up to be a novel. :)

(3) User Logon

I want to touch a little bit about User Logon here. Most of you who are familiar with Active Directory will know what a samAccountName is. This is the logon name we carried forward from the old NT world. It has to be unique in a Windows domain. Even though we have revolved to allow longer logon name in later version of Windows, we still need to specify a samAccountName, a mandatory attribute. It usually follow the logon name. If my logon name is John, then my samAccountName is also John.

I can login to the environment using either username@domain or DOMAIN\<samAccountName>. This is not a big problem in a normal corporate environment. When it comes to a multi-tenant environment, this maybe a bit of a problem because we may have John in AlpineSkiHouse.com and we may also have John in Contoso.com.

In order to achieve the uniqueness, HMC recommends users to logon using the UPN as a start, which is username@domain such as johnc@alpineskihouse.com. You will notice in HMC allows that by modifying the upnSuffixes attribute of the Tenant OU. Now, all these things are done automatically when you create a HMC Tenant through the Microsoft Provisioning System.

With that, it solves the user logon uniqueness problem. However, it does not resolve the samAccountName uniqueness. To ensure that the samAccountName is also unique, HMC appends the domain of the Tenant to the logon name but because samAccountName can't have more than 20 characters, it will get truncated if it is more than 20 characters. For example,

Logon Name: johnc@alpineskihouse.com will have the samAccountName of johnc_AlpineSkiHouse
Logon Name: bob@alpineskihouse.com will have the samAccountName of bob_AlpineSkiHouse.c

So, hopefully that will help to explain the naming convention of the user object.

(4) Active Directory Attributes

HMC did not introduce any schema change to Active Directory. Sometimes, I wish it does because it would have made things little bit easier but it didn't. Instead, it uses what I would call the poor man's schema extension, the otherWellknownObjects attribute.

From the Exchange standpoint, this attribute is not used but for the provisioning process (MPS) to work properly, understanding this attribute is essential. And at times, I also think it is important to know this because it can tell you the Global Address List, Address List, Offline Address List that the specific HMC Tenant is using.

Here is a LDP dump of the otherWellKnownObjects attribute of one of the HMC Tenant.

12> otherWellKnownObjects: B:32:3841BDA6D81C4095B9BBB838808F5A55:CN=Services,CN=_Private,OU=AlpineSkiHouse,OU=ConsolidatedMessenger,OU=Hosting,DC=hmc45,DC=com; B:32:D22DFCC5B73645E99E16C9AD3D61F34F:CN=AlpineSkiHouse OAL,CN=Offline Address Lists,CN=Address Lists Container,CN=HMC45,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=hmc45,DC=com; B:32:9E444526CB6F4D5C9A59C9A84E26B627:CN=AlpineSkiHouse AL,CN=All Address Lists,CN=Address Lists Container,CN=HMC45,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=hmc45,DC=com; B:32:89FB25B7DF784FC198A493E2E8A0EE7E:CN=AlpineSkiHouse GAL,CN=All Global Address Lists,CN=Address Lists Container,CN=HMC45,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=hmc45,DC=com; B:32:7DEF010C6019A1458068D74AD1A3C1FA:CN=FolderUsers@AlpineSkiHouse,OU=AlpineSkiHouse,OU=ConsolidatedMessenger,OU=Hosting,DC=hmc45,DC=com; B:32:4619BE598BF441DB8C9DB0482E62E386:CN=_Private,OU=AlpineSkiHouse,OU=ConsolidatedMessenger,OU=Hosting,DC=hmc45,DC=com; B:32:EA755D448CE64157A20E82B7CCBE14B0:CN=customer,CN=WatOrgTypes,CN=_Private,OU=Hosting,DC=hmc45,DC=com; B:32:3B6FF4FA8AA248039AD8F9493A43B704:CN=CSRAdmins@AlpineSkiHouse,OU=AlpineSkiHouse,OU=ConsolidatedMessenger,OU=Hosting,DC=hmc45,DC=com; B:32:65F37ECB46704F0E9300E1FB48E1096E:CN=Admins@AlpineSkiHouse,OU=AlpineSkiHouse,OU=ConsolidatedMessenger,OU=Hosting,DC=hmc45,DC=com; B:32:58888CFC8F7F430C8183102CD5758D81:OU=ConsolidatedMessenger,OU=Hosting,DC=hmc45,DC=com; B:32:CC016CF08DEF4EA4A05C9C54B198785A:OU=AlpineSkiHouse,OU=ConsolidatedMessenger,OU=Hosting,DC=hmc45,DC=com; B:32:A276E3A170F0C24699770F593818501E:CN=FolderAdmins@AlpineSkiHouse,OU=AlpineSkiHouse,OU=ConsolidatedMessenger,OU=Hosting,DC=hmc45,DC=com;

As you can see from above, we have a unique GUID number to identify a specific category of relationship.

In addition, here are some of the Key attributes to Exchange:

homeMDB: DN of the users mailbox store - no customizations for HMC.
mailNickname: This is the HMC Users mailbox alias. - no customizations for HMC.
msExchMailboxFolderSet: Defines the folders available to the user through OWA, this attribute is controlled by the HMC User Plans.
msExchQueryBaseDN: Provides a baseDN from which LDAP Searches are performed, in HMC this attribute is set to the DN of the Tenant effectively limiting LDAP searches in OWA, ActiveSynch etc to the Tenant OU.
protocolSettings: Defines which protocols a user can use to access Exchange, this attribute is controlled by the HMC user Plans
showInAddressBook: Multi-valued attribute includes the DNs for the Tenant specific GAL and AL as well as the Default GAL, this attribute must be set correctly and the GALs ACL'd correctly in order for a user to logon to Outlook as well as for them to be restricted to the appropriate address list.
msExchUseOAB: DN of the Tenant specific OAL object, this attribute directs Outlook to the appropriate OAB to download for this user.

Let's briefly talk about the e-mail address stamping here too. In Exchange 2003, we have this process called Recipient Update Services. In Exchange 2007, this whole process has been embedded into the cmdlets and Email Address Policy is being used to determine the e-mail address of those users during the creation. It should be noted that, in HMC, right after it created an email user, a distribution group or an Exchange resource, HMC immediately performs another task to set the EmailAddressPolicyEnabled attribute to False. This means,

It does not rely on the Email Address Policy at all. That's why you will see that HMC will not create any Email Address Policy at all.

HMC will use its own mechanism to stamp the SMTP addresses.

So, that's it. It is straightforward so far, eh? We will talk more about other customizations introduced by HMC for Exchange in the new few parts.