(This post was published on the original team blog in February 2010.)

Are you troubleshooting AD RMS and AD FS integration?  This post will help you get started but it is not a comprehensive look at the specific issues you may encounter. 

Which fields are case-sensitive when installing AD FS?

There are several case-sensitive fields when configuring AD FS to work with AD RMS.  Your organization's Federation Service URI value, located in the Trust Policy Properties box, must match the Federation Service URI value your partner configures in the Add Partner Wizard.  These two fields are shown in the following image:

   

The client computers in the external (FS-A) domain contain the following registry entry: HKEY_Local_Machine/Software/Microsoft/MSDRM/Federation/FederationHomeRealm whose value is the Federation Service URI of the FS-A.  This value also must match the URI value of the FS-A exactly.

Custom claim names are also case sensitive.  In AD FS and AD RMS integration, custom claims can be created for the ProxyAddresses attribute and you should ensure that the cases match when creating the custom claim, extracting the claim after creating the Active Directory account store, and matching the ProxyAddresses claims when configuring your AD FS partner.

Finally the application URLs for the AD RMS certification and licensing pipelines, which are configured in the Add Application Wizard, are case sensitive. 

What network port does AD FS use?

By default AD FS uses port 443 for all communications between the federation servers, federation proxy servers, and clients.

What is the flow of communication in AD FS and AD RMS?

This diagram shows the flow of communication for a user in an AD FS account forest to successfully consume protected content for the first time:

Additional Resources:

The AD FS team has developed the AD FS Diagnostic Tool to help troubleshoot AD FS.  The AD FS Diagnostic Tool can help you diagnose common configuration issues that occur when setting up an AD FS deployment, including certificate issues.  For more information or to download the tool visit the AD FS team blog: http://blogs.technet.com/adfs/archive/2007/11/01/adfs-diagnostic-tool.aspx.