Hi

Have you noticed the Office 365 directory sync database just getting larger and larger… If so try the following to clear the sync runs

1. Open the MIISClient.exe and click the Operations tab.

2. From the Actions menu, select "Clear Runs"

3. On the Clear Runs dialog

a. Uncheck the option "save runs before clearing them"

b. Click to select "Clear runs before: " radio button

c. Select a date that will clear runs starting with the oldest, not trying to clear more than about a week at a time.

IMPORTANT: This is pretty important as this operation will cause the transaction log file to grow temporarily. When trying to clear too many runs all at once, the transaction log file could grow a lot, possibly running out of disk space on the drive hosting the file.

4. Wait for it to finish the clear operation

a. This will be apparent when the list of runs has been modified to remove the runs just cleared.

5. Repeat as necessary

Written by Daniel Kenyon-Smith

Whilst working on a few O365 engagements over the last month or so I have seen various issues that have been caused by on-premise infrastructure/setup and which have not allowed the HCW (which was introduced in Exchange 2010 SP2) to complete successfully. These issues range from not correctly publishing autodiscover to not having the correct patches/updates installed, so I thought I’d share my experiences.

 

HCW Issues:-

I have seen issues with the HCW failing on Get-FederationInformation, this can be caused by various things such as:-

·         The customer did not publish the autodiscover endpoints correctly to allow O365 to make federated autodiscover lookups

·         There is pre-authentication set on the TMG/ISA or UAG. By connecting to https://autodiscover.company.com/autodiscover/autodiscover.xml for example as shown in Figure 1 - Autodiscover pre-auth authenticating - UAG and Figure 2 - Autodiscover pre-auth authenticating - TMG, you can see that UAG/TMG is prompting for pre-auth instead of the auth being directly on exchange, as shown in Figure 3 - No pre-authentication

Figure 1 - Autodiscover pre-auth authenticating - UAG

Figure 2 - Autodiscover pre-auth authenticating - TMG

Figure 3 - No pre-authentication

·         You can also run the command in powershell with the –verbose switch to get more detail, such as get-federationinformation……-verbose

·         If you are using TMG you can follow this article for configuring TMG with hybrid setups

·         You can also check autodiscover and other O365 endpoints (such as AD FS) have been published and are reachable from the internet using the ExRCA

 

Another issue I have seen is with certificates and as you run through the HCW, under Manage Hybrid Configuration – Mail Flow Security, you are asked to select the certificate that will be used for TLS mail flow between on-prem on and the cloud. If you are using a wildcard cert such as *.company.com then you will need to ensure you are running at least Exchange 2010 SP2 RU1. If you are not running SP2 RU1 then you most probably see an empty box when you get to the certificate page as shown in Figure 4 - HCW certificate blank below.

 

Figure 4 - HCW certificate blank

Of course you need to also ensure the certificate has been installed onto the exchange hybrid server(s) via the EMC or powershell, you can check using get-exchangecertificate from powershell.

 

Hybrid steps

Below are the detailed steps that run behind the scenes when you start the hybrid wizard:-

 

Create Federation Delegation and Organizational Relationships

Creates a new Delegation Federated Trust to 'Microsoft Federation Gateway'

Creates new 'On Premises to Exchange Online Organization Relationship'

Creates new 'Exchange Online to on premises Organization Relationship'

Enables MRSProxy on the Exchange 2010 Hybrid Servers

Configure the 'On Premises to Exchange Online Organization Relationship' to set:-

MailboxMoveEnabled 'True'

FreeBusyAccessEnabled 'True'

FreeBusyAccessLevel 'LimitedDetails' –

ArchiveAccessEnabled 'True'

MailTipsAccessEnabled 'True'

MailTipsAccessLevel 'All'

DeliveryReportEnabled  'True'

TargetOwaURL 'http://outlook.com/owa/<company.com>

Configure the 'Exchange Online to on premises Organization Relationship' to set:-

FreeBusyAccessEnabled 'True'

FreeBusyAccessLevel 'LimitedDetails'

MailTipsAccessEnabled 'True'

MailTipsAccessLevel 'All'

DeliveryReportEnabled 'True'

 

Create Send and Receive Connectors

HCW creates a new On Premise Send Connector -Name 'Outbound to Office 365' and below is the full output from the send connector created by the HCW:-

 

AddressSpaces                : {smtp:TenantName.mail.onmicrosoft.com;1}

AuthenticationCredential     :

Comment                      :

ConnectedDomains             : {}

ConnectionInactivityTimeOut  : 00:10:00

DNSRoutingEnabled            : True

DomainSecureEnabled          : False

Enabled                      : True

ErrorPolicies                : DowngradeAuthFailures

ForceHELO                    : False

Fqdn                         : Company.com

HomeMTA                      : Microsoft MTA

HomeMtaServerId              : ServerName

Identity                     : Outbound to Office 365

IgnoreSTARTTLS               : False

IsScopedConnector            : False

IsSmtpConnector              : True

LinkedReceiveConnector       :

MaxMessageSize               : 10 MB (You can increase from 10MB which is the default in 2007 and 2010 to 25MB which is set in the service)

Name                         : Outbound to Office 365

Port                         : 25

ProtocolLoggingLevel         : None

RequireOorg                  : False

RequireTLS                   : True

SmartHostAuthMechanism       : None

SmartHosts                   : {}

SmartHostsString             :

SmtpMaxMessagesPerConnection : 20

SourceIPAddress              : 0.0.0.0

SourceRoutingGroup           : Exchange Routing Group (DWBGZMFD01QNBJR)

SourceTransportServers       : {ServerName}

TlsAuthLevel                 : DomainValidation

TlsDomain                    : outlook.com

UseExternalDNSServersEnabled : False

 

 

HCW creates a new On Premise Receive Connector 'Inbound from Office 365' on each of the Hybrid HT servers and below is the full output from the receive connector created by the HCW:-

 

AuthMechanism                           : Tls, Integrated, BasicAuth, BasicAuthRequireTLS

Banner                                  :

BinaryMimeEnabled                       : True

Bindings                                : {LocalIP:25}

ChunkingEnabled                         : True

DefaultDomain                           :

DeliveryStatusNotificationEnabled       : True

EightBitMimeEnabled                     : True

BareLinefeedRejectionEnabled            : False

DomainSecureEnabled                     : False

EnhancedStatusCodesEnabled              : True

LongAddressesEnabled                    : False

OrarEnabled                             : False

SuppressXAnonymousTls                   : False

AdvertiseClientSettings                 : False

Fqdn                                    : Company.com

Comment                                 :

Enabled                                 : True

ConnectionTimeout                       : 00:10:00

ConnectionInactivityTimeout             : 00:05:00

MessageRateLimit                        : unlimited

MessageRateSource                       : IPAddress

MaxInboundConnection                    : 5000

MaxInboundConnectionPerSource           : 20

MaxInboundConnectionPercentagePerSource : 2

MaxHeaderSize                           : 64 KB (65,536 bytes)

MaxHopCount                             : 60

MaxLocalHopCount                        : 12

MaxLogonFailures                        : 3

MaxMessageSize                          : 28 MB (29,360,128 bytes)

MaxProtocolErrors                       : 5

MaxRecipientsPerMessage                 : 200

PermissionGroups                        : AnonymousUsers

PipeliningEnabled                       : True

ProtocolLoggingLevel                    : None

RemoteIPRanges                          : {RemoteIP Ranges}

RequireEHLODomain                       : False

RequireTLS                              : True

EnableAuthGSSAPI                        : False

ExtendedProtectionPolicy                : None

LiveCredentialEnabled                   : False

TlsDomainCapabilities                   : {outlook.com:AcceptOorgProtocol}

Server                                  : LocalServerName

SizeEnabled                             : Enabled

TarpitInterval                          : 00:00:05

MaxAcknowledgementDelay                 : 00:00:30

AdminDisplayName                        :

ExchangeVersion                         : 0.1 (8.0.535.0)

Name                                    : Inbound from Office 365

DistinguishedName                       : CN=Inbound from Office 365,CN=SMTP Receive Connectors,CN=Protocols,etc…

Identity                                : LocalServerName\Inbound from Office 365

Guid                                    : 9feef51e-1bd9-4aa4-9202-0614a1fcc0dd

ObjectCategory                          : company.com/Configuration/Schema/ms-Exch-Smtp-Receive-Connector

ObjectClass                             : {top, msExchSmtpReceiveConnector}

OriginatingServer                       : ServerName

IsValid                                 : True

 

There are also some connectors created in Forefront Online Protection for Exchange (FOPE). The connector created in FOPE will be called ‘Hybrid Mail Flow Inbound Connector’ and will have the following settings (assuming you are routing mail back on-premise):-

 

Description: The Hybrid Mail Flow inbound connector was created when hybrid mail flow was configured. This connector cannot be modified.
Sender Domains: *.*
Sender IP Addresses: 
Transport Layer Security (TLS) Settings: Forced TLS, and certificate matches specified domain

The recipient certificate matches: mail.company.com

Filtering Settings: Add these IP addresses to the safelist and only accept mail from these IP addresses for the domains specified above
IP Reputation Filtering: Disabled
Spam Filtering: Disabled
Policy Rules: Disabled

 

The other connector created in FOPE will be called ‘Hybrid Mail Flow Outbound Connector’ and will have the following settings:-

 

Description: The Hybrid Mail Flow outbound connector was created when hybrid mail flow was configured. This connector cannot be modified.
Recipient Domains: mail.company.com,*.*
Message Delivery Settings:

Fully Qualified Domain Name: mail.company.com

Transport Layer Security (TLS) Settings: Forced TLS, and certificate matches specified domain

The recipient certificate matches: mail.company.com

 

Create Remote Domains

Create new Remote Domain 'Hybrid Domain – company.com' set with:-

TrustedMailInbound 'True'

Create new Remote Domain 'Hybrid Domain - TenantName.mail.onmicrosoft.com' set with:-

 TrustedMailOutbound 'True'

TargetDeliveryDomain 'True'

AllowedOOFType 'InternalLegacy'

AutoReplyEnabled 'True'

AutoForwardEnabled 'True'

DeliveryReportEnabled 'True'

DisplaySenderName 'True'

NDREnabled 'True'

TNEFEnabled 'True'

Create new Remote Domain 'Hybrid Domain - mail.company.com '

DomainName 'mail.company.com'

TrustedMailInbound 'True

 

Setup Hybrid Mailflow

Set Hybrid Mailflow to:-

SecureMailEnabled 'True'

CentralizedTransportEnabled 'True'

OnPremisesFQDN 'mail.company.com '

CertificateSubject 'mail.company.com '

InboundIPs <>

OutboundDomains <>

 

 Set Address Policies

Update Default Recipient Policy to add <alias>@TenantName.mail.microsoft.com

Apply the updated Default Recipient Policy immediately

 

 

Written by Daniel Kenyon-Smith

I’ve been working with a customer who wanted to lock down access to O365 so users can access all the services from anywhere apart from browser based access, which can only be accessed from their corporate managed devices. Here’s a quick rundown of what we did…

We used the basic setup of 2 AD FS proxy servers in the DMZ and 2 internal federation servers to give us high availability. We had TMG sat in front of the AD FS proxy boxes, an example of the rule is shown below:-

Create a TMG Firewall Policy for AD FS

1. Log in to TMG as Domain Admin and launch Forefront TMG Management from the Start menu

2. Select Firewall Policy and select Publish Web Sites

3. Web publishing rule name: AD FS and click Next

4. Select Allow and click Next

5. Select Publish a single Web site or load balancer and click Next

6. Select Use SSL to connect to the published Web server or server farm and click Next

7. Internal site name: <your-federation-service-name

8. Check the checkbox to Use a computer name or IP address to connect to the published server

9. Computer name or IP address: <your-adfs-proxy> and click Next

10. Path: /* and click Next

11. Accept requests for: This domain name (type below):

12. Public name: <your-federation-service-name>

13. Path: /*

14. Click Next

15. Click New

16. Web listener name: AD FS Listener and click Next

17. Select Require SSL secured connections with clients and click Next

18. Check the checkbox next to External and click Next

19. Click Select Certificate, select your SSL cert certificate, click Select, and click Next

20. Select how clients will provide credentials to Forefront TMG: No Authentication and click Next and Next and Finish

21. Click Next, select No delegation, but client may authenticate directly, and click Next

22. Make sure All Users is displayed and click Next and Finish

23. Right-click the AD FS policy and select Configure HTTP

24. On the General tab, uncheck the checkbox for Verify normalization and click OK

25. Right-click the AD FS policy and select Properties

26. Select the Link Translation tab, uncheck the checkbox for Apply link translation to this rule, and click OK

27. Click Apply at the top of the TMG console

See here for full details/images - http://social.technet.microsoft.com/wiki/contents/articles/limit-access-to-office-365-via-adfs-with-threat-management-gateway.aspx and follow this link for ISA - http://blog.c7solutions.com/2011/06/publishing-adfs-through-isa-or-tmg.html and it’s also worth noting that Forefront UAG works only with the WS-Federation Passive protocol (e.g. browser access, such as the portal and OWA) not the active (e.g. Exchange ActiveSync, Outlook 2007, Outlook 2010, IMAP, POP, SMTP and Exchange Web Services) or MEX (e.g. Office subscriptions and Lync) see here for more details.

We also just used the Windows Internal Database (WID) as we don’t require more than 5 federation servers in the farm. The WID uses a primary/secondary model to replicate the database to all of the nodes in the federation farm. Because only one server is configured as the primary server, configuration changes can only be made on that server. The federation services on the secondary nodes are not able to modify the database. In this configuration, there are two features that cannot be implemented because they rely on database changes across all nodes. These two features are Token Replay Detection and SAML Artifact Resolution (for more details see here).

We followed the TechNet article ‘Limiting Access to Office 365 Services Based on the Location of the Client’ to create our own custom rule to lock down access to O365. The rule we created was ‘Allow all external access to Office 365, except Browser based access’ this is the rule we created:-

exists([Type == "http://schemas.microsoft.com/2012/01/requestcontext/claims/x-ms-proxy"]) &&

NOT exists([Type == "http://schemas.microsoft.com/2012/01/requestcontext/claims/x-ms-forwarded-client-ip",

Value=~"\b1\.2\.3\.4\b"]) &&

exists([Type == "http://schemas.microsoft.com/2012/01/requestcontext/claims/x-ms-endpoint-absolute-path", Value == "/adfs/ls/"])

=> issue(Type = "http://schemas.microsoft.com/authorization/claims/deny", Value = "True");

If we break down the rule we can see what it’s doing and it’s a little confusing as it uses double negatives! When the Value = True in the claims deny (last line in the rule) and the expression is set to NOT exist it allows that expression through!

So looking at our rule we are allowing ‘x-ms-forwarded-client-ip’ with a value IP address of 1.2.3.4 which is the public IP address from our internet facing TMG (so the NAT’d IP from internal to external).

We are not allowing ‘x-ms-proxy’ or ‘x-ms-endpoint-absolute-path’ the absolute path is what’s blocking the browser access. So when we try to access OWA for example from a none managed device (e.g. not tunneled through the corporate network using the internet proxy servers and appearing to come from 1.2.3.4) but instead coming from another IP, such as home broadband, we get access denied as below.

Access Denied

There was a problem accessing the site. Try to browse to the site again.

If the problem persists, contact the administrator of this site and provide the reference number to identify the problem.

You are not authorized to access this site. Contact your administrator for more information.

Reference number: 11111111-1111-1111-1111-111111111111

You can then lock down POP and IMAP using the following link if required

Oh and dont forget we now have the Client Access Policy Builder to automate the creation of these rules for the most common scenarios as listed in the TechNet article above.

Written by Daniel Kenyon-Smith

Please find a list of migration issues and resolutions that I captured during my last customers migration:-

Flood mitigation on Threat Management Gateway (TMG) - http://community.office365.com/en-us/w/exchange/office-365-move-mailbox-fails-with-transient-exception.aspx

 

 

Issue	Resolution
The operation couldn't be performed because object couldn't be found on 'xxx.prod.outlook.com'	The object has not been replicated to Office 365, check Dirsync error logs, dirsync email errors. Ensure these errors have all been resolved before the users are added to migration list and ensure the object has been sync’d to O365
A large item was encountered: Item (IPM.Note) Subject:"4600562666", Size: 42.07 MB (44,113,161 bytes), Folder:"Inbox A large item was encountered: Item (IPM.Note) Subject:"Here's My Card", Size: 42.43 MB (44,493,004 bytes), Folder:"Inbox" A large item was encountered: Item (IPM.Note) Subject:"", Size: 48.49 MB (50,848,177 bytes), Folder:"PRIVATE EMAILS"	PFDAV ExMerge - http://support.microsoft.com/kb/328202 Search-Mailbox – Needs to be a 2010 mailbox http://technet.microsoft.com/en-us/library/dd298173.aspx To be able to use the -DeleteContent Parameter you need to have "Mailbox Import Export" permissions. Without these permissions, the parameter -DeleteContent is not available for the Search-Mailbox Command. Instructions: 1. Create a new Security Group 2. Enter the following command in Exchange Management Shell (replace the Security Group name accordingly): New-ManagementRoleAssignment -Name "Import Export Support" -SecurityGroup ImportExport -Role "Mailbox Import Export" *See below for the command to skip large items
Domain xxx.co.uk is not an accepted domain for your organization	Remove all SMTP addresses from mailboxes that are not registered in O365 (or register the domain names)
Target user already has a primary mailbox. Mailbox already migrated	User has already been migrated, ensue no users are duplicated
The target mail user doesn't have an SMTP address that matches the target delivery domain 'Service.company.com'	Add the target domain SMTP (Service.company.com) to all mailboxes before migration (and ensure they have been sync’d to O365)
Warning: Failed to clean up the source mailbox after the move. Error details: MapiExceptionNotFound: Unable to delete mailbox. (hr=0x8004010f, ec=-2147221233)	The issue occurs when migrated accounts cannot clear homemdb, homemta, msexchhomeservername and msexchguid attributes from the source server and set the TargetAddress on the source side which it is supposed to do after mailbox move completes (flipping of the attributes starts after 95% completion of the move). A script can be created or apply this hotfix - http://support.microsoft.com/kb/940012 Pretty much the same issue as large item count http://support.microsoft.com/kb/2584294
Error: This mailbox exceeded the maximum number of large items that were specified for this request. --> MapiExceptionMaxSubmissionExceeded: IExchangeFastTransferEx.TransferBuffer failed (hr=0x80004005, ec=1242)	Pretty much the same issue as large item count http://support.microsoft

 

*new-MoveRequest -identity $_.UserPrincipalName -Remote -RemoteHostName 'mail.company.com' -RemoteCredential $cred -TargetDeliveryDomain 'service.company.com' -BadItemLimit 50 -LargeItemLimit 40

 

If you receive the following error when trying to migrate to O365, the you will need to check that your EWS endpoints are published correctly. 

The call to 'https://mail.company.com/EWS/mrsproxy.svc' timed out. Error details: The request channel timed out while waiting for a reply after 00:00:00.Increase the timeout value passed to the call to Request or increase the SendTimeout value on the Binding. The time allotted to this operation may have been aportion of a longer timeout. --> The HTTP request to https://mail.company.com/EWS/mrsproxy.svc' has exceeded the allotted timeout
of 00:00:00.0010000. The time allotted to this operation may have been a portion of a longer timeout.--> The operation has timed out

Without stating the obvious, there is a timeout issue! I have seen before if you haven’t published EWS correctly to the internet. Ensure when you connect to https://mail.company.com/EWS/mrsproxy.svc that you get prompted for authentication from the exchange server and there is no pre-authentication at TMG/UAG for example. Once EWS is published to the internet correctly you should be able to start migrations to O365!

 

Written by Daniel Kenyon-Smith

 

Whilst working with a customer here in the UK on their O365 project, they asked about using Forefront Online Protection for Exchange (FOPE) (the version that comes with O365) how to setup rules to block credit cards details. There are various rues you can configure, Basic or RegEx, more details can be found here.

We setup up the following rules also using this site:-

The credit card format is like this 1234-1234-1234-1234, so we set up rues to cover various scenarios, such as:-

Credit Card Format	FOPE Rule
1234-1234-1234-1234	\d\d\d\d\-\d\d\d\d\-\d\d\d\d\-\d\d\d\d
1234 1234 1234 1234	\d\d\d\d\s\d\d\d\d\s\d\d\d\d\s\d\d\d\d
1234123412341234	\d\d\d\d\d\d\d\d\d\d\d\d\d\d\d\d

The top 2 rules for example can be added into a single rule, as shown below and in the screenshot

\d\d\d\d\s\d\d\d\d\s\d\d\d\d\s\d\d\d\d|\d\d\d\d\-\d\d\d\d\-\d\d\d\d\-\d\d\d\d

To configure and create new rules, follow the steps below:-

1. Login to FOPE admin center - https://Admin.messaging.microsoft.com (also as a side note, this is the site to access quarantine messages - https://Admin.messaging.microsoft.com, but also note that global admins can’t manage quarantine by default - http://support.microsoft.com/kb/2587698)

2. Select Administration

3. Policy Rules

4. New Policy Rule, on the right hand side under Tasks

5. Then configure the rule accordingly, so in this example we using body to match the rule

Since we are in the UK we are not worried about social security numbers, but we did add in American Express (Amex) formats too, as show below:-

Credit Card Format	FOPE Rule
1234 123456 12345	\d\d\d\d\s\d\d\d\d\d\d\s\d\d\d\d\d
1234-123456-12345	\d\d\d\d\-\d\d\d\d\d\d\-\d\d\d\d\d
123412345612345	\d\d\d\d\d\d\d\d\d\d\d\d\d\d\d

Written by Daniel Kenyon-Smith