Exchange Online Protection will now evaluate both the P1 and P2 headers in a message against an end users safe and blocked senders list. I know, I’m super excited too! Previously only the P1 header of a message was compared to these lists. Not only does this make blocking or safe listing within Outlook or OWA easier for an end user, the process is much more intuitive.

If this makes sense to you then you can probably skim or skip the rest of the article. But if you would like more information on what this means and why we have made this change then read on.

Note: This also applies to on-premises mailboxes in Exchange 2010 or newer where the Directory Synchronization tool is being used to sync the safe and blocked sender lists to EOP. For the syncing to work with Exchange 2007 some manual configuration is required which is beyond the scope of this article.

Background in Safe and Block senders

In the Outlook client (and in OWA), there are two important lists in the Junk E-mail options. Safe Senders and Blocked Senders.

 
Senders placed in the Safe Senders list will never be marked as spam by the Outlook client and senders placed in the Blocked Senders list will always be moved to your Junk Mail folder.

When using EOP, these lists will be respected in the cloud. For example. If one of your end users has added andrew@contoso.com to their Safe Senders list, any email from andrew@contoso.com which are destined to ONLY this user will skip spam filtering in EOP. These messages are easy to identify as you will see SFV:SFE and SCL:-1 in the X-Forefront-Antispam-Report header for that message.

On the other side of the fence, if an end user has added andrew@contoso.com to their Blocked Senders list, EOP will mark the message with SFV:BLK andrew SCL:6 in the X-Forefront-Antispam-Report header and will take the spam action defined in the content filter. If the spam action is quarantine message, then the message will be moved to the quarantine.

Note: The above only affects the single end user that added the sender to their safe or blocked senders list. An end user cannot safe or block a sender for the entire organization. That would be crazy!

Background on P1 and P2 headers

Before I continue, let’s quickly talk about P1 and P2 headers. I’m going to use an actual paper letter (I know, ghetto!) as an analogy for easier understanding. The P1 address is what’s seen on the outside of the envelope, where the P2 address is what we see on the paper inside. Often these are the same, but they don’t have to be.

On the technical side, the P1 header refers to the MAIL FROM and RCPT TO commands in the SMTP conversation. The P2 header refers to the FROM and TO commands in the SMTP conversation.

The FROM address in the P2 header is what you see in your email client when looking at the sender of a message. Typically the sender is the same between P1 and P2 headers, but in some cases can be different. If a marketing company is sending mail on-behalf of another company you may see a difference between these two headers.

Consider the following SMTP command sequence.

P1 header
mail from: p1@fabrikam.com
rcpt to: astobart@tailspintoys.com
data

P2 header
from: p2@contoso.com
to: andrew@tailspintoys.com
Subject: P1 and P2 headers are different
The P1 and P2 headers will be different in this message.

In my Outlook client, here is what the message looks like.

 
Notice how Outlook shows the sender as being the address that was specified in the P2 header. My P1 sender address is nowhere to be found… on the surface that is.

Digging into the headers we find that there is a Return-Path listed which shows the P1 address. Here is an excerpt from the header of the above message.

From: p2@contoso.com
To: andrew@tailspintoys.com
Return-Path: p1@fabrikam.com

Here we can clearly see that the P1 and P2 headers are different in this message. If the sender is the same in both the P1 and P2 header then you typically won’t see a Return-Path in the header.

With all this in mind let’s loop back and think of the end user that received the above message. I right click on this message in Outlook and choose “Never block sender.” This will add p2@contoso.com to my Safe Senders list. Before the change to EOP, if a subsequent message was sent with the same P1 and P2 headers as above, it would still be scanned by the EOP spam filter and my safe list not respected. This is because EOP would look at the P1 header which is p1@fabrikam.com and since this address isn’t in my Safe Senders list, the message would not be treated as safe.

In this scenario an end user would need to go through the headers of a message, find the Return-Path, and then add this address to their safe or blocked senders list. This is definitely not a great end user experience.

Now, because EOP compares both P1 and P2 headers, it would see that p2@contoso.com is on my safe senders list and so this message would be marked with SFV:SFE, SCL:-1, and would skip spam filtering. Hence, this is now much more intuitive for an end user. They no longer need to hunt through the header of a message if the P1 and P2 headers are different.

In the past why did EOP only check the P1 header?

I’m sure this will be the next question on your mind. The reason why EOP used to only evaluate the P1 header for safe and blocked sender lists comes down to security. The SPF check is only performed on the P1 header. By only checking the P1 header when evaluating Safe and Blocked Sender lists, we can reduce the chance of spoofing because this is the domain we will perform an SPF check on. The following is from the header of the above message.

Received-SPF: Neutral (protection.outlook.com: 167.220.24.155 is neither permitted nor denied by domain of fabrikam.com)

We can see that the SPF check was done against fabrikam.com which was the sender domain in the P1 header.

With this change, a spammer could present a P1 Mail From address that will pass the SPF check, and then spoof the P2 From address to be a domain or sender which an end user has added to their Safe Senders list and EOP will skip spam filtering. On the flip side, an end user that chooses to block a sender who sends with different P1 and P2 addresses now no longer needs to hunt through a header to find the Return-Path value to add to their Blocked Senders list.

While it’s true that with this change we are forgoing some security, the gain in end user usability far outweighs this loss.

Final thoughts

This change is still rolling out so if you don’t see EOP respecting P2 headers yet, just be patient. Final note, EOP started comparing both P1 and P2 headers to end users Blocked Senders list a few months ago, but only started comparing both P1 and P2 headers to end users Safe Senders list this month.

Resources

Manage safe sender lists for bulk mailers
Anti-spam message headers

Update Dec 23, 2014 – I’ve revised this article to make a couple of points clearer. On the mail flow side there are no issues with daisy chaining. However, if the intent is to evaluate the effectiveness of EOP filtering, having a service sit in front of EOP will not allow EOP to show it's full abilities.

Daisy chaining filtering services is something I often see as a temporary configuration when an organization is transitioning from a previous filtering solution to Exchange Online Protection. I also see this when customers are testing EOP and don’t want to take their previous filtering solution out of the mail loop. Here’s what mail flow looks like in these situations.

Daisy chaining services with EOP in this manner is a fine, however there are some important behavioral changes with EOP that you’ll need to be aware of. On the mail flow side this setup is fine, however if the intent is to test EOPs Spam filtering abilities, you won’t get the full picture of EOPs abilities unless your MX is pointed directly towards EOP.

Why you ask? Great question, and in the next section we will dive into the technical reasons behind this.

How Daisy Chaining Effects EOPs Filtering Abilities

EOP evaluates the connecting IP of all messages that are received. The IP is evaluated against the IPs in the tenants Connection Filter along with IPs that are on Microsoft’s safe and block lists.

Here’s the important part, EOP will only evaluate the connecting IP for these checks. For example, you have added 1.1.1.1 to the Blocked List in the IP Connection Filter. A message is inbound from 1.1.1.1 and first arrives at the Third Party Filtering Service, this service then relays the message to EOP. EOP will only see the IP of the Third Party Filter Service because this is the connecting IP. EOP will not see the originating IP of the message and so won’t block it based on entry of 1.1.1.1 in the blocked list in the connection filter.

Another scenario where this is impacted

Consider the following mail flow.

This is typical for customers that have an on-premises spam filtering solution, but are slowly moving mailboxes to Exchange Online. In this scenario, the end goal is to have their MX record pointing towards Exchange Online, but initially this is usually what mail flow looks like. For inbound mail, if the recipient mailbox in on-premises it’s delivered directly, and if it’s in the cloud, on-premises will relay it to the cloud.

In this scenario, on-premises mailboxes are obviously not protected by EOP, but cloud based mailboxes are. For Exchange Online mailboxes, you still lose EOPs IP based edge blocks. In this scenario, EOP will only ever see the connecting IP, which is the IP of the on-premises mail environment, so any IP based evaluations, EOP will be evaluating the on-premises IP and not the originating IP.

Edge Blocking Stats

Even for customers that don’t populate the Connection Filter, EOP identifies a large amount of spam based on the connecting IP, and when you put a hop in front of EOP, you lose out on this ability. EOP maintains an IP block list that will always apply at our edge (unless you have white listed the IP in the connection filter). This IP block list is a combination of a list that we maintain, along with third party lists that we subscribe to.

Consider the following slide which I showed in my EOP presentation at TechEd North America this year.

The numbers in this slide are only averages and change month to month. However, you can see that EOP does a large amount of spam identification based on the originating IP. When you introduce a hop in front of EOP, you lose out on this IP based spam detection. Hence, unless your MX is pointed directly towards EOP, you won’t be seeing the full power of the service.

Real world example

I worked with a customer that had the mail flow I described at the start of this article.

Internet --> Third Party Spam Filter --> Exchange Online

This was a temporary configuration, but in this setup their cloud mailboxes were seeing a large amount of spam arriving in their inboxes. I had them send me twenty examples of spam messages that had been missed by EOP. Out of the twenty submissions, 18 had originated from IPs on our block list which weren’t detected because of the hop in front of EOP. The number of spam messages slipping through EOP decreased greatly once the MX was changed to point towards EOP.

In this situation, the customer had disabled filtering on their third party spam filter and was relying on EOPs filters. The problem was that EOP was unable to filter out spam based on the sending IP.

Summary

On the mail flow side there is nothing wrong with daisy chaining services with EOP. Be aware though that if your intent is to evaluate the effectiveness of EOP, you should point your MX directly towards EOP otherwise you won’t see the full power of the product.

I recently worked with a customer who had a configuration in their EOP outbound connector that broke inbound mail for a newly added domain. I want to share this tale in hopes that you not only learn more about EOP partner connectors, but that you decrease the change of breaking your own mail flow. So grab your favourite caffeinated beverage, tilt back in your office chair, and read along.

To begin, let’s first look at what the inbound mail flow looks like.

It is against best practices to daisy chain services like this (ie. Pointing your MX towards a 3rd party filtering service which then relays to EOP). I won’t get into the disadvantages of this setup here but will write about it soon. For EOP to accept mail from the 3rd Party spam filter, the following connector was created in Office 365 (note: this connector is actually not required in this situation, more on this further down in the article).

General tab

Security tab

Scope tab

Mail flow with this setup worked fine. The MX records for all accepted domains pointed towards the third party filtering service, and the IP in the Partner Connector is the IP that this 3rd party service used to relay the message to EOP.

Everything was perfect… until a new domain was added, let’s use contoso.com to protect the innocent. The MX record for contoso.com was set to point towards EOP and not the third party filtering service (the plan was to have all MX records pointed towards EOP, but this had not yet happened, except for contoso.com). All mail sent to contoso.com would be directed to EOP, which would result in an NDR (sent from EOP) back to the sender which included the following error.

BN1BFFO11FD051.mail.protection.outlook.com gave this error:
Relay Access Denied ATTR1

Your message wasn't delivered due to a permission or security issue. The address may only accept email from certain senders or another restriction may be preventing delivery. For more tips to resolve this issue see DSN code 5.7.1 in Exchange Online. If the problem continues contact your help desk.

This is certainly not an ideal situation, unless of course you don’t want anyone to email you.

What’s happening here?

How come EOP is rejecting inbound mail which is destined for contoso.com? Let’s take a look back at the EOP partner connector. Two things here, this connector has the following configuration.

• Connector type: Partner
• Domain restrictions: Restrict domains by IP addresses
• Sender domains: *

All externally originating mail that comes in to EOP will hit this connector because the type is set to partner and the sender domain is set to *. This connector will then check if the message came from the IP(s) listed in the “sender IP addresses” field of this connector and if there is not a match then EOP will reject the message. For mail that was being relayed from the 3rd party filtering service everything was fine because the IP of this service was in the partner inbound connector. For mail that arrived from a different IP, EOP would reject the message, as was the case with contoso.com because the MX for that domain was pointed directly at EOP.

The solution

In this particular case there were two options if this connector was going to remain enabled. Either set the Domain Restrictions to None, or remove the * from Sender domains and manually type the domains in. Since this connector is picking up all inbound mail, typing in all the domains on the Internet is not exactly an option. Instead, we need to remove the Domain Restrictions on this inbound connector and set this option to none.

Typically Partner Inbound connectors are used to force TLS with a subset of domains so it is very rare to see both an IP Address restriction along with a * for the sender domains. In this scenario, a connector in EOP DOES NOT need to be created to accept mail from the Internet as EOP will accept all mail destined for your domain. This particular customer had created this connector to work around an issue in EOP which has since been fixed.

References

Configure custom mail flow by using connectors
Inbound and Outbound connector FAQ
Decide which connector to use (great article with examples)