We get two different common requests for features that we have solutions for, but it's funny that they are incompatible. I accidentally discovered this on my own Exchange 2003 server at home.
Feature 1: “Catchall” mailbox
The idea with this one is that you have people mailing you at email@example.com, firstname.lastname@example.org, etc. and you don't want to miss a potential sales opportunity just because the sender mistyped your address. So you designate a “catchall” mailbox, where all mail sent to *@company.com goes to this one mailbox. This is an example of the kind of thing we couldn't do with Exchange 5.5 because we didn't have an extensible architecture. With Exchange 2000 and beyond, you could use an event sink to do this. We have a well-used sample in a KB article. Now these days (this wasn't so true when we first wrote this article) you really don't want to do that, because spammers who use dictionary attacks against your domain will make your life a living hell. Of course the solution to that is a good spam filter, which makes this reasonable.
Feature 2: Validate recipients in the directory before accepting message
This is a feature that was an endemic part of sendmail for a long time, as long as most people used sendmail with a local password file. Many implementations of sendmail that use an LDAP directory instead did not offer this feature, and we did not either for a long time. If you don't have this feature, and you get a lot of spam to random recipients, then your machine can spend a fair amount of effort generating Delivery Status Notifications (DSNs) aka Non-Delivery Reports (NDRs) that are sent to the sender of the spam. Of course, this sender is usually bogus, which results in the NDR “NDRing”, and when Exchange generates an NDR to an NDR, it has nowhere to put that NDR, so it puts it in the “badmail” directory.
Anyway, finally in Exchange 2003, we added this feature. To turn it on, fire up Exchange System Manager (ESM), under Global Settings, get properties on “Message Delivery”. Go to the “Recipient Filtering” tab. Check the box next to “Filter recipients who are not in the Directory”. Like other filtering, make sure that on your SMTP VS that accepts mail from the Internet, turn on the filtering. To do this, click the “Advanced...” button next to the IP address on the General tab, click “Edit”, and enable “Recipient Filtering”. With this feature, at the RCPT command, the server looks up the recipient in the AD and if no one has the email address as one of their addresses, the RCPT is denied, and the message is never accepted. Which is nice because it's up to the sending server to generate the NDR.
What is funny is that if you turn on recipient filtering, and a message comes in to email@example.com, even though the catchall sink would have caught this message and redirected it to my mailbox, because the recipient filtering feature above happens at the SMTP RCPT command, the message never gets accepted. I do need to get the KB above updated to reflect this find! (or maybe some smart person will find a workaround)
Exchange-faq.dk - Din portal til Microsoft Exchange Server information
I'm wondering how this 'validation' will affect performance on the AD. There is a Anti-SPAM server in front of our Exchange environment, and to prevent the NDR NDRing activating this on our incoming Exchange box (next in line) doesn't help. The Anti-SPAM box must do this. I've never used the Anti-SPAM server's LDAP lookup option to avoid an AD performance decrease, but in the future there will be a point were I must start using it. Hundreds of NDR each day...
You're absolutely right that turning on this option will cause more AD traffic. The Exchange server does have a local directory lookup cache that will help to some extent, but it could cause quite a load on the GC that your Exchange server is using. (to check which GC your Exchange server is using, use ESM and look at the properties of the server) It's a trade-off: do you want to spend the effort in doing AD lookups, or generating NDRs. The latter is probably more expensive overall.
Another thing to consider, that I didn't cover above but I did cover in a previous post about the recipient lookup feature, is that enabling recipient lookup makes a dictionary attack against your domain much easier.
In the previuos post you mention that doing a brute-force dictionary attack could be solved by "tarpitting" where the <x>th RCPT command and higher all add a sleep period to each next one. If this is a MSX SMTP virtual setting (or SMTP Internet Mail Connector setting) I'm missing something.
I can start another connection for each incoming message exceeding <x> recipient, but not slow it down AFAIK.
(Digging in the Anti-SPAM server SMTP settings to see if this box can do it...)
Unfortunately there is no built-in tarpitting in Exchange 2003. It is something that we're looking at doing for future releases. It would be theoretically possible to do it using a protocol event sink, although I don't think we have ever tested doing that to know what other side effects it might have. For instance, it might slow down other concurrent SMTP connections to your server, depending how you write the sink.
Maybe you could, in a future post, explain how features are determined for upcoming releases of Exchange Server. I tested the version 4 Beta release way back in time, and do test for several other current Beta products, but this mainly involves discovering bugs. If features request are made through a Beta program, it's almost always triggered by something that doesn't work as expected in the Beta, not something completly new.
How do you come up with a list of new features that are to be included in a new release?
In a 'Meet the Expert' session here in the Netherlands last December, I discussed a problem for which ' the expert' knew a ' feature design request' was already on the list: how did it get there? Not my single PSS call I assume..
On topic: An event sink might be the solution (isn't that the case for almost anything that's not build into Exchange server). The problem with this sink would be testing: I hate to fiddle with the production environment, and testing this in the lab won't tell much about slowdown effects. Most labs (mine for example) run less speedy hardware and are unable to generate huge amounts of traffics. That's what you need to test this type of sink.
Would the Check Recipients also catch domains that you are forwarding to?
Oooh, haven't looked at this feature with domain forwarding in mind. For sure with AD contacts this works, they can be validated with a LDAP lookup. But does recipient filtering look at configured address spaces and routing configurations? I would like to know that too.
I checked with the PM for the feature and he confirmed my assumption that check recipients only checks recipients in "local" domains. That is, domains that have "this exchange organization is authoritative for this domain". Any other "remote" domains, for which Exchange just relays, are not checked. If it did, then the feature would be broken for most users.
David Lemson said:
It's a trade-off: do you want to spend the effort in doing AD lookups, or generating NDRs.
I don't see the tradeoff. If you generate NDRs, don't you still have to do the AD lookup to know it's an NDR? Therefore, isn't just doing AD lookups always less work than doing AD lookups AND generating NDRs? Or am I missing something?
I was pretty excited to see the new feature in 2003 so that we can ease the strain on our Exchange server from all the NDR generating.
I didn't make it clear above. You're right, you have to do the AD lookup eventually, but the tradeoff is whether you do the AD lookup during the SMTP protocol, making the client wait, or just accept the message and then do the lookup later and potentially expend more effort to generate the NDRs. The other tradeoff is whether you want spammers to know immediately if a recipient is invalid vs the effort to generate the NDRs to the bogus address for invalid user mail.
That makes sense. We're a small company, so we only have one AD domain and one Exchange server. In addition, we have a mail gateway sitting in front of our Exchange server, so the traffic is all local. In our case, the "client" is always our gateway, so I don't think making it wait while the Exchange server does its AD lookup will be a problem. Plus the spammers are contacting the gateway (which accepts mail to all users, valid or invalid), not talking directly to the Exchange server, so that won't be a problem either.
Question - do you know if spammers gather NDR's to determine if a recipient is invalid? If so, then does it really matter whether they get an immediate "user unknown" in the SMTP conversation, as opposed to waiting a little bit to get the NDR?
They probably rarely gather NDRs that are sent back later, firstly because that requires them to have given a real address in the MAIL FROM: of the SMTP conversation, which would receive that NDR. If the client is notified that a recipient is invalid, the server tells it immediately in the SMTP conversation "that recipient is invalid", so the server does not generate an NDR (or in SMTP terms, a DSN). It is the CLIENT that actually generates the DSN in this case.
So for yourself, your "mail gateway" is the machine that is generating the NDR/DSN back to the spammer. I would say that in your situation, turning on the "resolve anonymous recipients" feature does you almost no good, because when you turn it on, you're just transferring the work of generating NDRs from the Exchange server to the "mail gateway". Also, you continue to get the traffic of those messages in and out. (if they're blocked at the RCPT command in SMTP, the data never gets sent).
What a useful discussion this is turning out to be for me. I follow what you're saying about the spammers. I also agree that if we shift the work of NDRs to our mail gateway, we still have the traffic in and our of our T1. However, we have reduced the amount of data our Exchange server receives (even though the gateway still has to receive it). Since our Exchange server is more than just an SMTP server (it is our primary information store as well), offloading work from it to the gateway seems to be a decent tradeoff, especially since the gateway is just sitting there passing mail along.
However, since you brought it up, do you have any other suggestions? It seems to me that you can't have it both ways. If we have our gateway block bad addresses at the RCPT command in SMTP, we will reduce the traffic in and our and force the work of the NDRs back to the sending server (which would be the "client" in this scenario). Sounds good so far. However, doesn't this allow the spammers to know immediately if a recipient is invalid?
So, in our case, if we assume we don't want the spammers to know immediately if a recipient is invalid, then it sounds like our best option is to at least shift the burden of NDRs from the Exchange server to the mail gateway. Am I getting close?