Evan Dodds - Microsoft Exchange Server Blog

Exchange, Exchange administration stuff, and other assorted ramblings

Blogs

IMF and the Junk E-mail folder in Outlook

  • Comments 23
  • Likes

I’ve posted several IMF-related blog postings in the past — here, here, and here. In these posts, I’ve focused mainly on the gateway behavior of the IMF. This is because that’s really what the IMF is, of course… just the gateway behavior. But when your mailbox server is running Exchange 2003 or better, there’s a whole different angle to consider: Store behavior.

As I’d mentioned, through the IMF UI in ESM, you can set the Store Action Threshold. This sets a value in the AD that is read by the Information Store on Exchange 2003 (or newer) and controls the server-side Junk E-mail folder behavior. What this means is that if the mail makes it to the Store (ie – is not stopped by IMF at the gateway), it will have an SCL stamped on it. The Exchange 2003 Information Store will detect that there is an SCL value associated with the message. The store will then make a determination on whether or not the message SCL value means it needs to be moved into the mailbox’s Junk E-mail folder.

Important to note that this IMF/SCL related Store Action is totally separate from and unrelated to the Outlook client-side Junk E-mail evaluation. It’s a bit of a confusing interaction, but they really are separate. Even if you never log onto your mailbox with Outlook 2003 in cached mode and set the Junk E-mail options to be enabled, you can still have the Exchange Junk E-mail processing move messages to the Junk E-mail folder. There’s a bit more information on Outlook 2003 client-side Junk E-mail behavior in KB.842510.

Bonus: Outlook doesn’t use the IMF provided SCL values for its client-side (cached-mode only) anti-spam determination. Instead, it does its own Junk E-mail evaluation and determines whether or not to move the mail to the Junk E-mail folder based on the settings within the “Tools->Options” Junk E-mail settings.

  • “Low” setting corresponds to approximately a store action threshold of 6 (ie – it will move messages with SCL or 6 or higher to the Junk E-mail folder).
  • “High” setting is more aggressive and corresponds to an SCL of between 4 and 5 (ie – it will move messages with SCL greater than somewhere between 4 and 5 to the Junk E-mail folder). Note that there’s not a 1:1 mapping between the four settings in Outlook Junk E-mail behavior and IMF SCLs, so that’s why impacted SCL values are “somewhere between 4 and 5” for Outlook.

Back to the StoreActionThreshold, much of the confusion I see in this area is around what has to be done to get the server-side Junk email processing to work. It’s important to understand that there is actually another requirement that has to be taken into account here. It’s not just the server realizing that the message processed by IMF has a particular SCL and moving it to the Junk E-mail folder… there actually has to be a special rule in place within each mailbox to make this move happen. The rule is made up of information about your junk email settings, safe senders, etc.. 

Now we’ve established that in order for the Store Action to work properly, you need to have both Exchange 2003 (or better) Information Store that knows to read the StoreActionThreshold value from the AD and ALSO a special rule set in the mailbox to tell the store how to process the Junk E-mail. That’s where most people run into trouble, and why sometimes it seems to work and sometimes it doesn’t. It’s all about how this special rule gets set (or doesn’t)!

  • If you fire up Outlook 2003 in cached mode, it will set this special rule in your mailbox automatically.
  • If you fire up Outlook 2003 in online mode, it will NOT automatically set this special rule, although you can go into the Tools->Options dialog and change the Junk E-mail configuration to ensure it is set.
  • If you fire up Outlook Web Access (OWA) 2003 and go into the Options page, you can check the checkbox to “Filter Junk E-mail”, which creates the special rule.

So, in short, if you’re not having all of your users login to the server with Outlook 2003 in cached mode already, the rule probably isn’t already created and won’t be automatically created just because you implement IMF.

What to do? If you’re not using Outlook 2003 or if you only have a handful of users affected by this behavior, the best option is probably to go into the affected mailboxes (or have the end-user do it) with OWA2003 and set the Filter Junk E-mail checkbox. This will set the special rule in the mailbox, and Junk E-mail folder will begin to function.

But what if you have a BUNCH of mailboxes to set this on? Unfortunately, there’s no built-in way to bulk set this rule on each mailbox directly. But one of my Microsoft colleagues (Jerry Wang) put together a VBScript that will automate the process of spinning through the mailbox in OWA and setting up the special rule. This script was subsequently modified a bit by Xavier Coppin to make it work in more end-user cases. Some credit also belongs with Glen who posted some of these details back in December: http://gsexdev.blogspot.com/2004/12/setting-outlook-2003-junk-email.html. This script might be useful if you’ve a need to set this on a large number of mailboxes. This script comes with no warranty, etc and please review it before you use it, and of course you use it at your own risk!

You feed this script a file containing a list of mailboxes (alias or SMTP address), one per line. If you can’t connect to the mailbox OWA with /exchange//">http://<servername>/exchange/<line-from-the-input-file>/ it won’t work, so construct the input file accordingly and change the strURL value in the script if you need to adjust the URL. The script will prompt for the server, username, password, etc if you don’t provide them according to the usage: cscript JunkEnable.vbs servername domain\user <file name> [Password]

SCRIPT UPDATED Feb 8, 2005


'=======================================================================
' Microsoft provides programming examples for illustration only,
' without warranty either expressed or implied, including, but
' not limited to, the implied warranties of merchantability
' and/or fitness for a particular purpose. This article assumes
' that you are familiar with the programming language being
' demonstrated and the tools used to create and debug procedures.
' Microsoft support professionals can help explain the functionality
' of a particular procedure, but they will not modify these examples
' to provide added functionality or construct procedures to meet your
' specific needs. If you have limited programming experience, you may
' want to contact a Microsoft Certified Partner or the Microsoft fee-based
' consulting line at (800) 936-5200. For more information about Microsoft
' Certified Partners, please visit the following Microsoft Web site:
' http://www.microsoft.com/partner/referral/
'=======================================================================

'==========================================================================
' VBScript Source File
'
' NAME: JunkEnable.vbs
'
' AUTHOR: Xavier Coppin  (http://xavblog.net/)
' DATE  : 8/Feb/2005
'
' COMMENTS:
' Enable Junk Email filter at mailbox level
'==========================================================================

Option Explicit
'==========================================
Dim ServerName, AdminUserName, Password, oArgs, ArgNumber, TextFileName
Set oArgs = Wscript.Arguments
ArgNumber=CInt(oArgs.Count)
If ArgNumber > 0 Then
 If oArgs(0)="/?" Or oArgs(0)="?"  Then
  Usage()
 End If
End If
'WScript.echo "oArgs.count=" & oArgs.count
If ArgNumber < 1 Then
 ServerName=InputBox("Can you please enter the Exchange Back-End server name ?","ServerName","ServerName")
Else
 ServerName=oArgs(0)
End If
If ArgNumber < 2 Then
 AdminUserName=InputBox("Can you please enter an Exchange Administrator account name ?","AdminUserName","DOMAIN\USERNAME")
Else
 AdminUserName=oArgs(1)
End If
If ArgNumber < 3 Then
 TextFileName=InputBox("Can you please enter the name of the plain text file ?","Text File Name")
Else
 TextFileName=oArgs(2)
End If
If ArgNumber < 4 Then
 Password=InputBox("Can you please enter the Password of " & AdminUserName,"Password")
Else
 Password=oArgs(3)
End If
Dim oFileObject, oLog
Dim strLine,strURL
Set oFileObject = CreateObject("Scripting.FileSystemObject")
Set oLog = oFileObject.OpenTextFile(TextFileName,1)
Do Until oLog.AtEndOfStream
 strLine = oLog.ReadLine
 strLine = Trim(strLine)
 strURL = "HTTP://" & ServerName & "/Exchange/" & strLine
 ForceJunkEmailFolder strURL, AdminUserName, Password
Loop
oLog.Close
Set oLog = Nothing
Set oFileObject = Nothing

'==========================================
' Print the usage
'==========================================
Public Sub Usage()
 WScript.Echo "Usage:"
 WScript.Echo "Enable Filter Junk Email at mailbox level ."
 WScript.Echo "  cscript JunkEnable.vbs servername domain\user <file name> [Password]"
 WScript.Quit 0
End Sub

'==========================================
'Logon to specified mailbox by OWA
'==========================================
Public Sub ForceJunkEmailFolder(strURL,strUserName,strPassword)
On Error Resume Next
Dim XmlHttpStr, ObjXmlHttp
Set ObjXmlHttp = CreateObject("Microsoft.XMLHTTP")
XmlHttpStr = ""
xmlHttpStr = xmlHttpStr & "Cmd=options" & vbLf
xmlHttpStr = xmlHttpStr & "junkemailstate=1" & vbLf
XmlHttpStr = XmlHttpStr & "cmd=savejunkemailrule" & vbLf
ObjXmlHttp.Open "POST", strURL,false,strUserName,strPassword
ObjXmlHttp.SetRequestHeader "Accept-Language","en-us"
ObjXmlHttp.SetRequestHeader "Cache-Control","no-cache"
ObjxmlHttp.setRequestHeader "Content-type:","application/x-www-UTF8-encoded"
ObjxmlHttp.setRequestHeader "Content-Length:", Len(XmlHttpStr)
ObjXmlHttp.SetRequestHeader "Accept", "*.*"
ObjXmlHttp.Send XmlHttpStr
If Err <> 0 Then
 WScript.Echo "Filter Junk Email failed at " & strURL & " Failed!"
 WScript.Echo "Error number" & Err.Number
 WScript.Echo "Description : " & Err.Description
 Exit Sub
End If
 
If ObjXmlHttp.Status <> 200 Then
 Wscript.Echo "Filter Junk Email failed at " & strURL & " Failed!"
 WScript.Echo "Http status code =" & ObjXmlHttp.status
Elseif  ObjXmlHttp.responsetext <> "" Then
     WScript.Echo "It seems an error occured during the Send Command. Are you sure you have the correct permissions on this mailbox ?"
     WScript.Echo "Response of the server was " & ObjXmlHttp.responsetext
  Else
    WScript.Echo "Filter Junk Email enabled for mailbox " & strURL
End If
 

Set ObjXmlHttpStr = nothing
End Sub

Comments
  • Evan posted on his blog an excellent article concerning IMF and the Junk Email f

  • Evan has taken a further look at IMF and it's relationship with the Outlook Junk E-mail folder.

  • Hi Evans, thanks for your excellent blog on this topic.

    I have found some interesting behaviour on the store threshold. If the Extended rule is set using Outlook 2003 in cached mode, email with SCL equal or higher than the threshold gets filtered. But if I were to turned on the rule via OWA 2003, only email with a value higher than the threshold gets filtered. I have done the test a few times and I get the same result all the time. Is it possible that you can verify this with someone on the Exchange team?

  • Lee -

    What you describe wouldn't surprise me at all! The original documentation all discusses treating the Extended rule for StoreActionThreshold as a ">" value, and it's only because Outlook treats it as ">=" rather than as ">" that we have this confusion. It seems quite reasonable to me that Outlook Web Access would treat it as originally documented, although I can see how this could lead to unpredictable behavior. Do you have a case opened with PSS on this issue? If it's causing you trouble in your environment, you may want to pursue that route and see if there's anything that can be done to stabilize this behavior for future versions...

    Evan


  • Hi Evan,

    I think these lines are missing in the ForceJunkEmail sub.

    xmlhttp = ""
    xmlhttp = xmlhttp & "Cmd=options" & vbLf
    xmlhttp = xmlhttp & "junkemailstate=1" & vbLf

    Best regards,

    Xavier
    http://xavblog.net

  • Thanks to Xavier for providing some changes to the script and allowing me to post the update. The script now listed in the blog post should now hopefully work for everyone.

    Evan

  • This works just great. We are going to test using this for a bunch of users. However, is there a way to mass turn junk email off if we don't like it? I tried setting "junkemailstate=0" but that didn't work.

    Thanks!

  • Not aware of any way to bulk disable the extended rule, although if anyone knows of a way feel free to post it or let me know. You could, of course, turn off the IMF at the gateway to stop the SCL from appearing on the message. This would serve as a sort of "bulk disable" for the extended rule.

    Why do you want to disable the rule in bulk? Remember that the more common case is that SOME users have it enabled already (cached mode Outlook 2003) and you realize that other users need to have it added to have consistency.

    Evan

  • The script worked when I ran it using the Administrator accont and on its corresponding maibox, but it fails with "Http status code=403" if I use on any other mailbox, even with the user's credentials. Help!!! ;)

  • Milton -

    I haven't run into the 403 errors from the script before, but it wouldn't surprise me at all if it was related to permissions not being in place to log into each user's mailbox. In order to set the options for each mailbox, you will need to have rights to login to the mailbox with the account you specify. Have a look at http://support.microsoft.com/kb/273642 for more information on how to set this up most effectively (it's the same permissions you would need to run Exmerge using this account).

    Evan

  • Thanks Evan! I tried it. There were a couple of Deny entries at the Organization level for the Administrator account which I removed. Still no go. I should have mentioned is that the Exchange site was set to require https and allow only basic authentication. I tried allowing http access and Windows authentication, but still no go, even after re-setting IIS on the server. Do the Exchange services need to be re-started for any of these changes to take effect? Also, what is the proper syntax for the server name, account name and mailbox name? I've tried the usual but I wonder if the methids being called require something different.

  • Hang on - I got it working after disabling basic authentication (I left it on after enabling Windows authentication). According to the script all mailboxes now have the extended rule. But ... no filtering seems to be taking place. I'll try restarting the Exchange services a bit later tonight. How I wish that wasn't necessary to make changes to the IMF settings (it makes the "I" in IMF stand for "Incomplete", "Inconvenient", and "Irritating").

  • Milton -

    You should only need to restart the store on the mailbox server after the IMF is first installed (ie - the store has to be cycled to recognize that the first IMF has been installed into the organization). Subsequent changes to the IMF levels/settings don't require a restart, as there is a notification set up with the AD.

    Probably the best way to troubleshoot this is to set up the "view the SCLs within Outlook" form (http://blogs.msdn.com/exchange/archive/2004/05/26/142607.aspx) and make sure the items arriving to the mailbox have the appropriate SCL in place such that they SHOULD be getting moved to Junk Email.

    Evan

  • Evan: thanks for the very helpful explanation that IMF setting changes do not require a re-start. The phrasing in the Admin Guide was a bit confusing.
    Now I'm seeing one out of several dozen messages stored in my Junk Email folder. I installed the form and added the SCL column to my Inbox. I see one message with an SCL of -1 (which happens to be from me to an "all users" list). The rest, including a few that arrived since the change, do not show anything at all in the SCL column. I get the feeling that all this I may be due to something very simple, but what could it be?