Configuring the RPC filter to support DCOM traffic is a particular pain point in ISA Server configuration. This entry provides a quick overview of the filter, the implications of the "Enable strict RPC compliance" setting, and some information on common issues with DCOM traffic.
ISA Server's RPC filter monitors RPC traffic between hosts, and sets up secondary connections as required for RPC traffic. For outbound RPC requests, ISA Server inspects the traffic flowing between the source and destination. For incoming requests to published RPC servers, ISA Server inspects the traffic flowing between the source and destination, and dynamically opens and closes ports on the external published listener based on the protocols used by the RPC client and server. The RPC filter cannot be applied to traffic tunneled over another protocol, such as RPC over HTTP. When a rule references a protocol that is bound to the RPC filter, then the filter is applied to traffic matching the rule. By default, ISA Server provides three predefined RPC protocols for use by inbound and outbound RPC traffic:
By default the predefined RPC protocols are bound to the RPC filter. You can also create custom protocol RPC definitions using the New RPC Protocol Definition Wizard. When you create a custom RPC protocol using the wizard, the following defaults are applied:
Note that traffic defined as "outbound" is not handled by ISA Server based on specific UUIDs, so it isn't possible to set up a custom protocol definition for specific UUIDs. For traffic defined as "incoming", you can create a custom protocol with specific UUIDs, either by selecting them from the endpoint mapper list, or by manually creating them.
Enable strict RPC compliance
For publishing rules ISA Server blocks DCOM traffic, and this setting cannot be modified. For access rules, a default "Enable strict RPC compliance" setting is configured on each RPC rule. With this setting in place, DCOM traffic is blocked. More specifically, any traffic (such as DCOM) that does not start an RPC exchange by communicating with the endpoint mapper is blocked. Turning off the "Enable strict RPC compliance" setting does not specifically allow DCOM traffic. It simply disables filtering for this traffic after the endpoint mapper requirements have been met. To allow DCOM traffic through an RPC access rule, either of the following is required:
Hints for Troubleshooting RPC Server Publishing
ISA Server Product Team
Excellenet article! Lots of interesting and useful information
Thank you for the article, very helpful.
Is this going to be fixed in the next release of ISA? It's a rather annoying bug...
Are there any issues with RPC publishing on ISA 2006 with Exchange 2007 and Outlook 2007? I can't 'stay' connected, it keeps dropping off and reconnecting. Outlook 2k/Ex07 works though.
Recently I had to re-install WinXP on a workstation. This caused non-critical login errors in the server security log, event ID 537. I eventually tracked these errors back to automatic certificate enrollment errors, "RPC Server is unavailable", at the workstation. The workstation was requesting a computer certificate.
In my first effort at debugging this problem I disabled the "strict RPC compliance", restarted the ISA 2004 firewall, and manually requested the certificate. No luck! Same error!
Next I disabled the RPC filter, restarted the firewall, and requested the certificate. The certificate request worked. I enabled the RPC filter and restarted the firewall. When I checked the logs today everything is normal. I have noticed in several forums that several people were successful at resolving their RPC issues only by turning off the RPC filter.
Where should I be looking to fix this problem?
1. I believe I have a fully patched ISA 2004 server.
2. The workstation is in the "protected networks" so all protocols should be available. 3. The ISA log does not show any error messages for this workstation. ISA does not "appear" to be preventing the connection.
Applications that want to talk to other servers will often use the Remote Procedure Call (RPC) infrastructure
I wonder why don't you distribute this article inside your team.. We spent a whole day on a Severity A case collecting and recollecting mutiple diagnostic data from ISA until someone suddenly said: "Hey, we don't support WMI, and we don't care if all other MS products rely upon it". Thank's a lot...
So, is it not possible to configure Windows EBS 2008 security server to connect through WMI? If it is possible, how do we connect? I created a filter and unchecked the "Strict RPC compliance" but that didnt help.
<a href="www.xgametop.com/">private traffic servers advertising</a>
Thanks for this article, if I only found it earlier..
I was struggling with an ISA2006 server which didn't want to be monitored by WMI..
The problem was with the Remote Management Group.... I was able to manage it remotely :D but monitoring.... hell no..
Modifying the rule did the trick..