RPC Endpoint Mapper in a network trace

RPC Endpoint Mapper in a network trace

  • Comments 6
  • Likes

Something to think about when looking at the Endpoint Mapper in a network trace.

That is some title, it took a bit to come up with. In most cases when I find myself writing a technical document or blog it has to do with a specific fix for a specific problem. This blog will be a bit different; I am writing this more as something to think about when you are troubleshooting. I will reference some things that I have found in traces and I hope this gives you a starting point for troubleshooting. So why am I writing this? Well I was working on an issue and noticed something in a trace, when I tried to find more information there was really nothing that helped, I hope that this information may help you. Because I am on a roll, let me start by telling you that reading a trace is part science and part art, combined with a bit of luck. The thing about reading traces is you can see what happened but sometimes the answer to why it happened is more difficult. To tell you the truth, after fixing the issue that you are going to read about here I still do not know for sure why it happened; I can only guess.

So let me tell you a story.

I was asked to help with an issue where a connection attempt from a SharePoint server to a Domain Controller for the purposes of getting user information would fail. Now, it only failed when it was using Domain Controllers in a specific site so the thought was that there was a network problem. This is where I came in; I was asked to review the network trace to determine where the problem was. Now remember that a network trace only shows me what happened and not why it happened.

Let's start with how an RPC connection should look.

We start with a three way handshake.

Tcp: Flags=......S., SrcPort=2913, DstPort=DCE endpoint resolution(135), PayloadLen=0, Seq=722369472, Ack=0, Win=65535 (  ) = 65535
Tcp: Flags=...A..S., SrcPort=DCE endpoint resolution(135), DstPort=2913, PayloadLen=0, Seq=1169857372, Ack=722369473, Win=16384 ( Scale factor not supported ) = 16384
Tcp: Flags=...A...., SrcPort=2913, DstPort=DCE endpoint resolution(135), PayloadLen=0, Seq=722369473, Ack=1169857373, Win=65535 (scale factor 0x0) = 65535

After the three way handshake we initiate an RPC Bind to the Endpoint Mapper.

RPC: c/o Bind:  UUID{E1AF8308-5D1F-11C9-91A4-08002B14A0FA} EPT  Call=0x1  Assoc Grp=0x0  Xmit=0x16D0  Recv=0x16D0
RPC: c/o Bind Ack:  Call=0x1  Assoc Grp=0xBCEDB  Xmit=0x16D0  Recv=0x16D0

After receiving a response we make an Endpoint Map request to the application (this is what we will be looking at).

Epm: Request: ept_map: NDR, DRSR {E3514235-4B06-11D1-AB04-00C04FC2DCD2} v4.0, RPC v5, 0.0.0.0:135 (0x87) [DCE endpoint resolution(135)]

We receive a response indicating the IP address and port that we will need to connect to.

Epm: Response: ept_map: NDR, DRSR {E3514235-4B06-11D1-AB04-00C04FC2DCD2} v4.0, RPC v5, 10.0.0.1:5000 (0x1388) [GENA(5000)]
Tcp: Flags=...A...., SrcPort=2913, DstPort=DCE endpoint resolution(135), PayloadLen=0, Seq=722369745, Ack=1169857609, Win=65299 (scale factor 0x0) = 65299

We close this connection and start a new connection with the host and port referenced in the Epm response, starting with a three way handshake.

Tcp: Flags=......S., SrcPort=2914, DstPort=GENA(5000), PayloadLen=0, Seq=505047819, Ack=0, Win=65535 (  ) = 65535
Tcp: Flags=...A..S., SrcPort=GENA(5000), DstPort=2914, PayloadLen=0, Seq=137124896, Ack=505047820, Win=16384 ( Scale factor not supported ) = 16384
Tcp: Flags=...A...., SrcPort=2914, DstPort=GENA(5000), PayloadLen=0, Seq=505047820, Ack=137124897, Win=65535 (scale factor 0x0) = 65535

We initiate an RPC Bind to the application.

RPC: c/o Bind:  UUID{E3514235-4B06-11D1-AB04-00C04FC2DCD2} DRSR  Call=0x1  Assoc Grp=0x0  Xmit=0x16D0  Recv=0x16D0
RPC: c/o Bind Ack:  Call=0x1  Assoc Grp=0x439C6D  Xmit=0x16D0  Recv=0x16D0

We continue working with the application from here.

So now we have made it to the issue that I encountered.

Here is what I saw in the Epm for the failing connection.

Epm: Request: ept_map: NDR, DRSR {E3514235-4B06-11D1-AB04-00C04FC2DCD2} v4.0, RPC v5, 10.0.0.1:135 (0x87) [DCE endpoint resolution(135)]
Epm: Response: ept_map: NDR, DRSR {E3514235-4B06-11D1-AB04-00C04FC2DCD2} v4.0, RPC v5, 10.0.0.1:5000 (0x1388) [GENA(5000)]

Notice that there is an IP address in the Epm Request above.

Here is the same thing from the working trace.

Epm: Request: ept_map: NDR, DRSR {E3514235-4B06-11D1-AB04-00C04FC2DCD2} v4.0, RPC v5, 0.0.0.0:135 (0x87) [DCE endpoint resolution(135)]
Epm: Response: ept_map: NDR, DRSR {E3514235-4B06-11D1-AB04-00C04FC2DCD2} v4.0, RPC v5, 10.0.0.1:5000 (0x1388) [GENA(5000)]

So this started the questions:

  • Why is there and IP address listed?
  • Where did the IP address come from?
  • Why would this cause a problem?

Well I could not find much information about this issue. What I did find was a reference in the MSDN technical reference indicating that the Epm request can contact a host address. The problem with this is it is application specific, so the application would need to be written to allow for it.

For the most part, my job with regard to finding a network problem was over; there was no problem. The communication was sent, received, and answered. Not being an expert in the application being used, I began to look for differences between the configuration when connecting to the working server and the failing server.

Let me make this very clear – the application is not important here and I am not suggesting that what I am about to tell you will fix any particular problem. What I am saying is that it fixed the one I had and it is something that you can look at to help troubleshoot a similar issue.

In reviewing the configuration, I found that the application could be configured to find Domain Controllers automatically or it could be configured to use a specific Domain Controller. In my case, both the working and non-working configuration was set to use a specific Domain Controller. The difference was that the working configuration had the FQDN of the Domain Controller (server.domain.com) and the non-working configuration had the IP address of the Domain Controller.

On the non-working configuration, we added an entry for the Domain Controller to the HOSTS file and changed the IP address to the FQDN and the problem was fixed.

Okay, so here we go again. I am not saying that I know for sure what was happening but the indication is that when you place an IP address in this configuration it sets the Epm request to use that IP. There is no reason that I can think of that this would not work but in my case it did not.

If you are having this type of problem and notice this behavior in the trace it might be a good idea to check if your application allows you to specify and IP address, and if so change it to an FQDN.

- Michael Andreacola

Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment
  • PingBack from http://virdep.wordpress.com/2009/02/04/active-directory-and-firewalls/

  • 214 Microsoft Team blogs searched, 93 blogs have new articles in the past 7 days. 212 new articles found

  • We are seeing a strange issue with or DB2 clients (XP) joined to the domain. Once joined to the domain the application authetication windows takes 40 seconda to pop up as againt 2 sec when it was in a workgroup. From the frames, it looks like for some reason when the application is clicked, the clients trying to get the Auth Window from the AD DC (see GENA (5000) frames instead of DB2 Server...which i think is adding the delay. Can you assist as to how i can mitigate this delay? (DB 2 Server is running on AIX Machine and is configured to listed on port 50000...its a direct IP connection i.e. no name resolution issues)

    I have attached the capture file for your info. Any help would be much appreciated.

    First time 3-way Handshake

    8 0.000000  {TCP:6, IPv4:5} DB2 Client     AD DC TCP TCP:Flags=......S., SrcPort=1074, DstPort=DCE endpoint resolution(135), PayloadLen=0, Seq=2880073679, Ack=0, Win=65535 (  ) = 65535

    9 0.000000  {TCP:6, IPv4:5} AD DC     DB2 Client    TCP TCP:Flags=...A..S., SrcPort=DCE endpoint resolution(135), DstPort=1074, PayloadLen=0, Seq=2774228518, Ack=2880073680, Win=16384 ( Scale factor not supported ) = 16384

    10 0.000000  {TCP:6, IPv4:5} DB2 Client     AD DC TCP TCP:Flags=...A...., SrcPort=1074, DstPort=DCE endpoint resolution(135), PayloadLen=0, Seq=2880073680, Ack=2774228519, Win=65535 (scale factor 0x0) = 65535

    RPC Bind to the Endpoint Mapper

    11 0.000000  {MSRPC:7, TCP:6, IPv4:5} DB2 Client    AD DC MSRPC MSRPC:c/o Bind:  UUID{E1AF8308-5D1F-11C9-91A4-08002B14A0FA} EPT  Call=0xD  Assoc Grp=0x0  Xmit=0x16D0  Recv=0x16D0

    12 0.000000  {MSRPC:7, TCP:6, IPv4:5} AD DC     DB2 Client MSRPC MSRPC:c/o Bind Ack:  Call=0xD  Assoc Grp=0x2481B8  Xmit=0x16D0  Recv=0x16D0

    End point Map Request to the Application

    13 0.000000  {MSRPC:7, TCP:6, IPv4:5} DB2 Client     AD DC EPM EPM:Request: ept_map:

    14 0.000000  {MSRPC:7, TCP:6, IPv4:5} AD DC DB2 Client EPM EPM:Response: ept_map:

    Second time 3-way Handshake

    15 0.000000  {TCP:8, IPv4:5} DB2 Client     AD DC     TCP TCP:Flags=......S., SrcPort=1075, DstPort=GENA(5000), PayloadLen=0, Seq=3342506103, Ack=0, Win=65535 (  ) = 65535

    16 0.000000  {TCP:8, IPv4:5} AD DC    DB2 Client     TCP TCP:Flags=...A..S., SrcPort=GENA(5000), DstPort=1075, PayloadLen=0, Seq=1515412576, Ack=3342506104, Win=16384 ( Scale factor not supported ) = 16384

    17 0.000000  {TCP:8, IPv4:5} DB2 Client    AD DC TCP TCP:Flags=...A...., SrcPort=1075, DstPort=GENA(5000), PayloadLen=0, Seq=3342506104, Ack=1515412577, Win=65535 (scale factor 0x0) = 65535

    RPC Bind to the Endpoint Mapper

    18 0.000000  {MSRPC:9, TCP:8, IPv4:5} DB2 Client    AD DC    MSRPC MSRPC:c/o Bind:  UUID{12345778-1234-ABCD-EF00-0123456789AB} LSARpc  Call=0xD  Assoc Grp=0x0  Xmit=0x16D0  Recv=0x16D0

    19 0.000000  {MSRPC:9, TCP:8, IPv4:5} AD DC     DB2 Client MSRPC MSRPC:c/o Bind Ack:  Call=0xD  Assoc Grp=0x363A17  Xmit=0x16D0  Recv=0x16D0

    End point Map Request to the Application

    20 0.000000  {MSRPC:9, TCP:8, IPv4:5} DB2 Client     AD DC             LSAT LSAT:LsarLookupNames4 Request, *Encrypted*

    21 0.000000  {MSRPC:9, TCP:8, IPv4:5} AD DC          DC DB Client     LSAT LSAT:LsarLookupNames4 Response, *Encrypted*

    After these frames only i see that the DB Client connecting to DB Server

    22 0.000000 Admin.exe {TCP:11, IPv4:10} DB2 Client     DB2 Server     TCP TCP:Flags=......S., SrcPort=1076, DstPort=50000, PayloadLen=0, Seq=2538123636, Ack=0, Win=65535 ( Negotiating scale factor 0x1 ) = 65535