John Howard - Senior Program Manager in the Hyper-V team at Microsoft

Senior Program Manager, Hyper-V team, Windows Core Operating System Division.

Hyper-V: Why does Hyper-V Manager not always work over VPN connection? Access Denied or RPC server unavailable errors.

Hyper-V: Why does Hyper-V Manager not always work over VPN connection? Access Denied or RPC server unavailable errors.

  • Comments 46
  • Likes

This post examines a problem several people have reported when running Hyper-V Remote Management tools over a VPN connection  - specifically hitting an error “Access denied. Unable to establish communication between ‘SERVER’ and ‘CLIENT’”. In some variations, I’ve seen RPC errors such as “RPC server unavailable. Unable to establish communication between ‘SERVER’ and ‘CLIENT’.”

vpn1 

And an example of an RPC error case:

vpn2

To be explicit up front, I am talking about this only occurring over a VPN/RAS connection – when connected using a wired or wireless connection without VPN, everything works normally. If things are not working on wired/wireless, follow my series of remote management posts to configure everything first. 

Diagnosing the issue took a bit of sleuthing. So let’s dive in. A big clue is in the first message – it implies there is some form of communication between the Hyper-V enabled server and the Remote Management client. Indeed, that is correct – there is a DCOM callback. So let’s start by looking at the IP configuration on the laptop machine I’m using for this walkthrough after the VPN connection has been established.

vpn3 

Note that the DHCP assigned address for the VPN connection is 192.168.200.6, and the DHCP assigned address for the Internet connection is 192.168.1.119.

Let’s run a network trace network trace on the Hyper-V enabled server to see what’s going on. I’m running the network trace while starting Hyper-V Manager on the laptop:

vpn4
The two highlighted lines show that the Hyper-V enabled server is making an attempt to connect to my local wireless IP address on my broadband connection, 192.168.1.119, rather than the DHCP assigned IP address for my machine on the internal network, 192.168.200.6.

What’s also interesting in the trace are ARP packets from the Hyper-V enabled server at 192.168.200.218 to “HPCRAPTOP”:

vpn5

Notice that the server is asking where 192.168.200.14 is, and netmon is resolving 192.168.200.14 to the IP address of the laptop. So that indicates all is not well with DNS since we know above that the DHCP assigned address on the VPN connection is 192.168.200.6. Let’s do an nslookup to examine the DNS entry for the laptop.

vpn6

Indeed, the laptop from a DNS perspective is incorrect and explains why netmon is resolving 192.168.200.14 to my laptop. (Although I didn’t mention it, I happen to know that this DNS entry, 192.168.200.14, was the IP address assigned to the laptop when it was last connected directly to the internal network.)

So as an experiment and first workaround, let’s edit \windows\system32\drivers\etc on the Hyper-V enabled server to add an entry for my laptop as 192.168.200.6, the current IPv4 address for VPN and see what happens.

vpn7

vpn8
Yes, that works. But it’s hardly what I could describe as a desirable or every-day-workable solution. If you’re walking through with me, remember to remove that entry hosts to see if there are any other workarounds.

Well there is one interesting workaround which I mentioned in my remote management configuration series. However, I absolutely do NOT recommend this one unless you really need to as you are lowering the security of your machine. Changing this setting is NOT necessary for remote management in a domain environment, but it is in a workgroup environment (my home environment I’m using for this is domain based).  Here are the settings to change in dcomcnfg on the management client:

vpn9

Why this works is related to WMI/DCOM fallback, but I’m far from claiming to be an expert here and will walk swiftly away from any further explanation. However, I re-iterate, I absolutely do not recommend you change this setting unless you need to.
So let’s step back a bit now and try and understand a bit more about the DNS issue. The obvious thing to think may be to run “ipconfig /registerdns” from an elevated command prompt on the remote management machine to correct the DNS registration. Let’s see what happens, while at the same time running a network trace on the ISA server with a filter for just the DNS protocol. 

vpn10

If you’re ahead of the game, you may notice this is a very interesting capture! Maybe not if you’re aware of my home setup, so let me explain why. 192.168.15.2 is the external internet address of my ISA server (in turn connected to a VOIP router). The destination being resolved to a host with name starting ‘ns’ is my ISP’s DNS server. Looking at the frame details, you can see the packet is a DNS update request for the laptop. Unsurprisingly, if you look at the response from the ISP in packet 3949, the response is “NotAuth”. Afterall, they’re not authoritative for DNS of my domain. I am!

This routing to the external network through ISA is normal expected behaviour. So I’m still yet to find a good solution. But all is not lost (of course). Let’s take a different tactic and look a little closer at the Vista SP1 inbox VPN client configuration (as in one which hasn’t been created by what-ever the equivalent of CMAK, or Connection Manager Administration Kit, for Vista – and no, I’ve no idea what the replacement technology is. But it does remind me to do some research for another day....).

I’m assuming you’re already familiar with configuring a PPTP or L2TP VPN connection in Vista – that’s a little outside of the scope of this post. But here’s the IPv4/Properties/Advanced/DNS dialog tab of the VPN connection I’ve created to connect back to my home network. Look at the bottom three items relating to DNS registration:

vpn11

Hmmmm. These look extremely promising . Logically, it sounds like I want all three: I want to specify a DNS suffix for this connection which is that of my internal domain; Yes, I want to register the connection’s address in DNS; and I’d like to use the DNS suffix in the DNS registration. So I changed it to look like this:

vpn12
After saving the changes, let’s run that DNS-filtered network trace on the ISA server again while re-establishing the VPN connection:

vpn13

Looks good as a DNS update was sent to the internal DNS servers, not to the external ISP. It shows the update for the IPv4 address of the remote management client as 192.168.200.4 with a success response in the following packet. And ipconfig on the client?

vpn14
This confirms the trace above – the remote management client has IP address 192.168.200.4. What about an nslookup of the laptop?

vpn15
Excellent. Everything is looking rosey – the DHCP assigned IP address of the laptop acquired from the VPN connection is in DNS on the internal servers. Therefore, the Hyper-V enabled server should be able to locate the laptop when making it’s DCOM callback, so let’s fire up Hyper-V manager and see what happens:

vpn16
Voila! Hope you found that useful.

Cheers,
John.

Comments
  • Thanks for the blog and utils. Still having a time of it. Running hyper-v server. I built the server at home with no domain. Got all the tools running finally using your tools, I did have to use the cmdkey utility to get the hyper-v management tool to work, even though both weren't in a domain. (vista x86 laptop)

    I moved the server back into the domain, and tried to go through all the same steps. Trying to manage it remotely through an SSL vpn from a non-domain laptop. I updated the domain dns to point requests for laptop back to the correct IP.

    I edited the hosts file on the server to do the same. I reran cmdkey. I can remotely connect with Computer Management and do most things without trouble. I get the RPC services cannot be contacted on my hyper-v server from the Hyper-V manager.

    I can RDP into the hyper-v box from vista

    I can use Computer Management from vista to Hyper-v

    hyper-v manager doesnt work though with the rpc error.

    Any clues?

    many thanks

  • I have had a heck of a time getting this going. I first built w/no domain and the only way i could get remote management going was to use cmdkey, even though both machines weren't attached to the domain.

    I then moved the hyper-v server into the domain, and tried to connect remotely from the same vista laptop and got the rpc errors. Connecting over an SSL VPN. I updated the internal dns to resolve the laptops hostname. Now i get, you do not have the required permissions.

    any ideas?

  • David A

    Does this work without the SSL VPN - with the info you've provided about it, it's not clear if that could be the cause. Do you have a way to connect on the same LAN directly to rule it out?

    For other troubleshooting Hvremote /show reveals all. If you can post that from both server and client side; PLUS: ipconfig /all on both machines; PLUS: results of attempt to ping by name the client from server and visa-versa, it will give me the info I need to determine if it's an external factor.

    cmdkey should only have been necessary in an all workgroup config if the usernames and passwords didn't match.

    To be clear though on your current scenario - you're now attempting to connect to a domain joined server from a workgroup vista client. It would be helpful if you could confirm if cmdkey is set correctly to authenticate with domain credentials to the server and that domain account has been granted remote access on the server.

    Thanks,

    John.

  • Thanks a lot. After following the instructions I still got access denied (connecting workgroup Vista to domain HyperV) until I used the "cmdkey /add:ServerComputerName /user:ServerComputerName\UserName /pass" command. Instead of servername\username in the /user: section I use domain\username - and then it all worked.

    Thanks

    Mark

  • A comment - this problem might also occur if the client machine is assigned a new IP address and the Hyper-V server still uses the old IP address. In that case you might want to run ipconfig /flushdns on the Hyper-V host.

    Cheers

  • Hello everyone! Very nice and helpfull post. My config is a windows 7 domain client and a hyper-v domain server. When I'm inside of our company netwok everthing is fine with the remote management of the hyper-v server. Now I am at home and connected with vpn over an ISA 2006. With the help of your articel it is possible to connect with the remote mmc to the hyper-v server. After I take the neccessary changes for ISA Server 2006 (disable RPC filter), I get the hyper-v server in the mmc, can create new guest systems, check virtual harddrives and so on. But I don't see any virtual guest system in the middel console window. There is only a message "RPC Server unavaible. Unable to establish RPC connection between Server and Client." I think it is the same problem like Paul descibes above. Perhaps somebody can help me.

  • Thanks a lot! I am also having the excact same problem as TPickhan. I can edit the settings in Hyper-V, but i cannot view any of hte machines and am getting the RPC error.

  • I've tried everything, and I'm still having these errors even though I'm on the same LAN (no VPN) between the Hyper-V management client and the Hyper-V Server 2008.  I'm in an all workstation environment without a Windows Domain Controller (just a consumer grade router with DNS on it).

    Both machines can resolve each other just fine on the local LAN though it is not via DNS but broadcast discovery.  I've even disabled the firewall on both machines, run the HVremote commands, and I still get these "unable to establish connection" errors.  I'm at wit's end here!

  • Hi John,

    I'm having the same problem as described above after following your suggestions. Actually, I also needed to flush dns on the Hyper-V server to pick up the newly-rtegistered addres and then the behaviour changed from the RPC error to the "No virtual machines were found on this server" error described by TPickhan and Steve above. I'm connnecting from Hyper-V Manager on Windows Server 2008 R2 Build 7100 to Hyper-V Server 2008 R2 Build 7100. RDP connects fine. All works normally when connected normally, the behavior only occurs over VPN. I can change Hyper-V server settings but can't see any of the child partitions over VPN.

    Thanks,

    Tristan

  • Tristan - curious, I've not heard of this having been reported before. I'd be interested to see the results of use wbemtest on the client to connect to \\server\root\virtualization and run a query "select * from msvm_computersystem". You should get multiple entries back - one for the server itself, and one for each VM configured on the system. Can you confirm also if you are using SCVMM in your environment?

    Thanks,

    John.

  • Hi John,

    Sorry about the delayed reply and thanks very much for your help. Yes, we are using SC VMM 2008 in our environment and I believe this server is VMM-managed, but I am not presently a user of VMM on this network so I can't confirm. I should be able to arrange tests if needed though.

    I tried the WBEMTEST query and I'm getting the seven results for the parent and the six children as you describe. The parent returns the server name and the children have GUIDs for names. Is this what you would expect? Can you suggest anything else?

    Cheers,

    Tristan

  • Oh, and I thought I should mention that I disabled the DNS cache (the DNS client service) on the Hyper-V server in order to eliminate the need for DNS flushing

  • Tristan - I confess to now knowing whether disabling that service has that effect. I'd still be sure though that it is a case of the server not being able to accurately resolve the IP address of the client. The wbemtest queries above only test client to server.

    Can you try pinging server to client -4 byname to verify it hits the right address? Maybe worth also running a netmon trace on the server while connecting to verify that the server is attempting to callback to the right IP address of the client.  

    To confirm also - are you saying that wbemtest on VPN returns instances for each of the VMs, but Hyper-V Manager shows no virtual machines, and no error? However, on non-VPN, using the same account, you get a list of VMs back in both Hyper-V manager and wbemtest? VMM puts VMs into scopes which are not generally accessible unless you are authenticating as the delegated user in VMM, or the local administrator on the Hyper-V machine itself. I'm wondering if it's actually working and not returning any virtual machines simply due to the use of VMM. If you can confirm there is a inconsistency using the same account in the VPN/non-VPN case.

    Thanks,

    John.

  • Hi John,

    Yeah, stopping (disabling) the DNS Client service has that effect: http://support.microsoft.com/kb/318803

    I found it was necessary to make this change on the Hyper-V server so that it would always pick up the newly registered IP address of my client machine when it registers its new address after logging on via VPN. Previously it would still have the old non-VPN address in its cache. This step is merely a convenience to avert the need for an IPCONFIG /FLUSHDNS on the Hyper-V server immediately after connecting the client to the VPN. I follow that it is independent from the WBEMTEST results.

    The ping tests confirmed that it was resolving to the old non-VPN IP address *prior to the DNS cache fix I describe above*. After making this fix the behaviour switched from "RPC server unavailable" error to not enumerating the VMs in the console. The scenario is as you describe above. WBEMTEST enumreates the child VMs and the root while Hyper Manager only allows managment of the root settings. There is no error per se. Hyper-V Manager attempts to connect. It quicky connects to the root and the Actions menu changes to reveal the new actions accociated with the root partition. However, the child partitions are never enumerated in the Virtual Machines pane. After about half a minute or so (from recollection) the "No virtual machines were found on this server" message appears.

    When connecting normally this isn't a problem. This is only a problem when connecting over VPN.

    Unfortunately I'm not sure how I'll run Netmon from the Hyper-V server since it is the Hyper-V Server edition, and therefor Core. Maybe that's possible and I'm just not aware of how to do it???

    I think your VMM suggestion makes sense. I'll persue that line of troubleshooting and let you know how we get on, as we have only recently introduced SC VMM on this network.

    Thanks again for all of your help with this.

    Cheers,

    Tristan

  • Tristan - another option which may help diagnosis is to turn on tracing for Hyper-V Manager. If you email me using the link at the top, I can forward on instructions.

    Thanks,

    John.

Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment