We have seen a lot of cases of customers trying to view the Virtual Machine Manager Self-Service Portal (SSP) UI from a computer running Windows Server 2008 R2 or Windows 7. If the portal is running VMM 2008, it is likely that you can't view the connections to a Virtual Machine and you are getting error "Virtual Machine Manager failed to connect to the virtual machine because the guest operating system's computer name XXX could not be resolved by the Domain Name System."
This error occurs because Windows 7 and R2 were not certified as client machines for the SSP of VMM 2008. VMM 2008 R2, which is in beta right now (beta is available from http://connect.microsoft.com) does support Windows 7 and Win2k8 R2 as a client of the SSP. The actual problem is because we are not identifying credssp correctly and we are trying to connect to the VM via regular RDP through its computer name (In this case the VM does not have a computer name that we can resolve with DNS).
To correct this issue, you can either deploy VMM 2008 R2, or you can simply copy two files from an VMM 2008 R2 installaton into your VMM 2008 installation. You just need to update the ActiveX that we use to enable remote connections to the VM to work. Here is how:
1. On your web server where the SSP is installed, open the location %programfiles%\Microsoft System Center Virtual Machine Manager 2008\wwwroot\activex
2. Replace the vmmctlx_amd64.cab and vmmctlx_i386.cab files with the versions from our VMM 2008 R2 Beta (you need to install this beta on one computer to extract these files. Only the installation of the SSP will put these files on disk on the same location as listed above).
a. You can install our VMM2008 R2 Beta from Microsoft Connect
3. Your Self Service users will then have to go to their “Programs and Features” and uninstall the “Virtual Machine Manager Self-Service Client”. Then when they view the SSP website again, the newer version of the ActiveX will be installed and the VM console connection should work