Recently, I was doing some Hyper-V physical hosts moving and I found that one Hyper-V host getting managed by VMM got in a state whereby it found itself “sick.” I describe sick meaning that Virtual Machine Manager 2008 R2 (VMM) could no longer manage the host (who was actually getting managed by another VMM instance) but as administrator I was unable to remove the host. Instead, the outcome was a physical host was shown in the legacy VMM instance as in Pending state like below -
If you attempt to remove the host, the VMM Service will crash within seconds of the job starting. You will receive the following information in the jobs detail after restarting VMM -
In short, the removal of this physical host isn’t possible using any technique within the VMM Administrators console or via a PowerShell command-let.
To help many of you understand how I put this server in the particular spot where it ended up being un-manageable yet un-capable of getting removed. I felt it important to explain how this server, called out as “Server Name” got into this state -
After this, I assumed that the “existing” VMM host would get removed from the old VMM instance or that I could manually remove it. This wasn’t the case though as when I attempted to remove the host removal process would start and within a few seconds it would crash VMM.
When using DebugView to do advanced debugging, the following was shown when this action was taking place -
For many of you, the root cause is less important and the resolution is the key to your happiness. Besides, this is the very reason you are reading this blog. I took sometime today to dig into how I could rid myself of this host and clean up my VMM environment. The following steps are used to remove yourself of the host.
NOTE: The first step you should do is backup your Virtual Machine DB before moving forward with this process. This will ensure that, in case of an undesirable outcome, you can restore your VMM instance. Please proceed at your own caution.
In this process, we will bounce around as we solve this complex problem but the end-result is you will now have the un-managable host out of VMM. To do this, we will alter the VMM database by doing the following -
This is an ordered approach as there are constraints all through the VMM VirtualManagerDB (as their should be to maintain data integrity) so it isn’t an easy 1..2..3 process.
The data you will need to collect from you broken host are the following -
The following steps must be followed exactly or you will receive errors that constraints have been violated. Start Here.
1: Select HostID,ComputerName from dbo.tbl_ADHC_Host
You should now locate the server name and the copy the HostID (you will use it a lot!).
This should clean up any virtual machine objects that are attached to this Host.
This will remove all references to the volumes that are available on this host, in my case it was the C:\ & D:\ drives.
This is where you will need to have the NIC MAC addresses from the host that is currently unable to be managed. This can be obtained using the ipconfig /all command in Windows.
Delete from dbo.tbl_ADHC_HostNetworkAdapter where MacAddress = '00:11:0A:5D:9C:06'
You should do this for each NIC that this server has in the table.
This removes any reference for this HostId for the Network (think Network Tags, etc.).
After all this, you’ve found your way to the last step and you should now be able to successfully delete the record related to the Host. To do this, go back to the first step we did earlier where you determined the HostId and now you can easily delete using the following SQL query -
Delete from dbo.tbl_ADHC_Host where HostId = 'e32a0641-0268-4ba5-b8fb-8249cce01642'
The command should execute completely and you’ve now cleaned up this rogue Hyper-V host. I mentioned, I hope, that this isn’t a supported method to do this as you shouldn’t be accessing the database directly but sometimes drastic times means drastic measures. The end result for my situation is that VMM is now clean and life is good!
Breathe. It’s over. This server is now history. You can now move on with life. In today’s post, I spent some time helping you understand how to forcibly remove a host from VMM 2008 R2. This is linear process that isn’t for the faint of heart and requires you to follow your detective-like leads and use them to obliterate this server from the VirtualManagerDB. Breathe. It’s over. This server is now history.
Brilliant article. Very clear and solved a major issue we were having with one of our clusters.