Hyper-V: MAC Address allocation and apparent network issues MAC collisions can cause

re: Hyper-V: MAC Address allocation and apparent network issues MAC collisions can cause

Polk — Wed, 16 Jul 2008 10:00:25 GMT

John, so if Hyper-V is immune to MAC duplication, how did you get the timeouts on MAC collision ping in the very first screenshot?

re: Hyper-V: MAC Address allocation and apparent network issues MAC collisions can cause

jhoward — Wed, 16 Jul 2008 17:51:08 GMT

Polk - yes, good question and one which I didn't make clear above. I was using a second server running the Hyper-V role with a virtual machine on it configured with a duplicate MAC.

Thanks,

John.

re: Hyper-V: MAC Address allocation and apparent network issues MAC collisions can cause

omar — Thu, 17 Jul 2008 02:14:29 GMT

So one question I cant seem to get answered- If we leave a Hyper-V guest network as dynamic MAC- will it ever change if that system is rebooted or shutdown, etc?

I have had issues with two Hyper-V servers on the same LAN with several guests a piece and I have had broadcast storms and strange issues that have yet to be explained but when I run an arp -a command all the host and guest have different MACs.

I was going to restart the hosts and see if these MACs remain the same or not.

re: Hyper-V: MAC Address allocation and apparent network issues MAC collisions can cause

jhoward — Thu, 17 Jul 2008 02:24:23 GMT

Omar - no, the dynamic MAC address once allocated will not change, so if all VMs have unique MAC addresses, I wouldn't expect to see what you are seeing. Is it possible that there are other Hyper-V boxes elsewhere on the network which are conflicting?

Thanks,

John.

re: Hyper-V: MAC Address allocation and apparent network issues MAC collisions can cause

Craig — Fri, 18 Jul 2008 15:57:25 GMT

Here is any idea, MS design a product that doesn't have these limitations. You don't have to worry about MAC duplication in a VMWare ESX environment. This is just typical MS "features" that are only half baked, but the MS bigots can see no faults.

re: Hyper-V: MAC Address allocation and apparent network issues MAC collisions can cause

jhoward — Fri, 18 Jul 2008 19:50:33 GMT

Craig

Thank you for your comment. Unfortunately, without a higher management layer spanning multiple ESX servers (eg Virtual Center server), ESX suffers from exactly the same problem. Of course, feel free to correct me if you believe me to be incorrect. I am more than open to a discussion - if you want to take this offline, you can email me directly using the email link at the top of my blog.

The reasons I believe you are incorrect are based on documentation from both VMWare, Cisco plus a thread on the VMWare communities forum:

VMWare Infrastucture 3 Online library - ESX Server 3 Edition (http://pubs.vmware.com/vi35/wwhelp/wwhimpl/js/html/wwhelp.htm)

- Expand nodes out through ESX Server 3 Configuration Guide

- Advanced Networking

- Setting up MAC Addresses

- MAC Address Generation.

The particular part is at the end of fourth paragraph: "The algorithm puts a limit on the number of running and suspended virtual machines at any one time on any given server. It also does not handle all cases when virtual machines on distinct physical machines share a subnet."

The full section:

MAC Addresses Generation

Each virtual network adapter in a virtual machine is assigned its own unique MAC address. A MAC address is a six-byte number. Each network adapter manufacturer is assigned a unique three-byte prefix called an OUI (Organizationally Unique Identifier) that it can use to generate unique MAC addresses.

VMware has the following OUIs:

• One for generated MAC addresses.

• One for manually set MAC addresses.

• One that was previously used for legacy virtual machines, but is no longer used with ESX Server 3.

The first three bytes of the MAC address that is generated for each virtual network adapter are comprised of the OUI. This MAC address-generation algorithm produces the other three bytes. The algorithm guarantees unique MAC addresses within a machine and attempts to provide unique MAC addresses across machines.

The network adapters for each virtual machine on the same subnet should have unique MAC addresses. Otherwise, they can behave unpredictably. The algorithm puts a limit on the number of running and suspended virtual machines at any one time on any given server. It also does not handle all cases when virtual machines on distinct physical machines share a subnet.

The VMware Universally Unique Identifier (UUID) generates MAC addresses that are checked for any conflicts. The generated MAC addresses are created by using three parts: the VMware OUI, the SMBIOS UUID for the physical ESX Server 3 machine, and a hash based on the name of the entity that the MAC address is being generated for.

After the MAC address is generated, it does not change unless the virtual machine is moved to a different location, for example, to a different path on the same server. The MAC address in the configuration file of the virtual machine is saved. All MAC addresses that have been assigned to network adapters of running and suspended virtual machines on a given physical powered-off virtual machine is not checked against those of running or suspended virtual machines. It is possible but unlikely that when a virtual machine is powered on again, it can acquire a different MAC address. This acquisition is due to a conflict with a virtual machine that was powered on when this virtual machine was powered off.

VMWare Infrastructure 3 in a Cisco Network Environment http://www.cisco.com/en/US/docs/solutions/Enterprise/Data_Center/vmware/VMware.html#wp696203

Without quoting the whole section, the relevant part under vNIC MAC Addresses, Bootup, VMotion Migration states "The vNIC MAC addresses include the Organization Unique Identifiers (OUI) assigned by IEEE to VMware. The ESX host and the configuration filename information is used to create a vNIC MAC address. The OUIs used by VMware are 00-50-56 and 00-0c-29. The algorithm used to generate the MAC address reduces the chances of a MAC address collision, although the process cannot guarantee a MAC address is unique"

Finally, on the VMWare communities site http://communities.vmware.com/thread/147630 this thread indicates how you configure a higher layer management server spanning multiple servers (Virtual Centre Server) using vpx generated MAC addresses.

This is the reason I mentioned in my article why managing MAC addresses spanning multiple virtualization servers (eg using SCVMM 2007 for Hyper-V/Virtual Server and ESX, Virtual Centre Server for ESX) is important.

Cheers,
John.

re: Hyper-V: MAC Address allocation and apparent network issues MAC collisions can cause

Bryan — Sat, 26 Jul 2008 01:18:17 GMT

So is it possible for the host machine to be a kind of "NAT" box, therefore the guest OS uses the physical NIC MAC address instead of it's own dynamically created one?

re: Hyper-V: MAC Address allocation and apparent network issues MAC collisions can cause

jhoward — Sat, 26 Jul 2008 01:50:24 GMT

Bryan - No, this is is not possible.

Thanks,

John.

re: Hyper-V: MAC Address allocation and apparent network issues MAC collisions can cause

�� — Sun, 23 Nov 2008 20:34:26 GMT

�� . �� . �� .

re: Hyper-V: MAC Address allocation and apparent network issues MAC collisions can cause

Chris — Wed, 04 Feb 2009 15:33:36 GMT

Thanks for the article John, I now understand what has been happening with the MAC address generation on my sysprep cloned hyper-v servers.

It seems that these registry entries were just copied from the template system, the MAC addresses as a result were all based on the template systems original IP address and were starting to come out with the same MAC addresses on different clones.

It looks like hyper-v doesn't currently check dynamically whether these max and min entries still match the current machine address.

re: Hyper-V: MAC Address allocation and apparent network issues MAC collisions can cause

jhoward — Wed, 04 Feb 2009 19:40:20 GMT

Chris - yes, that would be correct. We don't have a sysprep provider implemented in Hyper-V v1 (2008), so you would have clashing ranges in each of the clones.

Thanks,

John.

re: Hyper-V: MAC Address allocation and apparent network issues MAC collisions can cause

Isambert Patrick — Sat, 07 Feb 2009 19:26:59 GMT

When connecting two physical machines with hyper-V for a demo (ie one network card shared for physical and virtual machine), ipconfig/all say that the two notebooks have the same MAC Address for physical computer.

To be sure, we have made a reservation in DHCP. When the first boot, it takes the reserved IP address. When the second boot, it take the same ip address....

Do you know this problem ? How can we fix it ?

re: Hyper-V: MAC Address allocation and apparent network issues MAC collisions can cause

jhoward — Mon, 09 Feb 2009 21:39:52 GMT

Isambert - I would not expect two physical computers to have the same MAC address - very unlikely. I can think of one situation where it is possible due to a bug in Hyper-V for the virtual NIC in the parent partition to take on a MAC address from the internal pool MACs for VMs that we maintain.

Can you confirm the MAC addresses which are clashing start 00-15-5D (the default range for our pool); that these are virtual NICs rather than physical nics in the parent (ipconfig /all will list the adapter as Microsoft Virtual Network Switch Adapter #n); the steps in Hyper-V Manager you went through to create the external virtual network switch (I suspect you created an internal virtual network, then changed it to external).

Thanks,

John.

re: Hyper-V: MAC Address allocation and apparent network issues MAC collisions can cause

RMiller — Wed, 18 Mar 2009 09:24:02 GMT

I am experiencing this issue within our Hyper-V Cluster (4 servers) running about 25 VM's. When we had the network down for maint, the cluster shut down and saved the VM's state. When the network was restored, we experienced intermittent issues with access to VM's and even the Hyper-V Parent Partition. We have MAC management set to automatic. The VM's primarily use the same subnet.

It appears that based on the above comments, we should be manually managing the MACs (BUMMER! )

re: Hyper-V: MAC Address allocation and apparent network issues MAC collisions can cause

Sachin — Mon, 06 Apr 2009 20:46:30 GMT

John-

Thanks for the article.

Also, can you let me know or point to some article which talk about where are these MAC address for VMs stored in WMI of Hyper V server.

re: Hyper-V: MAC Address allocation and apparent network issues MAC collisions can cause

jhoward — Mon, 06 Apr 2009 21:39:34 GMT

Sachin - take a look at the WMI documentation for msvm_virtualswitchmanagementservice http://msdn.microsoft.com/en-us/library/cc136938(VS.85).aspx.

Specifically the CreateInternalEthernetPort* methods.

Thanks,

John.

New in Hyper-V Windows Server 2008 R2 Part 2 – MAC Spoofing

John Howard - Hyper-V and virtualization blog — Thu, 21 May 2009 21:42:56 GMT

Our Virtual Switch got smarter in Windows Server 2008 R2. In Windows Server 2008, VMs are susceptible

New in Hyper-V Windows Server 2008 R2 Part 2 – MAC Spoofing

News — Sun, 24 May 2009 17:51:44 GMT

Our Virtual Switch got smarter in Windows Server 2008 R2. In Windows Server 2008, VMs are susceptible

New in Hyper-V Windows Server 2008 R2 Part 2 – MAC Spoofing

HyperVoria — Tue, 26 May 2009 09:13:54 GMT

John Howard: Our Virtual Switch got smarter in Windows Server 2008 R2. In Windows Server 2008, VMs are susceptible to MAC spoofing. MAC spoofing is where a (generally) malicious machine pretends to be another machine on a network (there are legitimate