This is not really a CCIE topic, but it is the sort of thing that you should be prepared for in real life. Microsoft NLB – “Network Load Balancing”. This allows an application to be served by multiple servers.
The way it works is by using layer-2 multicasts. When a client wants to talk to a server, (or in this case a virtual server) it puts out an ARP request for the server’s IP address. The server (one or both, I don’t know) responds with a multicast MAC address. From then on each frame from the client to the application is addressed to the multicast MAC address.
There are a number of things to consider:
- The servers generate IGMP for the IP group corresponding to the MAC multicast address. If the switch is running IGMP snooping, then this ensures that the multicast frames are sent to the servers and nowhere else. If the switch is not running IGMP snooping, then the frames are flooded to all ports on the VLAN – the scheme still works, but at the expense of flooding all the client-to-server traffic.
- IGMP snooping filters only those packets that are strictly IP, i.e. the ones that have EtherType 0×0800. There is also a keepalive between the servers, also addressed to the multicast MAC destination, at a rate of 2 packets per second per server. The Ethertype is 0×886F. These are flooded to all ports on the VLAN, regardless of IGMP snooping.
- If you think about it, this is not really Network Load Balancing, but CPU load balancing. All client frames go to both servers, and then the servers decide between themselves which packet each server is handling, and which are left to the partner.
- It does not work too well through a router. When a router gets a MAC address in an ARP response, it does not believe it, so it discards it. The only way I have found to get round this is with a static ARP entry in the router.
- Even if you do put a static ARP entry in the router, does it balance the load from the router? I suppose it depends what algorithm the servers use to distribute the load. If it is based on the source MAC address, then it won’t work to well through a router! On the other hand, if it is based on the source IP address, then that means that both servers have to process all packets all the way up to layer-3. The devil and the deep-blue sea.
I have seen many posts by people trying to make windows NLB work but never any true here’s how you do it right write-ups. I have seen people with sucess with cobbled hubs and unicast but never anything good. Seems that a Cisco 3750 and servers set to Multicast with IGMP would work just fine out of the box (since IMGP is enabled by default). Any other good sources of how to on this subject?
Comment by cciewannabe — 02 Oct 2008 @ 17:53
I’m afraid I don’t know any good sources for this. I learned about it the hard way.
As for the 3750 out of the box, that should work on the single VLAN. That is, the router will do its bit for the IGMP, and so IGMP snooping will work. But beware of trying to access the servers from another VLAN – in that case I found you need the static ARP entry as per #4.
Comment by dorreke — 02 Oct 2008 @ 22:39
Microsoft’s load balancing is not compliant with the ARP RFC because they reply to ARP requests with a multicast MAC. Cisco IOS does not accept ARP replies containing a multicast MAC, preventing it from being installed in the ARP table. Without a resolved ARP entry, traffic will not be routed to the NLB cluster IP.
As outlined in earlier comments, IOS does permit a static mac to be installed.
Comment by Phillip — 30 Oct 2008 @ 03:39
Am I right that microsoft NLB Clustering software use packets with multicast MAC and unicast IP?
And this is a real problem for router when it try to populate ARP table…
can someone capture traffic to analise etehrnet and IP headers.
Comment by Daniel — 21 Nov 2008 @ 10:14
That is right. I have have not found any way to persuade a router to accept a multicast MAC in an ARP response. The best I can do so far is to configure static ARP entries for each NLB service. That is not as difficult as it sounds, because the MAC adress is directly related to the unicast IP address. Specifically, the MAC address is 0100.5e7f.xxxx, where xxxx corresponds to the last two octets of the unicast IP address (trsnslated into hex of course).
Comment by dorreke — 21 Nov 2008 @ 10:42
I’m wondering what are the differences between Multicast and IGMP Multicast when configuring MS NLB? What I know is when choosing Multicast, on the 3750 switch, ’show ip igmp snooping groups’ shows nothing. I’m hoping that by selecting IGMP Multicast, and the fact that IGMP Snooping is enabled by default, I would be able to see a new group entry when issue command ’show ip igmp snooping groups’.
Comment by Vincent — 14 Jan 2009 @ 01:07
I have not tried the two configuration options, but I guess it is simply multicast with or without IGMP.
If you have IGMP snooping configured on your switches, then the IP multicast will only go to those ports that have expressed an interest in the multicast by sending an IGMP, plus the router ports). So, if you have IGMP snooping, then you will have to use the IGMP multicast option, otherwise the servers will no see each other. Put it this way: “IGMP snooping” places a filter on the multicast forwarding, and the way to punch through the filter is to generate an IGMP packet.
OTOH, if you do not have IGMP snooping enabled on your switches, then all multicasts will be flooded anyway, so there is no need for the servers to generate the IGMP packets, so use the non-IGMP multicast option.
So you are right: if you use the IGMP multicast option, and you have IGMP snooping on the switches, you should see the servers in the “show”.
Comment by dorreke — 14 Jan 2009 @ 14:30
Hey guess what, I’ve configured IGMP Multicast option on MS NLB and sure enough, nothing comes up on my Cat 3750 switch that shows the switched auto detected the IGMP Multicast. I ended up having to configure two commands on the switch for NLB to work properly. The first is to configure the static MAC address of the NLB group and the appropriate forwarding interface/s. Once this is done, configure the switch (Cat 3750 L3 switch) to ARP the IP and MAC addresses of the NLB group. The commands are:
- arp 10.10.51.89 0100.5e7f.3359 ARPA alias
- mac-address-table static 0100.5e7f.3359 vlan 15 interface GigabitEthernet2/0/5 GigabitEthernet1/0/5
I think it might be that MS is not using standard IGMP Multicast hence the switch cannot detect the address. Thank for your help anyway.
Comment by Vincent — 21 Feb 2009 @ 09:19