VMware Networks, Bridged vs. Nat vs. Host

I have a customer out in Ohio who has had a horrible year. His software vendor convinced him to buy a $6,000 server to run their new Windows version of their software; he had all kinds of Windows configuration and hardware problems; his users hated the new software; he finally gave up and went back to the ancient, unsupported SCO Unix version.

What to do with the $6,000.00 server? I suggested putting Linux and VMware on it - heck, it's a big, powerful box, it seems a shame to have it go to waste (and we both thought that running Windows 2003 Server was definitely a "waste").

So he did. And immediately ran into all kinds of networking problems. Unfortunately, most of times when he called me about it, I was on the road or otherwise tied up and couldn't help him much. Add to that is that I am no VMware expert, so he was getting nowhere. He basically had everything working, but the VMware Windows instance couldn't talk to the Linux host that ran the VM ware. It could talk to everything else, but not its own host. He tried fixing it, but made things worse and when he called me Saturday afternoon, Windows wasn't talking to much of anything.

My poor wife.. we have our daughter and son-in-law coming Monday morning to stay for the week; we are pretty much ready but there are still a few things she needs me to do, and now I'm tied up on the phone. She knows customers have to come first but sheesh - I'd already been out in the morning to help a neighbor and now this? She sighed..

I sshed to the box while keeping my customer on the phone. As I said, I'm no VMware expert, but I saw a few things that bothered me. First of all, he had 4 NIC's in the machine. Given the size of his business and the network traffic, I saw no reason for that and I had already realized that he was confused as to which card was which. I asked him to cut it back to two cards: one for the inside, private IP, one for its public interface. That would make it much easier to figure out where cables needed to go.

The second thing I realized as I looked things over was that eth0 was configured to 192.168.1.4 and that eth1 was 192.168.1.3. As he was going to disable eth1, I would need to add an alias for .3m, so I added "ifconfig eth0:1 192.168.1.3" to rc.local. However, there was something more that I had not caught on to in our phone conversations: the Windows machine was configured to be 1.4 also.

You can do that with VMware, but it's not the default. By default, VMware works in "bridged" mode - you use the 192.168.1.4 card, but you put a free address on the Windows virtual card - say 1.5 or 1.6. That's why Windows couldn't talk to 1.4 or 1.3, it had to go through eth0 to do that. Frankly, I'm surprised it could talk to other IP's on the 192.168.1 network.

Bridged mode lets the virtual machine share the host's Ethernet connection, while appearing as a separate machine with its own MAC and TCP/IP address. NAT mode nats through the hosts NIC, much like the 192.168.1.x machines nat through your router to the internet (yes, yes, I know what the deep networking geeks are about to complain about: nat vs. masquerading). The VMware server assigns DHCP addresses to the virtual network cards, and the NAT system takes it from there through the real NIC (though that in turn is obviously natting or masquerading itself). Finally, Host mode only allows the virtual machine to talk to the hosting machine and other virtual machines configured the same way, but nothing else.

After making the changes and resetting Windows to be .5, everything worked as it should. If he wants to add back the other NICS, he can, but at least he's starting from a known working configuration.