Once upon a time a company I worked for was migrating their systems to VMware in a consolidation effort. They had a SAN, they had a BladeCenter, and they had VMware. And we were all pretty pumped about the VMware, let me tell you. It was like the second coming.
But there was a problem, you see. We were using RedHat Cluster on Debian Sarge. And RedHat Cluster didn't support fencing machines through VMware. What's fencing, you ask? Well, in cluster-land, when a node becomes unreachable or out of sync with the others, it is 'fenced'. This is a mechanism to ensure data integrity by forcably removing the node from the cluster and the cluster storage. In RedHat Cluster, this can be achieved by either removing the node from the storage in the switch, or by rebooting the node via a LOM card or a addressable power supply. If we did either of those in VMware, we'd take everyone else down with us, and that would be very, very bad.
Well luckily for us, VMware provides a Perl API to use to connect to the host machine. I used this API to modify the fence_apc script, which is used to telnet to an APC Masterswitch and fence a node, to a version that would work for VMware Server or ESX.
First you must set up a fence.ccs file.
fence_devices {
esxblade1 {
agent = "fence_apc"
ipaddr = "10.0.0.1"
login = "fence"
passwd = "password"
}
esxblade2 {
agent = "fence_apc"
ipaddr = "10.0.0.2"
login = "fence"
passwd = "password"
}
}
This defines two fence devices, esxblade1 and esxblade2. It lists their IP addresses, a login ID and a password, and the agent to use, which in this case is fence_apc. I placed my script in /sbin/fence_apc and moved the original out of the way to make sure there wouldn't be a problem. The login ID and password should be an account on the VMware host system that has access to reboot, startup, and shutdown the VMs in question.
Next, we create a nodes.ccs file.
nodes {
node1 {
ip_interfaces {
eth0 = "10.1.0.1"
}
fence {
vmware {
esxblade2 {
port = "/home/vmware/node1.vmx"
}
}
}
}
node2 {
ip_interfaces {
eth0 = "10.1.0.2"
}
fence {
vmware {
esxblade1 {
port = "/home/vmware/node2.vmx"
}
}
}
}
}
This specifies the public interfaces for two machines, known as node1 and node2. It also defines a fencing mechanism for node1 and node2, vmware. The vmware fence script takes as its argument a 'port', which is really the path on the host system's filesystem to the .vmx file for the virtual machine.
The script for fence_vmware is available here.
You should be able to use this, in conjunction with the documentation for RedHat Cluster, to set yourself up a test VMware cluster. If not, feel free to shoot me an email and I'll try and help you out.
