One of the questions we’re frequently asked by clients is how to do hot replication of running VMware virtual machines (VMs) without spending a small fortune on VMware management tools. The good news is that it’s possible—the bad news is that it requires a little work on your part.

The general concept is to gain Secure Shell (SSH) access to the ESX/ESXi host machine [1], use the command-line tools and standard *nix automation tools to create a clone of the running virtual machine and to copy it to another host, and then to add it to the inventory of the second host machine. This leaves the VM in a state that is similar to the situation in which a VM crashes or hangs, such as when the guest operating system detects that something went wrong and attempts to do a file system check or database transaction log replay. This obviously is not as desirable as properly shutting down the guest operating system and ensuring that the entire system is in a consistent state, but that is not possible for mission-critical systems in many cases.

The following is a simple *nix shell script illustrating this concept:

#!/bin/bash

# ESXi hosts
HOSTS=(10.150.100.16)

# For each host, indicate vm datastore name in order
DS=(datastore1)

# Backup datastore
BDSTORE="x.x.x.x::vmbak-casc-dc"

for (( i = 0 ; i < ${#HOSTS[@]} ; i++ )) do
 host=${HOSTS[$i]}
 dstore=${DS[$i]}

 # Get vms list
 ssh root@$host 'vim-cmd vmsvc/getallvms' > /tmp/vmlist-temp
 sed 1d /tmp/vmlist-temp > /tmp/vmlist

 # Iterate backup sequence
 exec 3</tmp/vmlist
 while read <&3; do
  line=$REPLY
  #echo "VM: $line"

  # Get VM id
  id=`echo "$line" | awk '{print $1}'`
  vmname=`echo "$line" | awk '{print $2}'`
  echo $id
  ts=`date +%y%m%d-%H%M`
  echo $ts
  # Create snapshot
  echo "ssh root@$host 'vim-cmd vmsvc/snapshot.create $id $vmname-$ts $vmname-$ts'"
  ssh root@$host "vim-cmd vmsvc/snapshot.create $id $vmname-$ts $vmname-$ts"

  # Copy snapshot to remote datastore
  #echo "ssh root@$host 'scp -i /.ssh/id_dsa -r /vmfs/volumes/${dstore}/$vmname $BDSTORE'"
  #ssh root@$host 'scp -i /.ssh/id_dsa -r /vmfs/volumes/${dstore}/$vmname $BDSTORE'

  # This is the rsync option
  echo "time ssh root@$host 'rsync -av /vmfs/volumes/${dstore}/$vmname $BDSTORE'"
  time ssh root@$host "rsync -av /vmfs/volumes/${dstore}/$vmname $BDSTORE"

  # Remove snapshot to merge back delta file
  echo "ssh root@$host 'vim-cmd vmsvc/snapshot.removeall $id'"
  ssh root@$host "vim-cmd vmsvc/snapshot.removeall $id"
 done
 exec 3>&-
done

If you have any questions about this approach, please don’t hesitate to contact us. We’d be glad to help and, if we can’t, we’ll get you in touch with someone who can. Send an email to info@cascadeo.com or call us at 206-577-1155. We are here to help!

[1] Note that ESXi has the SSH server disabled by default. It is possible to re-enable it, but it may violate VMware licensing rules and, as such, we can only formally endorse ESX for this solution. ESXi is known to work, however, with the SSH server re-enabled.