In this example we use VirtualBox 4.2 and Ubuntu Server 12.04 to host our own virtual servers. After playing with VirtualBox I thought it would be a good idea to host some virtual servers with VirtualBox in headless mode. After all this is quite the same as some sell you as Infrastructure as a Service (IaaS) or private cloud solution. To be honest to play with the big guys you have to find a solution how you can handle elasticity, scalability and automation.
For our homegrown private cloud solution we will install an Ubuntu Server which hosts our VirtualBox virtual machines and a load balancer which will distribute the work load to different virtual servers. The whole network setup should look like this:
- The VirtualBox server atlas with IP 10.0.0.50
- virtual server zeus1 with IP 10.0.0.71
- virtual server zeus2 with IP 10.0.0.72
- A load balancer which distributes our workload for port 80 (http) to zeus1 and zeus2
Install Ubuntu server as VirtualBox server
This server will be our VirtualBox server which hosts our virtual machines. Install Ubuntu Server with the following package.
- OpenSSH Server
For this example our VirtualBox server has the following IP address and hostname.
- hostname: atlas
- IP: 10.0.0.50
Change the network settings in /etc/network/interfaces. You may also adapt the gateway and dns-nameserver settings.
auto eth0
iface eth0 inet static
address 10.0.0.50
netmask 255.255.255.0
network 10.0.0.0
broadcast 10.0.0.255
gateway 10.0.0.1
dns-nameservers 10.0.0.1
Also change the hostname in /etc/hostname and don’t forget to update the /etc/hosts file.
After a fresh installation we need to to upgrade Ubuntu.
sudo apt-get update sudo apt-get upgrade
sudo apt-get install linux-headers-$(uname -r)
Install VirtualBox
Now we need to install VirtualBox. For more details see the official instructions. For Ubuntu 12.04 add the following lines to your /etc/apt/sources.list:
# # VirtualBox for Ubuntu 12.04 (Precise Pangolin) # deb http://download.virtualbox.org/virtualbox/debian precise contrib
Next we need to add the public key, so that we can install the VirtualBox package from a third party package provider.
wget -q http://download.virtualbox.org/virtualbox/debian/oracle_vbox.asc -O- | sudo apt-key add -
Now we can install the newest VirtualBox version from the VirtualBox package repository.
sudo apt-get update sudo apt-get install virtualbox-4.2
Also install the dkms package to ensure that the VirtualBox host kernel modules (vboxdrv, vboxnetflt and vboxnetadp) are properly updated if the Linux kernel version changes.
sudo apt-get install dkms
We should also install the VirtualBox extension pack which extends the functionality of the VirtualBox base package. Functionality like the VirtualBox Remote Desktop Protocol (VRDP). When you have problems with a virtual machine you can start the virtual machine with VRDP support and start a connection with rdesktop.
sudo VBoxManage extpack install Oracle_VM_VirtualBox_Extension_Pack-4.2.12-84980.vbox-extpack
Create the user vbox which will be used to start our virtual machines. The user must be in the follwing groups so that VirtualBox can access the cdrom and other devices.
sudo adduser vbox sudo usermod -a -G vboxusers,cdrom,sudo,dip,plugdev,dialout,lpadmin,adm vbox
The group vboxusers should be automatically created during the installation of VirtualBox.
Create a virtual machine
Now its time to create a Virtual machine with VirtualBox and export the virtual machine as OVA file. I created a Virtual machine from a Ubuntu server iso image. The easiest way is to create a virtual machine with VirtualBox on a computer with a graphical user interface.
Create a new virtual machine with the name zeus1, operation system type Linux and version ubuntu.
Select the amount of base memeory for the virtual machine.
Add a new virtual harddisk.
Select VDI as virtual disk image.
Select dynamically allocated disk space.
Select the virtual disk space.
Add the ubuntu server iso image as CD.
Change network to bridged mode (bridged adapter).
Boot up your virtual machine and start with the Ubuntu installation.
When you did not select the Open SSH server during the installation, install it now with apt-get for our virtual machine.
sudo apt-get install openssh-server
An easy way to shutdown a virtual machine is with the VirtualBox command line tool which can send an ACPI shutdown event to the virtual machine. (e.g. VBoxManage controlvm zeus1 acpipowerbutton). To handle these events we have to install the acpid package.
sudo apt-get install acpid
For a working ACPI configuration which can handle the power button event you have to edit /etc/acpi/events/powerbtn config file.
event=button[ /]power action=/etc/acpi/powerbtn.sh
To test the load balancing we also install the apache server.
sudo apt-get install apache2
We change now the network configuration to the following settings:
- hostname: zeus1
- IP: 10.0.0.71
Change the network settings in the file /etc/network/interfaces
auto eth0
iface eth0 inet static
address 10.0.0.71
netmask 255.255.255.0
network 10.0.0.0
broadcast 10.0.0.255
gateway 10.0.0.1
dns-nameservers 10.0.0.1
After we have changed the network settings also change the hostname to zeus1 in /etc/hostname and don’t forget to update the /etc/hosts file too.
Before we clone our image we have to remove some udev config files.
- Remove all entries in /etc/udev/rules.d/70-persistent-net.rules because the interfaces (eth0, etc.) are bound to the MAC address and after import of a virtual machine with VirtualBox the MAC address is changed. So when you import virtual machine and start it up the configured interface eth0 is not configured.
- Also remove all entries in in /etc/udev/rules.d/70-persistent-cd.rules. I had some problems to boot the virtual image. After I have removed this entries the virtual machine booted up without problems.
To test the acpi package, shutdown zeus1 with the poweroff command.
sudo poweroff
After the shutdown is complete, export the virtual machine zeus1 as OVA file.
Copy the exported OVA file to your virtualbox server. As you know we have installed openssh on our VirtualBox server (atlas). So you can copy the virtual machine with scp over the newtork.
scp zeus1.ova vbox@10.0.0.50:/tmp
Configure VirtualBox server
Login to our virtualbox server (atlas) with ssh as user vbox and import the virtual machine
ssh vbox@10.0.0.50 VBoxManage import /tmp/zeus1.ova
Now we clone our virtual machine zeus1 and create a identically one with the name zeus2.
VBoxManage clonevm zeus1 --name zeus2 --register
To show all available vms run “VBoxManage list vms”. The output should now display zeus1 and zeus2.
"zeus1" {fa974e98-6411-4768-9cd9-017176e47e81}
"zeus2" {81953acc-59d0-4fa9-87f1-42e08bbbdb78}
To start up our virtual server automatically we create for both virtual machines the start script vm_zeus1.sh and vm_zeus2.sh. You have to change the vm_name variable to the name of the VirtualBox machine. The following example shows the start script vm_zeus1.sh. Copy this script and change the vm_nam variable to zeus2 for the script vm_zeus2.sh.
#!/bin/bash
# VirtualBox vm name
vm_name=zeus1
# VirtualBox username
vbox_user=vbox
vbox_command="sudo -u $vbox_user -i VBoxManage"
case "$1" in
start)
echo "starting vm vm_name"
$vbox_command startvm --type headless $vm_name
;;
shutdown)
echo "sending ACPI shutdown to vm $vm_name"
$vbox_command controlvm $vm_name acpipowerbutton
;;
pause)
echo "pause $vm_name"
$vbox_command controlvm $vm_name pause
;;
resume)
echo "resume $vm_name"
$vbox_command controlvm $vm_name resume
;;
stop)
echo "savestate $vm_name"
$vbox_command controlvm $vm_name savestate
;;
*)
echo "usage: $0 (start|stop|pause|resume|shutdown|help)"
;;
esac
exit 0
The vm start script uses VBoxManage controlvm for operating with virtual machines. For more details what these commands do have a look in the official manual.
Install both start script vm_zeus1.sh and vm_zeus2.sh with update-rc.d. The scripts must start after the VirtualBox service vboxdrv and stop before it. So we add the start sequence number 25 and kill sequence number 10. Adapts this settings when you have problems during the startup or shutdown of the atlas server.
chmod a+x vm_zeus1.sh sudo cp vm_zeus1.sh /etc/init.d/ sudo update-rc.d vm_zeus1.sh defaults 25 10
Now you can make the same configuration steps with the virtual machine zeus2.
Configure virtual machine zeus2
Start up the virtaul server zeus2. Be sure that zeus1 is not running, because zeus2 have still the same IP as zeus1.
sudo /etc/init.d/vm_zeus2.sh start
Log into zeus2 with ssh and change the IP adresse and hostname.
- hostname: zeus2
- IP: 10.0.0.72
auto eth0
iface eth0 inet static
address 10.0.0.72
netmask 255.255.255.0
network 10.0.0.0
broadcast 10.0.0.255
gateway 10.0.0.1
dns-nameservers 10.0.0.1
Change the hostname in /etc/hostname to zeus2 and don’t forget to update the /etc/hosts file.
To test the acpi package, shutdown zeus2 with the poweroff command.
sudo poweroff
Now you can adjust some seetings with VBoxManage modifyvm when you need to allocate more memory or add an additional cpu.
Configure the load balancer
We use crossroads as our load balancer. You should considering to install the load balancer on a different Linux box, which is connected to the Internet and redirects the request to your virtual servers. But in this example I have installed crossroad on our Ubuntu Server atlas.
sudo apt-get install crossroads
Create a start sript /etc/init.d/crossroads.sh. This scripts redirects HTTP requests to 10.0.0.71 and 10.0.0.72. For configuration details look up the crossroads documentation.
#!/bin/sh
XR=/usr/bin/xr
SERVER="-s http:0:80"
BACKENDS="-b 10.0.0.71:80 -b 10.0.0.72:80"
ALGORITHM="-dl"
HTTP_FLAGS="-x -X"
TIMEOUTS="-t 10"
CHECK_CALLS="-c 10"
DEBUGGING="-v"
case "$1" in
start)
$XR $SERVER $BACKENDS $ALGORITHM $HTTP_FLAGS $TIMEOUTS $CHECK_CALLS $DEBUGGING 2>&1 | logger -t crossroads &
;;
stop)
killall $XR
;;
*)
echo "usage: $0 (start|stop|help)"
esac
exit 0
The last step is to install the load balancer start up script.
chmod a+x crossroads.sh sudo cp crossroads.sh /etc/init.d sudo update-rc.d crossroads.sh defaults
Start up the whole infrastructure
So now its time to startup the load balancer and the virtual servers.
sudo /etc/init.d/crossoads.sh start sudo /etc/init.d/vm_zeus1.sh start sudo /etc/init.d/vm_zeus2.sh start
Test the load balancer
Now when you connect to atlas (10.0.0.50) with a web browser you should be redirected to zeus1 (10.0.0.71) or zeus2 (10.0.0.72). To test the load balancer just shutdown one of the zeus vm’s and try to connect to http://10.0.0.50:80, now shutdown the other vm and start the previous stopped virtual machine. Refresh the browser and you should not notice any difference, expect that the loading time of the page is longer for the first time.
Enable remote desktop for a virtual machine
When a virtual machine behaves not as expected you can connect to the virtual machine over the remote desktop protocol. You have just to enable remote desktop for the specific virtual machine. First we have to shutdown the virtual machine an specify on which port the VirtualBox should listen for remote desktop connections.
sudo /etc/init.d/vm_zeus1.sh shutdown VBoxManage modifyvm zeus1 --vrde on --vrdeport 3389 --vrdeauthtype external sudo /etc/init.d/vm_zeus1.sh start
When the virtual machine is started yo can connect from your client with rdesktop to the VirtualBox server over the previous specified port. Add the parameter “-p -” so that rdesktop ask for a password.
rdesktop 10.0.0.50:3389 -u vbox -p -
Or just add the password in the command line so that rdesktop don’t aks for it.
rdesktop 10.0.0.50:3389 -u vbox -p your_password
After you have no use for the remote desktop connection you should disable the remote desktop option for this virtual machine.
sudo /etc/init.d/vm_zeus1.sh shutdown VBoxManage modifyvm zeus1 --vrde off
And now?
The next thing do would be to look how you can replace the human tasks with scripts. For example:
- You could write a script for cloning a virtual machine, change the IP address and add the new server to the load balancer.
- Write a script which adds or remove dynamically new virtual machines to the load balancer when the load increase or decreases.
I hope this post was useful and have fun by building your own homegrown private cloud.










