Build your own private cloud solution with Ubuntu Server and VirtualBox

In this example we use VirtualBox 4.2 and Ubuntu Server 12.04 to host our own virtual servers. After playing with VirtualBox I thought it would be a good idea to host some virtual servers with VirtualBox in headless mode. After all this is quite the same as some sell you as Infrastructure as a Service (IaaS) or private cloud solution. To be honest to play with the big guys you have to find a solution how you can handle elasticity, scalability and automation.

For our homegrown private cloud solution we will install an Ubuntu Server which hosts our VirtualBox virtual machines and a load balancer which will distribute the work load to different virtual servers. The whole network setup should look like this:

  • The VirtualBox server atlas with IP 10.0.0.50
  • virtual server zeus1 with IP 10.0.0.71
  • virtual server zeus2 with IP 10.0.0.72
  • A load balancer which distributes our workload for port 80 (http) to zeus1 and zeus2

Install Ubuntu server as VirtualBox server

This server will be our VirtualBox server which hosts our virtual machines. Install Ubuntu Server with the following package.

  • OpenSSH Server

For this example our VirtualBox server has the following IP address and hostname.

  • hostname: atlas
  • IP: 10.0.0.50

Change the network settings in /etc/network/interfaces. You may also adapt the gateway and dns-nameserver settings.

auto eth0
iface eth0 inet static
        address 10.0.0.50
        netmask 255.255.255.0
        network 10.0.0.0
        broadcast 10.0.0.255
        gateway 10.0.0.1
        dns-nameservers 10.0.0.1

Also change the hostname in /etc/hostname and don’t forget to update the /etc/hosts file.

After a fresh installation we need to to upgrade Ubuntu.

sudo apt-get update
sudo apt-get upgrade
Install the kernel headers for the current kernel version. The command “uname -r” will give you your current kernel version. The kernel headers are needed for the VirtualBox kernel modules.

sudo apt-get install linux-headers-$(uname -r)

Install VirtualBox

Now we need to install VirtualBox. For more details see the official instructions. For Ubuntu 12.04 add the following lines to your /etc/apt/sources.list:

#
# VirtualBox for Ubuntu 12.04 (Precise Pangolin)
#
deb http://download.virtualbox.org/virtualbox/debian precise contrib

Next we need to add the public key, so that we can install the VirtualBox package from a third party package provider.

wget -q http://download.virtualbox.org/virtualbox/debian/oracle_vbox.asc -O- | sudo apt-key add -

Now we can install the newest VirtualBox version from the VirtualBox package repository.

sudo apt-get update
sudo apt-get install virtualbox-4.2

Also install the dkms package to ensure that the VirtualBox host kernel modules (vboxdrv, vboxnetflt and vboxnetadp) are properly updated if the Linux kernel version changes.

sudo apt-get install dkms

We should also install the VirtualBox extension pack which extends the functionality of the VirtualBox base package. Functionality like the VirtualBox Remote Desktop Protocol (VRDP). When you have problems with a virtual machine you can start the virtual machine with VRDP support and start a connection with rdesktop.

sudo VBoxManage extpack install Oracle_VM_VirtualBox_Extension_Pack-4.2.12-84980.vbox-extpack

Create the user vbox which will be used to start our virtual machines. The user must be in the follwing groups so that VirtualBox can access the cdrom and other devices.

sudo adduser vbox
sudo usermod -a -G vboxusers,cdrom,sudo,dip,plugdev,dialout,lpadmin,adm vbox

The group vboxusers should be automatically created during the installation of VirtualBox.

Create a virtual machine

Now its time to create a Virtual machine with VirtualBox and export the virtual machine as OVA file. I created a Virtual machine from a Ubuntu server iso image. The easiest way is to create a virtual machine with VirtualBox on a computer with a graphical user interface.

Create a new virtual machine with the name zeus1, operation system type Linux and version ubuntu.

Select the amount of base memeory for the virtual machine.

Add a new virtual harddisk.

Select VDI as virtual disk image.

Select dynamically allocated disk space.

Select the virtual disk space.

Add the ubuntu server iso image as CD.

Change network to bridged mode (bridged adapter).

Boot up your virtual machine and start with the Ubuntu installation.

When you did not select the Open SSH server during the installation, install it now with apt-get for our virtual machine.

sudo apt-get install openssh-server

An easy way to shutdown a virtual machine is with the VirtualBox command line tool which can send an ACPI shutdown event to the virtual machine. (e.g. VBoxManage controlvm zeus1 acpipowerbutton). To handle these events we have to install the acpid package.

sudo apt-get install acpid

For a working ACPI configuration which can handle the power button event you have to edit /etc/acpi/events/powerbtn config file.

event=button[ /]power
action=/etc/acpi/powerbtn.sh

To test the load balancing we also install the apache server.

sudo apt-get install apache2

We change now the network configuration to the following settings:

  • hostname: zeus1
  • IP: 10.0.0.71

Change the network settings in the file /etc/network/interfaces

auto eth0
iface eth0 inet static
        address 10.0.0.71
        netmask 255.255.255.0
        network 10.0.0.0
        broadcast 10.0.0.255
        gateway 10.0.0.1
        dns-nameservers 10.0.0.1

After we have changed the network settings also change the hostname to zeus1 in /etc/hostname and don’t forget to update the /etc/hosts file too.

Before we clone our image we have to remove some udev config files.

  • Remove all entries in /etc/udev/rules.d/70-persistent-net.rules because the interfaces (eth0, etc.) are bound to the MAC address and after import of a virtual machine with VirtualBox the MAC address is changed. So when you import virtual machine and start it up the configured interface eth0 is not configured.
  • Also remove all entries in in /etc/udev/rules.d/70-persistent-cd.rules. I had some problems to boot the virtual image. After I have removed this entries the virtual machine booted up without problems.

To test the acpi package, shutdown zeus1 with the poweroff command.

sudo poweroff

After the shutdown is complete, export the virtual machine zeus1 as OVA file.

Copy the exported OVA file to your virtualbox server. As you know we have installed openssh on our VirtualBox server (atlas). So you can copy the virtual machine with scp over the newtork.

scp zeus1.ova vbox@10.0.0.50:/tmp

Configure VirtualBox server

Login to our virtualbox server (atlas) with ssh as user vbox and import the virtual machine

ssh vbox@10.0.0.50
VBoxManage import /tmp/zeus1.ova

Now we clone our virtual machine zeus1 and create a identically one with the name zeus2.

VBoxManage clonevm zeus1 --name zeus2 --register

To show all available vms run “VBoxManage list vms”. The output should now display zeus1 and zeus2.

"zeus1" {fa974e98-6411-4768-9cd9-017176e47e81}
"zeus2" {81953acc-59d0-4fa9-87f1-42e08bbbdb78}

To start up our virtual server automatically we create for both virtual machines the start script vm_zeus1.sh and vm_zeus2.sh. You have to change the vm_name variable to the name of the VirtualBox machine. The following example shows the start script vm_zeus1.sh. Copy this script and change the vm_nam variable to zeus2 for the script vm_zeus2.sh.

#!/bin/bash

# VirtualBox vm name
vm_name=zeus1

# VirtualBox username 
vbox_user=vbox

vbox_command="sudo -u $vbox_user -i VBoxManage"

case "$1" in

start)
        echo "starting vm vm_name"
        $vbox_command startvm --type headless $vm_name
        ;;

shutdown)
        echo "sending ACPI shutdown to vm $vm_name"
        $vbox_command controlvm $vm_name acpipowerbutton
        ;;
pause)
        echo "pause $vm_name"
        $vbox_command controlvm $vm_name pause
        ;;

resume)
        echo "resume $vm_name"
        $vbox_command controlvm $vm_name resume
        ;;

stop)
        echo "savestate $vm_name"
        $vbox_command controlvm $vm_name savestate
        ;;


*)
        echo "usage: $0 (start|stop|pause|resume|shutdown|help)"
        ;;
esac

exit 0

The vm start script uses VBoxManage controlvm for operating with virtual machines. For more details what these commands do have a look in the official manual.

Install both start script vm_zeus1.sh and vm_zeus2.sh with update-rc.d. The scripts must start after the VirtualBox service vboxdrv and stop before it. So we add the start sequence number 25 and kill sequence number 10. Adapts this settings when you have problems during the startup or shutdown of the atlas server.

chmod a+x vm_zeus1.sh 
sudo cp vm_zeus1.sh /etc/init.d/
sudo update-rc.d vm_zeus1.sh defaults 25 10

Now you can make the same configuration steps with the virtual machine zeus2.

Configure virtual machine zeus2

Start up the virtaul server zeus2. Be sure that zeus1 is not running, because zeus2 have still the same IP as zeus1.

sudo  /etc/init.d/vm_zeus2.sh start

Log into zeus2 with ssh and change the IP adresse and hostname.

  • hostname: zeus2
  • IP: 10.0.0.72
auto eth0
iface eth0 inet static
        address 10.0.0.72
        netmask 255.255.255.0
        network 10.0.0.0
        broadcast 10.0.0.255
        gateway 10.0.0.1
        dns-nameservers 10.0.0.1

Change the hostname in /etc/hostname to zeus2 and don’t forget to update the /etc/hosts file.

To test the acpi package, shutdown zeus2 with the poweroff command.

sudo poweroff

Now you can adjust some seetings with VBoxManage modifyvm when you need to allocate more memory or add an additional cpu.

Configure the load balancer

We use crossroads as our load balancer. You should considering to install the load balancer on a different Linux box, which is connected to the Internet and redirects the request to your virtual servers. But in this example I have installed crossroad on our Ubuntu Server atlas.

sudo apt-get install crossroads

Create a start sript /etc/init.d/crossroads.sh. This scripts redirects HTTP requests to 10.0.0.71 and 10.0.0.72. For configuration details look up the crossroads documentation.

#!/bin/sh

XR=/usr/bin/xr
SERVER="-s http:0:80"
BACKENDS="-b 10.0.0.71:80 -b 10.0.0.72:80"
ALGORITHM="-dl"
HTTP_FLAGS="-x -X"
TIMEOUTS="-t 10"
CHECK_CALLS="-c 10"
DEBUGGING="-v"


case "$1" in

start)


	$XR $SERVER $BACKENDS $ALGORITHM $HTTP_FLAGS $TIMEOUTS $CHECK_CALLS $DEBUGGING 2>&1 | logger -t crossroads &
	;;
stop)
	killall $XR
	;;
*)
    echo "usage: $0 (start|stop|help)"
esac

exit 0

The last step is to install the load balancer start up script.

chmod a+x crossroads.sh
sudo cp crossroads.sh /etc/init.d
sudo update-rc.d crossroads.sh defaults

Start up the whole infrastructure

So now its time to startup the load balancer and the virtual servers.

sudo /etc/init.d/crossoads.sh start
sudo /etc/init.d/vm_zeus1.sh start
sudo /etc/init.d/vm_zeus2.sh start

Test the load balancer

Now when you connect to atlas (10.0.0.50) with a web browser you should be redirected to zeus1 (10.0.0.71) or zeus2 (10.0.0.72). To test the load balancer just shutdown one of the zeus vm’s and try to connect to http://10.0.0.50:80, now shutdown the other vm and start the previous stopped virtual machine. Refresh the browser and you should not notice any difference, expect that the loading time of the page is longer for the first time.

Enable remote desktop for a virtual machine

When a virtual machine behaves not as expected you can connect to the virtual machine over the remote desktop protocol. You have just to enable remote desktop for the specific virtual machine. First we have to shutdown the virtual machine an specify on which port the VirtualBox should listen for remote desktop connections.

sudo /etc/init.d/vm_zeus1.sh shutdown
VBoxManage modifyvm zeus1 --vrde on --vrdeport 3389 --vrdeauthtype external
sudo /etc/init.d/vm_zeus1.sh start

When the virtual machine is started yo can connect from your client with rdesktop to the VirtualBox server over the previous specified port. Add the parameter “-p -” so that rdesktop ask for a password.

rdesktop 10.0.0.50:3389 -u vbox -p -

Or just add the password in the command line so that rdesktop don’t aks for it.

rdesktop 10.0.0.50:3389 -u vbox -p your_password

After you have no use for the remote desktop connection you should disable the remote desktop option for this virtual machine.

sudo /etc/init.d/vm_zeus1.sh shutdown
VBoxManage modifyvm zeus1 --vrde off

And now?

The next thing do would be to look how you can replace the human tasks with scripts. For example:

  • You could write a script for cloning a virtual machine, change the IP address and add the new server to the load balancer.
  • Write a script which adds or remove dynamically new virtual machines to the load balancer when the load increase or decreases.

I hope this post was useful and have fun by building your own homegrown private cloud.