Recently, I had a need to create a multi-node OpenStack Folsom deployment with Quantum. I needed to test out some deployment scenarios for a customer. To make things even more interesting, I wanted to test it out with the recent VXLAN changes in Open vSwitch which went upstream. I thought others may be interested in this as well. I’m planning to document this for Grizzly as well, but the steps should be mostly the same. Also, I’ve opened a blueprint for the Grizzly release to enable the selection of either GRE or VXLAN tunnels when using the Open vSwitch plugin with Quantum.

First Steps⌗

To get started, you’ll need to setup two machines you can use for this. I chose Fedora 17, but Ubuntu 12.04 will work just as nicely. I also chose to install Fedora 17 into virtual machines. And just a quick plug for deployment options here: If you’re not using something like Cobbler in your lab to automate Linux installs, you really need to. I’ve got Cobbler setup to automate installs of Ubuntu 12.04, CentOS 6.3, and Fedora 17 in my lab. I can PXE boot VM images or physical machines and with a simple menu selection walk away and come back 30 minutes later to a fully installed system. When you’re spinning up a large number of devstack installs, this turns out to be very handy. Colin McNamara has a great blog post to get you started with Cobbler.

Make sure to give each VM 2 virtual interfaces, if you go that route, or that your physical hosts have 2 interfaces. The first one will be used for management traffic, the second one will be used for the external network to access your tenant VMs. I’ll assume eth0 and eth1 here.

At this point you should have your 2 VMs or physical hosts up and running with Fedora 17 or Ubuntu 12.04.

Upgrading Open vSwitch on Your Hosts⌗

To enable VXLAN tunnels in Open vSwitch, you need to pull the latest from master, build it, and install it. I’ll show the instructions for Fedora 17 below, which include building RPMs, but for Ubuntu it should be similar except for the RPM building part. I did this as root, to build the kernel module that seems to work best.

At this point, reboot your host and you should have the latest Open vSwitch installed. Copy the RPMs from this build host over to your other host, install them the same way, and reboot that host. On each host, the output of “ovs-vsctl show” should indicate 1.9.90 as below:

devstack⌗

Getting devstack installed and running is pretty easy. Here’s how to do it. Make sure you do this as a non-root user. Make sure you add passwordless sudo access for this user as well (add “ ALL=(ALL)      NOPASSWD: ALL” to /etc/sudoers before running devstack).

At this point, you should have a Folsom version of devstack setup. You now need to populate your localrc files for both your control node as well as your compute node. See examples below:

Control node localrc⌗

compute node localrc:⌗

For the compute localrc, make sure you change SERVICE_HOST to be the IP on your control node you want to use. Also, pick an appropriate floating IP range if you want to use floating IP addresses. On the compute node, make sure to change SERVICE_HOST and SERVICE_HOST_NAME appropriately. Also, once you’ve run devstack on each host, you can uncomment the “OFFLINE=True” to speed it up on subsequent runs.

Post devstack tasks⌗

I had to do the following tasks on my setup to workaround a few things. Fedora 17 does not come with nodejs installed by default, so Horizon will not work out of the box. To install nodejs, follow these instructions. I performed these as root as well, but sudo would work with the “make install” step as well.

Next, to work around a Nova metadata issue I was having, I added some IP configuration to eth1 by doing “sudo ifconfig eth1 up 169.254.169.254″. I also added eth1 to br-ext on the control node. This is the interface which will be used for external access to your tenant VMs via their floating IP addresses.

You will also need to apply a small patch to Quantum on the control node. This is to make Quantum create VXLAN tunnels instead of GRE tunnels. The patch is below and you should be able to apply it manually quite easily:

Running devstack⌗

At this point, you should be ready to run devstack. Go ahead and run it on the control node first (cd devstack ; ./stack.sh). Next, run it on the compute host (cd devstack ; ./stack.sh).

To access the consoles of your devstack installs, execute “screen -r stack” on each host. This pops you into a screen session with each session handling the output of a particular OpenStack component. To move around in the screen window you can use “ctrl-a-p” and “ctrl-a-n” to do move to the previous and next windows. “ctrl-a-ESC” will freeze the window and let you use vi commands to scroll back. “ESC” will unfreeze it.

Summary: You’re a Cloud Master Now!⌗

If you’ve followed this guide, you should have an OpenStack Folsom Cloud running in your lab now with the Open vSwitch Quantum plugin running and VXLAN tunnels between hosts! A followup post will show you how to create multiple tenants and verify Quantum is segregating traffic by utilizing VXLAN tunnels between hosts with a different VNI for each tenant.

Welcome to the world of cloud computing on OpenStack!