Advanced OpenStack Networking – Configuring an OpenStack VM with Multiple Network Cards

 

We have discussed OpenStack networking extensively in previous posts.  In this post, I’d like to dive into a more advanced OpenStack networking scenario.
Many cloud images are not configured to automatically bring up all network cards that are available. They will usually only have a single network card configured. To correctly set up a host in the cloud with multiple network cards, log on to the machine and bring up the additional interfaces.
On an Ubuntu Image, this usually looks like this:

Networks in the cloud
A complex network architecture is a mainstay of modern IaaS clouds. Understanding how to configure your cloud-based networks, and hosts, is critical to getting your application working in the cloud. This is especially true with Cloudify, the open source cloud orchestration platform I work on.

Network orchestration for any cloud with Cloudify. Get Started.  Go

The cloud, like the world, used to be flat
It was not that long a time ago that most IaaS providers only supported flat networks – all of your hosts were in one large network. Separation between services running in the cloud was enforced in software or with firewalls/security-groups. But technically, all of the hosts were connected to the same network and visible to each other.
The flat network model is simple, and therefore easy to reason and understand. It was a good choice for the early days of the IaaS cloud and no doubt helped with getting applications into the cloud in the first place. It was one of the things that made EC2 so easy to use for anyone just starting out with the ‘cloud’. This model is in fact still available on Amazon Web Services under the title ‘EC2-Classic’. And for many applications, a flat network is good enough.
But as cloud adoption increases, more complex applications are moving into the clouds, and issues like network separation, security, SLA and broadcast domains make more complex networks models a must. Software Defined Networks (SDN) fill that gap. They are now a staple of most major IaaS clouds. AWS has AWS-VPC, OpenStack has the Neutron project and there are many other implementations.
Working with SDN requires knowing a bit more about how information moves around between your cloud resources. In this post I am going to discuss how to set up a host in the cloud so it will play nice with complex networks. I’ll be using OpenStack, but the concepts are similar for other cloud infrastructures.
Openstack configuration
I am going to start with an empty tenant, only the public network is available.
First, lets set up out networks and router:

Note the network IDs:

We’ll start with a standard Ubuntu cloud image:

Create the keypair and security group:

Let’s spin up an instance connected to both our networks:

And set up floating IPs for the first network:

Note how we matched the VM’s IP to its port, and associated the floating IP to the port. I wish there was an easier way to do this from the CLI…
If everything worked correctly, you should have the following setup:

Let’s make sure ssh works correctly:
ssh -i demo-keypair.pem ubuntu@192.168.15.31 hostname
demo-vm
Cool, ssh works. Now, we should have two network cards, right?

Huh?! The VM only has one working network interface! Where is my second NIC? Was there a configuration problem with the OpenStack network setup? The answer is here:

The second NIC exists, but is not running.
The issue is not with the OpenStack network configuration – it’s with the image. The image itself should be configured to work correctly with multiple NICs. All we have to do is bring up the NIC. So we ssh into the instance:
ssh -i demo-keypair.pem ubuntu@192.168.15.31
And run the following commands:

The second NIC should now be running:

And there you go – your VM can access both networks.
This issue can make life complicated when setting up a complex, or even a not very complex, application. When will this issue hurt you? Well, imagine a scenario where you have a web server and a database server. The web server is connected to both Network1 and Network2, and the database server is only connected to Network2. Network1 is connected to the external world over a router, and Network 2 is completely internal, adding another layer of security to the critical database server. So what happens if the web server only has one network card? If only the NIC for Network1 is up, the web server can’t access the database. If only the NIC for Network2 is up, the web server can’t be reached from the external world. Even worse, if this web server is accessed via a floating IP, this IP will also not work, so you won’t be able to access the web server and fix the issue. Tricky.
In conclusion
The above commands will bring up your additional network card. You will of-course need to repeat this process for each additional network card, and for each VM. You can use a start-up script (a.k.a. user-data script) or system service to run these commands, but there are better ways. I’ll discuss how to automate the network setup in a follow-up post.

This was originally posted at Barak’s blog Head in the Clouds, find it here.