OpenStack Orchestration with Heat + Devstack

Today in order to bootstrap a Cloudify manager you have to take care of the OpenStack compute, networking and security services yourself, essentially all of the OpenStack orchestration (this is still a good reference).  Each of these is comprised of a number of configurations.
Just to name a few:

• When we talk about compute services, these can include provisioning the right instance type, memory and CPU characteristics, image type, and more.
• Networking services include networks, ports, subnets, association with floating IPs as well as an establishment of a router and binding its interfaces correctly.  See this post to learn more about OpenStack Networking.
• Security services include the creation of security groups to define which network ports are open and whom can access them, as well as the creation of security keys to SSH into the manager machine, and future instances machines. There is a different key for the manager machine and and a different key for the launched application instances.

Below is an example of the bootstrapped Cloudify manager after all the above IaaS services are provisioned.

The Heat Stack

Moreover, when creating the Cloudify manager in the usual way you have to install a CLI (command line interface), which in turn connects to Openstack endpoints like the Identity endpoint and Neutron endpoint, and communicates with them over the network using JSON-based commands. This works fine, but since there are more components involved like installing a CLI, it may take more time and is at times more error prone.
To simplify this, you can create an OpenStack Heat stack, and launch the stack which will create everything automatically. Later on I’ll show a real example of how to launch the Cloudify manager as a Heat stack.

OpenStack orchestration made easy. Test drive Cloudify.     Go

First let’s talk about Heat. Heat is based on a HOT template – (which we first discussed in this post on OpenStack Heat if you need a refresher),  which resembles the TOSCA standard. The template is composed of three main sections:

• The parameters: Where you can define various parameters and constants to be referenced later when creation and accessing resources.
• The resources section: Where you define and provision resource like compute resources, network, subnets, etc.
• The output section: Where you can output variables like the web page URL, public IPs, etc.

Heat stacks can be nested, and one stack can reference a second stack. This is what is called Template Composition with Amazon CloudFormation.
As we shall see next, you can define IaaS services and dependencies between them, for example network creation comes before a port creation or a subnet creation.

Cloudify Manager provisioned as a Heat Stack

You can see above a real example of how to install the Cloudify manager as a Heat stack. In the parameters section we define several parameters, like the key to SSH to the manager machine, instance type, image ID, the manager network in CIDR notation, the gateway address, start/end network allocation pool, as well as the public network ID.
In the resources section, we first create the network, then the subnet . You can see how we reference the parameters defined in the previous section like gateway_ip. We also reference the Google name servers as the DNS name servers to resolve names to IP addresses. Next, we provision the router and bind its interfaces. And now after all of the IaaS services are provisioned, and are in place, we can provision the Cloudify manager with the right image_id, flavor, and key_name. In the user data we create a bash script which installs the manager packages. First it downloads the right packages from the net, and then installs them.
In the output section we retrieve and present the manager’s floating IP address.
Below are two charts, the first shows how the heat stack looks graphically, with all of the connected nodes and dependencies.
The second chart shows the provisioned resources:   Networks, subnets, Floating IPs and Cloudify manager VM.


In order to access the Cloudify manager from an external browser I’ve created port forwarding on the Devstack machine, so that when you access port 8080 of the Devstack VM hosted on HP Cloud, you will be forwarded to port 80 on the Cloudify manager provisioned VM (this is specific to to this current example, but this can easily be provisioned on any Openstack cloud). The commands to create the port forwarding are listed below.

In the chart below you can see a screenshot of how the bootstrapped Cloudify manager looks when it’s up and running.