Installing Cloudify 4 on AWS and OpenStack – A Simple Step by Step Tutorial

The most up-to-date tutorial on environment setup can be found at this link

Introduction

There are generally two paths to installing Cloudify Manager on your cloud – either via a Cloudify Manager Pre-baked Image or bootstrapping (which gives the user the option of choosing what is installed on the machine). In an effort to make the installation of Cloudify Manager as simple as possible, this tutorial will walk you through the more basic path to provisioning your AWS or OpenStack infrastructure and get your first Cloudify Manager up and configured – using a pre-baked image. In the next post, we will walk you through bootstrapping Cloudify Manager on your cloud of choice, including Azure (which we don’t have an image for yet) and we will follow that up with deploying your first demo web application.

Keep in mind that this method of installing Cloudify Manager is not necessarily ideal, but it will give you the best understanding of what is required to get your environment set up in about five minutes and should work the very first time you try it (barring unforseen circumstances based on your environment or cloud setup).
One last note: We will be installing the Open Source Community Edition of Cloudify in this tutorial.
To ask a question or report an issue, please visit the Cloudify users group.

Check out our NFV and Enterprise Cloud Orchestration Whitepapers!  Go

Prerequisites

There are a few things to keep in mind and prepare before getting started, so please read through this post carefully at least once and then begin.

• Python 2.7 installed on your computer.
• Virtualenv installed on your computer.

• IaaS Cloud provider and API credentials and sufficient permissions to provision network and compute resources (a new, clean environment is always best):
• AWS Credentials
• Openstack Credentials – skip step 5 in those instructions — do not “source” the file.

Getting Started

It is highly suggested that users run Cloudify in a virtual environment such as virtualenv. Once you have created your environment in the terminal (or command prompt or Powershell), continue with the Cloudify CLI installation.

1. Install Cloudify CLI on your computer by downloading the binary file for your operating system (Linux, Ubuntu, or Windows) and install. If you are using Mac, just run pip install cloudify in the terminal and it will install for you.

2. Download and extract this blueprint archive to the directory (folder) of your choice AND make sure your terminal is currently in that directory.

3. Install your environment’s infrastructure by executing one of the example commands below, inserting your account credentials where indicated. The simplest way to do this is to copy the text to a text editor, add the details, and the paste them into the terminal.

NOTE: AWS users – this process will automatically be run on US-EAST-1 (N. Virginia). For slightly more advanced users who may want to use a different region, open the “aws-blueprint.yaml” file in a text editor to customize the inputs.

For AWS run:

$ cfy install cloudify-environment-setup-latest/aws-blueprint.yaml -i aws_access_key_id=[INSERT_YOUR_AWS_ACCESS_KEY] -i aws_secret_access_key=[INSERT_YOUR_AWS_SECRET_ACCESS_KEY] --task-retries=30 --task-retry-interval=5 --install-plugins

For OpenStack run:

Before running this command, make sure you have uploaded a Cloudify QCOW image to your OpenStack. You can find the Cloudify QCOW image toward the bottom of the page here. Once uploaded, click into the image and use the ID in the below command. You should also be able to find an Ubuntu Trusty image for that input. Also, your “image_flavor” will be dependent on your OpenStack. Contact your IT admin if you are unsure of any of these.

$ cfy install cloudify-environment-setup-latest/openstack-blueprint.yaml -i username=[INSERT_YOUR_OPENSTACK_USERNAME] -i password=[INSERT_YOUR_OPENSTACK_PASSWORD] -i tenant_name=[INSERT_YOUR_OPENSTACK_TENANT_NAME] -i auth_url=[INSERT_YOUR_OPENSTACK_V2.0AUTH_URL] -i region=[INSERT_YOUR_OPENSTACK_REGION] -i external_network_name=[INSERT_YOUR_OPENSTACK_EXTERNAL_NETWORK_NAME] -i cloudify_image_id=[INSERT_YOUR_CLOUDIFY_IMAGE_ID] -i ubuntu_trusty_id_examples=[INSERT_YOUR_OPENSTACK_UBUNTU_TRUSTY_IMAGE_ID] -i small_openstack_image_flavor=[INSERT_YOUR_OPENSTACK_SMALL_IMAGE_FLAVOR_ID] -i large_openstack_image_flavor=[INSERT_YOUR_OPENSTACK_LARGE_IMAGE_FLAVOR_ID] --task-retries=30 --task-retry-interval=5 --install-plugins

4. Get info to configure Cloudify Manager by running ‘cfy deployments outputs’ in your terminal.
The output should look like this:

For the purpose of this tutorial, you will only need to follow the “Configuration” steps. Ignore the “Bootstrap” and “Demo” sections. This will ready our environment to run the webapp we will deploy in a future post.
5. Configure your manager:
At this stage, it is suggested to wait 5 minutes for all of the services to synchronize.

Initialize the manager CLI profile:

You need to initialize a manager profile in order to control your manager. Copy the text from your outputs in the previous step and paste it in your terminal. It will look like this:

$ cfy profiles use -s cfyuser -k ~/.ssh/cfy-manager-key -u admin -p admin -t default_tenant **.**.***.***

Upload the plugins for your manager:

Note: the exact plugins you need to upload will vary. For this example, you will be shown the plugins to upload in your outputs.

Create your secrets:

Adding secrets to your manager make your deployments more secure. The exact secrets you add also vary by clouds. Again, copy and paste from your previous step outputs and paste into your terminal.
Note that in the last command, the double-quotes are unescaped:
The deployment output was like this:

$ cfy secrets create agent_key_private -s "$(<~/.ssh/cfy-agent-key)"

But you will need to remove the on either side of the quotes so it looks like this:

$ cfy secrets create agent_key_private -s "$(<~/.ssh/cfy-agent-key)"

Your manager is now installed and configured!

6. When you are ready to uninstall your environment, run:

$ cfy profiles use local
$ cfy uninstall --allow-custom-parameters -p ignore_failure=true --task-retries=30 --task-retry-interval=5

Watch the tutorial video below to see this in action:

Troubleshooting

502 Bad Gateway

• If cfy profiles use ... fails with this output, the service has started in error state. Try restarting the VM.

<head><title>502 Bad Gateway</title></head>
<body bgcolor="white">
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.8.0</center>
</body>
</html>

[Errno 61] Connection refused

• If cfy profiles use ... fails with the following output, then the server is refusing your connection because of too many requests. Most likely this is an issue with your network. Expect this issue to come up a lot until your network service improves.

Attempting to connect...
HTTPConnectionPool(host='**.***.***.***', port=80): Max retries exceeded with url: /api/v3/provider/context (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x10d9d0590>: Failed to establish a new connection: [Errno 61] Connection refused',))