Continuous Deployment with Architecture and Infrastructure Changes Using Cloudify

In our recent release of Cloudify 4.4, we made some really solid improvements and added great new features to make the user experience a breeze. Read about them here. One particular enhancement that is quite significant, especially for production deployments, is the Deployment Update feature, which is a built-in workflow that can be run from the UI or CLI.

Test-drive Cloudify’s Multi-Cloud Lab Free!    Get Lab

About Deployment Update

The deployment update feature enables users to add, remove, and connect new nodes in your existing deployment, while retaining the state of your current settings, instead of creating a new deployment.
A deployment can be updated in any of these three ways:

• Upload a new blueprint and replace it with the current one
• Change Inputs for the deployment, either with an inputs YAML file or providing each input value directly
• Set specific Actions to be taken upon deployment update like enabling/disabling Install/Uninstall workflows on specific node instances

Updating Inputs in the UI, such as the web server URL or port endpoint, and setting specific Actions are simply a matter of changing the information and clicking “Update”. Those are trivial compared to migrating to a new service.
When an architectural change is required (such as in the example below), there is a lot more to consider. Therefore, we need to go back to the blueprint in order to make the necessary changes.
The update workflow execution occurs roughly in this order:

• The unlink operation is executed in regard to each removed relationship.
• The uninstall workflow is executed on each of the removed nodes.
• Old plugins that were outdated are being uninstalled (if necessary).
• The install workflow is executed on each of the added nodes.
• The establish operation is executed in regard to each added relationship.
• New plugins that were updated are being installed (if necessary).
• The reinstall workflow (uninstall and install) is executed on each of the modified nodes and each of the nodes that were explicitly marked for reinstall.

The use case

In order to get a better understanding of what the Deployment Update does, let’s start with the assumption that you currently have a basic Continuous Integration pipeline. Github, Jenkins, a testing tool, artifact repo, and then Cloudify for deployment and lifecycle management of your infrastructure, application, network, etc. We will also take it as a given that you have deployed your application – in this example, an e-commerce site – with all the VMs, DBs, LBs, Firewall, and all that.
After doing some research, a decision has been made to migrate from the current local database running on AWS to Amazon RDS. But how can this be done in an automated way? After all, we will need to update infrastructure resources, application resources, networking resources.
In this example, you would be required to change infrastructure by moving from a VM running your application to the new service, create a new node to connect to the DBaaS ensuring zero downtime, decommission the old database nodes, update relationships to the existing nodes, and update inputs such as the URL and port.

How the magic happens

So, let’s see how it all goes down step-by-step in Cloudify.
First, we start with the below initial (sample) blueprint which will spin up the required VMs in AWS and installs the e-commerce app, DB, and FW. We also have our inputs file with the DB information – let’s call it inputs.yaml.

tosca_definitions_version: cloudify_dsl_1_3
imports:
  - http://www.getcloudify.org/spec/cloudify/4.4/types.yaml
  - http://www.getcloudify.org/spec/awssdk-plugin/2.4.1/plugin.yaml
inputs:
  db_url:
  db_port:
  db_password:
node_templates:
  host:
    type: cloudify.nodes.aws.ec2.Instance
  local_db:
    type: cloudify.nodes.DB
    properties:
      db_url: { get_input: db_url }
    relationships:
      - type: cloudify.relationships.contained_in
        target: host
  app:
    type: cloudify.nodes.WebApplication
    relationships:
      - type: cloudify.relationships.contained_in
        target: host
      - type: cloudify.relationships.connected_to
        target: local_db
  fw:
    type: cloudify.nodes.Firewall
    properties:
      rules:
        - port: { get_input: db_port }

app_local_db.yaml

db_url: http://10.0.20.34
db_port: 7876
db_password: "***********"

Initial inputs.yaml file

Initial blueprint and input settings

Upload, Deployment, Install

Option 1: Using the Console UI, you can go to the Blueprints page and upload the blueprint, create a deployment (we’ll name it ecom_app), and execute the install workflow.
Option 2: From the terminal, run the cfy install command which will upload the blueprint to the manager, create the deployment, and run the install workflow: cfy install app_local_db.yaml -i inputs.yaml -n ecom-app

ecom_app deployment

Deployment Update

Now that we have our running deployment, we will create the new blueprint (app_rds.yaml) with the changes we need to make and upload it to the manager. Use the link above to upload the blueprint or run cfy blueprints upload app_rds.yaml from the terminal.

tosca_definitions_version: cloudify_dsl_1_3
imports:
  - http://www.getcloudify.org/spec/cloudify/4.4/types.yaml
  - http://www.getcloudify.org/spec/awssdk-plugin/2.4.1/plugin.yaml
inputs:
  db_url:
  db_port:
  db_password:
node_templates:
  host:
    type: cloudify.nodes.aws.ec2.Instance
  dbaas:
    type: cloudify.nodes.aws.RDS
    properties:
      db_url: { get_input: db_url }
  app:
    type: cloudify.nodes.WebApplication
    relationships:
      - type: cloudify.relationships.contained_in
        target: host
      - type: cloudify.relationships.connected_to
        target: dbaas
  fw:
    type: cloudify.nodes.Firewall
    properties:
      rules:
        - port: { get_input: db_port }

app_rds.yaml

Note: For simplicity’s sake, we will show the Deployment Update from Cloudify Console only. To run the Deployment Update from the terminal, see the docs.
Go to the Deployments page, click the hamburger menu on the right-hand side of your deployment, and choose Update.

New settings for deployment update

Once there, you can insert the ID of the blueprint being used for the deployment update and click on the Update button.
As you can see, we are updating a few things here:

• Starting from the top, the new blueprint name, app_rds, tells Cloudify to check that blueprint
• We also manually changed the db_port and db_url inputs
• We ticked the Run install workflow first Action. This Action ensures that our new database is fully installed and configured before we remove the uninstall the node being changed, which means our application will effectively have zero downtime.

Deployment after update with tooltip notification

• The deployment update is then started, and then the magic happens.
• The new RDS service is installed and configured
• The endpoint URL and port are updated
• The firewall is updated with new rules to allow connectivity to the new RDS service
• The new webapp relationship target is updated to connect to the RDS service
• The old database is uninstalled

Details of deployment update

And that’s it. We just orchestrated the infrastructure migration of our VM-based database to AWS RDS using Cloudify!
As a bonus, we have a fantastic video detailing this feature and many others which you can watch below.

You rock, Tali!