Cloudify’s latest 5.1 release brings a new approach that simplifies multi-cloud orchestration.  

As demand for our ‘Environment as a Service’ increases, Cloudify’s 5.1 release introduces a new service composition technology with out-of-the-box integrations for all cloud environments (VMware, OpenStack, AWS, Azure, GCP) and existing toolchains (CI/CD, Terraform, Kubernetees). Cloudify 5.1 enables modernizing on-prem cloud environments, and manages deployments on public and cloud-native architecture – all with no lock-in.

What’s New

This document describes the new features and updates introduced in Cloudify v5.1. 

The key themes of v5.1 include:

• Environment as a Service (EaaS)
  Extending the Cloudify EaaS experience to any enterprise domain, private or public, automated or not, with zero throw-away code and complete integration into the ecosystem.
  This theme introduces features and capabilities allowing Cloudify’s seamless integration into existing CI/CD workflows while interacting with existing automation tools and extending native abilities.
• Self-service experience 2.0
  Providing internal & external service consumers a complete self-service portal allowing a fully customizable menu of certified services & environments for self-provisioning.
• Operability
  This major theme introduces enhancements intended for the system operators. Our key objective in this theme – which builds on top of the Cloudify 5.0.5 operability enhancements – is to reduce the level of expertise required from the ops team while at the same time empowering them with easier access to the information they require, allowing simpler and smoother operations.
  Other aspects of this theme include improved maintenance flows for the Cloudify product itself.
• Ease of development
  Focused towards the service designer and developer, v5.1 introduces improvements to the Composer, streamlining blueprint editing, with new code completion & IDE integration schema
• Compliance & Security
  Updating all packages & attending to security patches are considered standard for every Cloudify release. Some items in 5.1 are worth a wider discussion though: Migration of the entire codebase to Python 3.x to keep compliance and be at the front end of the supported versions, and ISO 27001-27701 are key examples. Check the complete list below.
• IT & Maintenance
  Improved maintenance flows for the Cloudify product, including clustering options, health check, certificate handling, and upgrade tools.

Cloudify 5.1 is a major Cloudify release containing over 250 developed stories and 359 resolved issues and tasks, improving all aspects from functionality to robustness and security.

Environment as a Service (EaaS)

Cloudify v5.1 extends the foundations set in previous releases allowing organizations to transform their on-prem infrastructure services into a private cloud that is managed as code and breaks automation silos across the private and public cloud by putting all automation frameworks under a common automation scheme also referred to as ‘Everything as Code’.

The release of Cloudify’s EaaS introduced a new ‘Orchestrator of Orchestrator’ that breaks the silos between different automation frameworks. Cloudify v5.1 includes many updates to the respective plugins such as  Kubernetes, Ansible, AWX, and Terraform that will allow users to continue using those frameworks of choice natively and use Cloudify as an overlay that will provide consistent workflow management and visibility across those frameworks as well as allowing interoperability between them.

VMWare Support Enhancements

Providing modern public Cloud management concepts to private clouds has been part of Cloudify from day one. With v5.1 extended functionality, orchestrating your VMWare infrastructure with Cloudify not only provides a top notch orchestrator capable of running Infrastructure as code across VMWare and any other Infra, but also Extend your VMWare capabilities by allowing all of your VMWare products to collaborate in a better way supporting one orchestrator building infrastructure, networking, applications, compliance and security.

Supporting that concept is done through a combination of

• Extended support for all legacy automation tools, domain specific or generic. (vSphere, VRO, NSX, …)
• Cloudify service composition modeling and DSL as an abstraction layer on top of services; thus provide simple intent based modeling that fits with DevOps best practices where  Everything is managed as Code.
• Use Cloudify Self-Service Portal to enable simple provisioning of pre-templatized services.

By combining these, Cloudify is able to not only seamlessly adopt all of the existing automation and offer an advanced interface to it, but to also extend the functionality of all existing services by bridging them together and allowing them to interact with each other.

5.1 introduces major enhancements to our VMware support including a new NSX-T plugin and extended support for VRO, vSphere, and other VMware services. Orchestrating your VMWare infrastructure, networks and services with Cloudify allows the creation of both parallel & serial tasks, getting much more from your automation by handing state attributes from workflow outputs and using them as inputs for the next workflow, thus generating a complete automation flow.

Tie that with the complete set of integrations Cloudify unlocks for combining any other automation tool, and the ability to streamline Cloudify into your CI/CD pipelines, a new potential can be realized.

v5.1 provides updates support for 

• vSphere 7:
  Cloudify vSphere plugin has been certified with vSphere 7, providing orchestration capabilities for both existing resources which were supported in v6.7.3 and extensions supporting NSX-T v3.0 SDN based platform.
  • vSphere v7 support using Cloudify existing python SDK for vSphere API
  • NSX-T v3.0 created virtual networks.
  • NSX-T v3.0 distributed switches
  • Attachment of multiple NSX-T virtual networks to a single VM with ipv4 & ipv6 support.
  • Extended life-cycle interfaces supporting a two phased – create / start (power-up) –  approach.
• NSX-T v3.0 support

The Cloudify NSX-T plugin provides SDN orchestration for NSX-T v3.0 with focus on creating overlay networks within transport zones inside datacenter infrastructure.

• • The features include lifecycle management of Segments, DHCP, Gateways (Tier0 & Tier1), and more. 
  • Built-in integration with the vSphere plugin allows Cloudify to execute complex tasks in very simple steps, for instance fixed IP binding configuration, where VM is created at vSPhere and static IP is assigned by an NSX-T DHCP function.

Learn more about Cloudify Network Orchestration of vSphere v7.0 & NSX-T

Kubernetes

Kubernetes is the popular choice and the go-to platform for most organizations on their cloud-native journey. Cloudify v5.1 extends its Kubernetes support for all key platforms with improvements and updates, supporting: 

• Native Kubernetes – direct orchestration via the Kubernetes plugin
• Public and private Kubernetes services; with support for: 
  • Amazon Managed Kubernetes Service – EKS
  • Azure Kubernetes Service – AKS
  • Google Kubernetes Engine – GKE
  • Red Hat OpenShift container platform and Kubespray.
• Optimizations:
  • Improve file resource type
  • Support update Cloudify deployment on Kubernetes file resource
  • Deprecate Kubernetes master node type, move client config to node type as in other plugins
  • Resolve resource uniqueness based on kind, name, and namespace.
  • Validate Kubernetes resources before the creation.

Helm

Helm 3 plugin has been developed. With the Cloudify Helm plugin you can create a Helm client for each deployment; add repositories, create releases and easily deploy an application on any existing Kubernetes cluster.

CI/CD Native

Cloudify 5.1 introduces a new set of CI/CD integration plugins which currently include Jenkins, CircleCI, and GitActions. Those plugins allow direct usage of Cloudify and any supported automation tool (e.g. Terraform, Azure ARM, CloudFormation) via the CI/CD tool, leveraging the native interfaces of the CI/CD tool manager.

This opens the door for anyone familiar with Jenkins, CircleCI, or GitActions to build orchestration into their pipelines with little to no learning curve.

It also allows for complete flow integration by handing outputs of one tool as inputs to the other thus orchestrating complex scenarios leveraging several automation tools in tandem.

Learn more about CI/CD integration

Terraform & Ansible Integration

Leveraging Cloudify as the orchestrator of orchestrators, two of the common automation tools managed by Cloudify are Terraform and Ansible.

Cloudify v5.1 not only integrates with these tools but also extends their capabilities and provides easier management and operation.

To top all that, Ansible AWX plugin is in the incubation phase and available as early access.

The Enterprise Challenge

Both tools are very easy to use on a departmental/team level; managing the usage of them across business units and at the global enterprise level, however, poses some challenges. Running multiple versions of your automation code by multiple teams, each requiring different modules and setup is not easy to manage.

Cloudify v5.1 allows on-the-fly construction of a Terraform or Ansible Docker container, equipped with the required version, and customized to include all the required modules and packages. This is done through a Cloudify blueprint and can be set, granularly per each environment or team using it. This container will run all required automation tasks and then tear down upon completion, saving resources and cost.

Understanding the Topology

Cloudify now offers a topology view of the Terraform template based on its state. This allows the (sometimes) inexperienced operator to get visibility into the structure of the automated services including all instances and the dependencies between them. This view, as expected, is embedded in the Cloudify Topology view to clearly display complex topologies containing a combination of Terraform and other automation tools working in tandem.

Terraform & Ansible CI/CD Enablement

Cloudify’s intimate integration with Terraform, Ansible, and other domain orchestrators such as CloudFormation and Azure ARM, combined with v5.1 native CI/CD capabilities, allows us to simplify the usage of new automation tools directly from GitActions, Jenkins, or CircleCI, and embed Terraform (or any other tool) template in the CI/CD pipeline for operators and users with zero Terraform/Ansible familiarity. 

Users can further use output of one tool as an input to another thus bridging the orchestration directly through their CI/CD tool of choice.

Self-Service Portal Experience 2.0

The self-service portal provides a simple interface for non-DevOps users who want access to a pre-canned environment of a particular service, infrastructure system, network, or any orchestrated environment. It provides an interface in which the DevOps team in the organization can publish these services to any internal or external consumer who may now reserve, create, and teardown such services on their own. 

For VMware / VRO users, the self-service portal provides a consistent experience with the VRA catalog service.

Service Catalog

Cloudify 5.1 provides an out-of-the-box enhanced service catalog that supports multiple catalog services and introduces grouping of services based on tabs. Enterprises may maintain multiple catalogs with a variety of services and expose them to their internal/external consumers based on roles and access rights leveraging Cloudify’s RBAC (role-based access control).

Easy End-User Experience

Providing a self-service portal allows all authorized consumers to provision their own resources. These users are not familiar with the orchestration flows, and therefore require a simplified interface for progress tracking, status review, easy actions, and error handling.

v5.1 provides all of these via the new operational improvements.

• Improved workflow execution task graph view providing easy tracking of the progress
• Context-sensitive action buttons providing quick handling based on the deployment state
• Direct access to automatically filtered logs for easy error tracking.
• Topology view sharing the infrastructure details and a visual representation of the service.
• RBAC allowing a filtered view according to user permissions.

Learn more

External Portal – ServiceNow Integration

To provide a complete service, Cloudify v5.1 also offers integration with ITSM (IT Service Management) tools such as ServiceNow for external process triggering. This is achieved in v5.1 via a set of ServiceNow REST calls.

White Labeling and Customization

Cloudify offers an open and highly customizable self-service portal based on the REACT framework. The portal, alongside the entire user interface, can be easily modified to use the enterprise color scheme, logos, and nomenclature.

Cloudify v5.1 also offers the complete flexibility to create or adjust user interface widgets and we offer complete documentation in the form of a new developer guide, a set of libraries that simplify the development of new UI pallets, alongside a set of examples how to achieve that.

Learn more

Operability

Deployment page re-designed to simplify common tasks

The deployment page is the main operational interface for the operators. In v5.1 we have made the data easier to access, and the common actions easier to perform.
Changing the page layout now provides a clear deployment status summary header, as well as direct access to dedicated task-oriented tabs:

• Last execution: 
• Deployment info
• Deployment history

Last Execution Tab

Providing real-time information regarding the current execution, easy identification of issues, and quick access to the relevant logs by clicking any failed node task. 

The last execution tab also provided quick contextual actions such as Resume, Kill Task, Show Errors, and more – thus providing the operator with everything needed – info & actions – to successfully execute tasks.

Deployment Info Tab

If the operator wants to understand the fine details of a deployment, the Deployment Info tab is the place. It contains an improved topology view – now displaying each node’s infrastructure type, all the deployment’s output, capabilities, runtime properties, inputs, and of course the detailed blueprint structure.

History Tab

Sharing the complete status of all past executed workflows, with a visual representation of each task, success & failure info, and complete logs with automatic task-click filtering.

Task Execution Made Simple

V5.1 introduces seamless transition through the management console walking the user through the flows.

Upon service selection (deploy a blueprint), the user can select the ‘deploy & Install’ option which runs both flows through a single wizard, then directing the user to the deployment execution page allowing visibility into the progress as well as applying runtime actions. In case of errors, a new ‘show logs’ option has been added to any task node allowing the user to click and get a filtered view of the log – pin-pointing them to the specifics they should handle.

Once fixed – the workflow can be resumed from that same page while progress monitoring continues.

Improved Visibility Into the Orchestration Tasks Execution 

Deploying a multi-domain clustered service may consist of multiple steps with many inter-dependencies. Cloudify 5.0.x introduced a fully detailed task execution graph clearly displaying the planned, nodes, execution steps & their order, and dependencies. V5.1 takes this view to the next level allowing for automatic tracking (play mode) and a complete facelift.

Customizing pages using the new Tabs object

Cloudify’s user interface is completely granular and allows for complete customization by adding or modifying any page and widget. v5.1 extends that by introducing tabs widgets. Tabs are fully customizable, may be placed on a page, and in turn, may contain multiple widgets.

Learn more about tabs. 

No More Erroneous Deletion of Services-In-Use

Service composition DSL has created the foundation for modeling a system that consists of multiple orchestration domains.

When running thousands of services with direct and indirect dependencies between them, expiring a service may lead to an outage of a dependant service.

V5.1 introduces component dependency identification functionality which detects direct and indirect dependencies between 

• Internal components
• Deployments applied through the same manager
• Deployment applied through multiple managers

The system detects deletion attempts of components and services that other services depend on and alerts the user accordingly.

Ease of Development

Examples and best practices:

Cloudify v5.1 provides a catalog of walk-through usage examples. The examples provide all the required basic knowledge including setting up the infrastructure, provisioning services and using third-party automation tools orchestrated by Cloudify. The examples cover both private and public cloud scenarios and are accompanied by a complete set of blueprints and documentation.

Check the new learning section

Cloudify Blueprint Schema for Popular IDEs

As we move to manage everything as code it only makes sense that the development of our environment and infrastructure blueprint would be done through your favorite development tool (IDE) such as Visual Studio, IntelliJ IDEA, etc.

Cloudify v5.1 includes new code completion and IDE integration support through the addition of a JSON Scheme. Unlike other proprietary integration by other tools, this approach comes with the benefit that it doesn’t turn the Cloudify DSL into yet another complex and unreadable script but at the same time allows more typesafe validation during the blueprint development phase and better developer experience. Another benefit of this approach is that it is based on an industry-standard that covers many other frameworks. 

Composer Quick Editing

The Cloudify Composer allows for drag-and-drop composing and editing of blueprints and generates blueprint TOSCA code on-the-fly. With v5.1, developers can edit a copy of any blueprint on the manager, modify it using the composer, and immediately push it back to update existing deployments. This simplifies the flow and provides a complete blueprint maintenance loop.

Security & Compliance

As with every Cloudify release, security improvements are included in v5.1. All used packages have been updated to accommodate all latest security patches, and all known issues handled in all previous Cloudify patches have been resolved in v5.1. 

Python 3

v5.1 is running Python 3.6, updated from Python 2.7 used in former releases. The migration of the Cloudify code to Python 3.x is crucial to keep up-to-speed with all Python fixes and patches as Python 2.x is soon to be declared ‘end-of-life’.

Python 2.x code must be modified to become Python 3 compliant and such changes were applied to the entire Cloudify code-base. To provide a seamless experience for Cloudify plugin users, all of Cloudify’s plugins were updated and are now Python 3 compliant and certified over Cloudify v5.1.

Customer-generated plugins must be Python 3 compliant to run over v5.1. Cloudify provides a detailed document containing the recommended approach, tools, tips, and best practices on how to modify Python 2 plugins to become Python 3 compliant, as well as a set of examples. Contact the Cloudify support for assistance with any migration question. 

ISO compliance

Cloudify is ISO 27001 (Security) and ISO 27701 (Data Privacy) certified.

IT & Maintenance

Node optimized cluster option:

Cloudify’s 5.x clustering architecture is based on an active-active model, and on service separation, allowing Cloudify to offer a high availability architecture while keeping the flexibility to leverage a managed external PostgreSQL database service or RabbitMQ messaging queue. The separation of services approach was introduced in v5.0.5 and improved the scaling ability and the robustness of the failover mechanism thanks to the active-active approach and to the usage of industry-standard replication and synchronization concepts.

V5.0.5 offered the above model leveraging complete service separation over dedicated VMs. 

v5.1 takes that to the next level and offers a 3 VM compact model still offering an active-active approach and complete high availability, yet leveraging just 3 VMs for simplified management and operational flows.

Certificate Handling

V5.1 makes certificate updates much simpler with a new command line allowing for a complete update of certificates in all cluster nodes, reducing the chance of human errors.

Prometheus Based Cluster Monitoring

Cloudify cluster health monitoring is improved in v5.1. In previous releases, health monitoring was done leveraging a proprietary monitoring solution.

In v5.1, monitoring the cluster health, be it single box deployment, 3 nodes compact cluster, or a full cluster, is done leveraging industry-standard Prometheus exporters. 

The monitoring endpoints are exposed using the same commands introduced in v5.0.5 with options for manager status and cluster status. The implementation, however, is Prometheus based.

Learn more about the new monitoring infra and creating new metrics

Upgrading to v5.1

Upgrading to v5.1 is supported from the following versions: v4.3.3, v4.4, v4.5, v4.5.5, v4.6, v5.0.5.

Read more about the upgrade process

Other Improvements

v5.1 includes many medium/minor enhancements and improvements:

• Smart Resume for install workflow – upon resume of a failed or cancelled install workflow, Cloudify will first uninstall all nodes in intermediate stage (e.g. ‘creating’) and only then apply the install on all non started nodes.
• Easier flow for user password modification through the account drop-down actions.
• New look and feel for the Cloudify Catalog page.
• Added zoom in-out and fit to view options to the Execution Task Graph.
• Lazy plugin load – Starting v5.1 plugins are installed on the manager/agent only upon first execution, not during plugin upload. The plugin upload command was extended to support plugin validation.
• Add the `-t, –tenant-name` option to `cfy agents` commands.
• New link in the Cloudify dashboard with quick access to the getting-started guide.
• The hello-world and quick start wizards no longer appear by default in the dashboard page.
• The Sites Map widget is available under both the Premium and Spire license.
• Modifying the objects location in the topology view can now be saved/reset. A view saved in the Composer will appear the same in the blueprint/deployment topology view.
• OpenStack plugin – added support for “A” record addition/deletion.
• Added command allowing cleanup of past executions from the logs.
• Ansible plugin – added a workflow for Ansible playbook reload
• Ansible plugin – support for existing inventory file.
• Terraform plugin – added workflows for Terraform resources refresh and Terraform template reload.
• In the deployment outputs widget, outputs and capabilities are listed. This view was updated to easily identify capabilities versus regular outputs.
• All deployments in the deployments page now have a light progress bar with color indication as to their last execution status.

Support Discontinuance

Product Features

1. Diamond plugin – Cloudify declared the end-of-life of the Diamond plugin with the release of Cloudify v5.0, hence it is no longer supported in v5.1.
   That said, customers running Cloudify v4.6 or earlier and using the Diamond plugin via their agents should not be concerned about modifying their blueprints to remove such support when upgrading to 5.1.
   While the Diamond functionality will not be available, a stub mechanism was put in place to guarantee the successful operation of all historic blueprints.
   Cloudify strongly recommends that new introduced blueprints will not mention the Diamond options.
2. The PSUtil plugin is no longer supported by Cloudify starting v5.1.
3. Cloudify deployment proxy functionality is declared as deprecated with Cloudify v5.1.
   All existing functionality will remain active and supported, however no new functionality will be added to the deployment proxy, and only critical issues will be resolved.
   Cloudify recommends using service components and shared resources, introduced with Cloudify v5.0, replacing and improving the deployment proxy functionality instead.

Supported Versions

Listed below are the support discontinuance dates for the recent Cloudify versions. As of these dates, the respective versions will no longer be supported under the standard Cloudify support agreement.

Release date: October 5, 2020