Cloudify 5.0.5 Release Notes
Release date: January, 2020
This page describes the new features and updates introduced in both v5.0.0 and v5.0.5, referenced as 5.0.x.
Cloudify at scale
A set of features and architectural changes improving Cloudify’s cluster scaling and load-sharing abilities alongside easier Cloud deployment with full support for private/public clouds databases as a service. Other aspects of this theme include better abilities to manage and control a large number of deployments over multiple locations.
Service composition made easier
Focused at the service designer and developer, with multiple new features simplifying the code and the user experience required for complex environments design and advanced services definition.
These new improvements open the door to huge time & cost saving through code reuse.
This major theme introduces enhancements intended for the system operators. Providing greater visibility into the processes happening under the hood such as task execution flows reduces the level of expertise required from the ops team while at the same time empowers them with easier access to the information they require and quick application of actions. Other aspects of this theme include improved maintenance flows for the Cloudify product itself.
Over 240 stories were introduced, and 370 issues and tasks were resolved as part of 5.0.x, improving all aspects from functionality to robustness and security, making it the best Cloudify orchestrator we have delivered to date.
Cloudify At Scale: Organizing The Chaos, Leveraging Site Management
Managing multiple locations with massive numbers of systems and services is challenging. Cloudify ‘Sites’ allow grouping of deployments by location, providing visibility into installations in your data centers, public cloud regions, branches, or edge sites.
The site’s concept with filtering and operations is partially available for all premium users, while the map view and UI widgets are limited to Spire license.
Enterprise-grade highly available topology
Cloudify’s high availability topology has been updated in v5.0.x to improve the availability and robustness of the Cloudify service in cases of malfunction. It further allows for easier scaling to get more work done by your Cloudify manager.
The new topology allows for either:
- Single, all-in-one managers (with no high availability or cluster option)
- Highly available cluster as described here, based on 3 services (on separate nodes) with Managers, database, and messaging queue each operating as a highly available cluster, on separate nodes.
Manager high availability
- The new cluster topology is based on an external database cluster, external RabbitMQ messaging broker, and two or more Cloudify managers set in an active-active mode such that all managers are sharing the workload.
Database high availability
- The database can be a managed database such as Amazon RDS / Aurora, Azure Database for PostgreSQL, or self-managed PostgreSQL cluster (read more)
- Alternatively, it can be a Cloudify deployed three-node PostgreSQL cluster in a Patroni based topology of leader, synchronous standby, & asynchronous standby.
Message queue high availability
- The message queue is a Cloudify deployed three-node-cluster of RabbitMQ in an active-active topology.
Public Cloud DBaaS As Cloudify’s Database
With version 5.0.5 Cloudify supports an external DBaaS. The certified solutions are Amazon Relational Database Service (RDS), Amazon Aurora, and Azure Database for PostgreSQL.
Setting up Cloudify with external DBaaS leverages public cloud support for mission-critical workloads and predictable performance, security, high availability, as well as dynamic scalability.
Cluster Status and Manager Status
Improved system health mechanism is introduced in v5.0.5, extending the manager status information with cluster status command and widget providing visibility into the operational status of the cluster (Active, degraded service, Service is down) with detailed information about each of the nodes by role type.
The cluster status command was designed to support Global Load Balancers (GLB) or other networking services as a means to determine the cluster health.
Load balancing among the active-active managers may be set through the usage of the manager status command reporting the operational health of each manager.
The manager’s health and cluster management (high availability) widgets have been updated and provide greater detail about the system’s health.
Scaling of the Cloudify manager (for more workload) is achieved via two methods: enhancing the cluster form factor (more compute power and memory) or setting more Cloudify managers in the cluster. Thanks to the new active-active approach, scaling is made easy, and each added manager will partake in the workload.
Read more about Cloudify sizing.
Flexible Deployment Options
Cloudify services are now available in the following flavors:
- rpm based deployment over RHEL/CentOS 7.6 (All-in-one / separate services)
- Container-based as docker images (All-in-one / separate services)
- All-in-one Openstack image
Download Cloudify 5.0.5
Cloudify as a Service
The Cloudify service is now available as a hosted service for trial purposes.
Service Composition Made Easy: Service Components And Shared Resources
Composing a service blueprint is easy. Keeping your blueprint code maintainable, readable and reusable is typically the challenge.
Services grow more and more complex, with a larger number of components and more complicated sub-services (e.g. a database cluster in highly available mode consisting of dozens of nodes, ports, VMs, security groups, etc. as a sub-component).
As a service designer, one should not have to describe every tiny object each time a service is required. Service components provide the answer to this.
A service component is a new node type that embeds a complete sub-service blueprint as a single node. Service components are easy to use across blueprints and keep your code organized, easy to read, maintainable, and most importantly reusable. Components can be updated independently from their hosting deployment making day 2 operations even easier.
Shared Resources are a special case of components and represent components that are used by multiple services. These can be referenced directly through the blueprint without the concern of setting them up or tearing them down as part of the deployment.
Cascading workflows from the hosting deployment to the component can be enabled/disabled, allowing for complete flexibility running day 2 operations on the relevant parts of the service.
Read more about Service components and Shared resources
Start creating your blueprints with the Cloudify Composer
5.0.x provides a makeover to the Cloudify Composer making it much easier to create your first blueprints through a drag-n-drop interface. The Cloudify composer provides a new project view organizing your blueprint packages and providing better visibility into the blueprint resources and properties.
Easy composition, switching from code to topology views, package import, and real-time code validation are available out-of-the-box.
Lifecycle operation dependency – Boost up deployment execution time
Creating a dependency between nodes sets the deployment order such that if node B depends on node A, node B will be executed after node A execution is complete. But what if that dependency is only important because node B requires some info from node A which is available at the beginning of node A’s execution. Why should we wait for node A to be completely up?
Setting a dependency on lifecycle operation allows to reduce that wait time and set more accurate dependencies thus reducing the overall execution time considerably.
Operability: Complete real-time visibility into the orchestration tasks execution
Deploying a multi-domain clustered service may consist of multiple steps with many inter-dependencies. Cloudify 5.0.x provides a fully detailed task execution graph clearly displaying the planned, nodes, execution steps and their order and dependencies. This visual tool makes it easy to track your executions in real-time, gain visibility into their progress & status, and easily identify issues and failures.
Gain A Better Understanding Of Changes Before They Take Place With Dry Run Preview
Cloudify’s intent-based model simplifies your day 2 operation and deployment maintenance and makes them extremely easy to maintain. With automation, though, comes some lack of visibility into the steps that the system is going to apply with every requested topology update. Getting a preview into the changes before they are applied provides that missing layer of visibility, allowing the operator to understand exactly what is going to take place and apply or correct if needed. Deployment update flow
Tie Into CI/CD Workflows With Secure Secret Migration
A key part of a healthy CI/CD pipeline is the ability to shift products through different environments – from Dev to QA to Staging, and to Production…
Integrating Cloudify into the pipeline allows for greater flexibility and reduces considerable time; it also requires that the Cloudify code (in the form of blueprints, plugins, and secrets) can be migrated through this process.
Secrets (secure properties) are a key part of every project & environment. A secure process for secret export from a manager (e.g. Dev) and import into another manager (e.g. QA) allows for a secure automatic flow of pipeline for Cloudify code.
In 5.0.5 we introduce a complete secret export-import flow supporting:
- A secure flow with data encryption, anti-tampering flow, and collision detection
- A mapping functionality allowing for different tenant and account names in different systems
- A new option for bulk secret import from a file
- A fully automated flow leveraging API/CLI.
Learn more about secret export & import
Reduce Human Errors Using Input Validation
An operator or any team member running a deployment is not always aware of the specifics of inputs required for that blueprint. Input constraints, allow the service designer to enforce a set of validations per input such that the user will have to comply with these. For example limit port numbers to an integer between 2500 and 5600. Specify specific values, base the validation on a pattern, etc.
Easy error correction through real-time modification of inputs and properties
The Cloudify 4.6 introduced option of resuming a failed workflow allowed the user to optimize re-run of workflows in cases of failures by resuming the workflow execution after correcting the problem from the point of failure, as opposed to teardown and re-install.
In 5.0.x we have added the option to select run-time analysis of inputs and properties, which means easier error correction during resume or other run-time workflows.
Agents In 5.0.5
Cloudify agents have been made more robust and easier to maintain through a series of improvements:
- Supporting Windows service accounts
- Installation timeout option
- Custom parameter based agent installation
- An improved agent installation script
- More detailed logging
Keeping Plugins Up-To-Speed
Plugins are the interfaces to your infrastructure and services. In an ever-changing environment, plugin changes are inevitable and should be considered as part of the ongoing maintenance of your systems.
Automatic update of plugins is now much simpler using two new 5.0.x capabilities:
- PEP440 format for plugin import is supported allowing the blueprint designer to specify a range of supported plugin versions (e.g. >=2.1, !=2.4, <3.0).
Cloudify will take the most recent available plugin matching that range.
- Bulk update of a new plugin version across multiple deployments can now be executed. Updating a plugin does not require a deployment update.
5.0.5 introduces enhancements to all of our key plugins, providing multi-cloud orchestration leveraging Ansible, Terraform, AWS (direct or via CloudFormation), Azure (direct or via ARM), OpenStack, VMWare and many more.
Cloudify Best practices
Cloudify 5.0.5 is accompanied by a series of best practices articles and examples, demonstrating the new service composition approach allowing for granularity and code reuse via components and shared resources, as well as many other recommended approaches and tricks.
Check our best practices section.
Upgrading to 5.0.5
Upgrade to 5.0.5 is supported from the following versions: v4.3.x, v4.4, v4.5, v4.5.5, v4.6, v5.0.
Read more about the upgrade process
5.0.x includes many medium/minor enhancements and improvements:
- The rest.conf and mgmt.conf configurations are now removed from the config files and are part of the Cloudify database. During an upgrade, all configurations are migrated to the database automatically. A set of CLI commands allow further setup.
- An execution token is introduced, replacing the REST token and used by workflows to access the REST service. This removed the 10 hour limit on execution time while keeping security.
- Script plugin supports implementation from URL.
- New interfaces added to the default lifecycle workflows: create/delete validation interface, precreate/prestop & poststart/postdelete lifecycle interfaces.
- Deployment logs can now be kept after the deployment is deleted. Read more
- Direct update of runtime properties is available via the CLI. Read more
- A timeout can be specified per each operation. Read more
- The ‘hooks’ mechanism now supports managed plugins allowing extensions for REST or other interfaces.
- New ‘Executions status graph’ widget displays the number of executions by status.
- The workflow execution menu is now categorized for easier access and use.