Booya – Cloudify 3.1 has landed and it is HOT!

The much awaited and anticipated version 3.1 has landed, and with it a number of core features that will deepen Cloudify’s orchestration, and extend our TOSCA support, which pretty much serves to provide all the rest of the goodies that are now packaged into 3.1.

TOSCA to the Max

We’ll start with our even more advanced TOSCA support. Cloudify 3.0 introduced the use of the TOSCA specification based on YAML DSL as its standard templating language, to enable the orchestration of even the most complex topologies using an industry standard format.
This version deepens this support and has added the TOSCA naming convention, as well as support for blueprint inputs, blueprint outputs, and intrinsic functions.  Aside from the importance of the standardized templating, this also enables everything topology-driven in Cloudify from Docker management & support, customized monitoring & metrics, which then makes the newly available custom policies and so forth and so on…you get the drift.  So this is one of the most important additions to this release, and we intend to align ourselves even more closely with TOSCA for upcoming releases.

Cloudify 3.1 is here. You’ll want to check this out.  Go

To remove the hurdle of coding a provider for each cloud in addition to the plugins, and learning another configuration file format, we converted our CLI to become a mini-TOSCA engine.  This now means that the bootstrap is just another blueprint that describes the manager as an application, and uses the same set of plugins you would then later use with your business applications.
We’ll have a dedicated post on all this TOSCA-ness coming very soon.

Topology-Driven Everything

We’ve spoken a lot about what’s coming in terms of topology-driven customizable monitoring and metrics and the entire policy engine that goes with it.  Well, it’s finally here, and it’s pretty nifty.
Cloudify’s topology-driven monitoring and management makes it possible to track not only the state of the application, but also the status of deployment, updates, and scaling processes through a single view. Cloudify allows users to install monitoring agents, such as Diamond or Collectd, configure them and collect metrics for all application components. These metrics will be sent to a built-in metrics database based on InfluxDB, and provide insights about application availability and performance. The Cloudify GUI embeds Grafana with a customizable, monitoring dashboard to allow users to view and correlate metrics of the same node or compare them across nodes.
By allowing users to define and push policies for application components to the policy engine through the topology-driven monitoring, the Cloudify policy engine provides a mechanism that can automatically react to any monitored event with appropriate corrective measures, including the ability to auto-scale and self-heal as needed.  This is all built on best of breed and extremely scalable frameworks, including Celery, RabbitMQ, Elasticsearch, Riemann.io and others to ensure this production-proven.
And we’re not talking small-scale either.
These policies will process large streams of events and metrics and apply actions to them in real-time. The rules can be based on metrics values as statutes of components and free text fields. The applied actions can be the triggering of a specific workflow or an invocation of a webhook, which means that the user can ultimately customize this to trigger literally any kind of action.

DOCKERDOCKEREVERYWHERE

With Docker being the word and just getting hotter, we just had to package in some Docker goodness into this release.  This release provides built-in Docker management and support on a number of levels. On top of our built-in Docker plugin and matching types, which allows users to use Docker containers as part of any topology (that essentially provides a full-blown orchestration and automation method for large-scale Docker-based systems), this release also comes with a *sweet* Docker management environment. This means you now get pre-made Docker images for deploying your management environment, and out-of-the-box cross Linux-distribution support for a more robust experience, easier & faster upgrades for your management environments, along with a faster bootstrap process.   This, of course, was built to mesh rather nicely with the Manager Blueprint approach and the Docker and Fabric plugins (which I’m getting to now) for a seamless management environment provisioning experience.

New Plugins

On top of the core functionality added to Cloudify as a product, we had to make sure we continue to expand our library of plugins to continue to improve the Cloudify experience.  This release introduced a few important plugins, the Script, SSH Remote Execution, and SaltStack plugins.

• Script plugin
  This plugin enables easy access to the node’s context from any programming or scripting language as long as it is executed as a sub-process of the plugin. You can develop your logic for Windows or Linux in any scripting language and gain access to any context list or map using simple client calls. This is done using ZeroMQ.
• Remote Execution plugin (Fabric based)
  An officially supported Fabric plugin will be provided which will allow users to remotely execute commands/tasks and remove the agent installation requirement.
  A user will be able to provide a set of shell commands hardcoded into the blueprint or, alternatively, a file with tasks. The tasks can then be mapped to the different nodes. This will expose an agentless manner of provisioning applications.
• SaltStack Plugin
  SaltStack is a cool python configuration management tool. Cloudify now offers an integration with Saltstack. The SaltStack plugin will install minions for you, login them into the master and make sure they get classified the way you need, so they will install the stack you need, in right the order and timing.

Stepping up OpenStack Support

Cloudify 3.1 will support the OpenStack Cinder API with Volume type. This will allow users to create and attach external disks to VMs, as well as detach and delete these volumes.

More stuff in this release

Since this post is getting long, I’ll just mention shortly some of the additional stuff to check out, and let you get to it.  In this release you’ll also be able to find:

• Agents support for any distribution on both Python 2.6.x and 2.7.x.  Briding the gap between 2x and 3x has been a top priority for us here.  And this release is moving us closer to doing that.  3.1 will provide a basic matrix of agents supporting the default Python version on the major distributions. Alongside those agents, a tool will be provided which will allow users to create agents for their specific distribution with either of the supported python versions.
• GUI Enhancements
• Cloud object pages. These pages give the user reports on cloud resources consumed across different deployments. The added value of using this versus a Cloud IaaS portal is the application context reported per resource. Cloudify will have such reports for VMs, networks, floating IPs, routers and more.
• Execution history. This view gives a history of workflow executions per deployment and easy access to relevant logs.
• Better progress tracing. Improving the workflow execution trace UX with an event console that has a more CLI-style of tracing.

You can download Cloudify 3.1 now, and check out the docs.  And definitely be sure to comment and give us feedback on what you think – and what you still need.