Cloudify 3.4 Release Notes

3.4.2 Service Pack (16-March-2017)

Download Release Notes in PDF

New Features

• CFY-6382 The service_name property to has been added to the Cloudify agent AgentConfig data type.

Fixed Issues

• CFY-6516 Deleting deployments in parallel no longer causes the REST service to be unresponsive.
• CFY-6504 The issue in which an offline bootstrap failed due to a dsl_resources input bug has been resolved.
• CFY-6498 It is now possible to specify a file path for the manager_resources_package input.
• CFY-6473 An infinite loop in dispatch no longer occurs if the workflow executor gets to an invalid state.
• CFY-6333 Snapshots no longer fail due to the snapshot file being too big.

Tasks

• CFY-6554 An openstack-windows-winrm-blueprint has been added to the cloudify-hello-world example.
• CFY-6381 A Cloudify agent storage directory has been created under %PROGRAMDATA%.
• CFY-6380 Cloudify agent installations on Windows are located under %PROGRAM FILES%.

3.4.1 Service Pack (15-January-2017)

Download Release Notes in PDF

Release Highlights

• CFY-5996 An automated process has been created that ensures that Cloudify agents connect to the correct Cloudify Manager IP in the event that Cloudify Manager has more than one private IP address.
• CFY-6162 – To optimize performance, as a response from the REST API, the maximum value that can be set for a pages is 1000 records.
• CFY-5108 Import resolver rules have been externalized as an input so you are able to add your own rules, without requiring to modify the Cloudify Manager Blueprint.
• CFY-6149 A number of changes have been implemented to prevent the situation in which Cloudify Manager has insufficient memory to support resources, causing it to freeze. The following functionality has been implemented.

• CFY-6161 The number of Gunicorn instances has been limited to a number that the system can handle.
• CFY-6162 A maximum limit has been set for data that can be returned from the Web server in response to a request by a client.
• CFY-6163 The reasonable default value set in CFY-6162 is configurable.
• CFY-6164 A maximum limit has been set for data that can be returned from an Elasticsearch request.
• CFY-6129 The system can detect the current available memory in Cloudify Manager. If the available size is less than the configurable size, the request is dropped.

Fixed Issues

• CFY-6251 –The issue in which Windows agents had a memory leak has been addressed.
• CFY-6052 – When copying key pairs, the existence of the destination directory is verified. If it does not exist, it is created.
• CFY-3902 – The issue in which the Diamond agent was not restarted when a VM was restarted has been addressed.
• CFY-4233 – Unicode characters are not supported in Blueprints. Strings in Blueprints are now checked to ensure there are no unicode characters.
• CFY-6210 – Paging of blueprints and deployments screens is no longer limited to five pages.
• CFY-6211 – A default sort is now implemented on the deployments screen.
• CFY-6212 – The names of fields/columns are used consistently across blueprints and deployments screens.
• CFY-6215 – ‘Node instances’ screen shows last deployment according to deployments screen where the sort is wrong and first in the table deployment is not last

Known Issues

• CFY-6301 – If you have a running deployment and you upgrade the Cloudify agent from 3.4 to 3.4.1, performing an uninstall and then install on this deployment will not work as expected. If you wish to perform uninstall and then install on the deployment, perform the workaround instructions after uninstall and before install.
  Workaround: Delete the following from the host machine on which the agent is deployed:

• /home/centos/cfy-agent/vm_xxxxx.json
• /home/centos/vm_xxxxx/

Release 3.4.0.2 Patch (22-SEP-2016)

Download Release Notes in PDF

Release Highlights

• CFY-5606 Increase auto-generated ID section’s range. Cloudify auto-generates 5 hexa characters to be appended to every node instance’s name, this may not be enough in some scenarios, both in testing and real-world ones. The range should be increased.

Fixed Issues

• CFY-5971 Windows 3.2.1 blueprints broken on new managers.
• CFY-5866 Deployment update resource removal isn’t in order. When removing a set of nodes with dependencies between them, the order of removal isn’t as it should be. E.g: trying to remove the “A connected_to B” graph, will result in trying to remove B first, and only then A. This should obviously be reversed.
• CFY-5782 Deployment update broken for deployments migrated from 3.2.1.
• CFY-5698 Management worker sporadically doesn’t restart after reboot.
• CFY-5640 Snapshot restore fails on missing broker key. When trying to restore a snapshot from a manager that has stuck deployments it has empty dicts which makes the restore process to fail.
• CFY-5632 Snapshot restore may fail on elasticsearch timeout. When a snapshot is restored using elasticseach.helpers.bulk on a large snapshot, the operation may fail due to read timeout.
• CFY-5631 Dereferencing anchors aren’t properly supported in blueprints. If an anchor holds some value which is overridden in the dereferenced object, the final value is sometimes the dereferenced one, and sometimes the overridden one.
• CFY-5493 Scale workflow failure when scaling a node template which is a part of a group.
• CFY-5453 Deployment’s ‘updated_at’ field isn’t updated on a deployment update. Deployment objects have a ‘updated_at’ field. Unfortunately, it doesn’t actually get updated on a deployment update.

Use the online HTML editor to maximize your web content composing efficiency.

Release 3.4.0.1 Patch (16-AUG-2016)

Download Release Notes in PDF

Release Highlights

• • Add Telecom Edition, for more information see.

Telco Related Packages:

• • Overlay Service Chaining
  • Netconf Plugin
  • TOSCA-YANG Data Modeling Interoperability
  • Clearwater vIMS Blueprint
  • F5 BigIP Plugin

Fixed Issues

• UI – fix bug where deployment topology view is incorrect
• UI – fix bug where blueprints -> plugins view doesn’t show any plugins

Release 3.4.0 (3-JUL-2016)

Download Release Notes in PDF

Feature Highlights

Orchestration

• Deployment Update – Cloudify now allows updating application deployments, offering application operators and developers the ability to make topology changes and include new resources to running TOSCA deployments.
• Blueprint Composition – Cloudify supports composition of blueprints which enhances the support for microservices architectures and allows decomposing large application blueprints into separate application and infrastructure blueprints.
• Resource Grouping – Modeling of groups is now supported. This allows users to scale one-to-one relationships, and groups of resource templates.
• NIC Ordering – Respecting relationship order in blueprints, enabling users to determine which network is assigned to which vNIC, including control over the management network.
• Infrastructure + OpenStack plugin support. Other plugin updates to follow.

Cloudify Manager

• In-Place Manager Upgrade – A new Cloudify Manager upgrade process was introduced, which allows in-place upgrades for Cloudify Manager components and offers easier maintenance of the manager.
• Manager Maintenance Mode – Allows the Cloudify administrator to halt running operations in a safe manner, while preventing users from triggering new executions, in order to maintain and upgrade the Cloudify manager.
• Thousands of Deployments Scalability – Overhaul of the manager workers architecture improves the scalability of Cloudify Manager by removing the dedicated deployment workers and consolidating all task executions to be handled by the Cloudify Management Worker

Manager Bootstrap

Installation of the Cloudify Manager is now less error prone with embedded python packages and single package of all components needed for bootstrap.

• Bootstrap Cloudify Manager using a single tar file instead of many archives.
• Improved and more clear bootstrap logs.
• Perform validations on bootstrap.
• Plugins are not automatically uploaded to the manager during bootstrap.
• In addition to existing Cloudify Manager bootstrap methods, Cloudify now offers AWS AMI images and a QCOW image for OpenStack for simpler and quicker installation.

Web User Interface

Cloudify Web UI layout was improved for better use of screen real estate, and support for Plugin management and snapshots were introduced. This includes:

• New and improved UI layout
• New Snapshots management view
• New Plugins management view
• Improved Composer topology auto-layout mechanism
• Support of Groups in topology view
• Executions can now be cancelled from executions tab in UI
• Update deployment topology
• Manager maintenance mode

Cloudify Composer

Cloudify Composer is a graphical editor for creating YAML blueprints dynamically using a modern, drag and drop interface. Cloudify Composer enables topology modeling for complex applications while providing a means to implement relevant lifecycle operations via external plugins and scripts. Additional features include:

• Script based installation for Linux/Mac
• Stencil management – In addition to the built-in node types, Cloudify Composer allows loading and managing additional node types.
• Auto layout improvements
• Layout persistency
• Import existing packaged blueprints in the composer canvas
• Resource management – Cloudify Composer allows managing resources accompanying the blueprint

Cloudify CLI

• Cloudify CLI Improvements – Improvements in the CLI make it easier to install blueprints using a single command – ‘cfy install’ – which eases onboarding of new applications. Commands ‘cfy status’ and ‘cfy logs’ were updated with more robust information.
• Improved CLI installation – Python is no longer needed as it is bundled in Cloudify CLI packages. Wagon is now included in Cloudify CLI.

Plugins

• • Azure Plugin – An all new Microsoft Azure plugin allows orchestration of Microsoft Azure IaaS with all of it’s resources, including storage, network and compute with support for Windows and Linux.
  • Amazon AWS Plugin improvements with support for VPC and additional enhancements

• VMware plugins including vCloud and vSphere are now open source – all VMWare plugins are now open source and available in the Cloudify Community edition.

New Features

The following new features are available in this release:

• • CFY-5568 – Hot-keys support has been added to snapshots page in UI.
  • CFY-3210 – Fabric-plugin v1.4.1 now supports passing a `hide_output` list to all execution methods (e.g. `run_script`, `run_commands`, etc..). The list can receive the output groups described here and will hide unwanted output from showing up. For instance, provided a list with a single item called `running`, any `run: …`, `put: …` or `get: …` messages will be hidden.
  • CFY-5382 – The manager blueprints now use fabric-plugin 1.4.1 and use its `hide_output` feature to clean up bootstrap logs. The hidden output is the `running` group only to hide any `put`, `get` and `run` commands.
  • CFY-5276 – Fabric-plugin v1.4.1 now supports passing a boolean `use_sudo` input to `run_commands` and `run_script`. This allows to run scripts or commands as sudo without stating “sudo” within the commands or scripts themselves. Coupled with the `sudo_prefix` env property in fabric, this can be used to execute a script or command using an alternative implementation of sudo.
  • CFY-5301 – URLs for manager resources are no longer exposed via the manager blueprints. Instead, only the single resources package’s url is exposed. All input files contain empty strings where the URLs of those resources previously resided. Now, supplying a URL in one of those resource fields, will override what’s in the manager resources package. One culprit is that you cannot currently provide URLs to files with the same name as they will override one another.
  • CFY-4855 – Bootstrap Validations

Note that you can ignore validations by changing the `ignore_bootstrap_validations` input to true

• • CFY-4904 – We now verify that the manager’s resources package is available for download
  • CFY-5006 – We now verify that the manager is being bootstrapped only on the supported distributions. (currently, Centos 7.x and RHEL 7.x)
  • CFY-5007 – We now verify that there is enough memory (3792MB) for the manager. These are MINIMAL requirements and can vary by usage.
  • CFY-4771 – We now verify that the heap size provided for elasticsearch does not exceed the machine’s memory minus some memory gap for other services
  • CFY-5345 – We now verify that there are 5GB of free disk space on the host.
  • CFY-5381 – We now verify that the manager’s host is running Python 2.7.
  • CFY-5336 – Validations are documented under Manager/Prerequisites and Manager/bootstrapping
  • CFY-5300 – Bootstrap Validation values for physical memory, disk space and heap size memory gap are configurable via manager blueprint inputs.
  • CFY-5335 – Wagon is now a dependency of the CLI. This means that you can use wagon’s CLI to install wagons in your local environment. After creating your wagon, simply run `(sudo) wagon install -s WAGON_PATH`
  • CFY-5100 – The ctx-py wrapper’s logger now has a `warning` severity level. Before, there was only `warn`.
  • CFY-5000 – Manager bootstrap improvements:
  • CFY-5008, CFY-5011 – Make marketplace image builder work on openstack as well as AWS.
  • CFY-5012 – Ensure SSH host keys are regenerated when marketplace image is configured.
  • CFY-5010 – Ensure all IPs in configurations are correctly updated when image builder is used.
  • CFY-5013 – Deploy specified user SSH keys to authorized_keys in image builder configuration blueprints.
  • CFY-4929 – Deployment update
  • CFY-5083 – Enable overriding default ‘update’ workflow with custom workflow.
  • CFY-5344 – skipping install/uninstall related operations in deployment update.
  • CFY-5352 – support of add/remove/modify deployment description.
  • CFY-5348 – supporting passing several entries through the same rest request body. All of the body will still be passed as a stream.
  • CFY-5087 – added resources under deployment context. These resources are a part of the deployment and not the blueprints. Each deployment has access to both its blueprint resources and deployment resource. If conflict arises, the deployment resource takes precedence.
  • CFY-4930 – support for intrinsic functions.
  • CFY-5088 – support for adding/removing and modifying a workflow.
  • CFY-5247 – support for add/remove/modify outputs.
  • CFY-5121 – Detect All the differences between the current deployment and the proposed deployment update. Then, ‘translate’ those differences into standard api requests that can be executed by the Cloudify Manager.
  • CFY-4927 – Create CLI commands for deployment update.
  • CFY-5277 – support for relationship ordering modifications.
  • CFY-5376 – add “Failed” state for deployment update object. Since no two updates can run concurrently, and the update workflow is executed asynchronously, if a user wishes to execute another update / rollback after a failed update, they must issue the update command with a force flag which will set the previous update into a failed state and continue to execute the next update.
  • CFY-4983 – Verify that every change detected by the deployment update step extraction mechanism is supported as part of the deployment update process. For example, currently, adding a node is supported, but adding a policy type is not.
  • CFY-5096 – After all the changes (‘steps’) that will be executed as part of the deployment update are extracted, they should be sorted by their order of execution. For example, we should remove a node only after removing all the relationships it is a part of.
  • CFY-5273 – The initial mechanism that determined what changes were a part of a deployment update was based on .yaml blueprint files and resided in the dsl parser. The updated mechanism for this purpose is based on .json deployment plans and resides in the rest service. As a result, the old mechanism was no longer needed.
  • CFY-4863 – Wagons that are uploaded to the manager using the REST API are now installed as part of the upload process (if applicable) and uninstalled on plugin delete.
  • CFY-5260 – Scale workflow has been updated to support scaling group names. The `node_id` parameters has been renamed to `scalable_entity_name` and the default value of `scale_compute` has been changed from true to false. The scale compute value is ignored when a group name is provided. The previous implementation has been kept to support older types.yaml versions.
  • CFY-4953 – Have preupgrade validations run before upgrade process begins: We added validation scripts to each node’s cloudify.interfaces.validation.creation interface, in the manager blueprint. Inside the script, an “is upgrade” check is used to only run the validations while upgrading, not while bootstrapping. The scripts perform basic checks, such as examining the directory structure, checking that required services are running.
  • CFY-5255 – per-node sanity checks on bootstrap/upgrade/rollback: We added some sanity checks to the end of each node’s start script (in the manager blueprint). The scripts perform basic checks, eg. the elasticsearch script runs a few queries against it, to see if it’s running and if the cloudify_storage index exists.
  • CFY-5166 – Set order of Openstack Server NICs according to relationship order: Changed the openstack VM creation code to use the order of the VM-network relationships, when passing networks to which a VM should be connected to. If not explicitly in a relationship, the management network will be automatically set as the first openstack network.
  • CFY-5285 – `cfy agents install` doesn’t respect ctrl+c: The `cfy agents install` command spawns several threads to run operations concurrently. The method of waiting on these threads that was used, made the CLI unresponsive to signals (eg. ctrl+C). We fixed it by using python’s daemon threads instead. No other CLI commands used that method – all the commands remain responsive to signals.
  • CFY-5286 – REST Service: GET / fails with an unhelpful error: The maintenance mode handler tried to examine the endpoint of unrouted request – which was breaking, turning 404s into 500s. We made the maintenance mode handler just ignore requests that couldn’t be routed (404s)
  • CFY-5308 – CLI: installing plugins can use the global pip: in some circumstances, installing plugins from the CLI might errorneously install them to the system python, instead of a virtualenv. We fixed it by using the correct way (examining sys.executable) to discover the pip executable.
  • CFY-5330 – Openstack: use the port from the relationship to connect vm to a network: Openstack VMs that have relationship to an openstack port, would be connected to the network this port was on, but we didn’t guarantee that this particular port would be used. We changed the VM creation code to explicitly use the port that was passed in the relationship.
  • CFY-5116 – manager sanity test.
  • CFY-5225 – Data migration for persistent services.
  • CFY-4955 – Rollback support for manager upgrade.
  • CFY-5332 – Expose private IP as output in manager blueprint.
  • CFY-5306 – Rollback CLI command.
  • CFY-5071 – Manager-CLI version compatibility validation.
  • CFY-5271 – Display group information in UI. Displaying in which group is the node, and allowing to select autocompleted groups when running scale workflow.
  • CFY-5342 – deployment update. Enabling running ‘update’ action on a deployment and showing update in process.
  • CFY-5366 – maintenance activating – poll running executions. Polling remaining executions that prevent maintenance, presenting and allowing to cancel them.
  • CFY-5284 – Add a Snapshots view to UI, which allows the user to create/import a snapshot, download or restore from a snapshot, or delete one.
  • CFY-4966 – Add a Plugins view to UI, which allows the user to see all available plugins in manager, upload a plugin, download or delete it
  • CFY-5002 – CLI package now brings its own python and plugins, no need to compile on target machine (except for vSphere and vCloud in part of the cases), single rpm that runs on centos 6.5+ and rhel 6.5+, single deb file that runs on debian variants 7+ including ubuntu 14+, a Mac OS package (untested and not automated yet).
  • CFY-3461/CFY-3630 – By default, Manager Blueprints now consume only one resource (tar.gz file) which contains all of the manager’s resources (+agents). The `manager_resources_package_url` inputs indicates its location. The `manager_resources_package_checksum_file` and `skip_checksum_validation` inputs allow to configure md5 checksum validation on this package. By default, md5 checksum validation does not occur as to not force users to make the checksum file accessible over the internet. Bootstrap should be faster now as only a single file is downloaded. Overriding of any package within the archive is possible by explicitly specifying another package in any of the different resource inputs (either Cloudify or External components or any of the agents). If the `manager_resources_package_url` is not provided, the previous bootstrap process takes place.
  • CFY-5160 – AWS Plugin 1.4.1 – Bundles VPC Manager + Client requests
  • CFY-4949 – AWS Plugin – Tag All Resources with UUID, Node Instance ID, Deployment ID + Optional Name property
  • CFY-5168 – When a user attaches a Volume to an Instance, the volume ID is now available in a runtime property of the instance (supports multiple volumes), and vice-versa. (also handles detach)
  • CFY-5161 – There was no added properties to the provider context for default subnet deployment. This will not be exploited by a new manager blueprint right now, since we are favoring being explicit over being implicit right now, but I think that offering these parameters via the provider context is still desireable.
  • CFY-5162, CFY-5163 – Similarly, when a user makes use of the provider context to create a security group or an Elastic IP, there is a problem with deploying it to VPC in 1.3.1-1.4. This adds logic to handle deployment to VPC (and to the correct VPC).
  • CFY-4892 – Handles an issue where security group needs to idempotently retry create operation before adding rules to the security group
  • CFY-5150 – Docker Plugin version 1.3.2 – Updated the Docker Py client library version, handles an error in the system tests connection to Docker host.
  • CFY-2216 – Relationship order is maintained for node instances and the install/uninstall workflows execute relationship operations in the order specified by the relationships defined in the blueprint. (uninstall works in reverse, scale and heal workflows are also properly handled)
  • CFY-2233 – Group resources for supporting one-to-one relationship semantics within a group.
  • CFY-1580 – node_templates.node_name.capabilities.scalable.properties and policies.policy_name.properties now accept get_input and get_attribute. This can be used to inject the number of instances for a node/group from inputs/properties
  • CFY-5221 – Using get_attribute when there are multiple node instances is now supported of the “calling” node instances and the “requested” node instance belong to some shared group in which the “requested” has a single node instance in that group instance.
  • CFY-5071 – Validate that CLI version and manager version are compatible.
  • CFY-4929 – Deployment update:
  • CFY-5036 – (CFY-5084, CFY-4965) adding/removing nodes and relationships.
  • CFY-4964 – adding nodes with multiple instances.
  • CFY-5085 – Adding/modifying/removing interface operations.
  • CFY-5089 – Adding/modifying/removing node properties.
  • CFY-5142 – Add the ability to store resources in the context of a deployment.
  • CFY-5106 – Enhance detailed view of ‘task_failed’ and ‘task_rescheduled’ events.
  • CFY-5034 – Add maintenance mode to UI.
  • CFY-5201 – Add padding to cloudify header
  • CFY-5031 – UI hotkeys for a complete keyboard experience.
  • CFY-4952 – Manager in-place upgrade support. Reuse the manager’s install workflow scripts to upgrade a running manager. As part of this process, we now save every resource/node input that was used during the install, so that it can be used for rollback in case of upgrade failure. In addition a ‘use_existing_on_upgrade’ flag has been added to the manager-types to allow reuse on properties from the install process during upgrade.
  • CFY-5203 – Rewrite Azure Plugin Epic – Total rewrite of the Azure Plugin – implements base classes for handling of Azure Client, multiple resource groups, Windows VM.
  • CFY-4937 – Implement a maintenance-mode command in the CLI.
  • CFY-4852 – Add feature to allow to download manager logs from CLI.
  • CFY-5082 – Only create Riemann core for a deployment if policies were actually used in the blueprint.
  • CFY-4168 – Improve UI/Composer topology auto layout mechanism.
  • CFY-5051 – CLI help shows default values for arguments.
  • CFY-4401 – CLI help is now consistent across commands with fixes in wording
  • CFY-4988 Blueprints difference utility is introduced as part of application deployment update feature, this tool helps summarizing the changes that would acure to the running deployment.
  • CFY-5079/CFY-5101 Support for creating tmux based collaborative ssh sessions.
  • CFY-4824 – Blueprint Composition
  • CFY-4825 – Extended CLI inputs functionality

You can now pass multiple –inputs flags (e.g. `cfy bootstrap … -i x.yaml -i y.yaml`)
You can now pass wildcards for passing multiple input files (e.g. `cfy bootstrap -i *.yaml`)
You can now pass directories containing input files (e.g. `cfy bootstrap -i my_inputs_file_dir/`)

• • CFY-4826 – You can now import `node_templates`, `inputs` and `outputs` multiple times.

Our docs reflect that you can now import `inputs`, `node_templates` and `outputs` and how to use the CLI to pass multiple flags.

• • CFY-4691 – Added native support for using ctx in the fabric plugin when using python scripts. You can now run Python scripts via the fabric plugin while using ctx the same way you would do when using the script plugin (i.e. `from cloudify import ctx`). There is some functionality missing like workflow parameters from cloudify.state, but it provides the basics for now. You can still call ctx for any bash command you need by calling `ctx(COMMAND_REF)` (e.g. `node_id = ctx(node id)`). See https://github.com/cloudify-cosmo/cloudify-plugins-common/blob/master/cloudify/ctx_wrappers/ctx-py.py
  • CFY-4853 – Converted all Bash scripts in the manager blueprints to Python. These use the new ctx wrapper mentioned above. The transition was made to allow us to perform more complicated actions and parse complex data structures. As a result of this refactoring, CFY-4606 (bootstrap idempotency bug) was solved. It is important to note that we in no way officially declare that our bootstrap process is idempotent yet.
  • CFY-4956 – .git folder will now be ignored when archiving the manager deployment during bootstrap; additionally, there’s now a validation over the size of the manager deployment to ensure no other unnecessary resources get archived (max 50MB).
  • CFY-4832 – improve executions page : Deployments execution page, now shows is the execution is a system workflow and what are his parameters.
  • CFY-4762 – management worker’s log severity level is now configurable via the `management_worker_log_level` input in the manager blueprint.
  • CFY-4876CLI now accepts several verbosity levels.
  • -v will display tracebacks
  • -vv will also display debug log messages
  • -vvv will also display debug logging for loggers defined in config.yaml
  • CLI commands that also display events/logs now accept an optional –json flag that will make each event/log be emitted as json in its own line, to be easily parsable by external processes
  • Debug log messages are now stored in elasticsearch. They can be viewed in the cli by passing -vv (verbosity level 2)
  • CFY-4860plugin.workdir added as an API for plugins to get a work directory that is unique to the (deployment, plugin) tuple

plugin.prefix can be used to get the prefix where the plugin was installed
These two features were used to fix the diamond plugin for 3.4 (which broke to some extent after smart deployments was added)

• • CFY-4931 Added Windows Support to Host Pool

added port specifier for Windows hosts
added proper agent inputs for Windows hosts

• • CFY-4940 Added init script, debug loggers, and new lifecycle scripts
  • CFY-4959 Host Pool Plugin/Service Major Rewrite
  • Rewrote API server
  • Added service logging
  • Changed backend to TinyDB
  • Added mixed-OS pool support
  • Removed requirement for config YAML file
  • Added non-daemon service option
  • Added flask-restful exception handling
  • CFY-5035 OS/Tags Filtering in Host Pool Service

Add support for adding os property and tags, as a list, to pool hosts and filtering during allocate.

Fixed Issues

The following new feature is available in this pre release:

• CFY-5574 – Enable cancel execution from executions tab – Execution tab in deployment view is now updating live and executions table is ordered from most recent to last with an added column thats enable users to cancel running executions.
• CFY-5642 – Wrong error reported when running “cfy node-instances list” for a non-existing deployment.
• CFY-5585 – Maintenance mode strings have been improved.
• CFY-5531 – cfy logs get is inconsistent with other download functions in CLI.
• CFY-5499 – Improved validation error output on bootstrap.
• CFY-5004 – Remove Plugins from Manager (blueprints).
• WAGON-41 – Wagon v0.3.1 broke creation and installation of Wagons on Windows in situations where `pip` is not available in the path but rather under the `scripts` dir under Python’s installation directory. V0.3.2 fixes this issue fixes it by referring to the correct pip path.
• CFY-5450 – Update wagon to 0.3.2 across the board
• CFY-5586 – Execution parameters schema in elasticsearch will no longer be indexed, allowing for different executions to have parameters of the same name with a different schema (including executions of different workflows and on different deployments)
• CFY-5536 – Several step types are no longer supported in the deployment update mechanism:
• Adding/removing/changing the target of a contained_in relationship. (or any relationship derived from contained_in)
• Changing node type.
• In both cases, in order to make those changes, renaming of the node is required. The old node would be deleted and a new one would be installed (This behaviour is consistent with those change – changing the contained in or modifying node type).

• • CFY-5559 – Adding a relationship between two added nodes sporadically failed. Now we add the relationships in regard to the topological order of the added nodes subgraph.
  • CFY-5516 – Can’t scale after dialog opened for the second time in UI.
  • CFY-5502 – Scale workflow in UI – only groups are shown for scaling. It will now update the view even if nodes are loaded after groups + will now present a proper message if loading scale entities fails.
  • CFY-5461 – “scale_compute” parameter in scale dialog. The parameter will not be shown a group is chosen.
  • CFY-5462 – Improve maintenance mode dialog message.
  • CFY-5384 – Made the “install_python_compilers” flag in the manager blueprint actually install gcc and python-devel – it is required for installing some plugins (eg. openstack) from source.
  • CFY-5469 – Made the rabbitmq systemd unit file more reliable, using rabbitmqctl and a pidfile for stopping and starting.
  • CFY-5405 – Fixed agent migration for deployments using old pre 3.3 types.yaml
  • CFY-5401 – Fixed bug related to manager plugins not getting installed on a new manager when using snapshots based manager recovery to upgrade from an old manager (3.2.1, 3.3.1) to a new manager (3.4)
  • CFY-5375 – Fixed minor bugs in agent migration.
  • CFY-5433 – Fixed a bug where a plugin defining both workflows and central deployment agent operations, failed to install during deployment creation.
  • CFY-5481 – Fixed a bug where “partially” installed deployment that were migrated from an old manager to a new manager, may cause uninstall to fail unexpectedly (when uninstall should basically ignore all errors)
  • CFY-5492 – Fixed a bug where the check for concurrent deployment modifications (scale workflows) was global rather than on a per deployment basis. It is perfectly fine for two deployments to run the scale workflow concurrently.
  • CFY-5505 – Bug fix minor improvement in the scale workflow – If the delta parameter is passed as a string, it will be properly converted to an integer so one can now do `cfy executions start -p ‘delta=-1’` where previously it would fail with `-1` being parsed as string
  • WAGON-36/CFY-5365 – Fixed a bug in wagon in which it would use the python that’s in the path instead of the one that’s relative to the python being used to create/install the package. We now use Wagon 0.3.1 in Cloudify.
  • WAGON-35, CFY-5294 – When creating a wagon in the same directory where the package you’re trying to create resides, wagon would fail stating that the directory already exists. Wagon now uses a temporary working directory to generate the archive so that shouldn’t happen anymore.
  • CFY-2792 – CFY-5335 implicitly solves the inability to install plugins in the CLI
  • CFY-5097 – We now delete the built in sysvinit config file if it exists and leave only the systemd config provided with Cloudify
  • CFY-5122 – Previously, we used `sys.exit` in some scripts of the manager blueprints. We no longer do that and only use `ctx.abort_operation`.
  • CFY-5259 – We no longer declare that our code is “proprietary” in the docs.
  • CFY-4439 – Our upsteam configuration in nginx now uses 127.0.0.1 instead of localhost to prevent IPv6 conflicts.
  • CFY-5216 – Nginx timeouts have changed across the board and read, write and connect timeouts are now set to 330s. This aims to solve the problem where while gunicorn allowed for 300s timeouts on REST requests, Nginx timed out. This affected deployment creation, for instance.
  • CFY-5340 – The UI’s texts in different areas should now be clearer, more consistent, and grammatically correct.
  • CFY-5326 – The CLIs help texts and all log messages were refined to be more consistent and elaborate. Many syntactical and grammatical issues were addressed.
  • CFY-5253 – Openstack manager blueprint now creates a class B management network subnet rather than a class C one – supporting more than 253 additional hosts on the subnet.
  • CFY-5331 – fixed a bug in cfy version command when trying to retrieve the manager’s version.
  • CFY-5009 – Fixed a bug where the manager’s file server was accessible on any port including via public IP.
  • CFY-4880 – fixed misleading message about execution timing out in the CLI.
  • CFY-4663 – added validation for successful operation on associating a floating IP with a server in the Openstack plugin.
  • CFY-5110 – fixed incorrect validations for keypair creation on Openstack.
  • CFY-4216 – fixed a bug where CLI “snapshots restore” command wouldn’t fail even if the restoration of the snapshot did.
  • CFY-4679 – better handling of failed shards in Elasticsearch.
  • CFY-4929 – deployment update fixes
  • CFY-5254 – the get deployment update endpoint was mistakenly deleted, CFY-5347 – cleanup endpoints and rename others
  • CFY-5299 – updating deployment failed to import local resources (only online ones).
  • CFY-5304 – removed operations were not deleted but set to an empty template operation. This is not the wanted behavior.
  • CFY-5412 – deployment update object aren’t deleted when deleting a deployment.
  • CFY-5295 – Before a task is executed for the first time during a workflow, the agent is “pinged” to verify it is alive and functioning. The call made to do so had a pretty short default time (1 second). The timeout has been increased to 30 seconds and this error is now considered recoverable so that retries are attempted in such cases.
  • CFY-5264 – Tracebacks of error raised in all operations and workflows are now logged to the deployment log file.
  • CFY-5187 – Safely closing AMQP connection
  • CFY-4683 – Elasticsearch clustering is now set to `false` by default to prevent clustering of nodes on the same network. There are now two inputs allowing users to control whether they want to enable multicast discovery (clustering) and on which port discovery should occur – `elasticsearch_clustering_enabled` and `elasticsearch_clustering_discovery_port`.
  • CFY-5128 – Fixed regression in manager-blueprints transition to Python where you couldn’t yum install packages not starting with http, https or ftp. This caused a problem when trying to `install_python_compilers`.
  • CFY-5127 – A bug in RabbitMQ’s configuration prevented the correct policies to be set. Only the highest priority policies were set as we set policies on the same queues sequentially when they actually needed to be set on the same call. This is now fixed.
  • CFY-4611/CFY-3493/CFY-5069 – A log rotation bug prevented logs from rotating at all due to bad permissions set on the logrotate config files. This is now fixed. In addition, we configured log rotation to rotate based on the amount of files present in the service’s directory. The rotation settings now allow for 7x100MB log files per service.
  • CFY-5190 – Change ‘nodes instances’ to ‘node instances’ in UI.
  • CFY-5212 – Fixed auto completion of metric names in UI’s Grafana.
  • CFY-5185 – Runtime properties may get lost if runtime properties of a node instance are being updated concurrently.
  • CFY-5056 – Improved error handling for the “cfy use” command.
  • CFY-5115 – Cloudify agent (Celery) is not pulling tasks after restart.
  • CFY-4393 – Workflow succeeded event not shown in CLI.
  • CFY-5111 – Long deployment names cause errors.
  • CFY-4987 – Accessing undefined runtime_properties behave differently in Python and ctx command.
  • CFY-5070 – Fabric plugin fails if VM’s IP address matches a previous VM.
  • CFY-4805 – Improve error messages raised by DSL parser.
  • CFY-5218 – Cloudify leaks uncleanly closed sockets.
  • CFY-5175 – Diamond plugin fails to restart diamond.
  • CFY-4759 – Some CLI commands had their `deployment-id` argument optional, whereas the REST client expected them to be required.
  • CFY-5032 – If the user tries to install a blueprint using the default blueprint path and this path doesn’t exist, she is advised to explicitly supply a path using the -p/--blueprint-path flag
  • CFY-4823 – Integrated script plugin’s unit tests @ Appveyor
  • CFY-4958 – investigated possible installation issues for psutil-plugin when not running under Administrator; the issue didn’t reproduce.
  • CFY-5045 – ui embed=true: adding embed=true to the url request parameters will now work, causing the header and footer to be display no content.

• CFY-5093 – Host Pool Plugin now has locking to prevent allocating the same host twice to concurrent requests.

Known Issues

• • CFY-5493 – Cannot scale a group of instances that are also contained in another node. (e.g. a group containing a web server where the webserver is also contained in a compute node, and that compute node is not part of the group, but rather “contains” it)
  • WAGON-41– Wagon v0.3.1 breaks creation and installation of Wagons on Windows in situations where `pip` is not available in the path but rather under the `scripts` dir under Python’s installation directory.
  • CFY-5676 – Failed Execution CLI Output exits before all events are printed
  • CFY-5672 – Token auth secret_key is hardcoded in manager blueprint
  • CFY-5670 – vSphere manager blueprint default RAM validation failure
  • CFY-5648 – deployment update – changing node contained in relationship type
  • CFY-5643 – REST Service: getting node instances for nonexistent deployment returns empty rather than 404
  • CFY-5642 – Error reporting in `cfy node-instances list` is incorrect
  • CFY-5630 – Openstack plugin fails with key path with no directories
  • CFY-5626 – Can’t update a deployment with no workflows
  • CFY-5614 – Can’t update deployment’s node’s type or parent node without changing the node’s name
  • CFY-5570 – Deployment Events panel stops refreshing on cancelled execution
  • CFY-5566 – Snapshots UI doesn’t report failure to restore snapshot on dirty master
  • CFY-5561 – deployment update claims “unsupported changes” when there are no changes
  • CFY-5540 – Update process not effecting relationships when used with AWS plugin
  • CFY-5523 – cfy scale is breaking deployment if canceled during execution
  • CFY-5511 – Getting gcc error on openstack plugin since no wagon
  • CFY-5507 – Using `get_attribute` on a multi-instance node in outputs provides bad result
  • CFY-5495 – install plugins should check if required plugins are already installed.
  • CFY-5488 – Can’t pass non-string value by direct input/parameter in CLI
  • CFY-5482 – 500 error from REST service when attempting to execute a workflow with a unicode name
  • CFY-5480 – Wildcard input path in deployment creation doesn’t work properly
  • CFY-5464 – Descending Sort by date fields in REST API doesn’t work
  • CFY-5459 – JSON provided on command-line isn’t parsed
  • CFY-5415 – Deployment update validations don’t verify number of instances (scaling) wasn’t modified
  • CFY-5246 – Non-string property value in cloudify manager blueprint (cloudify_agent.env) causes exception when installing deployments

• CFY-1428 – agent VMs will have the Default security group on Openstack when there’s no provider context and when VM is connected to another security group

Deprecated Features

Breaking Changes

The following changes should be taken into consideration when upgrading to this release:
CFY-5315 – Running diamond plugin versions prior to 1.3.3 in “central_deployment_agent” mode is not compatible with this release.
The following features are deprecated and will be removed in future versions:

• “cloudify_agent” property in cloudify.nodes.Compute type. “agent_config” should be used instead.
• “install_agent” property in cloudify.nodes.Compute type. “Install_method” in the “agent_config” property should be used instead.
• “transient_deployment_workers_mode” property in manager blueprints “manager_configuration” node. Transient workers mode is no longer relevant after an internal architecture change took place as part of improving deployments handling in the manager.
• “cloudify.interfaces.host” interace in the cloudify.nodes.Compute type.
• “instances -> deploy” property in cloudify.nodes.Root. “scalable” capability should be used instead.
• “ctx.related” – ctx.source/target should be used instead.
• “ctx.capabilities” – ctx.source/target.instance.runtime_properties should be used instead.
• “ctx.plugin” no longer returns the current plugin’s name but a data structure containing more information about the current plugin. Plugin name should be retrieved using: “ctx.plugin.name”.
• “cloudify.utils.setup_default_logger”. Use “cloudify.utils.setup_logger” instead.