An In-Depth Look at Cloudify 4.0 Security – Multi-Tenancy, RBAC, LDAP and more

Introduction

Just two short weeks ago, we unveiled Cloudify 4.0. This release is a huge deal in the evolution of Cloudify, and our CTO, Nati Shalom, detailed how this product is working toward a new era in Cloud Management, with orchestration-first, model-driven principles. Security is a key component in any software, and in this post, I wanted to give you an even more detailed view at the specifics of how security works for Cloudify Manager.

Test drive Cloudify Labs now and see RBAC and multi-tenancy in action!

Security, in the context of Cloudify Manager, means securing the communication with the manager and controlling who is allowed to use it to execute various operations. Secured communication is achieved by using SSL, which allows clients to validate the authenticity of the manager as well as ensure the data sent to and from it is encrypted.
Controlling access to the manager and permissions to take actions is implemented via Flask-Security, to support user authentication and authorization. We will get into more details on Cloudify’s SSL and Access Control implementation and configuration a bit later.

Watch our Cloudify 4.0 webinar on demand – the next-gen CMP.  Go

Early Stages of Cloudify Manager Security

Cloudify’s security for client access focuses on the REST service, as this is the first and only client access point to Cloudify Manager. All requests to the manager are authenticated and authorized before reaching their endpoint. For example, when a Web-UI user attempts to upload a new blueprint, a request is sent to the REST service’s “/blueprints” endpoint via port 80/443. The request will only reach the endpoint if the user is authenticated and authorized to upload blueprints.
Similarly, a user that executes the CLI command ‘cfy deployments list’ triggers a request to execute GET on “/deployments”, which will only be successful if it includes valid credentials, identifying an authorized user.
Requests generated by other HTTP clients, such as curl, must also include valid credentials – username and password (or token, as described below) – and in future versions (4.0 onwards), a tenant name as well. If credentials are missing, invalid, or represent an unauthorized user, the request fails with a “401: Unauthorized User” error.

New in Cloudify 4.0 Security

Moving forward to Cloudify 4.0, we wanted make sure security was one of our top priorities. Below is a list of the main security features in this version, which we will go through in more detail:

• Cloudify Manager is secure by default and cannot be bootstrapped in an insecure manner
• Communication between agents and the manager is encrypted
• LDAP integration is improved
• User, group, and tenant management is available (locally / LDAP)
• Multi-tenancy for isolation between tenant resources
• RBAC – private resources, plus built in roles for user and admin
• Cloudify services can be configured to run under the Cloudify user and not root

Authentication

In Cloudify 4.0, authentication is mandatory for any level of access to Cloudify Manager. Authentication is handled in one of the following ways:

1. Username and Password – Can either be local or from a user management system the manager is configured with (LDAP-based integration is native)
2. Using a token – This can be generated by Cloudify and is valid for 10 hours

Authorization

A combination of roles and permissions, and multi-tenancy, provide the framework for authorization and resource isolation. Let’s look at these in detail:
Roles & Permissions
Cloudify includes built-in user roles which users are associated with: Administrator and User. Each role has different permissions, which ensures operation-based RBAC. For example, the role User cannot perform Administrator operations, such as with snapshot management, where the User role cannot perform any actions.
Isolation
Cloudify supports the concept of users, user groups, and tenants. These can be defined locally in Cloudify or taken from an external user management system (LDAP integration is native). In the latter case, passwords are not stored in Cloudify – authentication is done via LDAP (or other system) and a token is generated and used for the user session.
A user can be associated with one or more groups, and one or more tenants. A group can also be associated with one or more tenants. An authenticated user only has access to resources that belong to that user’s tenants.
Resource isolation is implemented for blueprints, artifacts, deployments, nodes, logs, events, and plugins. An additional layer of permission control is implemented on resources, allowing private resource configuration. A resource created as private will only be visible to the user who created the resource, and not to other users within the tenant. The exception is the administrator role – which has full access to all system resources. All REST APIs, except admin APIs and the version API, require a tenant, and the operation is associated with the provided tenant.
In the case of read operations, only information of the tenant will be provided, in case of write operations the resource is added to the tenant provided. Admin APIs are for the following resources (and unavailable for users):

1. 1. Tenant management (CRUD)
   2. User management (CRUD)
   3. User group management (CRUD)
   4. Snapshot management (CRUD)
   5. Cluster management (configuration of manager HA)
   6. Maintenance mode activation/deactivation
   7. Upgrade/rollback commands

RabbitMQ isolation is achieved through the usage of virtual hosts and the association between hosts and users, enabling authorization at the queue/exchange level and resulting in isolation of queues between tenants. In this configuration it is impossible for a host VM from “tenant A” to access/request operations on host VMs that belong to “tenant B”. **This addition is expected in 4.1.

Encryption

Scope

1. Communication from the outside world to the manager and its SSL/TLS configuration will be the user’s responsibility (CA/host verification etc..), where the endpoints include the UI and REST API.
2. Communication between agents and the manager, as well as within the manager, is Cloudify’s responsibility and hence will be determined by Cloudify. Cloudify will generate the necessary certificates for the internal communication.
3. Credentials will not appear in log files (cloud / RabbitMQ / Cloudify)

Communication Channels
In order to make the architecture simpler, the number of internal communication channels was reduced in the following ways:

• Agents will poll for task execution requests by connecting to the RabbitMQ server on the manager.
• Access to file server or REST API will be done through a secured port (authn, authz, encryption) which will be controlled by Cloudify.
• Accessing REST API internally will not be handled by Cloudify – if a user enables SSL/auth over port 80 and decides to use a REST client, either from a plugin or a script, the user should configure it correctly.

Certificate Propagation
Cloudify creates the private/public keys for the transport which will be used by both RabbitMQ and file server access. The certificate is used to identify the manager, there are no agent-host certificates. The manager certificate is propagated automatically to the agent host as part of the agent installation.
Certificate propagation depends on agent installation:

• SSH/WinRM – On agent installation, Cloudify will upload the certificate to the VM running the agent. The only limitation here is that WinRM is not encrypted in Cloudify and may pose a security risk in 4.0, but this will be resolved in 4.1
• Cloud-init/Userdata – Inject the certificate as part of the agent installation script injected into the VM.
• Provided – User will put the certificate in a static location on the VM.

Nonrepudiation

SSL is enabled for agent-manager communication, as described above. In addition, using SSL for client-server communication is possible and ensures:

1. Privacy – All communications between the client and server are encrypted
2. Trust – When a connection is established, Cloudify Manager presents a signed certificate to the client. The client can use that certificate to validate the authenticity of the manager.

Requests to the manager can be addressed to its public or private IP address. By default, internal requests (i.e. requests sent from the manager itself or from agent hosts) are sent to the manager’s private IP address, while external requests (i.e. requests originating from other, external clients) should be sent to the manager’s public address.
Each of the server’s IP addresses has a different SSL key pair, created with the matching address as its CN value. Sending a request to the wrong address could therefore fail, since the manager might present the wrong SSL certificate to the client.

Secret Store and More

Starting with 4.0, all service required by Cloudify will run under the Cloudify (and not root) user in the manager VM. The only exception is the parent process of Nginx which runs as root in order to enable usage of port 80. It is not recommended to change this behavior.
A secret store is implemented inside the Cloudify DB (Postgres), providing an encrypted, tenant-wide variable store:

• Through usage of the secret store a user can ensure all secrets (such as credentials to IaaS environments, passwords, etc) are kept in a secured manner, and adhere to isolation requirements between different tenants.
• Secrets can be added to the store by a set function, and retrieved via get.
• Export as environment variables enables agents to use secrets (not in 4.0).
• Plugins can get secrets to leverage those when communicating with IaaS environments.
• Managers should be secured via client-side SSL to ensure secrets are not passed on an unencrypted communication channel.
• Usage of Postgres ensures replication of secrets across all managers within a cluster as part of HA.

You are always welcome to check out the docs for any other information and feel free to contact us via the user group or on Slack for any questions.