Overview of the different components Cloudify is comprised of
This is aimed at advanced users to understand Cloudify’s architecture. It provides no user-functional information whatsoever. It does, however, provide information for understanding how Cloudify’s architecture supports currently implemented and even potential future flows. Operational knowledge assumed.
Cloudify’s Manager comprises (mainly) of the following open-source components:
Cloudify’s code and the components’ configuration is what makes Cloudify.. well.. Cloudify.
Rather than specifying the ports in each component’s overview, ports are specified here so that you can easily review network requirements.
By default, there are two external networks from which Cloudify’s Management Environment is accessed:
Therefore, Cloudify aims to have only two entry points to its Management Environment:
Port 5672 for application access via RabbitMQ.
cfy sshwill not work.
Internally, the following ports are exposed:
Nginx is a highly performant web server.
In Cloudify’s Manager, Nginx serves two purposes:
The fileserver served by Nginx, while tied to Nginx by default, is not logically bound to it. That is, while we currently access it directly in several occurences (via disk rather than via network), we will be working towards having it completely decoupled from the management environment itself so that it can be deployed anywhere.
Gunicorn and Flask together serve as Cloudify’s REST service. The REST service itself is written using Flask while Gunicorn is the server. Nginx, is the proxy to that server. Essentially, Cloudify’s REST service is what makes Cloudify tick and is the integrator of all parts of the the system.
Elasticsearch is a JSON based document store.
In Cloudify’s Manager, Elasticsearch serves two purposes:
Elasticsearch is initially provisioned with two indices:
cloudify_storage- Used for storing the data model (Blueprints, Deployments, Runtime Properties, etc..)
cloudify_events- Used for storing logs and events (We will probably split to two indices at some point in the future.)
The indices and their mappings are generated at build time and are provided within the Docker image(s). To keep the indices and their data persistent, they are mapped to a Data Container.
Logstash is a data handler. It can pull/push messages using several inputs, apply filters and output to different outputs.
Logstash is used by Cloudify to pull log and event messages from RabbitMQ and index them in Elasticsearch.
RabbitMQ is a queue based messaging platform.
RabbitMQ is used by Cloudify as a message queue for different purposes:
Currently not all requests between Cloudify’s Manager and the hosts it manages go through RabbitMQ. We aim to make it so.
Riemann is an event stream processor used mainly for monitoring.
Riemann is used within Cloudify as a policy based decision maker. For more information on policies, refer to the policies section.
Celery is a distributed task queue.
Cloudify’s Management Worker, the Deployment Specific agents and the host agents are based on Celery.
deployment workflow agent and the
deployment agent drawn in the diagram are deployment specific. For every deployment created, two of these agents are spawned.
deployment workflow agentexecutes deployment specific workflows.
deployment agentexecutes API calls to IaaS providers to create deployment resources or submits tasks to RabbitMQ so that host agents can execute them.
Note that all agents (Management, Deployment Specific, Host) are actually the same physical entity (a virtualenv with Python modules - Cloudify plugins installed in them).
An entity removed from the diagram is a management agent containing a Cloudify plugin able to spawn the aforementioned deployment specific agents. This agent is provided within the Docker image and is run during the bootstrap process.