Chef Integration Documentation

Cloudify supports the Opscode Chef platform as an additional method of conveniently deploying and configuring application services.
Chef provides a powerful multi-platform framework for configuring operating systems and services. It complements Cloudify recipes by handling the scope of intra-machine configurations. Using Chef, the lifecycle of services can be simplified as Chef takes care of keeping service configurations up-to-date with current specifications as defined by roles and cookbooks.
Before you proceed, make sure you are familiar with the basic concepts of Chef. For information, see Chef Basics.
In order to use Chef cookbooks for service installation, Cloudify provides a configurable base Chef recipe that does most of the Chef bootstrapping work. Typically you will only need to extend this recipe and take care of the pre and post deployment phases that Chef doesn’t handle.

Sections of the page:

Try Out Cloudify and Chef Online

You can expereince the way the Cloudify and Chef integration works online without having to download Cloudify.
The demo will allow you to setup a Chef Server, Install Cookbooks, and deploy a Clouidfy service that uses that Cookbook.
You can also expereince the set of command line and web management tools that will allow you to monitor the deployment process and invoke the Chef comands through Cloudify.
The online demo is available here.

How Does Cloudify Use Chef?

The following diagrams depict the Cloudify/Chef integration flow:

  1. Cloudify provisions machines and installs Cloudify agents as part of the deployment.
  2. Cloudify executes a parameterized Cloudify Chef client recipe. This recipe installs and runs the Chef client.
  3. Chef executes the Chef cookbook and installs the required software.

Chef Installation

Cloudify installs the chef client once the chef recipe triggers the install step of the service lifecycle, as follows.

The ChefBootstrap.getBootstrap method is a factory method pattern that selects the correct bootstrap strategy depending to the current operating system. The install method invoked on the factory, installs the Chef client using gem or yum install.

Using Chef with Cloudify

The next sections of this guide assume that you have Cloudify installed and that you have bootstrapped a Cloudify environment, ready to install an application. For a quick introduction to setting up a Cloudify cloud, see Cloudify’s Quick Start Guide

If you’d like to use Cloudify for installing a Chef server, and using 2.2 miletone 4 or lower, please make sure to use a Ubuntu 11.10 or 12.04 Cloud Driver template, since the Chef server recipe has only been certified for Ubuntu at this point.

Downloading Cloudify Chef Recipes and Related Artifacts

To obtain the basic Chef recipes provided by Cloudify, download the following files and save them under <cloudifyRoot>/recipes/services:

In addition, download the simple mySQL example and save it to the same location.

You can also clone the entire CloudifySource recipes repository from Github using the command git clone

Running the MySQL Example

Deploying the Chef Server

To use a cookbook that leverages the Chef Server (as opposed to Chef Solo cookbooks), you can use Cloudify to deploy a Chef Server.

To deploy the Chef Server:

  1. Connect to the Cloudify Management machine: From the Cloudify shell prompt, type: connect <cloudify REST gateway URL>
  2. Deploy the Chef Server: From the Cloudify shell prompt, type: install-service <cloudifyRoot>/recipes/services/chef-server (the recipe location can be any location on your local disk to which you’ve downloaded the Cloudify Chef server recipe).

Once the Chef server is deployed, its location is published as the global attribute chef_server_url in Cloudify’s attributes store.

Alternatively, you can configure Cloudify to use an existing Chef Server by setting this attribute manually using the Cloudify shell, as presented below. This can be a Chef server you installed separately or the Hosted Chef service.

Loading the mySQL Cookbook

The Cloudify Chef Server recipe adds a custom command to the Chef Server that allows users to upload cookbooks to the Chef server. You can point to a HTTP URL of a cookbooks tarball, a git or SVN repository URL (more on this below).

This is done by invoking the updateCookbooks custom command from the Cloudify shell.

Installing the mysql_vanilla Recipe

Assuming you’ve succesfully uploaded the mysql cookbook using the custom command above, type the following from the cloudify shell prompt:
install-service <mysql vanilla recipe path>

That’s all! Cloudify will start installing the mySQL server recipe.
You can trace the installation progress using the Cloudify shell or the Cloudify Web UI.

Running Additional Cookbooks

Creating Recipes based on Chef Cookbooks

If you are not familiar with Cloudify recipes, please read Anatomy of a Recipe first.

To create a Cloudify Chef recipe:

  1. Create an application recipe folder (e.g. “petclinic”).
  2. Add the application level recipe file describing the service dependencies in your application
  3. Create a sub folder for each service (e.g. “tomcat” or “mysql”)
  4. For each service you’d like to install using Chef, create a service recipe (<serviceName>-service.groovy) file that extends the chef recipe, specifying the service name, icon, number of instances, etc.

Optionally, you can specify the following sections in your recipe:

  • * Monitoring : Describes the custom performance indicators (metrics) to collect.
  • * Scaling Rules : Describes the performance indicators and thresholds to use to scale your service.
  • * Custom Commands : Describes custom commands that add functionality. The commands can be invoked using the Cloudify shell at any time. They are usually used for various maintenance and application upgrade tasks.
  • In the same folder, create a service properties file (<serviceName>, specifying a property called runParams. For example, you can assign a groovy structure that gives an array of the Chef recipes to use, the name of the cookbook, and the default attributes.

Loading Cookbooks

The Chef Server requires cookbooks to enable clients to install different services. Cookbooks can be loaded in the following ways:

  • Using knife: Use the command: knife cookbook upload [COOKBOOKS...] (options). For more information, see Managing Cookbooks With Knife.
  • Adding a cookbooks tarball or pointing to a source code repository in the chef-server recipe properties file, as follows:
  • Pushing recipes using custom commands add to the service recipe file. For example, the chef-server recipe exposes the following commands:

Running your Recipe

Chef based Cloudify recipes are executed in the same way as other Cloudify recipes are. You use must use the Cloudify shell to connect to your Cloudify REST gateway, and then use the install-application command with the full path to your application recipe folder. You can monitor the installation progress using the Cloudify shell and Web UI.

Chef Version
By default, the Cloudify Chef recipe uses the latest stable version of the Chef client and server. This is defined by the property chef.version in the file of the base Chef recipe. If you want to use a different Chef version, change this property to the right version. Note however that the Cloudify Chef integration is only tested with the default version in the above file. When installing Chef server using the chef-server Cloudify recipe, chef-solo is used. The recipe downloads the relevant cookbooks from a tarball, whose URL is specified in the property chef.bootstrapCookbooksUrl in the same properties file above. If you want to use a different version of one of those cookbooks, you will need to modify this property accordingly.

Running Chef

The Chef service recipe uses the run_chef.groovy script to run Chef with the correct run list.
This script gets the location of the Chef server from the Cloudify attributes store. Similarly, it gets the client certificate required for communication with the server. The run list is calculated using the Cloudify service name, or using the properties file of the service.

Chef then executes the cookbook in the normal way (completely independent of Cloudify).