We recently deployed an Amazon EC2 instance and I was tasked with installing and configuring Red5 and Openfire on the system. As I often do, I took careful notes of what I did on a wiki, which is useful to have in case another instance needs to be brought up later. Manual installation using notes, however, can be an arduous and error prone process. This can be alleviated by adding automation, typically through shell scripts. The problem with writing shell scripts is that you can only run those scripts on a specific system. And often, running the same script twice can leave your system in an inconsistent state.

Enter Chef , an open source systems integration framework written in Ruby. You can easily manage and configure your servers using what Chef calls cookbooks. A cookbook contains all the things you need to automate the configuration of your server and basically consists of one or more recipes. The easiest way to familiarize yourself with Chef is by running it with Chef Solo, which is what I ended up doing. Running it this way allows you to run cookbooks without running a Chef server and client. You can read more about deploying Chef in this configuration by reading the Chef Architecture documentation.

Before writing my cookbook, I had to have all the dependencies for Chef Solo installed on my Ubuntu 10.04 (Lucid Lynx) EC2 instance. Lucid Lynx already ships with Ruby so I didn’t need to install that. All I needed to do to get myself going was run these commands to install Chef Solo:

% sudo apt-get update
% sudo apt-get -y install ohai chef rubygems

You should read the Chef Installation documentation if you’re on a different system.

The first thing I did was to create a directory to contain my cookbooks and configuration files:

% mkdir -p ~/chef/cookbooks
% mkdir -p ~/chef/config

The chef-solo command comes with two main options:

• -c <configuration_file>
• -j <json_attributes>

The -c option tells chef-solo which cookbooks to pick up. If not specified, chef-solo will read the configuration file at /etc/chef/solo.rb. The -j option specifies which recipes chef-solo should run, as well as set or override any recipe attributes. The format of the file that the -j option reads in is JSON.

I wanted chef-solo to pick up my recipes, so I wrote my own configuration file:

% cd ~/chef/config
% vim solo.rb
home_dir = File.expand_path(File.dirname(__FILE__) + "/..")
file_cache_path  home_dir
cookbook_path [ "#{home_dir}/cookbooks" ]

The cookbook_path specifies where my cookbooks are going to exist, which is an array of paths. The file_cache_path is the directory where remote cookbooks will be downloaded and installed into. Chef solo has a -r option to download remote recipes into your system. Both cookbook_path and file_cache_path need to be full paths.

I needed to install Openfire and Red5 on my system, which meant there was a Java dependency. I knew there was a Java Chef recipe, so I downloaded it and installed it in my cookbooks directory:

% cd ~/chef/cookbooks
% wget https://s3.amazonaws.com/opscode-community/cookbook_versions/tarballs/194/original/java.tar.gz
% tar zxf java.tar.gz

One of the great things about Chef is that a whole slew of recipes are available to be used. You can find them at http://cookbooks.opscode.com.

I was now be able to install Java, but before doing that, I needed to create an attributes file telling chef-solo I wanted to run java as a recipe:

% cd ~/chef/config
% vim node.json
{
    "recipes": [ "java" ]
}

My first chef-solo run:

% cd ~/chef
% chef-solo -j config/node.json -c config/solo.rb

Voila! That installed Java on my system.

I was finally ready to write my first cookbook:

% cd ~/chef/cookbooks
% mkdir -p openfire/recipes
% cd openfire/recipes
% vi default.rb
tarball = "openfire_3_6_4.tar.gz"
execute "wget" do
 tarball_url = "http://www.igniterealtime.org/downloadServlet?filename=openfire/#{tarball}"
 cwd "/tmp"
 command "wget #{tarball_url}"
 creates "/tmp/#{tarball}"
 action :run
end
remote_file "/tmp/#{tarball}" do
 source "http://www.igniterealtime.org/downloadServlet?filename=openfire/#{tarball}"
 mode "0644"
 checksum "bc43e96d23dc865b624fd3ed2531b2a8a8ef3f0483e829dd087b08688d92ede2"
end
execute "tar" do
 user "ubuntu"
 group "ubuntu"
 installation_dir = "/home/ubuntu"
 cwd installation_dir
 command "tar zxf /tmp/#{tarball}"
 creates installation_dir + "/openfire"
 action :run
end

The openfire/recipes/default.rb file is the main recipe file that’s executed when the Openfire cookbook is referenced.  If you look in the Java cookbook, you’ll find java/recipes/default.rb. Each cookbook has a default.rb recipe.

A recipe is essentially a list of commands or, in Chef speak, resources. A resource is a command abstraction that can be run on any platform. For example, there’s a file resource that allows you to create or delete a file, a package resource that let’s you to install, remove or upgrade a package in your system. In the Openfire cookbook, I’m using the execute resource which runs a specific shell command. I found the terminology in Chef often confusing and turned to a blog post called The Chef Way Episode 2: Chef Speak to explain it all. I highly recommend using it as reference while reading the Chef documentation.

What the wget execute resource does is:

• Changes the working directory to /tmp (cwd “/tmp”)
• Runs wget to fetch the tarball (command “wget #{tarball_url}”)
• Says that this action creates the tarball in the /tmp directory and, if it exists, not execute the resource (creates “/tmp/#{tarball}”
• Lastly, says that the action for this command is to be run (action :run)

The other valid action for execute is :nothing which basically tells the resource not to run the command. Each resource has its own set of actions and valid attributes. For more details, check out the resources wiki.

The only difference with the tar execute resource are the user and group attributes which says the resource should change to a specific user and group id before running the command.

What the remote_file resource does is fetch the tarball from the remote server into the /tmp directory and run a checksum. The tar execute resource:

• Sets a specific user  id to run the command (user “ubuntu”)
• Sets a specific group id to run the command (group “ubuntu”)
• Changes the working directory (cwd installation_dir)
• Runs tar to untar the tarball (command “tar zxf /tmp/#{tarball}”)
• Says that this action creates the openfire directory in the installation directory. If the directory already exists, the resource will not execute. (creates installation_dir + “/openfire”)
• Lastly, says that the action for this command is to be run (action :run)

Each resource has its own set of actions and valid attributes. For more details, check out the resources wiki.

There were a lot of hard-coded values in the cookbook. So I decided to extract them out as much as possible into attributes. You can do that by creating a file in the attributes directory:

% cd ~/chef/cookbooks/openfire
% mkdir attributes
% vim attributes/openfire.rb
tarball = "openfire_3_6_4.tar.gz"
set_unless[:openfire][:installation_dir] = "/home/ubuntu"
set_unless[:openfire][:tarball_url] = "http://www.igniterealtime.org/downloadServlet?filename=openfire/#{tarball}"
set_unless[:openfire][:tarball_checksum] = "bc43e96d23dc865b624fd3ed2531b2a8a8ef3f0483e829dd087b08688d92ede2"
set_unless[:openfire][:tarball] = tarball
set_unless[:openfire][:dirname] = "openfire" # directory name of extracted tarball
set_unless[:openfire][:user] = "ubuntu"
set_unless[:openfire][:group] = "ubuntu"
% vim recipes/default.rb
tarball = node[:openfire][:tarball]
execute "wget" do
 cwd "/tmp"
 command "wget #{node[:openfire][:tarball_url]}"
 creates "/tmp/#{tarball}"
 action :run
end
remote_file "/tmp/#{tarball}" do
 source "#{node[:openfire][:tarball_url]}"
 mode "0644"
 checksum "#{node[:openfire][:tarball_checksum]}"
end
execute "tar" do
 user node[:openfire][:user]
 group node[:openfire][:group]
 installation_dir = node[:openfire][:installation_dir]
 cwd installation_dir
 command "tar zxf /tmp/#{tarball}"
 creates installation_dir + "/" + node[:openfire][:dirname]
 action :run
end

An attribute is a property that can be accessed via a hash called node in any recipe. When Chef runs, every single file in the attributes directory is read in alphabetical order. The attribute values set in those files are set in node. You can set attributes using the set or set_unless methods: set assigns the value even if the attribute has already been set, set_unless only assigns the value if it hasn’t been set already. When running chef-solo, you can override attributes using the JSON attributes file. Check out the attributes wiki for more information.

The Red5 cookbook is very similar to the Openfire cookbook, so I won’t include it here for brevity’s sake. To install Java, Openfire and Red5 using chef-solo:

% cd ~/chef/config
% vim node.json
{
    "recipes": [ "java", "openfire", "red5" ]

}
% cd ~/chef
% chef-solo -j config/node.json -c config/solo.rb

In this case, only the Openfire and Red5 cookbook will run because we installed Java before. Chef is smart enough not to rerun installations that have already run.

These cookbooks are all checked into github at http://github.com/blazingcloud/chef if you want to check it out. I also recommend reading Cooking with Chef 101 for a great Chef Solo tutorial.

I know the cookbooks here could be factored out better and be less platform dependent on Ubuntu, but for the purposes of instruction, I hope this was useful. I’ll leave it at that for my first Blazing Cloud blog post. Thanks for reading!