jtimberman's Code Blog

Chef, Ops, Ruby, Linux/Unix. Opinions are mine, not my employer's (CHEF).

Guide to Writing Chef Cookbooks

I wanted smartmontools installed to monitor the disk health of my LAN server at home. This is not an uncommon thing to want to do, so I thought I’d write and share a Chef cookbook for it. I also took this opportunity to write up the experience so I can illustrate how easy it is to write a cookbook for Chef.

The first thing to do when writing a cookbook is to create the cookbook directory structure with knife cookbook create. This command will create a README.rdoc by default, and I prefer Markdown, so I specify the -r md option.

knife cookbook create smartmontools -r md

By default, metadata and the default recipe are created with boilerplate content for author and copyright. I have configured the values in my knife.rb:

cookbook_copyright "Joshua Timberman"
cookbook_license   "apachev2"
cookbook_email     "cookbooks@housepub.org"

The resulting directory structure will be created:

% tree cookbooks/smartmontools
cookbooks/smartmontools
├── README.md
├── attributes
├── definitions
├── files
│   └── default
├── libraries
├── metadata.rb
├── providers
├── recipes
│   └── default.rb
├── resources
└── templates
    └── default

10 directories, 3 files

README Driven Development

I’m a big fan of Tom Preston-Werner’s blog post on README driven development. I don’t write the complete README before I start writing code for a new cookbook. I do write it as I go.

In order to write a proper README for a cookbook, and to write the cookbook itself, we’ll need to know a bit more about the software we’re installing. The best way to do that depends on the software, but often it is as simple as merely installing the package on a test system such as a virtual machine and explore its contents.

apt-get install smartmontools
dpkg -L smartmontools

Of note for smartmontools, documentation is in /usr/share/doc/smartmontools and configuration is in /etc. In particular, /etc/smartd.conf and /etc/smartmontools.

For now assume that the cookbook README.md is being written along the way.

List of Resources

One of the things I do when I am writing a new cookbook and exploring the contents of a package is to be mindful of Chef Resources I want to manage in the recipe(s). In the case of smartmontools, at this point I have determined I need a few specific resources.

Install the Package

First, as I’ve installed the package, I clearly need a package. I’m pretty confident that this particular package will not break backwards compatibility, and can be safely upgraded to the latest version if necessary.

Configuration files are templates

The next resources in the recipe are the configuration files. I want to dynamically configure these, so I am going to use templates.

I’m not going to write these from scratch. Instead, I will copy the source files from the installed package on my test system. These will go into templates/default in the cookbook.

% tree templates
templates
└── default
    ├── smartd.conf.erb
    └── smartmontools.default.erb

Templates are dynamically generated using ERB, and they can use Node attributes. I can use the automatically detected attributes, or I can set new attributes for the node in the cookbook.

Attributes Used in the Templates

The attributes go in the attributes/default.rb file in the cookbook. The ones I use are:

Attributes are definitely something to document in the README.

In templates/default/smartd.conf.erb, I check if there’s a list of devices to monitor, and if so iterate over the list passing in the default options (device_opts). The cookbook doesn’t at this time support per-device options - the same ones are applied to all devices. If devices is empty, then the configuration will use DEVICESCAN.

<% if node['smartmontools']['devices'].length > 0 -%>
<%   node['smartmontools']['devices'].each do |device| -%>
<%=    "/dev/#{device} #{node['smartmontools']['device_opts']}" %>
<%   end -%>
<% else -%>

DEVICESCAN -m root -M exec /usr/share/smartmontools/smartd-runner
<% end -%>

In templates/default/smartmontools.default.erb, the smartmontools daemon will be enabled based on the start_smartd attribute. Additional options will be passed per smartd_opts:

start_smartd=<%= node['smartmontools']['start_smartd'] %>

<% if node['smartmontools']['smartd_opts'].length > 0 -%>
smartd_opts="<%= node['smartmontools']['smartd_opts'] %>"
<% else -%>
#smartd_opts="--interval=1800"
<% end -%>

Generally speaking when creating attributes and using them in a template, I use the default values that are found in the configuration file dropped off by the package. I also try to use the default settings over all.

However, a cookbook is a place to express opinions. I made some with this cookbook, such as enabling DEVICESCAN if there are no devices, despite the configuration file’s comments indicating that the option shouldn’t be used. The best thing about attributes is they allow other people using the cookbook to change the behavior based on their preferences much easier than manually modifying things dropped off by a package.

I strongly recommend documenting where a cookbook has behavior that is not default for the installed package or upstream documentation. I did this in the README for this cookbook discussing the DEVICESCAN option and otherwise where appropriate.

Static Files

This cookbook has only one static file which will be deployed to /etc/smartmontools/run.d/10mail. The smartmontools package allows creating a number of scripts that go in the directory, and I have created an attribute for the list of scripts. These generally don’t cookbook_file resources. Since the list is an attribute, I iterate over that with Ruby’s Array#each loop.

The attribute:

The loop of resources:

Each filename in the array (currently only ‘10mail’) needs to have a corresponding file in the files/default directory of the cookbook.

% tree files
files
└── default
    └── 10mail

This file’s contents are not particularly exciting, I used the same one that came out of the package.

Why would I want to manage a static file that came from the package? Perhaps I want to modify the script in some way that doesn’t make sense in an attribute. In this case I don’t, however creating the attribute and iterating over it makes it easy to extend this functonality in the cookbook.

Manage the Service

Smartmontools comes with a service. That is, there’s an init script that can be enabled and started to monitor disk devices. This is actually the whole point of the cookbook, so I’ll make sure there’s a resource to manage the service.

In this resource, I used the meta parameter supports for the service. I found out what the init script can do by simply running it with no options on my test system.

% /etc/init.d/smartmontools
Usage: /etc/init.d/smartmontools {start|stop|restart|reload|force-reload|status}

The various options passed to the init script manage it in the familiar way. Telling Chef about it has a specific effect on the way the service provider functions when Chef manages it.

  • status: Chef will use /etc/init.d/SERVICE_NAME status to determine if the service is running. If the return code is 0, its running. Otherwise, Chef checks the process table for a process running with the name of the service.
  • reload: Chef can only take the reload action for a service if it actually supports reload.
  • restart: When the restart action is sent to a service, if it supports restart then Chef will use /etc/init.d/SERVICE_NAME restart. Otherwise, Chef will use stop and start.

Earlier when we wrote the template resources, we notified the service to reload. Since the resource supports reload, we can do this. Also note that the resource has an array of actions. Each of these actions will be taken if necessary when Chef manages it. In this case, it will be enabled at boot time, and then started if it is not already running.

Let’s Use the Recipe

Now that we’ve written a nice recipe and understand what it’s about to do, let’s actually use it on a node we’d like to have smartmontools. Before uploading, I remove the boilerplate directories that were created by knife. The actual cookbook contents for upload are:

% tree cookbooks/smartmontools
cookbooks/smartmontools
├── README.md
├── attributes
│   └── default.rb
├── files
│   └── default
│       └── 10mail
├── metadata.rb
├── recipes
│   └── default.rb
└── templates
    └── default
        ├── smartd.conf.erb
        └── smartmontools.default.erb

6 directories, 7 files

I didn’t mention the metadata yet as I haven’t modified it yet. The knife command will create a default metadata.rb file as mentioned before. It will also populate it with some boilerplate content. The main thing I’m going to modify is the version and the platforms supported.

Now it is time to actually upload the cookbook and apply it to a node.

% knife cookbook upload smartmontools
Uploading smartmontools             [0.5.0]
upload complete

% knife node run list add virt1test 'recipe[smartmontools]'
run_list:
    role[ubuntu]
    recipe[smartmontools]

Run Chef

I’ll use knife ssh to run chef-client on the nodes that have the smartmontools recipe applied.

Verify results

We can verify the results of running the recipe by examining the resources on the target system(s). Chef’s contract with you is that it will configure the resources in the manner specified in the recipe. You can be confident that it will completely configure every resource if it exits cleanly (which it did from the output above).

% cat /etc/default/smartmontools
...
start_smartd=yes
...
% cat /etc/smartd.conf
...
DEVICESCAN -m root -M exec /usr/share/smartmontools/smartd-runner
...
% sudo service smartmontools status
 * smartd is running

Now we can take this same recipe and apply it to other systems. I tested smartmontools on an Ubuntu system. Notice earlier that I had an ubuntu role on my node. I actually modified that role to include the smartmontools recipe, and then all my Ubuntu nodes were configured for smartmontools when they ran Chef again.

Conclusion

I hope this guide was helpful. I shared the cookbook on the Chef Community web site, and the source code is available on Chef Community web site. Note that while I said this would be easy, it certainly isn’t trivial. There are a lot of steps involved in making cookbooks that are dynamic, easily customized and shareable with others. It takes practice, but after a few you get the hang of it. End to end, this cookbook took me about 2 hours to write, test, tweak (fix bugs) and document.