jtimberman's Code Blog

Chef, Ops, Ruby, Linux/Unix. Opinions are mine, not my employer's (CHEF).

Chef 0.10’s Knife Plugins

Earlier today^Wyesterday, Opscode previewed a new feature of the upcoming Chef 0.10, Knife Plugins. I’ve been excited about this feature since Dan DeLeo first created the ticket for it. I already created the “grep” plugin that Dan illustrates on the blog post, and it is quite useful already (and I only have a few systems!).

A few months ago, I wrote a silly library that will retrieve the list of Canonical’s released AMIs. They make this available as a text file per distribution codename (lucid, maverick, etc), and my library supports that. For even greater silliness, but perhaps usefulness, I wrote a knife plugin to retrieve the AMI data. The code is really simple.

The run method processes the name_args (see the Knife Plugin Documentation) for the distribution and optionally a type. Lets see an example of what this means.

First, usage:

% knife ec2 amis ubuntu -h
knife ec2 amis ubuntu DISTRO [TYPE]
    -s, --server-url URL             Chef Server URL
    -k, --key KEY                    API Client Key
        --color                      Use colored output
    -c, --config CONFIG              The configuration file to use
        --defaults                   Accept default values for all questions
    -e, --editor EDITOR              Set the editor to use for interactive commands
    -E, --environment ENVIRONMENT    Set the Chef environment
    -F, --format FORMAT              Which format to use for output
        --no-color                   Don't use colors in the output
    -n, --no-editor                  Do not open EDITOR, just accept the data as is
    -u, --user USER                  API Client Username
        --print-after                Show the data after a destructive operation
    -V, --verbose                    More verbose output. Use twice for max verbosity
    -v, --version                    Show chef version
    -y, --yes                        Say yes to all prompts for confirmation
    -h, --help                       Show this message

The name_args we can pass are DISTRO and TYPE, TYPE being optional. The rest of the options are built into knife and available to every sub-command. Let’s pick a distro. My favorite these days is lucid:

% knife ec2 amis ubuntu lucid
ap_northeast_large:      ami-580fa459
ap_northeast_large_ebs:  ami-5e0fa45f
ap_northeast_small:      ami-4c0fa44d
ap_northeast_small_ebs:  ami-5c0fa45d
ap_southeast_large:      ami-c692ec94
ap_southeast_large_ebs:  ami-f092eca2
ap_southeast_small:      ami-2492ec76
ap_southeast_small_ebs:  ami-f292eca0
eu_west_large:           ami-631f2b17
eu_west_large_ebs:       ami-3d1f2b49
eu_west_small:           ami-a11e2ad5
eu_west_small_ebs:       ami-311f2b45
us_east_large:           ami-fa01f193
us_east_large_ebs:       ami-3202f25b
us_east_small:           ami-7000f019
us_east_small_ebs:       ami-3e02f257
us_west_large:           ami-e1bfefa4
us_west_large_ebs:       ami-f5bfefb0
us_west_small:           ami-19bfef5c
us_west_small_ebs:       ami-ebbfefae

For the DISTRO, I can use any of the codenames that Canonical makes available, including karmic, lucid, maverick and even natty at the time of this writing. The value on the left is the TYPE. Lets see what happens when I pass one of those:

% knife ec2 amis ubuntu lucid us_west_small
ami-19bfef5c

This returns just the AMI. We can feed this into an environment variable or just copy and paste it into a “knife ec2 server create” command for the “-I ami” option.

A note about the naming, I classed it under the ec2 sub-command space since it is most relevant there. Further, I used “amis ubuntu” rather than “ubuntu amis” because theoretically other plugins could be written to retrieve other platforms. There’s a couple bugs I know about already and will fix them later. Enjoy!

Display Chef Server Search Indexes

Anyone who runs a Chef Server or uses Hosted Chef has a wealth of search indexes available to them for data stored on the server. However, there isn’t a direct subcommand for displaying the search indexes. The following are automatically created on the server or Platform organization at creation and we know about them already:

  • node
  • role
  • client

Also, all data bags are created as a search index for the items they contain. All of these indexes are available through the API, and can be retrieved with Knife.

This uses knife’s “exec” sub-command. The subcommand takes a script argument, which is a Ruby script. It is executed in the context of the knife configuration, so if you have a .chef/knife.rb, it will be able to connect to a Chef Server. Let’s look at the contents of this script, it’s pretty simple.

The ‘p’ simply prints output of the argument. Then I’m calling the “get” method on the “api” object. This is a preconfigured object that knows how to talk to the Chef Server API through the aforementioned configuration. This takes an argument, which I pass the search API endpoint. The results that come back from the API are a Ruby hash, so I print out just the keys by calling the “keys” method. As we can see, I have a number of data bags in my environment, including DNS entries, users and Samba (shares and permissions for the curious).

Managing My Workstations With Chef

Update: This post is old and outdated.

Update: New cookbook, Lion and more!

This post describes how I use Chef and Opscode Hosted Chef to manage my workstations, Apple Macbooks. I have three:

  • Company issued Macbook Pro
  • Personal Macbook Air
  • Family Macbook Pro

Note that while I do use this on my work laptop, Opscode has a different set of recipes to set up development and internal company workstations. At some point there may be a post on the Opscode Blog.

Before I get started with Chef itself, I have to install a few things to “bootstrap” the systems to be ready. I’m not going to include the commands to install all these “pre-requisites”, as you can learn how to do that from their respective web sites.

Xcode 3.2 (or 4.0)

I wish I could purchase a Macbook with Xcode preinstalled. In fact, I would pay $50 to have this over iLife on new machines, so the $5 cost for version 4 is really minor. However, I don’t have any use for whatever 4.0 has to offer, since I only want Xcode because its the most sane way to get GCC and friends on OSX.

http://developer.apple.com/xcode/

Ruby Version Manager

I want to use Ruby 1.9.2 instead of the default that comes with OSX, and RVM makes it easy to do that. I also like the gemset feature, which allows me to sandbox and play with various configurations of RubyGems.

http://rvm.beginrescueend.com

Chef

I generally just install the latest released Chef gem from RubyGems.org once I have Ruby installed.

https://rubygems.org/gems/chef

Once Chef itself is installed, it’s time to get cooking! I have the following already set up:

  • Opscode Community userid.
  • Opscode Hosted Chef organization.
  • Chef Repository

Let’s take a look at what I put into the Chef Repository to configure my workstations. First, I created the repository and the listed components, then I upload everything to my Opscode Hosted Chef organization.

Cookbooks

I have a number of cookbooks for my personal repository, but the relevant ones for workstations are:

I wrote the dmg cookbook to automatically install OSX applications that are delivered to users via DMGs. After writing cookbooks for skype, and tunnelblick, I noticed some common patterns and abstracted those out to a lightweight resource and provider, “dmg_package”. This does the heavy lifting of:

  1. Retrieve a .dmg file from a remote URL.
  2. Mount the disk image.
  3. Copy the .app directory to /Applications.
  4. Unmount the disk image.

The homebrew cookbook was written by Graeme Mathieson, and it will install Homebrew in /usr/local and use it as the default package provider on OSX.

Both the dmg and homebrew cookbooks were downloaded to my local chef-repo with the “vendor” pattern supported by knife.

The workstation cookbook consists of a number of recipes. Many of them contain commands lifted from Pivotal Labs experimental GitHub repository, wschef. I don’t use any of the scaffolding they wrote in that repository, but I grabbed lots of the executes. However, that repository does not have any mention of a license, so I haven’t published the workstation cookbook I use. I can publish the recipe I wrote, which will be part of the cookbook when the license of their recipes is sorted out.

Roles

I use two roles on my workstations that tell Chef what recipes to apply. First, is role[mac_os_x]:

Next is the actual workstation recipe.

Data Bags

Chef is data driven, and I have written a data bag item for my workstation. It simply lists the gems, dmg’s and packages to install with the above recipe.

Client Configuration

I run Chef as my normal user, so it will need to be able to write out cache and other files in a location my user has write access. I replace “workstation_hostname” for the node name depending on the host. For example, doppelbock is the hostname and node name of my Macbook Air.

Nodes

I use the following run list on my workstations.

Upload Chef Repository

Now that all the victims are assembled, it is time to upload the Chef repository, copy the configuration file and validation certificate and run Chef.

Then I simply run ‘chef-client’ as my normal user and my laptop is configured. It does take some time because several of the DMGs are large, and homebrew compiles from source. All in all, it took about 1 hour to do the initial run of Chef on my Macbook Air 13” when I first set it up.

Obligatory First Post

My intention is to post about Linux/Unix system administration, Ruby development, Opscode’s Chef and even more philosophical topics like devops, cloud computing, or really anything that strikes my fancy.

I’m probably going to post a lot about Chef. Most of this is my own personal exploration of the framework and testing for my personal usage. While I work for Opscode and you’re free to ask me any time about Chef and the Opscode Platform, I post on this blog in my personal time. Not all the information here is canonical or even reflects the best way to go about doing anything, let alone with Chef.

 

Debian/Ubuntu Package for Ruby 1.9.2 With Fpm

Jordan Sissel created fpm a few months ago. It is a quick and easy way to create packages for Debian, Ubuntu and Red Hat family Linux distributions. I created a Ruby 1.9.2 package for my personal server.