I moved my blog from Posterous to GitHub Pages. Posterous isn’t a bad system and service. It just didn’t fit the way I wanted to manage my site and the content. It was entirely adequate to get going but I was dissatisfied with the web form for creating new posts. It also loaded pages fairly slow.

GitHub is awesome.

Many others have migrated their technical blogs from services like Posterous, Wordpress, Tumblr, Blogger and so on over to GitHub Pages. Most often, this is because these services are database backed systems, use a web form for creating posts, and don’t have a revision control system that we all know and love under the covers.

GitHub Pages is a great service by GitHub that allows you to serve up static content out of a Git repository. You can do simple static HTML pages, or you can use a static page generation framework to generate the content out of Markdown (or Textile, or HAML) files. There’s a number of options for this, such as Nanoc or Jekyll.

I chose Jekyll.

“Jekyll is a blog-aware, static site generator in Ruby” – http://jekyllrb.com/

Jekyll fits the bill for me. It supports different markup languages, such as Markdown or Textile, and GitHub Pages has easy instructions on how to get going with it. Jekyll also includes a fantastic Posterous importer, which flawlessly imported all my old posts. Now I can start creating new posts as Markdown in Emacs, instead of using a web form, or editing HTML directly. I can also save revisions using Git, instead of saving a draft and monkeying with an interface to recover drafts or publish them.

I sat down a few days ago and imported my old posts. Then I ran through trying to get them to look alright with a decent CSS theme. However, I’m not a front end designer/developer, and I never remember how do do anything useful with CSS. So I went looking for canned ways to build a nice interface/theme for this blog.

I remembered Twitter’s Bootstrap project, and thought that might be interesting. After looking at it I was confused about what to use, since it is a very complete solution, including iOS mobile layouts in addition to normal web pages. After looking elsewhere, I saw mention of Octopress, a framework built on top of Jekyll. I queried some people that had experience with it, found that it would probably meet my needs, and gave it a whirl.

Naturally if you’re familiar with Octopress, you can see the result of that evaluation. It is a really nice system for managing the site.

I like my blogging workflow.

I like my new blogging workflow. It is very natural to me as a system engineer, as it uses Emacs, Rake, a Web Service, and Git. I’ll walk through the steps I took in creating this post. I’m going to skip the setup of Octopress, as that is documented elsewhere.

% rake new_post'[Blog Moved to GitHub Pages]'
Creating new post: source/_posts/2011-09-29-blog-moved-to-github-pages.markdown

Then I load up the source file created by the rake task and enter all this content you see, saving my work along the way. I did, but could choose, to commit to the git repository as well. In order to preview the content, I can actually run the entire blog with Octopress’s built in rake task:

% rake preview
Starting to watch source with Jekyll and Compass. Starting Rack on port 4000
[2011-09-29 23:06:20] INFO  WEBrick 1.3.1
[2011-09-29 23:06:20] INFO  ruby 1.9.2 (2011-07-09) [x86_64-darwin11.1.0]
[2011-09-29 23:06:20] INFO  WEBrick::HTTPServer#start: pid=92373 port=4000
Configuration from /Users/jtimberman/Development/jtimberman.github.com/_config.yml
Auto-regenerating enabled: source -> public
[2011-09-29 23:06:21] regeneration: 107 files changed
>>> Compass is watching for changes. Press Ctrl-C to Stop.
127.0.0.1 - - [29/Sep/2011 23:06:24] "GET / HTTP/1.1" 200 50231 0.0181

Now I can browse to http://localhost:4000 and see exactly what the blog post will look like, and it loads lightning fast. This is of paramount importance for me, as I travel a lot and would really like to develop blog entries while I’m on airplanes with no internet access.

Once I am satisfied with the content and how it looks, I commit to the repository and push the source branch.

% git add source/_posts
% git commit -m 'Blog moved to github pages'

Then, it is time to deploy the blog to GitHub Pages. Octopress makes this easy with another rake task:

% rake deploy

This takes care of all the repository work required, and pushes to GitHub. Then I push the actual source branch.

% git push origin source

Nice things about Octopress

Besides the rake tasks and that works with Jekyll, Octopress has some really nice features.

First of all, the default theme is really nice. One thing I wanted out of CSS on top of Jekyll was something large enough that I could read easily. I think this theme looks pretty good and is quite readable with my default browser settings. It is also nice and easily readable on my iPhone.

Second, Octopress is very customizable. I can easily modify the layout by editing the _config.yml file. Of course I can change the theme, but I can add JavaScript, layouts and more. I modified the footer to include my Creative Commons license for the content of this site.

Third, I really like the default layout in this theme. The recent posts, GitHub repo links and my last few tweets are a nice touch. Plus I have integrated Disqus for comments and set up Google Analytics easily, thanks to the ease of customizing the configuration.

Conclusion

In closing, I’m very happy with this solution. I now have a blogging workflow that I’m comfortable with, and that will hopefully result in more posts.

Update: This post is old and outdated. I’ll have another post in 2015 about workstation management with Chef.

It has been a few months and several thousand page views since my original post on how I manage my workstations with Chef. I have made some changes to my repository, and have a few additional notes for working with Mac OS X Lion. I also added a new system in my network, an iMac.

Mac OS X Cookbook

First, I would like to point out the cookbook I created, mac_os_x. In the original post, I discussed using Pivotal Lab’s workstation recipes, which use the Mac OS X user defaults system to update system preferences from the command-line (with an execute resource). One of the first things I did in the mac_os_x cookbook is create a lightweight resource and provider for managing user defaults. It can be used, for example, like this:

mac_os_x_userdefaults "dont show hard drives on the desktop" do
  domain "com.apple.finder"
  key "ShowHardDrivesOnDesktop"
  value "false"
  type "bool"
end

While this does involve more typing than just entering the command, it has a couple advantages. First, it is easier to read for people not totally familiar with the defaults command. Second, behind the scenes it will check if the setting is already set and not update the resource if it isn’t. This isn’t a huge deal in terms of system resource usage, but depending on your Chef setup might end up with extra reporting on things that didn’t need to change if you’re using a report handler, which I do use.

This cookbook also includes a lightweight resource/provider for managing plist preferences files for ~/Library/Preferences. All plists in OS X can be manipulated through the defaults system, but it can be cumbersome for highly customized applications such as your Adium or 1Password configuration. Usage is very simple:

mac_os_x_plist_file "ws.agile.1Password.plist"

This is in my local 1password cookbook, which has the file in files/default/ws.agile.1Password.plist. If I make modifications to the preferences, I do have to copy it over to the cookbook and re-upload to the Chef Server, but since I now have 4 computers to change preferences on, this is much easier than remembering everywhere I clicked. I use this for managing my preferences for ghmac (GitHub for Mac), iterm2 and alfredapp.

Mac OS X Lion

Lion brought a lot of changes. In particular along with Xcode 4, Apple changed the gcc compiler to llvm. I’m not a C programmer, and don’t really understand the differences yet, I just know that a number of things in Homebrew fail to build. The main thing I installed with Homebrew that wasn’t working with Lion is Emacs. I actually only have Lion on my iMac, so I haven’t updated any recipes for installing it via Chef. In fact, I actually removed it from my “workstation” recipe and *gasp* installed it manually. The story there is enough for another post though, so stay tuned and I’ll write up my experience.

Relevant to how I manage workstations with Chef, I had to make sure that on Lion, I take care of some additional new preferences using my handy-dandy “mac_os_x_userdefaults” LWRP. These are in the mac_os_x cookbook recipe, “lion_tweaks.” In my workstation recipe, I include this one only on Lion.

Conclusion

If you haven’t read it yet, go back and read my post on how I manage my workstations with Chef. I hope if you’re using Chef to manage a Mac that the mac_os_x cookbook is useful to you. Also, stay tuned for an update later this week about my experiences installing Emacs on Lion.

I wanted smartmontools installed to monitor the disk health of my LAN server at home. This is not an uncommon thing to want to do, so I thought I’d write and share a Chef cookbook for it. I also took this opportunity to write up the experience so I can illustrate how easy it is to write a cookbook for Chef.

The first thing to do when writing a cookbook is to create the cookbook directory structure with knife cookbook create. This command will create a README.rdoc by default, and I prefer Markdown, so I specify the -r md option.

knife cookbook create smartmontools -r md

By default, metadata and the default recipe are created with boilerplate content for author and copyright. I have configured the values in my knife.rb:

cookbook_copyright "Joshua Timberman"
cookbook_license   "apachev2"
cookbook_email     "cookbooks@housepub.org"

The resulting directory structure will be created:

% tree cookbooks/smartmontools
cookbooks/smartmontools
├── README.md
├── attributes
├── definitions
├── files
│   └── default
├── libraries
├── metadata.rb
├── providers
├── recipes
│   └── default.rb
├── resources
└── templates
    └── default

10 directories, 3 files

README Driven Development

I’m a big fan of Tom Preston-Werner’s blog post on README driven development. I don’t write the complete README before I start writing code for a new cookbook. I do write it as I go.

In order to write a proper README for a cookbook, and to write the cookbook itself, we’ll need to know a bit more about the software we’re installing. The best way to do that depends on the software, but often it is as simple as merely installing the package on a test system such as a virtual machine and explore its contents.

apt-get install smartmontools
dpkg -L smartmontools

Of note for smartmontools, documentation is in /usr/share/doc/smartmontools and configuration is in /etc. In particular, /etc/smartd.conf and /etc/smartmontools.

For now assume that the cookbook README.md is being written along the way.

List of Resources

One of the things I do when I am writing a new cookbook and exploring the contents of a package is to be mindful of Chef Resources I want to manage in the recipe(s). In the case of smartmontools, at this point I have determined I need a few specific resources.

Install the Package

First, as I’ve installed the package, I clearly need a package. I’m pretty confident that this particular package will not break backwards compatibility, and can be safely upgraded to the latest version if necessary.

Configuration files are templates

The next resources in the recipe are the configuration files. I want to dynamically configure these, so I am going to use templates.

I’m not going to write these from scratch. Instead, I will copy the source files from the installed package on my test system. These will go into templates/default in the cookbook.

% tree templates
templates
└── default
    ├── smartd.conf.erb
    └── smartmontools.default.erb

Templates are dynamically generated using ERB, and they can use Node attributes. I can use the automatically detected attributes, or I can set new attributes for the node in the cookbook.

Attributes Used in the Templates

The attributes go in the attributes/default.rb file in the cookbook. The ones I use are:

Attributes are definitely something to document in the README.

In templates/default/smartd.conf.erb, I check if there’s a list of devices to monitor, and if so iterate over the list passing in the default options (device_opts). The cookbook doesn’t at this time support per-device options - the same ones are applied to all devices. If devices is empty, then the configuration will use DEVICESCAN.

<% if node['smartmontools']['devices'].length > 0 -%>
<%   node['smartmontools']['devices'].each do |device| -%>
<%=    "/dev/#{device} #{node['smartmontools']['device_opts']}" %>
<%   end -%>
<% else -%>

DEVICESCAN -m root -M exec /usr/share/smartmontools/smartd-runner
<% end -%>

In templates/default/smartmontools.default.erb, the smartmontools daemon will be enabled based on the start_smartd attribute. Additional options will be passed per smartd_opts:

start_smartd=<%= node['smartmontools']['start_smartd'] %>

<% if node['smartmontools']['smartd_opts'].length > 0 -%>
smartd_opts="<%= node['smartmontools']['smartd_opts'] %>"
<% else -%>
#smartd_opts="--interval=1800"
<% end -%>

Generally speaking when creating attributes and using them in a template, I use the default values that are found in the configuration file dropped off by the package. I also try to use the default settings over all.

However, a cookbook is a place to express opinions. I made some with this cookbook, such as enabling DEVICESCAN if there are no devices, despite the configuration file’s comments indicating that the option shouldn’t be used. The best thing about attributes is they allow other people using the cookbook to change the behavior based on their preferences much easier than manually modifying things dropped off by a package.

I strongly recommend documenting where a cookbook has behavior that is not default for the installed package or upstream documentation. I did this in the README for this cookbook discussing the DEVICESCAN option and otherwise where appropriate.

Static Files

This cookbook has only one static file which will be deployed to /etc/smartmontools/run.d/10mail. The smartmontools package allows creating a number of scripts that go in the directory, and I have created an attribute for the list of scripts. These generally don’t cookbook_file resources. Since the list is an attribute, I iterate over that with Ruby’s Array#each loop.

The attribute:

The loop of resources:

Each filename in the array (currently only ‘10mail’) needs to have a corresponding file in the files/default directory of the cookbook.

% tree files
files
└── default
    └── 10mail

This file’s contents are not particularly exciting, I used the same one that came out of the package.

Why would I want to manage a static file that came from the package? Perhaps I want to modify the script in some way that doesn’t make sense in an attribute. In this case I don’t, however creating the attribute and iterating over it makes it easy to extend this functonality in the cookbook.

Manage the Service

Smartmontools comes with a service. That is, there’s an init script that can be enabled and started to monitor disk devices. This is actually the whole point of the cookbook, so I’ll make sure there’s a resource to manage the service.

In this resource, I used the meta parameter supports for the service. I found out what the init script can do by simply running it with no options on my test system.

% /etc/init.d/smartmontools
Usage: /etc/init.d/smartmontools {start|stop|restart|reload|force-reload|status}

The various options passed to the init script manage it in the familiar way. Telling Chef about it has a specific effect on the way the service provider functions when Chef manages it.

• status: Chef will use /etc/init.d/SERVICE_NAME status to determine if the service is running. If the return code is 0, its running. Otherwise, Chef checks the process table for a process running with the name of the service.
• reload: Chef can only take the reload action for a service if it actually supports reload.
• restart: When the restart action is sent to a service, if it supports restart then Chef will use /etc/init.d/SERVICE_NAME restart. Otherwise, Chef will use stop and start.

Earlier when we wrote the template resources, we notified the service to reload. Since the resource supports reload, we can do this. Also note that the resource has an array of actions. Each of these actions will be taken if necessary when Chef manages it. In this case, it will be enabled at boot time, and then started if it is not already running.

Let’s Use the Recipe

Now that we’ve written a nice recipe and understand what it’s about to do, let’s actually use it on a node we’d like to have smartmontools. Before uploading, I remove the boilerplate directories that were created by knife. The actual cookbook contents for upload are:

% tree cookbooks/smartmontools
cookbooks/smartmontools
├── README.md
├── attributes
│   └── default.rb
├── files
│   └── default
│       └── 10mail
├── metadata.rb
├── recipes
│   └── default.rb
└── templates
    └── default
        ├── smartd.conf.erb
        └── smartmontools.default.erb

6 directories, 7 files

I didn’t mention the metadata yet as I haven’t modified it yet. The knife command will create a default metadata.rb file as mentioned before. It will also populate it with some boilerplate content. The main thing I’m going to modify is the version and the platforms supported.

Now it is time to actually upload the cookbook and apply it to a node.

% knife cookbook upload smartmontools
Uploading smartmontools             [0.5.0]
upload complete

% knife node run list add virt1test 'recipe[smartmontools]'
run_list:
    role[ubuntu]
    recipe[smartmontools]

Run Chef

I’ll use knife ssh to run chef-client on the nodes that have the smartmontools recipe applied.

Verify results

We can verify the results of running the recipe by examining the resources on the target system(s). Chef’s contract with you is that it will configure the resources in the manner specified in the recipe. You can be confident that it will completely configure every resource if it exits cleanly (which it did from the output above).

% cat /etc/default/smartmontools
...
start_smartd=yes
...
% cat /etc/smartd.conf
...
DEVICESCAN -m root -M exec /usr/share/smartmontools/smartd-runner
...
% sudo service smartmontools status
 * smartd is running

Now we can take this same recipe and apply it to other systems. I tested smartmontools on an Ubuntu system. Notice earlier that I had an ubuntu role on my node. I actually modified that role to include the smartmontools recipe, and then all my Ubuntu nodes were configured for smartmontools when they ran Chef again.

Conclusion

I hope this guide was helpful. I shared the cookbook on the Chef Community web site, and the source code is available on Chef Community web site. Note that while I said this would be easy, it certainly isn’t trivial. There are a lot of steps involved in making cookbooks that are dynamic, easily customized and shareable with others. It takes practice, but after a few you get the hang of it. End to end, this cookbook took me about 2 hours to write, test, tweak (fix bugs) and document.

I recently had a chance to sit down and implemented an encrypted data bag in my personal environment. This should translate nicely to anyone that wants to use encrypted data bags in their environment.

The use case

I send mail out through an SASL authenticated SMTP server. My local network has a postfix SMTP relay that connects to the SASL auth relay. I’m using the Opscode postfix cookbook with the sasl_auth recipe, since I wrote it originally for this use case.

The postfix::sasl_auth recipe is applied in an “operations master” role. The attributes for configuring the user and password for SASL are attributes. Relevant lines from the role:

name "ops_master"
run_list("recipe[postfix::sasl_auth]")
override_attributes(
  "postfix" => {
    "relayhost" => "[smtp.example.com]:587",
    "smtp_sasl_auth_enable" => "yes",
    "smtp_sasl_passwd" => "AWESOME!!",
    "smtp_sasl_user_name" => "MYUSER"
  }
)

Encrypted Secrets

Chef’s data bags are a great way to store infrastructure wide, but not role or node specific information. Encrypted data bags are a great way to store sensitive information, like passwords. Here are the steps I followed to get the encrypted data bag set up.

First, I created the secret key file that is used to encrypt the contents of the data bag item. This file will not be stored in source control, as it is highly sensitive, and only gets copied to the systems that need it.

openssl rand -base64 512 > ~/.chef/encrypted_data_bag_secret

Next, I created the actual data bag.

knife data bag create secrets

Next I created the data bag item using the secret key. This is created directly on the Chef Server, rather than a plain text file.

knife data bag create secrets postfix --secret-file ~/.chef/encrypted_data_bag_secret

I’m not saving the plaintext data bag item, but will store the encrypted item, so I’ll retrieve it from the Chef Server and redirect the output to a JSON file.

mkdir data_bags/secrets
knife data bag show secrets postfix -Fj > data_bags/secrets/postfix.json
cat data_bags/secrets/postfix.json
{
  "id": "postfix",
  "user": "encrypted string here",
  "passwd": "encrypted string here"
}

The current recipe doesn’t support using an encrypted data bag item, so I had to modify it. First, load the encrypted data bag item.

postfix_creds = Chef::EncryptedDataBagItem.load("secrets","postfix",
  Chef::EncryptedDataBagItem::DEFAULT_SECRET_FILE)

This access the “secrets” data bag for the “postfix” item, and uses the default value for the secret key file (which is “/etc/chef/encrypted_data_bag_secret”.

Next, I update the sasl_passwd template to pass in the user and password from the data bag item.

template "/etc/postfix/sasl_passwd" do
  #...
  variables(:smtp_sasl_passwd => smtp_sasl['passwd'],
            :smtp_sasl_user_name => smtp_sasl['user'])
  #...
end

Finally, the template is updated to use the new values.

<%= node[:postfix][:relayhost] %> <%= @smtp_sasl_user_name %>:<%= @smtp_sasl_passwd %>

After uploading the cookbook and the role (I removed the username and password), I copied the secret key file over to the node I needed to run Chef on, then ran Chef Client.

sudo cat /etc/postfix/sasl_passwd
[smtp.example.com]:587 MYUSER:AWESOME!!

Yay!

The full recipe is:

The library file (cookbooks/helper_libs/libraries/encrypted_data_bag_item.rb) is:

I wanted to dynamically set my run list for testing with Vagrant without editing the Vagrantfile. Since the Vagrantfile is just Ruby, and the run list is an array, I can split a string into an array and use it.

If I don’t specify the run list, it will still configure the node with my base Ubuntu configuration. Then bring up a Vagrant box with a comma separated string of items in the run list.

% CHEF_RUN_LIST="role[ubuntu],recipe[apache2]" vagrant up

Earlier today, GitHub released their native OS X client application. As I automate everything on my Macs with Chef, I wrote a cookbook for Github for Mac.

In my chef-repo I did:

knife cookbook site install ghmac
knife cookbook upload ghmac
#... add 'recipe[ghmac]' to my workstation role ...
knife ssh role:workstation 'source .rvm/scripts/rvm && chef-client'

Since my Mac OS X workstations all have role[workstation], they all got GitHub for Mac installed automatically. I did have to do the configuration steps through the UI, and that made me cry a little, but I got over it. The application itself is quite nice.

I wrote the following as a non-technical analogy for a variety of “as a service” cloud type providers.

AWS gives you the entire play ground, for the most part. You can do pretty much anything you want, and they give you some really cool pre-made toys to put in it.

Engine Yard is more like a sandbox. You can come play in the sandbox, but they make the rules and you have to use their buckets and shovels.

Heroku is just the bucket. You can’t look inside, you just put your sand in and they mold it into the sandcastle.

Rackspace is more like the entire park where the playground is. There’s other stuff there, and you can make use of it while you visit. They have a few sandboxes, and you can build your own section.

Opscode is like the construction company. We provide the tools and materials needed to build all the sandboxes, buckets, and parks. Each of the others can use Opscode in whole or in part of building their playground or park.

There’s a rumor floating around that I automate everything on my laptop with Chef, including window maximizing. While that’s not entirely true, I do automate as much as possible on my workstations with Chef, as I have three laptops I use regularly. One application I like to have installed is VirtualBox, because I do some Chef recipe testing with it and Vagrant.

I did the most natural thing ever, I wrote a cookbook to automate installation of VirtualBox on OS X. The disk image file that Oracle distributes for VirtualBox on OSX uses an “mpkg” installer package, so I couldn’t just use the dmg_package LWRP I wrote to install it in /Applications. I extended the dmg_package provider to allow a type to be specified and pushed a new release of the dmg cookbook (v.0.6.0). The recipe in my cookbook builds the URL, which may seem convoluted (maybe it is), but it makes it more dynamic and customizable with fewer assumptions. The actual usage looks like this:

One thing to note about the mpkg type, is that these generally want to install in /Applications, which often requires root privileges. The dmg_package LWRP will use sudo when performing the mpkg installation, so if you’re like my and want to run chef-client as a non-root user for most things, it will prompt for your password, unless you unlock sudo in the same shell. Output of VirtualBox being installed:

INFO: Processing dmg_package[Virtualbox] action install (virtualbox::default line 41)
INFO: Processing remote_file[/Users/jtimberman/.chef/cache/Virtualbox.dmg] action create (/Users/jtimberman/.chef/cache/cookbooks/dmg/providers/package.rb line 36)
INFO: Processing execute[hdid /Users/jtimberman/.chef/cache/Virtualbox.dmg] action run (/Users/jtimberman/.chef/cache/cookbooks/dmg/providers/package.rb line 42)
INFO: execute[hdid /Users/jtimberman/.chef/cache/Virtualbox.dmg] sh(hdid /Users/jtimberman/.chef/cache/Virtualbox.dmg)
/dev/disk1                  Apple_partition_scheme
/dev/disk1s1                Apple_partition_map
/dev/disk1s2                Apple_HFS                              /Volumes/VirtualBox
INFO: execute[sudo installer -pkg /Volumes/Virtualbox/Virtualbox.mpkg -target /] sh(sudo installer -pkg /Volumes/Virtualbox/Virtualbox.mpkg -target /)
Password: <-- unlock sudo before running to avoid this prompt. "sudo -l" should be sufficient.
installer: Package name is Oracle VM VirtualBox
installer: Upgrading at base path /
installer: The upgrade was successful.
INFO: execute[hdiutil detach '/Volumes/Virtualbox'] sh(hdiutil detach '/Volumes/Virtualbox')
"disk1" unmounted.
"disk1" ejected.
INFO: Processing file[/Applications/Virtualbox.app/Contents/MacOS/Virtualbox] action create (/Users/jtimberman/.chef/cache/cookbooks/dmg/providers/package.rb line 55)
ERROR: file[/Applications/Virtualbox.app/Contents/MacOS/Virtualbox] (/Users/jtimberman/.chef/cache/cookbooks/dmg/providers/package.rb line 55) had an error: Operation not permitted - /Applications/Virtualbox.app/Contents/MacOS/Virtualbox
INFO: Chef Run complete in 36.219113 seconds
INFO: Running report handlers
INFO: Resources updated this run:
INFO:   execute[hdid /Users/jtimberman/.chef/cache/Virtualbox.dmg]
INFO:   execute[sudo installer -pkg /Volumes/Virtualbox/Virtualbox.mpkg -target /]
INFO:   execute[hdiutil detach '/Volumes/Virtualbox']
INFO: Report handlers complete

After writing and testing, the next thing I wanted to do is share the cookbook on the Opscode Chef Community site. However, I discovered that the good Chris Peplin already shared one! Since the site has a flat namespace, I would have to name mine differently. However, I think it would be better to collaborate in true Open Source fashion with Chris, and we’ll collaborate on this. 

I have a confession to make.

For those who follow me on Twitter, you already know. For those who read the title of this post, you can make an educated guess.

After ~16 years of using Vi/Vim/MacVim, I’m switching my primary editor to Emacs. Over the years, I have tried to use Emacs without much progress. I never really gave enough effort to learn it. In fact, only in the last year did I learn all the movement keys (C-f, C-b, C-n, C-p). To say I made uneducated disparaging comments about Emacs would be an understatement.

I have a highly customized Vim/MacVim setup, with lots of bells and whistles like NERDtree, Command-T, language plugins and more derived from Adam Jacob’s repository. I’ve spent countless hours tweaking and customizing it further to enhance productivity.

So how did I come around and completely switch editors? It wasn’t easy, that’s for sure. First there’s the resistance to a new tool. However, I wanted to keep a truly open mind, and give it a real, honest attempt. I bought the O’Reilly Learning GNU Emacs book and started reading, with the editor open to one of my favorite things to edit, my ~/chef-repo.

Actually the first thing I did was get a sensible and sane configuration started. I forked Phil Hagelberg’s excellent Emacs Starter Kit. I also thought AquaMacs would be the sensible place to start, since I’m using a Mac, and it uses a lot of familiar OS X hotkeys for copy/paste, opening and closing tabs and more. However, some fellow Opscode coworkers advised against AquaMacs, especially with the starter kit, because it leads one down a path of really “non-standard” Emacs settings. That isn’t entirely awful, since I manage my workstation(s) with Chef and have my Emacs config available first thing on a new machine (now that its added in, anyway :)). However, I quickly became annoyed with some erratic behavior. For example, opening a tab, then having something like a completion window means that switching tabs will keep the completion window persisted, even though it was split on “a different tab”. 

So I switched to GNU Emacs, and have been using that for the last week. I have no measure of whether my productivity is increased or not, but I’m getting pretty comfortable with the editor, and able to do most of what I was really good at in Vim. I sometimes miss Command-T, but the default Emacs file fine (C-x C-f for those that don’t know) is really good. Some of the things that I have found to be totally essential and are what keep me using the editor:

Buffers are totally sane, very fast to switch between, and easy to navigate between open files. Also related, the half hour or so I spent learning and practising the basics - create horizontal and veritical split, opening files in new windows, and more was totally worth it. I’m actually getting to be faster than the panels used in Vim, which I’ve been using for ~4 years.

File navigation is awesome. I really liked ”-T” in the short time I used TextMate, and I loved FuzzyFileFinder and Command-T plugins for Vim. However the flexiblity of the built-in default file navigation in Emacs is really intuitive for me.

With the Emacs starter-kit, some good modes are installed and available by default without modification. Most of the time I’m using Ruby, Markdown or Javascript modes. I anticipate starting in on org-mode at some point, but as I’m still getting comfortable with the basics, we’ll take some baby steps :).

Emacs really is easy to customize. I never really did much with Vim. I certainly cargo culted other configs, went with what worked, and left it at that. With Emacs, I’m doing that, but the code I’m looking at is more sensible to me. I hope this doesn’t mean I’m going to go learn Lisp :). As I’ve forked emacs-starter-kit on GitHub, and made my own customizations, I’m also sharing those back on the site, so if you want to see what changes I’ve made and additional modes I’ve installed, take a peek. If you look, there’s a chef.el the “jtimberman” directory with some shortcuts to drop off common Chef resources. As I learn more, I plan to make that a bit more useful as snippets or maybe even a chef-mode.

Speaking of custom modes, even if I didn’t use it for anything else, Emacs is totally worth using for confluence-mode. We have two Confluence wikis, one for Chef, and an internal company wiki. Confluence mode seems to assume you only have one wiki, and I haven’t sorted out the details on how to easily switch between the two. For now I just put the confluence-url setting in the *scratch* buffer and do M-x eval-region to “switch”. That said, the ability to quickly search for a page and load the confluence wiki markup page in a buffer, then save the content back out to the wiki directly is *awesome*. No more 4 step process to copy/paste the markup into an editor window and back out to the form. Also, while there’s a Vim syntax plugin for Confluence, I had no end of error trying to use it.

As I’ve only been using Emacs for a week, I’m still a novice. However, I’ve gotten to the point where I can get what I need done in writing Chef cookbooks and documentation. I’m enjoying the learning process, and pleased that I’m at a point where the editor is not only out of the way, but helping me get more done.

The potential to devolve into an editor war is high, so please, no comments about how other editors are better or that I should switch back to Vim. I firmly believe that everyone should use tools that they enjoy, and that help them be productive. Snarky comments can be made on twitter, of course :).

Exception and report handlers are a pretty cool feature of Chef. They have been around for several versions (first appearing in 0.9.0), but haven’t received a lot of attention. In the Chef Advanced training class, we go into greater detail about writing handlers, so I had a play with them a bit recently. Also, the default logging output for a chef-client run is a bit more verbose in Chef 0.10 than it was in previous versions. I’ve been convinced this is for the better overall, though all the “Processing…” lines can get in the way of things that are actually being configured. To make it clearer of what resources were actually configured in the Chef run, I wrote a very simple handler:

To use this in my chef-client, I drop it off as a file, say “/var/lib/chef/handler/updated_resources.rb” and then configure /etc/chef/client.rb:

Then when I run chef-client, after the run is complete it will display something like this:

[Sun, 24 Apr 2011 22:50:58 -0600] INFO: Chef Run complete in 15.297228 seconds
[Sun, 24 Apr 2011 22:50:58 -0600] INFO: Running report handlers
[Sun, 24 Apr 2011 22:50:58 -0600] INFO: Resources updated this run:
[Sun, 24 Apr 2011 22:50:58 -0600] INFO:   execute[apt-get update]
[Sun, 24 Apr 2011 22:50:58 -0600] INFO:   execute[index-gem-repository]
[Sun, 24 Apr 2011 22:50:58 -0600] INFO:   remote_directory[/srv/gems/gems]
[Sun, 24 Apr 2011 22:50:58 -0600] INFO: Report handlers complete

Update: I released a gem for this handler. You can now install the gem and then use require 'chef/handler/updated_resources' in /etc/chef/client.rb