This quick tip brought to you by the letters “p,” “r,” and “y.”

You can start up a pry session in the context of a Chef recipe easily by using chef-apply. The pry gem is bundled with the Chef omnibus package, so it’s immediately available.

1

chef-apply -e 'require "pry"; binding.pry'

This will result in a prompt like this:

1

2.1.6 (#<Chef::Recipe>):0 >

This is a recipe! Use pry’s ls command:

1
2
3
4
5
6
7
8
9
10
11
12

2.1.6 (#<Chef::Recipe>):0 > ls
Chef::EncryptedDataBagItem::CheckEncrypted#methods: encrypted?
Chef::DSL::DataQuery#methods: data_bag  data_bag_item  search
Chef::DSL::PlatformIntrospection#methods: platform?  platform_family?  value_for_platform  value_for_platform_family
Chef::DSL::IncludeRecipe#methods: include_recipe  load_recipe  require_recipe
*** SNIP ***

2.1.6 (#<Chef::Recipe>):0 > cd node
2.1.6 (node[localhost.example.com]):1 > ls
*** SNIP ***
2.1.6 (node[localhost.example.com]):1 > exit
2.1.6 (#<Chef::Recipe>):0 >

Write a resource:

1
2
3

2.1.6 (#<Chef::Recipe>):0 > file "/tmp/hello_world" do
2.1.6 (#<Chef::Recipe>):0 *   content "I'm in pry!"
2.1.6 (#<Chef::Recipe>):0 * end

This will return the file[/tmp/hello_world] resource. It doesn’t run Chef, but we can do that in one of two ways: exit pry, or send the create action to the resource.

1
2
3
4
5
6
7
8

2.1.6 (#<Chef::Recipe>):0 > resources("file[/tmp/hello_world]").run_action :create
Recipe: (chef-apply cookbook)::(chef-apply recipe)
  * file[/tmp/hello_world] action create
    - create new file /tmp/hello_world
    - update content in file /tmp/hello_world from none to 3a5417
    --- /tmp/hello_world  2015-09-01 08:35:18.000000000 -0600
    +++ /tmp/.hello_world20150901-42665-11bwurx   2015-09-01 08:35:18.000000000 -0600
    @@ -1 +1,2 @@

If we write multiple resources, we’d have to send that action to every one of them. Exiting pry will work, but then we are, of course, no longer in the pry session. This is not ideal, but hey, it’s not like we’re in chef-shell.

The chef-apply program runs in “solo mode.”

1
2

2.1.6 (#<Chef::Recipe>):0 > Chef::Config.solo
=> true

However, it may be useful to debug things through the Chef Server API. We will want to do two things. First, load a config file like .chef/knife.rb. We can verify the Chef Server we want is configured by checking Chef::Config[:chef_server_url].

1
2
3
4
5
6

2.1.6 (#<Chef::Recipe>):0 > Chef::Config[:chef_server_url]
=> "https://localhost:443"
2.1.6 (#<Chef::Recipe>):0 > Chef::Config.from_file('.chef/knife.rb')
=> "client"
2.1.6 (#<Chef::Recipe>):0 > Chef::Config[:chef_server_url]
=> "https://api.opscode.com/organizations/joshtest"

Cool. Now let’s borrow chef-shell’s helper methods for interacting with the API.

1
2

require 'chef/shell/ext'
Chef::Shell::Extensions.extend_context_object(self)

And now we can use, for example, api.get or nodes.all.

1
2

2.1.6 (#<Chef::Recipe>):0 > api.get('/users')
=> [{"user"=>{"username"=>"joshtest"}}, {"user"=>{"username"=>"jtimberman"}}]

Of course, we can get a lot of this functionality by starting up chef-shell and loading pry, but I think this was more fun :–).

Learn more about pry.

I’m currently updating my vagrant cookbook, and adding ChefSpec coverage. Each of the different platform recipes results in slightly different resources to download the package file and install it. To support this, I have helper methods that calculate the download URI, the package name, and the SHA256 checksum based on the version of Vagrant (node['vagrant']['version']), and the platform (node['os'], node['platform_family']).

The outcomes I want to test are that for a given platform: the correct recipe is included, the correct file is downloaded, and the correct package resource installs the downloaded file. Those tests look like this (using Ubuntu/Debian example first):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

it 'includes the debian platform family recipe' do
  expect(chef_run).to include_recipe('vagrant::debian')
end

it 'downloads the package from the calculated URI' do
  expect(chef_run).to create_remote_file('/var/tmp/vagrant.deb').with(
    source: 'https://dl.bintray.com/mitchellh/vagrant/vagrant_1.88.88_x86_64.deb'
  )
end

it 'installs the downloaded package' do
  expect(chef_run).to install_dpkg_package('vagrant').with(
    source: '/var/tmp/vagrant.deb'
  )
end

I’ve set the version attribute in the chef_run block for the ChefSpec run to 1.88.88 to ensure that it doesn’t use the value set in the attributes file, and then I can test this specifically in the source for the calculated URI, even if the attribute changes – hopefully Vagrant doesn’t have a 1.88.88 version some day ;).

When I run rspec spec, I get exceptions, however.

1
2
3
4

1) vagrant::default debian includes the debian platform family recipe
   Failure/Error: end.converge(described_recipe)
   OpenURI::HTTPError:
     404 Not Found

This is because in the attributes/default.rb, the checksum is retrieved using the vagrant_sha256sum method:

1

default['vagrant']['checksum'] = vagrant_sha256sum(node['vagrant']['version'])

The exception is happening when Chef loads the attributes file, and because the version is not valid. The solution here is to stub out the return value from vagrant_sha256sum. This can be anything at all really, because that specific attribute is to make sure we don’t have to re-download the package to compare its checksum on later Chef runs. In this cookbook, the helper methods are not namespaced under a module, they’re bare methods in libraries.helpers.rb. This poses some challenges when trying to stub them in ChefSpec. I won’t rehash all the ways I attempted to get this to work, and instead focus on the final solution that got the tests passing:

1
2
3

before(:each) do
  allow_any_instance_of(Chef::Node).to receive(:vagrant_sha256sum).and_return('')
end

When Chef loads cookbook attributes files, it is evaluating them in the context of a Chef::Node, so those library helper methods are sent to the Chef::Node object. Similarly, if this were inside a recipe, I would use Chef::Recipe, and if it were inside a resource (e.g., package), Chef::Resource.

I put this before block at the describe 'vagrant::default' level, not within any context blocks, so it will be done for each of the various per-platform tests. The results in my debian context are now:

1
2
3
4
5
6
7
8
9
10

% rspec spec --color -fd

vagrant::default
  debian
    includes the debian platform family recipe
    downloads the package from the calculated URI
    installs the downloaded package

Finished in 7.04 seconds (files took 7.93 seconds to load)
3 examples, 0 failures

Six issues have been filed against ChefSpec about this. Hopefully this can result in fewer inquiries.

Happy testing!

As I indicated on Twitter earlier tonight, I’m working with the new Policyfile feature of ChefDK. While converting my personal systems’ repository to use Policyfile instead of roles, I found myself writing this Policyfile:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

name 'home_server'
default_source :community
run_list(
         'build-essential',
         'packages',
         'users',
         'sudo',
         'runit',
         'ntp',
         'openssh',
         'postfix'
        )

cookbook 'build-essential'
cookbook 'packages', git: 'https://github.com/mattray/packages-cookbook', branch: 'multipackage'
cookbook 'users', path: '../housepub-chef-repo/cookbooks/users'
cookbook 'sudo'
cookbook 'runit'
cookbook 'ntp'
cookbook 'openssh'
cookbook 'postfix'

No big deal, but I found the repetition… redundant. Several of these cookbooks are fine floating on the latest version from Supermarket – everything but packages and users. So I thought, “wouldn’t it be great if entries in the run list were automatically added as dependencies?”

Then, I added chef-client-runit to the run list, but I didn’t add it as a cookbook entry, performed the chef update, and reran my chef provision command, and wound up with chef-client-runit being converged.

To illustrate this with a really simple example, I confirmed with zsh:

% cd ~/Development/sandbox/test
% chef generate policyfile

Then edit the Policyfile.rb:

1
2
3

name "example_application"
default_source :community
run_list "zsh"

Note that there is no cookbook line here.

% chef install
Building policy example_application
Expanded run list: recipe[zsh]
Caching Cookbooks...
Using      zsh 1.0.1

Lockfile written to /Users/jtimberman/Development/sandbox/test/Policyfile.lock.json

And if we examine the Policyfile.lock.json, we see:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

{
  "revision_id": "e8b5b48d35f4a8efcd037ef6c9cc8e34f901561ffef160bd0a57ca1b612a1179",
  "name": "example_application",
  "run_list": [
    "recipe[zsh::default]"
  ],
  "cookbook_locks": {
    "zsh": {
      "version": "1.0.1",
      "identifier": "b512ef33af29b8d34ad7e4e9b6ad38d42dea4945",
      "dotted_decimal_identifier": "50967789358229944.59473511204894381.62483954551109",
      "cache_key": "zsh-1.0.1-supermarket.chef.io",
      "origin": "https://supermarket.chef.io/api/v1/cookbooks/zsh/versions/1.0.1/download",
      "source_options": {
        "artifactserver": "https://supermarket.chef.io/api/v1/cookbooks/zsh/versions/1.0.1/download",
        "version": "1.0.1"
      }
    }
  } <SNIP>

Yay! Shoutout to Dan DeLeo for preemptively implementing features I didn’t even know I wanted yet :–).

Earlier today, ChefDK 0.6.0 was released. In this post, I will illustrate a fairly simple walkthrough using Amazon EC2, based on information in the document. This example will include Policyfile use, too. Let’s get started.

First, install ChefDK 0.6.0. You can get it from the Chef Downloads page.

Next, setup AWS credentials. Ensure that they are exported to the current shell environment.

AWS_ACCESS_KEY_ID=secrets
AWS_SECRET_ACCESS_KEY=secrets
AWS_DEFAULT_REGION=us-east-1
AWS_SSH_KEY=your_ssh_key_name
AWS_ACCESS_KEY=secrets
AWS_SECRET_KEY=secrets

I’m going to use Hosted Chef as my Chef Server. I already have my user API key and configuration in ~/.chef, and I’m going to rely on the automatic configuration detection in the chef command for that.

Generate a new repository using the chef generate command. Further commands run from this directory.

chef generate repo chefdk-provision-demo
cd chefdk-provision-demo

Generate a provision cookbook. This is the required name, and it must be in the current directory.

chef generate cookbook provision

Edit the default recipe, $EDITOR provision/recipes/default.rb.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

context = ChefDK::ProvisioningData.context
with_driver 'aws::us-west-2'
options = {
  ssh_username: 'admin',
  use_private_ip_for_ssh: false,
  bootstrap_options: {
    key_name: 'jtimberman',
    image_id: 'ami-0d5b6c3d',
    instance_type: 'm3.medium',
  },
  convergence_options: context.convergence_options,
}
machine context.node_name do
  machine_options options
  action context.action
  converge true
end

To break this down, first we get the ChefDK provisioning context that will pass in options to chef-provisioning. Then we tell chef-provisioning to use the AWS driver, and in the us-west-2 region. The options hash is used to setup the instance. We’re using Debian 8, which uses the admin user to log in, an SSH key that exists in the AWS region, the actual AMI, and finally the instance type. Then, we’re going to set the convergence options automatically from ChefDK. This is the important part that will ensure the node has the right run list.

Generate a Policyfile.rb.

chef generate policyfile

And edit its content, $EDITOR Policyfile.rb.

1
2
3
4

name            "chefdk-provision-demo"
default_source  :community
run_list        "recipe[libuuid-user]"
cookbook        "libuuid-user"

Here we’re simply getting the libuuid-user cookbook from Supermarket and applying the default recipe to the nodes that have this policy.

The next step is to install the Policyfile. This generates the Policyfile.lock.json, and downloads the cookbooks to the cache, ~/.chefdk/cache/cookbooks. If this isn’t run, chef will complain, with a reminder to run it.

chef install

Finally, we can provision a testing system with this policy:

chef provision testing --sync -n debian-libuuid

This will result in output similar to this:

Uploading policy to policy group testing
Uploaded libuuid-user 1.0.1 (c3220a49)
Compiling Cookbooks...
Recipe: provision::default
  * machine[debian-libuuid] action converge
    - Create debian-libuuid with AMI ami-0d5b6c3d in us-west-2
    - create node debian-libuuid at https://chef-api.example.com/organizations/testing
    -   add normal.tags = nil
    -   add normal.chef_provisioning = {"hash" => "of options"}
    - waiting for debian-libuuid (i-251f1cec on aws::us-west-2) to be connectable (transport up and running) ...
    - been waiting 60/120 -- sleeping 10 seconds for debian-libuuid (i-251f1cec on aws::us-west-2) to be connectable ...
    - debian-libuuid is now connectable        - generate private key (2048 bits)
    - create directory /etc/chef on debian-libuuid
    - write file /etc/chef/client.pem on debian-libuuid
    - create client debian-libuuid at clients
    -   add public_key = "-----BEGIN PUBLIC KEY-----\n..."
    - Add debian-libuuid to client read ACLs
    - Add debian-libuuid to client update ACLs
    - create directory /etc/chef/ohai/hints on debian-libuuid
    - write file /etc/chef/ohai/hints/ec2.json on debian-libuuid
    - write file /etc/chef/client.rb on debian-libuuid
    - write file /tmp/chef-install.sh on debian-libuuid
    - run 'bash -c ' bash /tmp/chef-install.sh'' on debian-libuuid
    [debian-libuuid] Starting Chef Client, version 12.3.0
                     [2015-05-15T22:42:43+00:00] WARN: Using experimental Policyfile feature
                     resolving cookbooks for run list: ["libuuid-user::default@1.0.1 (c3220a4)", "libuuid-user::verify@1.0.1 (c3220a4)"]
                     Synchronizing Cookbooks:
                       - libuuid-user
                     Compiling Cookbooks...
                     Converging 1 resources
                     Recipe: libuuid-user::default
                       * user[libuuid] action create
                       - create user libuuid

                     Running handlers:
                     Running handlers complete
                     Chef Client finished, 1/1 resources updated in 5.29666495 seconds
    - run 'chef-client -l auto' on debian-libuuid

I’ve started working with the audit mode feature introduced in Chef version 12.1.0. Audit mode allows users to write custom rules (controls) in Chef recipes using new DSL helpers. In his ChefConf 2015 talk, “Compliance At Velocity,” James Casey goes into more of the background and reasoning for this. For now, I wanted to share a few tips with users who may be experimenting with this feature on their own, too.

First, we need to update ChefDK to version 0.5.0, as that includes a version of test kitchen that allows us to configure audit mode for chef-client.

1

curl -L https://chef.io/chef/install.sh | sudo bash -s -- -P chefdk

Next, create a new cookbook for the audit mode tests.

1
2

chef generate cookbook audit-test
cd audit-test

Then, modify the audit-test cookbook’s .kitchen.yml:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

---
driver:
  name: vagrant

provisioner:
  name: chef_zero
  client_rb:
    audit_mode: :audit_only

platforms:
  - name: ubuntu-12.04
  - name: centos-6.5

suites:
  - name: default
    run_list:
      - recipe[audit-test::default]
    attributes:

This is the generated .kitchen.yml with client_rb added to the provisioner config. Note that we must use the Ruby symbol syntax for the config value, :audit_only. The other valid values for audit_mode are :enabled and :disabled. This will be translated to an actual Ruby symbol in the generated config file (/tmp/kitchen/client.rb):

1

audit_mode :audit_only

Next, let’s write a control rule to test. Since we’re using the default .kitchen.yml, which includes Ubuntu 12.04 and uses SSH to connect, we can assume that SSH is running, so port 22 is listening. The following control asserts this is true.

1
2
3
4
5
6
7

control_group 'Blog Post Examples' do
  control 'SSH' do
    it 'should be listening on port 22' do
      expect(port(22)).to be_listening
    end
  end
end

Now run kitchen converge ubuntu to run Chef, but not tear down the VM aftward – we’ll use it again for another example. Here’s the audit phase output from the Chef run:

% kitchen converge ubuntu
Synchronizing Cookbooks:
  - audit-test
Compiling Cookbooks...
Starting audit phase

Blog Post Examples
  SSH
    should be listening on port 22

Finished in 0.10453 seconds (files took 0.37536 seconds to load)
1 example, 0 failures
Auditing complete

Cool! So we have asserted that the node complies with this control by default. But what does a failing control look like? Let’s write one. Since we’re working with SSH already, let’s use the SSHd configuration. By default in the Vagrant base box we’re using, root login is permitted, so this value is present:

1

PermitRootLogin yes

However, our security policy mandates that we set this to no, and we want to audit that.

1
2
3
4
5
6
7
8
9
10
11

control_group 'Blog Post Examples' do
  control 'SSH' do
    it 'should be listening on port 22' do
      expect(port(22)).to be_listening
    end

    it 'disables root logins over ssh' do
      expect(file('/etc/ssh/sshd_config')).to contain('PermitRootLogin no')
    end
  end
end

Rerun kitchen converge ubuntu and we see the validation fails.

Starting audit phase

Blog Post Examples
  SSH
    should be listening on port 22
    disables root logins over ssh (FAILED - 1)

Failures:

  1) Blog Post Examples SSH disables root logins over ssh
     Failure/Error: expect(file('/etc/ssh/sshd_config')).to contain('PermitRootLogin no')
expected File "/etc/ssh/sshd_config" to contain "PermitRootLogin no"
     # /tmp/kitchen/cache/cookbooks/audit-test/recipes/default.rb:8:in `block (3 levels) in from_file'

Finished in 0.13067 seconds (files took 0.32089 seconds to load)
2 examples, 1 failure

Failed examples:

rspec  # Blog Post Examples SSH disables root logins over ssh
[2015-04-04T03:29:41+00:00] ERROR: Audit phase failed with error message: Audit phase found failures - 1/2 controls failed

Audit phase exception:
Audit phase found failures - 1/2 controls failed

When we have a failure, we’ll have contextual information about the failure, including the line number in the recipe where itwas found, and a stack trace (cut from the output here), in case more information is required for debugging. To fix the test, we can simply edit the config file to have the desired setting, or we can manage the file with Chef to set the value accordingly. Either way, after updating the file, the validation will pass, and all will be well.

We can put as many control_group and control blocks with the it validation rules as required to audit our policy. If we have many validations, it can be difficult to follow with all the output if there are failures. Chef’s audit mode is based on Serverspec, which is based on RSpec. We can use the filter_tag configuration feature of RSpec to only run the control blocks or it statements that we’re interested in debugging. To do this, we need an RSpec.configuration block within the control_group – due to the way that audit mode is implemented, we can’t do it outside of control_group.

For example, we could debug our root login configuration:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

control_group 'Blog Post Examples' do
  ::RSpec.configure do |c|
    c.filter_run focus: true
  end

  control 'SSH' do
    it 'should be listening on port 22' do
      expect(port(22)).to be_listening
    end

    it 'disables root logins over ssh', focus: true do
      expect(file('/etc/ssh/sshd_config')).to contain('PermitRootLogin no')
    end
  end
end

The key here is to pass the argument focus: true (or if you like hash rockets, :focus => true) on the it block. This could also be used on a control block:

1
2
3

control 'SSH', focus: true do
  it 'does stuff...'
end

Then, when running kitchen converge ubuntu, we see only that validation:

Starting audit phase

Blog Post Examples
  SSH
    disables root logins over ssh (FAILED - 1)

Failures:

  1) Blog Post Examples SSH disables root logins over ssh
     Failure/Error: expect(file('/etc/ssh/sshd_config')).to contain('PermitRootLogin no')

This example is simple enough that this isn’t necessary, but if we were implementing audit mode checks for our entire security policy, that could be dozens or even hundreds of controls.

As of this writing, audit mode is still under development, and is considered an experimental feature. There will be further information, guides, and documentation about it coming to the Chef blog and docs site, and I’ll have a post coming soon with something I’m working on, so stay tuned!

One of my home projects while I’m on vacation this week is rebuilding my server with Fedora 21 (Server). In order to do this, I needed to add Fedora support to the runit cookbook, since I use runit for a number of services on my system. That’s really neither here nor there, as the topic of this blog post isn’t specific to Fedora, nor runit.

The topic is actually about an issue with transitive dependencies and how Chef dependency resolution works.

Here’s the scenario:

1. I run chef on my node
2. The runit cookbook is synchronized
3. An older version of the runit cookbook is downloaded
4. The new changes I expected were not made
5. WTFs ensue

So what happened?

The runit cookbook itself is updated to use Ian Meyer’s Package Cloud repository – at least, the version I want to use is, which is on GitHub. When I submitted the PR for adding this repository for RHEL platforms, Ian had not yet added Fedora packages. That’s okay because Fedora is not listed as supported in the cookbook. However, I wanted to use it, and figured folks in the community using Fedora Server might benefit too.

I digress. Ian pushed a Fedora package earlier today, so I added “fedora” to the various platform family conditionals, in the cookbook and opened the PR linked earlier. All was well in test kitchen. So I change my local repository’s Berksfile:

1

cookbook 'runit', github: 'hw-cookbooks/runit', ref: 'jtimberman/fedora-21'

Then a quick berks update runit and berks upload runit, and I was in business.

Or so I thought. Enter scenario listed above. The problem is, that when I did the berks upload, it only uploaded runit. However in the latest runit cookbook from that branch, it also adds a dependency on Computology’s packagecloud cookbook, since it uses that to add the repository. When the runit cookbook is specified on the berks upload command, it doesn’t upload the transitive dependencies when the location is a git URI. This appears to be by design.

What happens in Chef is that the server solves the graph, seeing that the node needs the runit cookbook. But the latest version of the runit cookbook depends on packagecloud, which hasn’t yet been uploaded. So the dependency solver looks for the latest version of the runit cookbook that meets the constraint (none), and doesn’t have the packagecloud cookbook. Thus, I end up with runit version 1.5.18 on my node, but it fails to converge because it doesn’t have the changes required for Fedora, which are in 1.5.20.

The simple solution here is to upload the packagecloud cookbook. This can be done with berks upload packagecloud, as it does exist in the Berksfile.lock and has been cached in the berkshelf. Alternatively, berks upload will also upload the cookbook, as that operates on all cookbooks in the Berksfile.lock.

I hope this helps anyone who’s faced this issue with transitive dependencies when working on a cookbook “in development.”

TL;DR, if you’re using Chef 11 and chef-sugar, upgrade chef-sugar to version 3.0.1. If you cannot upgrade, use the following in your chef_gem resources in your recipes:

1

compile_time true if Chef::Resource::ChefGem.instance_methods(false).include?(:compile_time)

As you may be aware, Chef 12.1.0 introduces a change to the chef_gem resource that prints out warning messages like this:

WARN: chef_gem[chef-vault] chef_gem compile_time installation is deprecated
WARN: chef_gem[chef-vault] Please set `compile_time false` on the resource to use the new behavior.
WARN: chef_gem[chef-vault] or set `compile_time true` on the resource if compile_time behavior is required.

These messages are just warnings, but if you’re installing a lot of gems in your recipes, you may be annoyed by the output. As the warning indicates, you can set compile_time true property. This doesn’t work on versions of Chef before 12.1, though:

NoMethodError: undefined method `compile_time' for Chef::Resource::ChefGem

So, as a workaround, we can ask whether we respond to the compile_time method in the chef_gem resource:

1
2
3

chef_gem 'chef-vault' do
  compile_time false if respond_to?(:compile_time)
end

This appears to get around the problem for most cases. However, if you’re using chef-sugar, you’ll note that until version 3.0.0, chef-sugar includes a compile_time DSL method that gets injected into Chef::Resource (and Chef::Recipe). This has been modified to at_compile_time in chef-sugar version 3.0.0 to work around Chef’s introduction of a compile_time method in the chef_gem resource. The simple thing to do is make sure that your chef-sugar gem/cookbook are updated to v3.0.1. However if that isn’t an option for some reason, you can use this conditional check:

1
2
3

chef_gem 'chef-vault' do
  compile_time true if Chef::Resource::ChefGem.instance_methods(false).include?(:compile_time)
end

Hat tip to Anthony Scalisi, who added this in a pull request for the aws cookbook. The instance_methods method comes from Ruby’s Module class. Per the documentation:

Returns an array containing the names of the public and protected instance methods in the receiver. For a module, these are the public and protected methods; for a class, they are the instance (not singleton) methods. If the optional parameter is false, the methods of any ancestors are not included.

If we look at this in, e.g., chef-shell under Chef 12.1.0:

1
2

chef:recipe > Chef::Resource::ChefGem.instance_methods(false)
 => [:gem_binary, :compile_time, :after_created]

And in Chef 12.0.3 or 11.18.6:

1
2

chef > Chef::Resource::ChefGem.instance_methods(false)
 => [:gem_binary, :after_created]

I am working on my presentation for ChefConf. I plan to have quite a lot of code samples. I’ve found the options for getting code samples with nice syntax highlight a lackluster endeavour, with various GUI editors like TextMate, Sublime, and Atom having “Copy as RTF” plugins, but none of them being easily customizable.

So I did a quick Google search and happened on a gist I hadn’t seen before. It describes the following steps:

1. Install Homebrew (done, I have that covered with Chef ;)).
2. Install “highlight” with brew install highlight.
3. Use highlight to transform the source code file to RTF and copy it to the clipboard.
4. Paste the clipboard into Keynote.app.

This isn’t much different than the other solutions, except one super cool thing I learned about highlight.

It has styles.

1
2
3

highlight -w

<OMG LIST OF EIGHTY TWO DIFFERENT STYLES!!!>

That’s right, at least at the time of this writing highlight has 82 different styles available. Including my favorite(s) solarized – both light and dark. Note that the --help output says that this option is deprecated in the version I’ve installed (3.18_1), but the styles are in /usr/local/Cellar/highlight/VERSION/share/themes.

Highlight knows the syntax highlighting for a lot of languages, these are in /usr/local/Cellar/highlight/VERSION/share/langDefs. For example I can get my Ruby recipe highlighted with this:

1

highlight -s solarized-light -O rtf recipes/client.rb | pbcopy

This will use Courier New as the font, and depending on the theme/style used, some of the highlighting may be bold or italic. This is easy enough to change in Keynote though.

For up to date documentation and information about highlight, visit the author’s page.

This quick tip is brought to you by my preparation for my ChefConf talk about using Chef Provisioning to build a Chef Server Cluster, which is based on my blog post about the same. In the blog post I used chef-zero as my Chef Server, but for the talk I’m using Hosted Chef.

In order for the Chef Provisioning recipe to work the provisioning node – the node that runs chef-client – needs to have the appropriate permissions to manage objects on the Chef Server. This is easy with chef-zero – there are no ACLs at all. However in Hosted Chef, like any regular Chef Server, the ACLs don’t allow nodes’ API clients to modify other nodes, or API clients.

Fortunately we can do all the work necessary using knife, with the knife-acl plugin. In this quick tip, I’ll create a group for provisioning nodes, and give that group the proper permissions for the Chef Provisioning recipe to create the machines’ nodes and clients.

First of all, I’m using ChefDK, and it’s my Ruby environment too, so install the gem:

1

chef gem install knife-acl

Next, use the knife group subcommand to create the new group. Groups are a number of users and/or API clients. By default, an organization on Hosted Chef will have admins, billing-admins, clients, and users. Let’s create provisioners now.

1

knife group create provisioners

The Role-based access control (RBAC) system in the Chef Server allows us to assign read, create, update, grant, and delete permissions to various objects in the organization. Containers are a special holder of other types of objects, in this case we need to add permissions for the clients and nodes containers. This is what allows the Chef Provisioning recipe’s machine resources to have their Chef objects created.

1
2
3
4
5
6
7
8
9

for i in read create update grant delete
do
  knife acl add containers clients $i group provisioners
done

for i in read create update grant delete
do
  knife acl add containers nodes $i group provisioners
done

Next, we need the API client that will be used by the Chef Provisioning node to authenticate with the Chef Server, and the node needs to be created as well. By default the client will automatically have permissions for the node object that has the same name.

1
2

knife client create -d chefconf-provisioner > ~/.chef/chefconf-provisioner.pem
knife node create -d chefconf-provisioner

Finally, we need to put the new API client into the provisioners group that was created earlier. First we need to get a mapping of the actors in the organization. Then we can add the client to the group.

1
2

knife actor map
knife group add actor provisioners chefconf-provisioner

The knife actor map command will generate a YAML file like this:

1
2
3
4
5
6
7
8
9

---
:user_map:
  :users:
    jtimberman: 12345678901234567890123456780123
  :usags:
    12345678901234567890123456780123: jtimberman
:clients:
  chefconf-provisioner: chefconf-provisioner
  jtimberman-chefconf-validator: jtimberman-chefconf-validator

This maps users to their USAG and stores a list of clients. More information about this is in the knife-acl README

At this point, we have a node, with the private key in ~/.chef that can be used with the Chef Server to use Chef Provisioning’s machine resource. We can also perform additional tasks that require having a node object, such as create secrets as Chef Vault items:

1

knife vault create secrets dnsimple -M client -J data_bags/secrets/dnsimple.json -A jtimberman -S 'name:chefconf-provisioner'

The entire series of commands is below.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

chef gem install knife-acl
knife group create provisioners

for i in read create update grant delete
do
  knife acl add containers clients $i group provisioners
done

for i in read create update grant delete
do
  knife acl add containers nodes $i group provisioners
done

knife client create -d chefconf-provisioner > ~/.chef/chefconf-provisioner.pem
knife node create -d chefconf-provisioner
knife actor map
knife group add actor provisioners chefconf-provisioner

knife vault create secrets dnsimple -M client -J data_bags/secrets/dnsimple.json -A jtimberman -S 'name:chefconf-provisioner'

Hopefully this helps you out with your use of Chef Provisioning, and a non-zero Chef server. If you have further questions, find me at ChefConf!

In this quick tip, I’ll explain why you may need to create resources to notify in a provider, even if the resource exists in a recipe, when using use_inline_resources in Chef’s LWRP DSL.

I’ll use an example cookbook, notif, to illustrate. First, I’ve created cookbooks/notif/resources/default.rb, with the following content.

1
2

actions :write
default_action :write

Then, I have written cookbooks/notif/providers/default.rb like this:

1
2
3
4
5
6
7

use_inline_resources

action :write do
  log 'notifer' do
    notifies :create, 'file[notified]'
  end
end

Then the default recipe, where I’ll use the resource automatically generated from the resource directory, notif.

1
2
3
4
5
6

file 'notified' do
  content 'something'
  action :nothing
end

notif 'doer'

When I run Chef, I’ll get an error like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

 Recipe: notif::default
   * file[notified] action nothing (skipped due to action :nothing)
   * notif[doer] action write

     ================================================================================
     Error executing action `write` on resource 'notif[doer]'
     ================================================================================

     Chef::Exceptions::ResourceNotFound
     ----------------------------------
     resource log[notifer] is configured to notify resource file[notified] with action create, but file[notified] cannot be found in the resource collection. log[notifer] is defined in /tmp/kitchen/cookbooks/notif/providers/default.rb:4:in `block in class_from_file'

     Resource Declaration:
     ---------------------
     # In /tmp/kitchen/cookbooks/notif/recipes/default.rb

      12: notif 'doer'

     Compiled Resource:
 ------------------
     # Declared in /tmp/kitchen/cookbooks/notif/recipes/default.rb:12:in `from_file'

     notif("doer") do
       action :write
       retries 0
       retry_delay 2
       default_guard_interpreter :default
       declared_type :notif

       recipe_name "default"
     end

To fix this, I define the file resource in the provider:

1
2
3
4
5
6
7
8
9
10
11

use_inline_resources

action :write do
  log 'notifer' do
    notifies :create, 'file[notified]'
  end

  file 'notified' do
    content new_resource.name
  end
end

Then when I run Chef, it will converge and notify the file resource to be configured.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

Recipe: notif::default
  * file[notified] action nothing (skipped due to action :nothing)
  * notif[doer] action write
    * log[notifer] action write

    * file[notified] action create
      - create new file notified
      - update content in file notified from none to 935e8e
      --- notified       2015-01-18 05:47:49.186399317 +0000
      +++ ./.notified20150118-15795-om5fiw       2015-01-18 05:47:49.186399317 +0000
      @@ -1 +1,2 @@
      +doer
    * file[notified] action create (up to date)

Running handlers:
Running handlers complete
Chef Client finished, 3/4 resources updated in 1.298990565 seconds

Why does this happen?

The reason for this is because use_inline_resources tells Chef that in this provider, we’re using inline resources that will be added to their own run context, with their own resource collection. We don’t have access to the resource collection from the recipe. Even though the file[notified] resource exists from the recipe, it doesn’t actually get inherited in the provider’s run context, raising the error we saw before.

We can turn off use_inline_resources by removing it, and the custom resource will be configured:

1
2
3
4
5

action :write do
  log 'notifer' do
    notifies :create, 'file[notified]'
  end
end

Then run Chef:

1
2
3
4
5
6
7
8
9
10
11

Recipe: notif::default
  * file[notified] action nothing (skipped due to action :nothing)
  * notif[doer] action write (up to date)
  * log[notifer] action write
  * file[notified] action create
    - update content in file notified from 935e8e to 3fc9b6
    --- notified 2015-01-18 05:47:49.186399317 +0000
    +++ ./.notified20150118-16159-r18q7z 2015-01-18 05:50:57.832140405 +0000
    @@ -1,2 +1,2 @@
    -doer
    +something

Notice that the file[notified] resource wasn’t updated at the start of the run, when it was encountered in the recipe, but it was when notified by the log resource in the provider action, changing the content.

Use inline compile mode!

The use_inline_resources method in the lightweight provider DSL is strongly recommended. It makes it easier to send notifications from the custom resource itself to other resources in the recipe’s resource collection. Read more about the inline compile mode in the Chef docs.

Also, define the resources that you need to notify when you’re doing this in your provider’s actions. A common example is within a provider that writes configuration for a service, and needs to tell that service to restart.