Getting started with Chef

These are my notes from the first time I set up Chef for configuration management, back when the company behind it was still called Opscode and the server was version 0.9. The commands are from then. The shape of the thing hasn't changed much, and that's the part worth understanding before you install anything, so I've written it up around that, with a note on what's different now.

The moving parts

• The repository (chef-repo) is a Git repository holding your cookbooks, roles and environments. This is the source of truth; you edit here and upload to the server.
• The server stores the uploaded cookbooks, the list of nodes and what each node should run. It doesn't push anything.
• knife is the command-line tool you run on your workstation to talk to the server: upload cookbooks, list nodes, bootstrap new machines.
• chef-client runs on each managed machine (a node). It authenticates to the server, pulls down its run list and the cookbooks it needs, and converges the machine to match.
• Keys. Every client, including your knife user, has an RSA key pair; the server keeps the public half. A shared validation key is used once, by a brand new node, to register itself and get its own key.

Start with the repository

Opscode published an empty skeleton with the directory layout Chef expects. Clone it, cut it loose from their remote and point it at your own:

apt-get install git-core
cd ~
git clone git://github.com/opscode/chef-repo.git
cd chef-repo
git remote rm origin
git remote add origin git@github.com:example/chef-repo.git

You need an SSH key on the account for the push to work:

ssh-keygen -t rsa -b 4096 -C "you@example.com"
cat ~/.ssh/id_rsa.pub

Paste the public key into your Git host's SSH keys page and test it:

ssh -T git@github.com

Hi example! You've successfully authenticated, but GitHub does not provide shell access.

Set your identity, make a change and push:

git config --global user.name "Your Name"
git config --global user.email you@example.com
nano README.md
git add README.md
git commit -m "First commit"
git push -u origin master

Installing the server

For Debian and Ubuntu, Opscode ran its own APT repository. Add it with its signing key and install the server, which pulls in the client and ohai (the tool that gathers facts about a machine: its IP addresses, memory, OS and so on):

apt-get install curl
echo "deb http://apt.opscode.com/ $(lsb_release -cs) main" > /etc/apt/sources.list.d/opscode.list
curl http://apt.opscode.com/packages@opscode.com.gpg.key | apt-key add -
apt-get update
apt-get install -y chef chef-server

The installer asks for the server URL. The API listens on port 4000 and the web UI on 4040, so I gave it the machine's full name on port 4000, e.g. http://chef.example.com:4000.

The 0.9 server was a collection of services (CouchDB for storage, RabbitMQ and Solr for search, the API and a Merb-based web UI) and each one starts separately. Watch the end of the install. On my first run everything came up except the web UI, which failed to start. The API is all knife needs, so I carried on and came back to it.

Configuring knife

knife needs a client of its own. The quickest way to make one is to run knife configure -i on the server, borrowing the web UI's admin key to create the new client:

mkdir -p ~/.chef
cp /etc/chef/validation.pem /etc/chef/webui.pem ~/.chef/
chown -R $USER ~/.chef
knife configure -i

Where should I put the config file? [~/.chef/knife.rb]
Please enter the chef server URL: [http://localhost:4000]
Please enter a clientname for the new client: [root] yourname
Please enter the existing admin clientname: [chef-webui]
Please enter the location of the existing admin client's private key: [/etc/chef/webui.pem] .chef/webui.pem
Please enter the validation clientname: [chef-validator]
Please enter the location of the validation key: [/etc/chef/validation.pem] .chef/validation.pem
Please enter the path to a chef repository (or leave blank): /root/chef-repo

It writes ~/.chef/knife.rb and a new key for your client. Check it can talk to the server:

knife client list

[
  "chef-validator",
  "chef-webui",
  "yourname"
]

knife cookbook list

An empty list is what you want at this point: the server is up, knife is authenticated, and there's nothing on it yet.

Look after the keys

validation.pem lets any machine that holds it register as a new node, and webui.pem is an admin key. Neither belongs in the Git repository. Add .chef/ to .gitignore before your first git add ., keep the copies on the server readable by root only, and once your own client exists you can delete the copy of webui.pem from your home directory.

Where next

From here the work is cookbooks: write or download one, knife cookbook upload it, add it to a node's run list and run chef-client on the node. The community cookbooks Opscode published on GitHub were the best way to learn how a good one is laid out.

Today

Most of the above has moved on. Opscode became Chef and the old APT repository is gone. The server is now Chef Infra Server, installed from a single package and set up with chef-server-ctl reconfigure; it's organised into organisations with per-user keys rather than borrowing the web UI's key. Workstation tools come as Chef Workstation, and chef generate repo replaces cloning the skeleton. GitHub dropped the unauthenticated git:// protocol, so clone over HTTPS, and generate an ed25519 SSH key instead of RSA. If you're starting fresh, the official Chef documentation is the place to begin; the concepts at the top of this page are the bit that carries over.