Chef is a gem which aims to help sys admins in the process of managing servers. Chef allows you to write recipes using ruby code rather than running commands on the servers to install and configure software, users and so on.
This way you can easily manage your servers and ensure — for example — that your staging server is a perfect clone of your production server. Once your recipes are ready, you are able to install or restore a server in a breeze.
I won’t pretend to be a Chef master but we’ll dive deep into the basics of the features it offers.
Chef runs a bunch of recipes to set up your servers. If you run it several times, Chef can guess what has already been done or what changed so you don’t have to care about accidental overwriting and things like that.
So if Nginx and gems you need are already installed, commands for installation will not be run again unless you change something like the version you want to be installed.
Chef has a standard directory structure to organize your recipes and provides many useful tools to automate tasks such as creating a backup strategy or monitoring alerts. There’s no limitation to what you can do on the server, you can really do whatever you want as if you were working interactively on the server via SSH.
You can create your own recipes matching your needs or re-use existing recipes. You have to be careful with existing recipes you borrowed from somewhere ‘cause you leave the door open to your servers when you execute Chef recipes. Be sure to check that the recipes don’t do anything nasty.
As Ezra Zygmuntowicz said «Chef is like unit tests for your servers», that is to say that Chef’s job is to ensure your servers respond to your needs.
Sure, you could write your own shell/ruby/whatever scripts to automate some sequences of commands on your server and install software but why bother? Chef will bring you readable code shipped with all the tools you need to efficently write your installation / configuration process even it’s an advanced one.
Once your recipes are ready and work on a given server, you can be sure that you will be able to repeat this process on any other server even if the unix distribution is not the same. If you’re not a sys-admin this is a huge advantage which will make you comfortable with deploying a new server.
Having Chef recipes also allows developers to install the same OS, software and services as on production server using a virtual machine with only one command. That’s a great way to ensure that the app you are working on will act the same way in both environments.
Chef uses a given directory organisation and specific terms to define things. Let’s see what the major concepts of Chef are:
The most simple usage you can do of Chef is to use chef-solo which doesn’t rely on Chef server so it is easier to start with it.
Let’s see what we can do with chef-solo.
Each server needs minimal requirements to be able to use Chef. We’re going to install Ruby and Chef. In my case, I’m using Debian based servers but the process is easy on other distributions too.
First you have to ssh into server(s) you want set up like so:
$ ssh root@my_server
Then you need to install an inital ruby version to get Chef running:
# apt-get -y update
# apt-get -y install ruby1.9.1-full
# gem install chef ruby-shadow --no-ri --no-rdoc
Note you can still install rbenv afterward using Chef then play with ruby version as usual.
Ensure all is ok:
# ruby -v
# chef-solo -v
The last thing to do on the server is to prepare the directory required by Chef to puts its cookbooks in:
mkdir /var/chef
Now you can go back to your local machine where you’ll write the cookbook.
You can, locally, create your main cookbook directory skeleton:
mkdir -p cookbooks/main/recipes
Note that I created only one main cookbook where we will do all our testing but Chef users like to have one cookbook per software to install or task to be done.
Open your editor of choice and create the chef-solo configuration file which specifies where to find the cookbooks and variables file:
cookbooks/solo.rb
cookbook_path File.expand("../cookbooks", __FILE__)
json_attribs File.expand("../node.json", __FILE__)
Next create the default recipe:
cookbooks/main/recipes/default.rb
package "git-core"
package "zsh"
These two lines tell our recipe to install git and zsh. This will use platform dependent installation that is to say apt-get for Debian.
Then you need to create a run list to define which recipes you want to run:
cookbooks/node.json
{
"run_list": ["recipe[main]"]
}
We have our minimalistic version of our cookbook, we’re now able to sync it on the target server(s) and run it. I like to copy my SSH key on the server so I don’t have to type in my password for every single command sent.
To copy this key, I use ssh-copy-id. I installed it using homebrew:
brew install ssh-copy-id
then
ssh-copy-id root@my_server
rsync -r . root@my_server:/var/chef
ssh root@my_server "chef-solo -c /var/chef/solo.rb"
You should see that git-core and zsh packages were installed. if you run chef-solo command again it first check current state and doesn’t try to install anything.
That’s it for the very basic stuff.
Chef offers some useful default cookbooks we’re going to use next to enhance our deployment strategy.
If you update anything in the recipes and run the chef-solo command again, all differences with current server state will be taken into account. Now if your Chef cookbooks are tracked using a SCM you can easily go back and forth in your server configurations.
First thing you definitively want to do on your server is to create a normal user to use for common tasks.
Chef comes with User resource to ease user creation and modification on the server.
So our user will need a password shadow hash we can create this way:
openssl passwd -1 "my_user_awesome_password"
Then you can edit your cookbooks/main/recipes/default.rb file to add:
user "Synbioz" do
comment "Synbioz User"
gid "users"
home "/home/synbioz"
shell "/bin/zsh"
password "$1$JJsvHslV$szsCjVEroftprNn4JHtDi."
supports manage_home: true
end
As you can see, you have fine-grained tuning for user handling and all the other resources available are as tunable. That’s a lot of comfort compared to a shell-script you write by yourself and need to maintain over the time.
Now we need to sync the cookbook on the server and run chef-solo again:
rsync -r . root@my_server:/var/chef
ssh root@my_server "chef-solo -c /var/chef/solo.rb"
We have a very simple cookbook for now but if you want something more complex I really encourage you to keep all the hardcoded values such as usernames, passwords and so on in a dedicated file.
You can use the existing cookbooks/node.json file to store all your variables:
cookbooks/node.json
{
"user": {
"name": "synbioz",
"password": "$1$JJsvHslV$szsCjVEroftprNn4JHtDi."
},
"run_list": ["recipe[main]"]
}
Now you have user informations stored in the json file, you still need to use it in recipes:
cookbooks/main/recipes/default.rb
user node[:user][:name] do
comment "#{node[:user][:name]} User"
gid "users"
home "/home/#{node[:user][:name]}"
shell "/bin/zsh"
password node[:user][:password]
supports manage_home: true
end
You’ll often need to create default configuration files on your server. Here come templates that allow you to create a skeleton for files and deploy it on the server.
Create a directory called coobooks/main/templates/default and add this at the bottom of coobooks/main/recipes/default.rb:
template "/home/#{node[:user][:name]}/.zshrc" do
source "zshrc.erb"
owner node[:user][:name]
end
The goal is to create a default .zshrc file for our user. The last step is to create the template file:
coobooks/main/templates/default/zshrc.erb
alias l='ls -lA1'
Don’t forget you’re in an erb file so you can use variables defined in node.json and generate content using Ruby code.
Another recurrent need on remote servers is to create cronjob for automated tasks:
Add this at the bottom of coobooks/main/recipes/default.rb:
cron "noop" do
hour "5"
minute "0"
command "/bin/true"
end
Sync your cookbook on the server and run chef-solo and a new cronjob will be added. This cronjob will run the /bin/true command at 5 every morning. You can also add some conditions for the cronjob to execute.
I can’t go through all the features of Chef in this post but be sure to check the documentation which contains a load of info.
You can for example discover how to handle services, use git, install gems, manage files, directories, users, groups and logs in an efficient and clean way.
The community cookbooks are a nice way to discover available cookbooks for databases, webservers, monitoring and more. Download the cookbook you need, let’s say “nginx”, copy it in your Chef coobooks directory then you can use it :
Add this at the bottom of coobooks/main/recipes/default.rb:
package "nginx"
You can and should explore cookbooks you download to understand what they can do and how you can structure your own cookbooks.
With this community cookbooks site, you can have a server up and running in a couple minutes. Adding a new clone server will be as fast as light and updates are as simple as changing your recipe and executing a single command for each of your servers.
If you’re a developper who likes to spend time writing code rather than doing sysadmin, I’m pretty sure you’ll love Chef.
I hope this quick introduction to Chef will be helpful for those of you who want to put servers up & running quickly in a repeatable manner.
I’ve barely scratched the surface and there’s a lot lot more to know about Chef. It’s a real server provisionning tool that can do anything on the server for you through Ruby code. Don’t be afraid to write your own cookbooks with custom resources to fit your needs.
The Synbioz Team.