[chef] chef-provisioning best practices for supporting multiple configurations

[Stefán Freyr Stefánsson < >]

Hello.

We're starting to look into using chef-provisioning and things are looking promising except maybe for lack of documentation on best practices. The documentation at https://docs.chef.io/provisioning.html is good but it doesn't really explain the best ways to structure your chef-provisioning code.

We have a system that can be set up on a cluster but it can also be set up on a single machine. We would like to use chef-provisioning to be able to set up multiple types of setups (single node, multiple nodes, using vagrant, using AWS, using our own EC2 compatable private cloud etc.). We are going to use these scripts to perform automatic tests, so when a new release of our software is issued, we want an automatic process to start and set everything up for us.

What I currently have are basically just some ruby files and an environment directory with json "chef environment" files.

The ruby files I have are:

• aws-minimal.rb
  
• vagrant-minimal.rb
  
• myapp-1node.rb
  

The myapp-single-node.rb is very simple, it sets up a single "machine" and applies the necessary recipes to it. The idea later on is to have myapp-4node.rb etc to set our system up on more nodes as needed.

The aws-minimal.rb file looks like this:

require 'chef/provisioning/fog_driver/driver'

with_machine_options({

  :bootstrap_options => {

    :flavor_id => 't2.medium',

   },

  :ssh_username => 'centos',

  :image_id => 'ami-96a818fe'

})

Basically this sets everything up for AWS as you can see.

The way I run this is:

chef-client -z provision/aws-minimal.rb provision/myapp-1node.rb

I also have a vagrant-minimal.rb file that looks like this:

require 'chef/provisioning/vagrant_driver'

vagrant_box 'centos7' do

  url 'https://cdn.nextcode.com/contrib/nextcode_centos-7-x86_64_virtualbox.box'

end

with_machine_options :vagrant_options => {

  'vm.box' => 'centos7'

}

And if I want to run my myapp-1node.rb and use vagrant instead of AWS I run this:

​chef-client -z provision/vagrant-minimal.rb provision/myapp-1node.rb

So as you can see, I'm keeping the driver configuration separate from my provisioning scripts, hoping that they can stay agnostic to what kind of machine is being set up and in what environment (vagrant or AWS).

So far so good, everything above is working for me.

But here's my question (finally!):

If I want to make a multiple node setup using chef-provisioning and some nodes need to be more powerful than others, how do I do that properly?

I assume I would be able to run something like the following:

​​chef-client -z aws-minimal.rb myapp-4node-db_web.rb aws-medium.rb myapp-4node-backend_worker.rb

The idea being that aws-minimal.rb sets up the machine type just as before, then the first two of the 4 machines would be set up (db and web) which only need a "minimal type" of server, then the AWS configuration is changed to use a beefier type of AWS server with aws-medium.rb and then finally the other two machines (backend and worker) would be set up using myapp-4node-backend_worker.rb which would then be set up on the more powerful AWS servers.

This seems a bit convoluted to me and I'm not really all that happy with it (and actually I haven't tested if this in fact works). For example, I am splitting my machine definitions into files depending on which machines should have the same specs and assuming that something came in between and set these specs correctly.

Does anybody have a better way to do this?

-Kind regards,

Stefan Freyr.​