[chef] Re: on the general usage of chef

[Dylan Northrup < >]

On Tue, Jan 14, 2014 at 7:02 AM, Sam Darwin < " target="_blank"> > wrote:

it seems like the ideal case of using chef, would be to run it as a cronjob
every day on the servers, to keep the servers in their proper and perfect
configuration.

In practice, we are currently not doing this.   there is more than just one
simple reason for it.

- the recipes would need to always be in a perfect and functional state.
let's say that you were experimenting with something.   and then every single
server in your environment grabs that recipe and runs it.    this could affect
the entire system, every single server.

If you pin recipes (directly in run_lists or indirectly in environments or roles), you can upload new versions of recipes without fear of having them propagate to unintended servers.  An operational pattern I've used in the past is to have dev servers with versions that aren't pinned (so they get the latest version of whatever cookbooks are out there) and have every other type of server (integration, reference, production, etc) have explicitly pinned versions.  This allows "testing" and "development" of cookbooks in the same chef organization/server (insuring you're don't have any "Dev is different than ref is different than int is different than prod" issues) without concern that new cookbook versions will affect production nodes (or any other ones, for that matter).

Two things you'll want (or need) to do if you put this into practice:

1. Verify your pinning all the way down for non-dynamic cookbooks.  We distribute user information (logins, ssh keys, sudoers, etc) via a generated cookbook.  The version of this cookbook will change every time it's newly generated.  Since we want to use the most recent version of this cookbook every time, we do not do any pinning for this cookbook.  But for cookbooks that are not dynamic, everything gets pinned explicitly or via dependent pinning.  So, the platform cookbook explicitly pins apt, bash, system ruby, etc; nginx cookbook explicitly pins the lua cookbook and any module cookbooks it depends on; tomcat cookbook explicitly pins the java cookbook version; and so on.  Make sure it's pinned turtles all the way down.  It's a pain if you have to retrofit, but worth it IMO.

2. Every time you upload a cookbook that will be pinned, it is frozen.  Every time, no exceptions.  What you don't want, after going through all the trouble of making sure you know exactly what version is going to go out on a specific type of node, only to have that version change.

Still doesn't mean you can't propagate an error to many machines, but with additional gates in place, it makes it more difficult for this to happen unintentionally.