[chef-dev] CHEF-2988 allowed_recipes, restricted_recipes, and override_recipes

[Jay Feldblum < >]

I think the key issue is that:

    application deployment =/= node convergence

Trying to treat application deployment as though it were just another part of node convergence possibly introduces friction into the configuration management, depending on the nature of the application deployment and the nature of the rest of the convergence.

For many people and for many applications, possibly most people and most aplications, this discrepancy is not a problem, and for them, application deploys during chef runs (node convergences) are perfectly fine.

Sometimes the discrepancy is a problem, and I think the best solution sometimes is to have a separate application-deployment mechanism. The application-deployment mechanism is delivered during the node convergence, but application-deployment actions are triggered out-of-band (i.e., not during convergence). A simple application-deployment mechanism is: building & packaging the application on a build server and uploading the built package (slug, deb, lxc-rootfs, etc.) to storage, a script dropped onto each application server during the node convergence that pulls the latest package and installs it, and knife-ssh to invoke the script. As Lusis mentions, Jenkins or similar build systems may also work.

I don't think trying to treat Chef as a glorified script runner is as useful as treating Chef as a declarative final-state convergence engine. Application-deployment may sometimes not fit within the declarative final-state convergence model, and when that happens, it may be wiser not to try to shoehorn it in.

On Mon, Mar 26, 2012 at 12:43 PM, Joe Williams < " target="_blank"> > wrote:

Hey gang,

Finally getting around to chiming in on this, as the client of Heavywater that commissioned them to build this feature (to some of you this will be of no surprise :) here are my thoughts and experiences.

Here's the main idea, we want regular, periodic converges to deal with all aspects of the system except for specific applications. Our application deployments are surprisingly similar to the idea of a Heroku slug. Everything the application needs bundled up in a tidy package. Any change to the that package is a new version of the application, this includes configuration changes. Not to get metaphysical but a configuration change is just the next iteration in a long line of changes in the life of an application. A configuration change changes what the application *is* just like changing code but we rarely version them that way. As a result we only want changes to configurations to happen on deploy, we never want our configs to change to happen due to a search index update when a new node comes online. Search is awesome and I want to make good use of it but we deploy often enough that it can wait until we ship more code and tell chef to update the config. So the end result is a steady state run_list that everyone can expect to keep the system in check and updated and then when developers do their deploys a single application recipe chef-client run with only that recipe in the run_list. Additionally, as someone else mentioned, this has the added benefit of avoiding any issues, bugs, whatever that my occur in recipes before/after in the run_list. I want my developers to deploy as often and as quickly as possible, they won't do that if they run into bugs I wrote in recipes causing their deploys to fail.

In the end this is an attempt at isolation of recipe converges and consolidation of any application changes to only deploy time. If there is a better way lets do that but this seemed like the least invasive and lowest risk to get the functionality I wanted.

-Joe

-- 

Name: Joseph A. Williams

Email: " target="_blank">

On Thursday, March 22, 2012 at 3:50 PM, Christopher Brown wrote:

Peter,

Good point, and I'd echo the same. We are working internally on design for just that feature (ad-hoc push execution of partial / alternative run lists) and want to share our ideas with the community once we have a better handle on it.

-Chris

On Mar 22, 2012, at 3:28 PM, Peter Donald wrote:

Hi,

It seems like this is an attempt to jury-rig something onto the chef

model rather than changing the model. It seems like the use case is

actually adhoc execution of commands/resource installation using the

chef syntax? If so why not add the ability to distribute a one-shot

command to chef clients when needed. Then these application deploys

and/or apt-get upgrades or whatever could be run through this system

and regular infrastructure maintenance can continue to use the

existing approach. I think trying to merge the two approaches may lead

to a bit of complexity when adopting chef.

The one place where I can see a useful to have an include/exclude is

in the resolution of cookbook dependencies. I can quiet easily see

that it is unnecessary to have a apt cookbook in a chef server if you

are only deploying to redhat infrastructure. It would be nice to add

an exclude and be done with it.

--

Cheers,

Peter Donald