[chef-dev] Re: Re: CHEF-2988 allowed_recipes, restricted_recipes, and override_recipes

[Bryan McLellan < >]

I'm allowing myself to think out loud, and I'm mixing design concerns
with implementation concerns. Sorry.

On Mon, Mar 26, 2012 at 1:29 PM, Jay Feldblum 
<
 >
 wrote:
> I think the key issue is that:
>
>     application deployment =/= node convergence

Configuring a system and deploying an application are different
things. Why? Experience from a world where different people managed
systems and applications? Because one often requires more
orchestration than the other, such as when updating a database scheme
with an application deployment?

We used to make a point about Chef being a systems integration
framework rather than a configuration management language because we
wanted to bring all the pieces of building infrastructure together. I
still think this is possible and we're getting there.

On Thu, Mar 22, 2012 at 6:28 PM, Peter Donald 
<
 >
 wrote:
> It seems like this is an attempt to jury-rig something onto the chef
> model rather than changing the model. It seems like the use case is
> actually adhoc execution of commands/resource installation using the
> chef syntax? If so why not add the ability to distribute a one-shot
> command to chef clients when needed.

Peter and CB are right that this needs to work ad-hoc runs in fully,
not just a one-off run on the client. While this could be a step in
that direction, we need to be sure we're not creating a hurdle for
that fuller implementation.

I don't like that in this patch we deal with allowed recipes and
restricted recipes in different parts of the code. That's not
necessarily the patches fault, but I think requires that if we went
down this road we did some refactoring.

A more easily correctable issue is that the logs regarding allowed and
restricted recipes both call them restricted recipes, making the
interaction between these features unclear.

On Mon, Mar 26, 2012 at 5:18 PM, AJ Christensen 
<
 >
 wrote:
> I want to reopen the original discussion with a response to your
> queries to try and get this moving some more:
>
> What the design gives us:
>
> - The ability to implement via local policy a non-ideal pragmatic
> approach to restricting run list components during various convergence
> phases

How would this functionality work with a larger policy? Would this
extend well into setting allowed and restricted cookbooks in a role?

> - A "straw man" fix to non-compatible recipe inclusion in the case of
> multi platform cookbook metadata dependency, e.g. apt & yum

It seems that the fix would be not including those cookbooks, and
rather including them based on node['platform'].

> - Easily deprecated or refactored later on for feature more accurately
> satisfying ad-hoc, orthogonal component failure zone separation

This gives the user more control over their run, which is good. It
might encourage them into bad habits, which is bad, but we must be
cautious not to judge their badness prematurely.

> Locks us into:
>
> - Clear test coverage of affected features. Code is clear and I have
> personally reviewed it and tested many times.

I think this belongs in the other column. :)

> - Potential problems for new users who do not anticipate the correct
> behaviour of the run list modifiers or use them by error, e.g. typo,
> or cargo-culted configuration template

We've got some sanity checking coming in from CHEF-1398 [1] for
RunListItems. Since this implementation of override run list sets
node.run_list, this should jump through that hoop properly.

This patch removes a run list item from the nodes run list, but what
if we want to restrict a dependent recipe that isn't in the run list?
Given a run list of "recipe[check_sl]" where the check_sl cookbook
depends on the install_sl cookbook, if we run "sudo chef-client -r
recipe[install_sl]" we still run the install_sl::default recipe
because it was not in the run list but rather was in the expanded run
list. This is pretty confusing because it doesn't do what you just
asked it to do, which was prevent that cookbook from running.
Shouldn't this fail from the start?

The dependency selector has already run on the server and given us a
set of cookbook versions that match the provided dependencies, but
what are the consequences of restricting a cookbook now? I think we're
likely to have recipes fail and it is going to be unclear why, a
situation that is prevented by the chef-server preventing a set of
cookbook versions that don't resolve the dependencies. It seems that
we would want to pass the run list by the server for a thumbs up
before going too far down a road that could lead to peril.

(Thinking about and testing how this affects node attributes reminds
me that attributes are still non-deterministically loaded. Ugh.)

This does lock us into the allowed/restricted features. While I think
there is general consensus that ad-hoc runs are a good thing, I
haven't heard anyone propose these before. My gut feeling is that the
unnecessarily complicate the creation of the run_list. If we decide we
want these, we need to ensure we've designed them correctly. Allowing
these features to be used via a role would likely change the final
composition of the run_list as the decisions could be made on the
server. This would change the result of a particular run between Chef
versions which is always of consequence to our design decisions.

I think that since this implementation came with an assumed design,
merging this would require getting some time in a engineering sprint
at Opscode where we could design the requirements and then see if this
patch would fit them.

Bryan

[1] http://tickets.opscode.com/browse/CHEF-1398