[chef] Re: Re: Re: Re: Environments vs. Metadata vs. Policyfile for locking cookbook dependencies

[Torben Knerr < >]

On Fri, Jun 20, 2014 at 6:18 PM, Daniel DeLeo < " target="_blank"> > wrote:

> >
> > In the current prototype, you’d name your policies something like $functional_role-$deployment_group where $functional_role is something like appserver/load balancer/database/etc. and $deployment group could map to your environments, groups within environments (for example, if you deploy to production on a cluster-by-cluster basis, you could have prod-cluster-a, prod-cluster-b, etc.) or whatever makes sense to you. Whether or not the deployment group concept becomes a bit more first class when we implement the APIs is something we haven’t decided yet.
>
> ​Does that mean that when the policy is named "appserver-prod" it would automatically apply the "prod" environment to the node?
>
> Or alternatively: will it be possible to bootstrap a node with a Policyfile AND a Chef environment (e.g. `knife bootstrap --policy "appservers" --environment "prod" ...`)?​

In the current “compatibility mode” implementation, you cannot use environments and Policyfiles at the same time. You must identify your policies with a single string, so you’d name them as $functional_role-$deployment_stage. This decision was forced by the requirement to use existing data bags as the storage mechanism and won’t necessarily be what we do in the final implementation.

​Guess that means ​that you are using environments via `berks apply` for locking a Policy's dependency graph under the hood?

 

>
> I would still vote for keeping environments though, because they allow you to set common attributes across a set of arbitrary nodes which might have totally different policies. ​
I understand the use case. Again, we haven’t made a decision here, but the things we’re thinking about are:

* Environments are global, so any change to them immediately affects all nodes in an environment. Is this a good thing?

​IMO yes. If something in the environment changes (e.g. ntp server) it affects all nodes in that environment. That should also be the criteria when deciding whether to put something into an environments vs. into a wrapper cookbook (which requires re-releasing the cookbook for changing attributes). ​

 

* How exactly do policies get promoted from stage to stage? Are they completely static or can they be customized as they’re promoted?

​From my understanding the Policy should is the static part which describes the functional role as a coherent whole, but not any environmental stuff surrounding it. They should be self-contained and independent of the environmentals IMO.

​

* Reducing the number of ways you can set attributes would make Chef easier to understand and debug.
* Can we do something other than environments that provides flexibility for use cases that require it? For example, some users need to customize data or behavior by both deployment stage and data center. If you make environments that are the conjunction of both (e.g., production-us-east), that causes a lot of the same problems that you have with “micro environments” (duplicated data, etc.).

​For me environments (for environmental stuff affecting all nodes in the env) and policies (functional roles with a locked dep graph) are two completely separate an orthogonal concepts. We should keep them selective and not mix them up, then​ they will be composable.

​One way to prevent "micro environments" would be to allow for a single node to be in multiple environments (e.g. `--environments prod,us-east,foo`), i.e. extending the current 1:1 relation to 1:N.

That would btw also let you use environments as the underlying mechanism for locking the dependency graph (think `berks apply`) while still providing the ability to express "deployment groups" like "prod" and "us-east" via the environment mechanism.

Making nodes:environments a 1:N relation and deprecating `cookbook_versions` and `cookbook` from environments (combined with Policyfile for sure) would probably be the most minimal change that would solve all the problems and use cases we discussed here and on github.

What do you think?

Cheers,

Torben