[chef] Re: Re: Re: Environments vs. Metadata vs. Policyfile for locking cookbook dependencies

[Daniel DeLeo < >]

On Thursday, June 19, 2014 at 5:54 AM, Torben Knerr wrote:

> Hi Daniel,
>  
> thanks for all the explanation, that makes my picture about Policyfile much 
> clearer.  
>  
> Few comments inline...
>  
>  
> On Mon, Jun 16, 2014 at 5:20 PM, Daniel DeLeo 
> <
 
>  
> (mailto:
 )>
>  wrote:
> >  
> > >  
> > > 3. How are the Policyfiles versioned, published and being referenced?
> > Policies won’t have versions (aside from what you do on your own in your 
> > version control system). The exact mechanism by which a node is assigned 
> > to a policy hasn’t been decided yet. We have some prototype code that 
> > uses data bags to store the policies, see:
> >  
> > https://github.com/opscode/chef/blob/master/lib/chef/policy_builder/policyfile.rb
> > https://github.com/opscode/chef/blob/master/spec/unit/policy_builder/policyfile_spec.rb
> > https://github.com/opscode/chef/blob/master/lib/chef/config.rb#L343-351
> >  
> > In the current prototype, you’d name your policies something like 
> > $functional_role-$deployment_group where $functional_role is something 
> > like appserver/load balancer/database/etc. and $deployment group could 
> > map to your environments, groups within environments (for example, if you 
> > deploy to production on a cluster-by-cluster basis, you could have 
> > prod-cluster-a, prod-cluster-b, etc.) or whatever makes sense to you. 
> > Whether or not the deployment group concept becomes a bit more first 
> > class when we implement the APIs is something we haven’t decided yet.
>  
> ​Does that mean that when the policy is named "appserver-prod" it would 
> automatically apply the "prod" environment to the node?  
>  
> Or alternatively: will it be possible to bootstrap a node with a Policyfile 
> AND a Chef environment (e.g. `knife bootstrap --policy "appservers" 
> --environment "prod" ...`)?​
In the current “compatibility mode” implementation, you cannot use 
environments and Policyfiles at the same time. You must identify your 
policies with a single string, so you’d name them as 
$functional_role-$deployment_stage. This decision was forced by the 
requirement to use existing data bags as the storage mechanism and won’t 
necessarily be what we do in the final implementation.
  
>  
>  
> >  
> > >  
> > > 4. Can you reference a Policyfile from within an environment?
> > With policies, all the environment version specification stuff goes away. 
> > We may keep them around as a place to store attributes, and it’s possible 
> > that nodes will associate to policies by policyname plus environment. 
> > Contrarily, we might use a different name for policy containers, or 
> > design the system in such a way that the “containers” are implicit.
>  
>  
> ​So basically that means deprecating the `cookbook` and `cookbook_versions` 
> from environments, right?
At the minimum, yes. But the deprecation process is going to be a long one. 
This means there’s a long period where it could be confusing if you set 
version constraints in an environment and they’re completely ignored.
  
>  
>  
> I would still vote for keeping environments though, because they allow you 
> to set common attributes across a set of arbitrary nodes which might have 
> totally different policies. ​
I understand the use case. Again, we haven’t made a decision here, but the 
things we’re thinking about are:

* Environments are global, so any change to them immediately affects all 
nodes in an environment. Is this a good thing?
* How exactly do policies get promoted from stage to stage? Are they 
completely static or can they be customized as they’re promoted?
* Reducing the number of ways you can set attributes would make Chef easier 
to understand and debug.
* Can we do something other than environments that provides flexibility for 
use cases that require it? For example, some users need to customize data or 
behavior by both deployment stage and data center. If you make environments 
that are the conjunction of both (e.g., production-us-east), that causes a 
lot of the same problems that you have with “micro environments” (duplicated 
data, etc.).

  
>  
>  
> > >  
> > > I assume (even though not explicitly mentioned) that the new Policyfile 
> > > mechanism would work for chef-solo as well, does it?
> > Exactly how it works with chef-solo is to be determined. Since chef-solo 
> > gets cookbooks from local disk, the question of supporting multiple 
> > versions with solo is pretty awkward.
>  
>  
> ​As a long time, happy Chef solo user​ I would hope that it would work in a 
> similar way like with a Berksfile today: just as `berks install` collects 
> all cookbook versions from `Berksfile.lock` and puts them into a separate 
> directory so it can be used as the cookbook repo for Chef solo, I would 
> expect that with `Policyfile` its working in a similar way.  
>  
> My main use case is Chef solo with Vagrant plus the awesome vagrant-omnibus 
> and vagrant-berkshelf plugins.
We’re planning to integrate ChefDK with chef-metal, which will do all the 
same things and provide a tunneled connection to a local chef-zero server.
  
>  
>  
>  
> Btw: what's the role of Berkshelf with Policyfile? Will it still be used 
> for resolving the dependency graph?
We’re integrating Berkshelf’s code into ChefDK, so the command line will be 
`chef`, but much of the underlying code is the same.
  
>  
>  
> Cheers,
> Torben

--  
Daniel DeLeo