Thanks for explaining your way of thinking.
As a user I would love to use Policyfiles to independently manage/version/release "subsets" of nodes where a subset is more of a logical group of nodes than an actual identical run list. For example my company would ship a new version for the Hadoop policy without touching others. But Hadoop has many different kind of nodes.
The Chef provision system would definitely match this in this specific case. I would have a separate git repo for each policy with a specific provision cookbook for my Hadoop policy. What I was looking for is a way to describe my policy (like Hadoop) in a provisioning recipe and decline this across various environments (aka policy groups) like CI where a dedicated vagrant cluster is built/destroyed, staging on AWS and prod on bare metal. All leveraging the same provisioning recipe. I would need to add some logic to manage the customizations across environments like instance types or number of nodes, but all this is easily doable with ruby and it is why I think this tool could be powerful.
Overall the duo of provisioning + Policyfile can be very interesting and looks promising to me.
On Wednesday, July 22, 2015 at 5:57 PM, Christine Draper wrote:
> I had the same reaction re the chef provision command... it seems targeted at letting me one or more 'identicalish' nodes, whereas my use of provisioning is typically to setup a variety of nodes that form a working system or solution.
I did think about clustering scenarios when I was writing `chef provision`, but it turns out that it can be complicated depending on what the exact use case is. Do you want a “throwaway” cluster to integration-test your cookbooks as a whole? How do you keep different developers’ throwaway clusters from conflicting with each other? How do you know what machines (or other resources) to destroy when the user requests to destroy stuff? How much responsibility should be placed on the user if they update their provisioning code and then run the destroy operation (which could leave a stray EBS volume, for example)? None of these problems are insurmountable, but it will take a lot of thinking to do this in a way that provides a great experience.
So, for the policyfile focused stuff, I opted to make it work more like `knife bootstrap` and `knife cloud create`, except using using Chef Provisioning so hopefully you don’t have the CLI options issues (i.e., you need 15 options you can never remember) that those commands have.
That said, I also felt that, regardless of whether you’re using Policyfiles or not, it wasn’t easy to pass “argument” type information into Chef Provisioning with any of the existing methods (environment variables being the easiest, but they have some unhelpful failure modes when you typo things). So I added the `chef provision --no-policy` mode of operation which loads up Chef and Chef Provisioning and then just runs your recipes. If you’re already happily creating clusters with Chef Provisioning, this might be the best way for you to use `chef provision` right now.
>
> Is there a way within a chef provisioning recipe to say 'set this machine up using this policyfile'? I couldn't see one. Maybe we need a policyfile resource (to load policies) and a policy attribute on the machine resource.
What `chef provision` does right now is, it adds `policy_name` and `policy_group` settings to the client.rb file via Chef Provisioning’s built-in option for this. You can do this manually as well.
At the moment this is the only way to tell chef-client which policyfile you want, because the node object doesn’t yet have these fields. In a future release of Chef Client and Chef Server, we will add these fields to the node object. After that’s done, it will be possible to add these to Chef Provisioning.
Policyfiles will be a bit trickier, since the process for generating the JSON and uploading everything to the server involves a lot of things (dependency resolution, local caching of cookbooks from supermarket, multiple cookbook uploads and then finally uploading the policy JSON) and there are some imperative operations involved (i.e., the user has to decide if they want to update dependencies or just take the lockfile as-is). I’m not ruling it out, but right now my thinking is there’s a bit of an impedance mismatch between the Chef Provisioning way of thinking and they way Policyfiles work otherwise. Maybe some more insight into user expectations here would be helpful
>
> Regards,
> Christine
>
> On Wed, Jul 22, 2015 at 1:16 PM, Maxime Brugidou < "> (mailto: "> )> wrote:
> > Hey,
> > Sorry might be wrong but from what I understand a Policyfile based workflow is pretty much like versioning a role with an associated Berksfile.lock.
> > I am not sure exactly how things are intended to be used but the associated provision cookbook should probably provision nodes from the given Policy name and nothing else. Each Policyfile would have their own provision cookbook (and maybe even git repo since they are versionned separately). This seem a bit extreme to me but could be greatly improved if we leverage named run lists in the Policyfile: then we can have multiple run_lists under the same Policyfile. I haven't tested that yet.
> > Maxime
> > On Jul 22, 2015 7:30 PM, "Chris Sibbitt" < "> (mailto: "> )> wrote:
> > > I've been experimenting with policyfile support lately, and I'm hoping someone can clarify some thinking around policyfile support in the "chef provision" command.
> > >
> > > `chef provision POLICY_GROUP --policy-name POLICY_NAME` lets me specify ONE policyfile and run a provisioning recipe. Policyfiles define a run_list, but one of my typical chef-provisioning recipes contains multiple machines with different run_lists.
> > >
> > > I'm not sure whether to take this as a suggestion that provisioning recipes should only do one machine each, or whether the tooling is just not quite meshing yet (I'm aware it's all very new and beta), or whether there is something conceptual missing from my thinking.
> > >
> > > Anyone else experimenting with this combination yet?
I addressed this above, you can use the —no-policy option to skip the policyfile part, which is probably the best way to do clusters. You’d have to manage uploading the policies and such yourself via the command line.
> > >
> >
>
>
>
> --
> ThirdWave Insights, LLC I (512) 971-8727 (tel:%28512%29%20656-7724) I www.ThirdWaveInsights.com (http://www.ThirdWaveInsights.com) I P.O. Box 500134 I Austin, TX 78750
--
Daniel DeLeo
Archive powered by MHonArc 2.6.16.