[chef] Re: RE: Re: RE: Re: berkshelf gem listed as a requirement in cookbook gemfile

[Ranjib Dey < >]

you are just recreating a chef server by all this. you can certain use git or any other SCM server itself, but this only mean you are using source code management server as an artifact server, which is fine, but then you are bound to face with other limitations (some of which you have already encountered). I am failing to understand the access control bits, they are same, its just now you have coupled it with source control (unless your git repos are public).

Regarding your versioning concerns, git sha != cookbook version. in general source code revision version !=  artifact version. Source code versions solves different problems than artifact versions (which can be compared, hence build tools can use them).

the approach you are taking is way more insecure and dubious, by your own arguments.

if you do take this route, you should just check in all dependency cookbooks inside your own repo (because by all means you are treating it as a artifact server), may be do some tooling with git sub-modules etc. This will solve your current problems, you wont need a build/assembly tool. In fact this is what most of us did with puppet or early days of chef. 

cheers

ranjib

On Mon, Jan 5, 2015 at 7:57 AM, Nico Kadel-Garcia < " target="_blank"> > wrote:

> -----Original Message-----
> From: Tensibai Zhaoying [mailto: "> ]
> Sent: Friday, January 02, 2015 5:21 PM
> Cc: ">
> Subject: [chef] Re: RE: Re: berkshelf gem listed as a requirement in cookbook
> gemfile
>
> Why the hell would you do a bundle install on a node ?
>
> Berkshelf is of use on your workstation to create a pack of cookbooks with
> correct versions to distribute as tarball or to upload to chef-server.

Or using it under chef-solo to pull and apply on the local host.

       cd /var/chef-solo
       git pull [ designated git repo, tag, branch, etc., etc. ]
           [ Modify locally in testable environments ]
           [ Or checkout out git tag for production ]
       berks install
       berks vendor
           [ Consistent Berksfile.lock allows use of hard-coded cookbook versions, including rollback to previous version without touching upstream ]
       chef-solo [ with configuration that looks in /var/chef-solo/berks-cookbooks ]
          [ test ]
          [ Record git changes ]
          [ Push git changes back upstream ]

'Deploy from tarball' involves maintaining tarballs and an available server to deploy them from, and potentially pushing tarballs from a development server to the chef server, which raises serious management and security concerns. This also gets really hairy when older cookbooks are incompatible with newer cookbooks or different chef releases, and some servers need one or the other. Recent architectural changes in the yum and mysql cookbooks are good examples where the chef server, itself, might need to have multiple cookbooks for multiple categories of servers. And Berksfile and a chef server do not trivially support deploying multiple cookbook versions, especially legacy ones that have not historically been deployed on that chef server. And they break *horribly* when different versions of the same numbered cookbook are in use on two development  hosts with the same chef server or the same "tarball building setup" as you mentioned, which happens when two different people are working on different parts of the same cookbook at the same time in the same chef server environment.

> If your day to day workstation is a lightweight Vm you recreate often, include
> chef-dk or do bundle install in the source template box...

"Recreating vm's" is a surprisingly resource expensive operation in many environments. It requires privileges to create VM's, and the sys-admin time and resources to manage those privileges to prevent mistakes which may corrupt critical resources. Being able to spin up  a VM or test environment with only local git privileges and *no* chef server privileges is much safer in many environments.

> I don't see any use case where you would do a bundle install from a cookbook
> dir within a chef run, you should install needed gems for a recipe with chef-gem
> resource if you need it inside your recipes.

Roles and node configurations and environments benefit from having consistent, locked cookbooks especially between QA, staging, and production environments. A "template box" is not good enough for production if you've not recorded your chef changes, especially including cookbook versions, and the 'Berksfile.lock' for chef-solo on individual test environments provides that.

There are trade-offs: you can't use "search" functions as you might on a chef server. But for many test environments, testing on a single chef-solo instance and then propagating it to the relevant staging or production environment is quite useful.

> Please enlight me if I missed your point...

I'm happy to do so: see above.


Nico Kadel-Garcia
Lead DevOps Engineer
">