[chef] Re: What to pin in environments?

[Steffen Gebert < >]

Hi Torben,

> 1) is far from ideal, but the best I could get so far. It's definitely
ugly
> that a part of the metadata.rb is generated. I would rather use includes
> than protected regions, but haven't found a clever way how to do so
yet. Or
> can we just use IO.read(transform("Berksfile.lock")) in metadata.rb and it
> gets properly rendered when the metadata.json is generated?!

That should be possible. The only thing to solve would be parsing the
Berksfile and transforming it into the "depends" statements.
Do you remember the old cookbook template that contained such a line?
> long_description IO.read(File.join(File.dirname(__FILE__), 'README.rdoc'))
That gives me the impression that any ruby code can be used there.

Yours
Steffen

On 29/01/14 18:49, Torben Knerr wrote:
> Hi Steffen, Peter,
> 
> Yes, there is some kind of duplication involved, but there should be always
> a single source of truth. Here's how we handle it:
> 
> 1) in the "top-level" cookbook we define the dependencies in the Berksfile,
> and the metadata.rb contains a protected region for the depends statements.
> You can fill this protected region either manually or via a rake task or
> similar during the build / test, which would ensure it is always consistent
> with what you specified in the Berksfile. You can use 'berks list' and
> transform that output to depends statements to get the fully
> Berksfile.lock'ed dependency graph.
> 
> 2) in any non-"top-level" cookbook we are much more lax with constraining
> dependencies so they can be easily mixed together. Here we define the
> direct dependencies (not the transitive ones) in metadata.rb, and the
> Berksfile only contains the 'metadata' keyword (plus git locations etc if
> any, but they had to be repeated in the top-level cookbook's Berksfile
> anyway...)
> 
> 2) is fine in terms of duplication, because there is essentially none :-)
> 
> 1) is far from ideal, but the best I could get so far. It's definitely ugly
> that a part of the metadata.rb is generated. I would rather use includes
> than protected regions, but haven't found a clever way how to do so yet. Or
> can we just use IO.read(transform("Berksfile.lock")) in metadata.rb and it
> gets properly rendered when the metadata.json is generated?!
> 
> Ideally I'd like some first-class Berkshelf integration for "top-level"
> cookbooks (e.g. 'berks apply_metadata' rather than 'berks apply
> <environment>'), but that's not a feature they'd like to support in
> Berkshelf. It probably can be done by extending / hooking in at the right
> place with a side-by-side installed gem, but my Ruby-Fu is too weak yet to
> do hack that up in a reasonable amount of time ;-)
> 
> P.S.: didn't hit CHEF-3921 yet because I'm mostly using chef-solo, but good
> to know...
> 
> Cheers,
> Torben
>  On Jan 29, 2014 8:30 AM, "Steffen Gebert" 
> <
 >
>  wrote:
> 
>> Hi Torben, Hi Peter,
>>
>> thanks for your replies!
>>
>> This knife-audit only returns a list of cookbooks in use, or also the
>> used versions? (couldn't find any information about the latter one in
>> the README).
>>
>> What you pointed out is what I had in mind and I guess it's atm the
>> easiest way to to. Having knife-audit to see, which nodes still use old
>> versions would be really helpful to e.g. make sure that some "bad"
>> configuration for one software is not isn't in use anywhere anymore.
>>
>> What still bothers me is the duplicate definition of dependencies in
>> metadata plus Berksfile. We haven't adopted Berkshelf completely, yet,
>> but to specify all deps twice smells a bit to me (but there's no other
>> way, not even with Berks 3, isn't it?).
>>
>> Thanks for your input!
>>
>> Yours
>> Steffen
>>
>> On 29/01/14 06:02, Peter Donald wrote:
>>>> And no, it's not that easy anymore to see which versions of the apache
>>>> cookbook are in use anymore (well it might be searchable via the chef
>> server
>>>> API?)
>>>
>>> We use a variant of knife-audit [1] to do this for us and it works well.
>>>
>>>> To me it seems that many people take the route of pinning everything in
>> the
>>>> environment (e.g. via `berks apply`). To me it seems wrong (or at least
>> as a
>>>> smell) to use environments for that, but that's just my view and there
>> might
>>>> be some good reasons to do it the other way around. This is just what
>> works
>>>> well for me...
>>>
>>> I would really like to use the technique you use and we did actually
>>> migrate towards the approach of only locking down top-level cookbooks
>>> in the environment files and then lock down supporting cookbooks in
>>> the top-level cookbooks metadata.
>>>
>>> Unfortunately we repeatedly ran in to the dependency resolver bug in
>>> Chef 11 (CHEF-3921) which would kill our chef server. So we were
>>> forced back to locking down everything in environments.
>>>
>>> [1] https://github.com/jbz/knife-audit
>>>
>>>
>>
>>
>>
>