[chef] Re: Re: Re: RE: Re: RE: Re: RE: How to manage cookbook versions more efficiently?

[Sean OMeara < >]

You can't have cookbooks without metadata.

The metadata is used by Chef itself
The Berksfile is used by your development tooling. (test-kitchen)

For example, a common thing to do with test-kitchen is apply the
apt::default recipe to the machine before running recipes on debuntu
platforms.

Like this: 
https://github.com/chef-cookbooks/mysql/blob/c2d49a98ba9566967f464009c59a557b23086907/.kitchen.yml#L21

Because apt is not a strict dependency for the mysql cookbook, it
needs to be listed in the Berksfile.

https://github.com/chef-cookbooks/mysql/blob/c2d49a98ba9566967f464009c59a557b23086907/metadata.rb#L20-L21

and

https://github.com/chef-cookbooks/mysql/blob/c2d49a98ba9566967f464009c59a557b23086907/Berksfile#L6

The other use case, is for when you're developing more than one
cookbook at once, and need to point at local disk paths or git repos.

https://github.com/someara/wordpress-uberdemo/blob/d8ed8d2ce53cb68518afc6cd2bfa1cda97140e63/Berksfile#L5-L6

^ After those two cookbooks are finished, they'll be released to an
artifact repository (supermarket or your chef-server), then the
Berksfile lines can be removed.

-s


On Thu, Apr 30, 2015 at 9:20 AM, Greg Damiani 
<
 >
 wrote:
> Is there any documentation or discussion around this approach, i.e. removing
> metadata.rb from your cookbooks? Can't seem to find a blog or thread
> anywhere.
>
> I've got cookbook developers asking me why they need to declare dependencies
> in two different places right now, so this would be a great convenience.
>
> One question I have at the outset is, without metadata.rb, where do cookbook
> name and version number get stored?  Most "knife cookbook" commands fail
> when I remove metadata.rb as a result.
>
> On Sat, Apr 25, 2015 at 12:08 PM, Bryan Baugher 
> <
 >
>  wrote:
>>
>> You can just remove the 'metadata' keyword in your Berksfile and use your
>> Berksfile as your metadata.rb.
>>
>>
>> On Fri, Apr 24, 2015 at 2:04 PM Fouts, Chris 
>> <
 >
>> wrote:
>>>
>>> Now this is an interesting idea. But isn’t this a chicken-egg problem?
>>> Doesn’t Berksfile.lock get generated based on the metadata.rb?
>>>
>>>
>>>
>>> Chris
>>>
>>>
>>>
>>> From: Bryan Baugher 
>>> [mailto:
 
>>> Sent: Friday, April 24, 2015 2:11 PM
>>> To: 
>>> 
 ;
>>>  
>>> 
 
>>> Subject: [chef] Re: RE: Re: RE: How to manage cookbook versions more
>>> efficiently?
>>>
>>>
>>>
>>> One thing I've played around with is have a cookbook's metadata.rb read a
>>> Berksfile.lock which looks something like this,
>>>
>>> require 'berkshelf'
>>>
>>> berksfile = ::Berkshelf::Berksfile.new 'Berksfile'
>>>
>>> berksfile.list.each do |dependency|
>>>
>>>  depends dependency.name, "= #{dependency.locked_version.to_s}"
>>>
>>> end
>>>
>>>
>>>
>>> The nice part about this is Berkshelf will calculate all of your
>>> dependencies, you don't have to figure them all out. Additional for your
>>> problem updating your cookbook's dependencies can be done with 'berkshelf
>>> update [COOKBOOK]'. So from here you could write a script to update all of
>>> them at once, or maybe even have your CI do it for you.
>>>
>>>
>>>
>>> On Fri, Apr 24, 2015 at 1:02 PM Fouts, Chris 
>>> <
 >
>>> wrote:
>>>
>>> Here’s an abbreviated description of our product.
>>>
>>>
>>>
>>> Say we have two different applications, say, a “cacheserver” and a
>>> “flexip” application, running in separate RHEL VMs. For each, I created a
>>> “cacheserver_role” and a “flexip_role” cookbooks to install their 
>>> respective
>>> application. They do however, share some “common” books, say a “java”
>>> cookbook. So role cookbooks’ metadata.rb files look like
>>>
>>>
>>>
>>> cacheserver_role
>>>
>>> name ‘cacheserver_role’
>>>
>>> depends ‘java’, ‘= 1.0.0’
>>>
>>> depends ‘cacheserver’, ‘= 1.0.0’
>>>
>>>
>>>
>>> flexip_role
>>>
>>> name ‘flexip_role’
>>>
>>> depends ‘java’, ‘= 1.0.0’
>>>
>>> depends ‘flexip’, ‘= 1.0.0’
>>>
>>>
>>>
>>> I want to purposely “pin” a specific version of the aggregate cookbooks
>>> in each role cookbook. This is an important requirement.
>>>
>>>
>>>
>>> As you can see, if I want to update my java cookbook, I have to modify
>>> (at least) two cookbooks. In my real product, I have more than just two 
>>> role
>>> cookbooks.
>>>
>>>
>>>
>>> I’m purposely moving away from environment.json and role.json files
>>> because they are NOT versioned, that is, they do NOT have a metadata.rb
>>> file.
>>>
>>>
>>>
>>> Chris
>>>
>>>
>>>
>>> From: Nathen Harvey 
>>> [mailto:
 
>>> Sent: Friday, April 24, 2015 12:48 PM
>>> To: 
>>> 
 
>>> Subject: [chef] Re: RE: How to manage cookbook versions more efficiently?
>>>
>>>
>>>
>>> I'm not sure I understand why you have a single role for each node in
>>> your
>>>
>>> infrastructure.  Could you help me understand that?  Or do you mean you
>>> have 25
>>>
>>> different types of nodes in your infrastructure where each type might
>>> have N
>>>
>>> instances?
>>>
>>>
>>>
>>> Have you considered using Environments [1] for pinning your cookbook
>>> versions?
>>>
>>> I think this might be a more sustainable approach for what you've
>>> described.
>>>
>>>
>>>
>>> -Nathen
>>>
>>>
>>>
>>>
>>>
>>> 1:  http://docs.chef.io/environments.html
>>>
>>>
>>>
>>>
>>>
>>> On Fri, Apr 24, 2015 at 12:25 PM, Fouts, Chris 
>>> <
 >
>>> wrote:
>>>
>>> " For complete consistency, you have to specify every single cookbook,
>>> including all dependencies for and on  your desired cookbooks in your 
>>> roles
>>> or run_lists. Your role based approach  gets very nasty if you start 
>>> mixing
>>> roles, and mixing conflicting cookbook versions or unintended 
>>> dependencies.
>>> There were big problems when the yum and mysql cookbooks were updated and
>>> were incompatible with many older, stable, tested, production cookbooks 
>>> that
>>> relied on them."
>>>
>>> That's why I asked the question, you just seconded it. :)
>>>
>>> Chris
>>>
>>> -----Original Message-----
>>> From: Nico Kadel-Garcia 
>>> [mailto:
 
>>> Sent: Friday, April 24, 2015 11:41 AM
>>> To: 
>>> 
 
>>>
>>> Subject: [chef] RE: How to manage cookbook versions more efficiently?
>>>
>>> From: Fouts, Chris 
>>> [mailto:
 
>>> Sent: Thursday, April 23, 2015 10:29 AM
>>> To: 
>>> 
 
>>> Subject: [chef] How to manage cookbook versions more efficiently?
>>>
>>> > I use role cookbooks to pin down versions of the specific versions of
>>> > the cookbooks they use. Since I have 25 nodes in my product and each 
>>> > node
>>> > has a role, I have at least 25 role cookbooks. I just then add my role
>>> > cookbooks to my nodes' run list. For example I have: the following. I DO
>>> > want to pin a particular cookbook version in my role cookbooks.
>>>
>>> > Any ideas on how to alleviate this situation?
>>>
>>> For complete consistency, you have to specify every single cookbook,
>>> including all dependencies for and on  your desired cookbooks in your 
>>> roles
>>> or run_lists. Your role based approach  gets very nasty if you start 
>>> mixing
>>> roles, and mixing conflicting cookbook versions or unintended 
>>> dependencies.
>>> There were big problems when the yum and mysql cookbooks were updated and
>>> were incompatible with many older, stable, tested, production cookbooks 
>>> that
>>> relied on them.
>>>
>>> This is one of my major reasons for giving up on the
>>> "chef-server/chef-client" model, and preferring "chef-solo" for small
>>> environments. I can lock down every single cookbook in Berkshelf in a much
>>> more controlled fashion than mixing and matching and unweaving roles,
>>> cookbook, or environment wrappers, and I can apply an updated or testing
>>> cookbook on a single host  with a locally updated or git branched
>>> Berkshelf.lock without potentially inflicting it on any other unexpected
>>> host. There are costs: using chefdk is a fast way to get a full Berkshelf
>>> enabled chef-solo environment, but it's not pre-built for all operating
>>> systems that Chef supports.
>>>
>>> Nico Kadel-Garcia
>>> Lead DevOps Engineer
>>> 
 
>>>
>>>
>
>
>
>
> --
>
> Greg Damiani | Senior System Administrator | @damianigreg
>
> BuzzFeed: The Social News and Entertainment Company
>
> 40 Argyll Street, 2nd Floor, London, W1F 7EB
>
>