[chef] Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Centralized cookbook-library repos vs distributed cookbook repos

[Hedge Hog < >]

On Fri, Apr 16, 2010 at 8:35 AM, Hedge Hog 
<
 >
 wrote:
> On Fri, Apr 16, 2010 at 7:43 AM, Adam Jacob 
> <
 >
>  wrote:
>> On Wed, Apr 14, 2010 at 7:08 PM, Hedge Hog 
>> <
 >
>>  wrote:
>>> This approach still preserves different cookbook-libraries as separate 
>>> silo's.
>>> Simply becasue it still ties dependency management to repo organisation.
>>
>> Yes - the places where a given cookbook lives as a unit of
>> collaboration is a separate silo.
>>
>> There is also a dependency management problem, but we have the
>> ground-work already to solve that.
>>
>> More on it in a sec.
>>
>>> OK, this is getting clearer to me :)
>>>
>>> I think we are getting side tracked by the fact that cookbook library
>>> repo organization is being used to proxy for a dependency management
>>> solution.
>>> However, library repos are being justified/defended as easy to
>>> maintain in terms of VCS effort and _not_ dependency management
>>> effort!
>>> In my mind the two activities should be separate but have become 
>>> conflated :(
>>> I don't think VCS used has anything to do with dependency management.
>>
>> I agree.
>>
>>> In reality the real reason for prefering library repos is _not_ that
>>> library repos are easier to _maintain_, rather cookbook dependencies
>>> are easier to _enforce_.
>>> Changing to cookbook repos will raise the same issues in CVS, hg, etc.
>>> - If you are not going to enforce cookbook-library silos, how do you
>>> manage cookbook dependencies?
>>>
>>> The cost of cookbook library repos is giving up distributed cookbook
>>> development and maintenance/admin burdens, having cookbook library
>>> silos develop indepently, etc all outlined in earlier emails
>>> The benefit  of cookbook library repos is trivial dependency magement:
>>> Solve dependencies by getting all cookbooks from this cookbook-library
>>> and no others.
>>>
>>> The proposal to elevate cookbooks to repos is to try and create a low
>>> pain path for cookbook contributions to flow _around_ the community,
>>> rather than just up-and-down different cookbook-library silos.
>>> To do this we'd need a dependency management approach that works for
>>> Chef's users and use cases.
>>
>> I think there is a third way, which is that we already have a
>> publishing platform for cookbooks, and the cookbooks themselves have
>> dependencies.  I've extended the vendor stuff I wrote before, so that
>> it now downloads and vendors the first cookbook, and then reads the
>> metadata and repeats the process recursively for all the dependencies:
>>
>> % ./knife cookbook site vendor wordpress 0.5.0 -d
>>
>> Now grabs wordpress and all it's dependencies, giving you a tree like this:
>>
>> % ls -la cookbooks
>> total 8
>> drwxr-xr-x   8 adam  adam  272 Apr 15 14:32 ./
>> drwxr-xr-x  11 adam  adam  374 Apr 15 14:32 ../
>> -rw-r--r--   1 adam  adam  151 Apr 15 14:32 README
>> drwxr-xr-x  10 adam  adam  340 Mar 10 16:43 apache2/
>> drwxr-xr-x  11 adam  adam  374 Apr 15 14:32 mysql/
>> drwxr-xr-x   7 adam  adam  238 Feb 26 14:35 openssl/
>> drwxr-xr-x   8 adam  adam  272 Apr 15 14:32 php/
>> drwxr-xr-x   8 adam  adam  272 Apr 15 14:32 wordpress/
>>
>> % git branch
>>  chef-vendor-apache2
>>  chef-vendor-mysql
>>  chef-vendor-openssl
>>  chef-vendor-php
>>  chef-vendor-wordpress
>> * master
>>
>> % git tag
>> chef-vendor-apache2-0.10.1
>> chef-vendor-mysql-0.15.0
>> chef-vendor-openssl-0.1.0
>> chef-vendor-php-0.7.0
>> chef-vendor-wordpress-0.5.0
>>
>> If we start publishing cookbooks and specifying dependencies in
>> metadata, and extend the cookbooks site to allow the upstream
>> developers to put the source control end-point for development in,
>> we'll have versioned cookbooks, with dependency management, along with
>> the mechanisms required to live on the bleeding edge.  And it'll be
>> discoverable.
>>
>> Sound good?
>>
>>> So it seems that the debate over cookbook repo approach really has
>>> little (nothing?) to do with the effort involved in managing repos and
>>> everything to do with the effort required by the dependency magaement
>>> solution adopted.
>>> Which really should be discussed in a separte thread:
>>> 'Freeing cookbook dependency management from source repo organization'
>>>
>>> Thoughts?
>>
>> I think we agree.  There are (at least) 3 things here:
>>
>> 1) How do you discover cookbooks you want to use
>> 2) How do you track them over time, and potentially make site-specific
>> changes, and track *those* over time.
>> 3) How do you track and resolve the dependencies one cookbook has on 
>> another
>>
>> I think I've got the 'knife cookbook site vendor' stuff far enough
>> that it does all 3 of these based on the cookbook site information
>> today, and we can focus on adding in the functionality we need to
>> cover the remaining use cases.
>>
>> You?
>>
>
> Sounds good.
> Regular work can be a real PITA - give me 24hrs and I should have an
> example alternative to discuss/evaluate :)

Better late than never,,,

http://hedgehogshiatus.com/carving-chefs-cookbooks

HTH
>
> Regards
>
>> Adam
>>
>> --
>> Opscode, Inc.
>> Adam Jacob, CTO
>> T: (206) 508-7449 E: 
>> 
 
>>
>
>
>
> --
> πόλλ' οἶδ ἀλώπηξ, ἀλλ' ἐχῖνος ἓν μέγα
> [The fox knows many things, but the hedgehog knows one big thing.]
>  Archilochus, Greek poet (c. 680 BC – c. 645 BC)
> http://wiki.hedgehogshiatus.com
>



-- 
πόλλ' οἶδ ἀλώπηξ, ἀλλ' ἐχῖνος ἓν μέγα
[The fox knows many things, but the hedgehog knows one big thing.]
  Archilochus, Greek poet (c. 680 BC – c. 645 BC)
http://wiki.hedgehogshiatus.com