[chef] Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Centralized cookbook-library repos vs distributed cookbook repos

[Hedge Hog < >]

On Fri, Apr 16, 2010 at 7:43 AM, Adam Jacob 
<
 >
 wrote:
> On Wed, Apr 14, 2010 at 7:08 PM, Hedge Hog 
> <
 >
>  wrote:
>> This approach still preserves different cookbook-libraries as separate 
>> silo's.
>> Simply becasue it still ties dependency management to repo organisation.
>
> Yes - the places where a given cookbook lives as a unit of
> collaboration is a separate silo.
>
> There is also a dependency management problem, but we have the
> ground-work already to solve that.
>
> More on it in a sec.
>
>> OK, this is getting clearer to me :)
>>
>> I think we are getting side tracked by the fact that cookbook library
>> repo organization is being used to proxy for a dependency management
>> solution.
>> However, library repos are being justified/defended as easy to
>> maintain in terms of VCS effort and _not_ dependency management
>> effort!
>> In my mind the two activities should be separate but have become conflated 
>> :(
>> I don't think VCS used has anything to do with dependency management.
>
> I agree.
>
>> In reality the real reason for prefering library repos is _not_ that
>> library repos are easier to _maintain_, rather cookbook dependencies
>> are easier to _enforce_.
>> Changing to cookbook repos will raise the same issues in CVS, hg, etc.
>> - If you are not going to enforce cookbook-library silos, how do you
>> manage cookbook dependencies?
>>
>> The cost of cookbook library repos is giving up distributed cookbook
>> development and maintenance/admin burdens, having cookbook library
>> silos develop indepently, etc all outlined in earlier emails
>> The benefit  of cookbook library repos is trivial dependency magement:
>> Solve dependencies by getting all cookbooks from this cookbook-library
>> and no others.
>>
>> The proposal to elevate cookbooks to repos is to try and create a low
>> pain path for cookbook contributions to flow _around_ the community,
>> rather than just up-and-down different cookbook-library silos.
>> To do this we'd need a dependency management approach that works for
>> Chef's users and use cases.
>
> I think there is a third way, which is that we already have a
> publishing platform for cookbooks, and the cookbooks themselves have
> dependencies.  I've extended the vendor stuff I wrote before, so that
> it now downloads and vendors the first cookbook, and then reads the
> metadata and repeats the process recursively for all the dependencies:
>
> % ./knife cookbook site vendor wordpress 0.5.0 -d
>
> Now grabs wordpress and all it's dependencies, giving you a tree like this:
>
> % ls -la cookbooks
> total 8
> drwxr-xr-x   8 adam  adam  272 Apr 15 14:32 ./
> drwxr-xr-x  11 adam  adam  374 Apr 15 14:32 ../
> -rw-r--r--   1 adam  adam  151 Apr 15 14:32 README
> drwxr-xr-x  10 adam  adam  340 Mar 10 16:43 apache2/
> drwxr-xr-x  11 adam  adam  374 Apr 15 14:32 mysql/
> drwxr-xr-x   7 adam  adam  238 Feb 26 14:35 openssl/
> drwxr-xr-x   8 adam  adam  272 Apr 15 14:32 php/
> drwxr-xr-x   8 adam  adam  272 Apr 15 14:32 wordpress/
>
> % git branch
>  chef-vendor-apache2
>  chef-vendor-mysql
>  chef-vendor-openssl
>  chef-vendor-php
>  chef-vendor-wordpress
> * master
>
> % git tag
> chef-vendor-apache2-0.10.1
> chef-vendor-mysql-0.15.0
> chef-vendor-openssl-0.1.0
> chef-vendor-php-0.7.0
> chef-vendor-wordpress-0.5.0
>
> If we start publishing cookbooks and specifying dependencies in
> metadata, and extend the cookbooks site to allow the upstream
> developers to put the source control end-point for development in,
> we'll have versioned cookbooks, with dependency management, along with
> the mechanisms required to live on the bleeding edge.  And it'll be
> discoverable.
>
> Sound good?
>
>> So it seems that the debate over cookbook repo approach really has
>> little (nothing?) to do with the effort involved in managing repos and
>> everything to do with the effort required by the dependency magaement
>> solution adopted.
>> Which really should be discussed in a separte thread:
>> 'Freeing cookbook dependency management from source repo organization'
>>
>> Thoughts?
>
> I think we agree.  There are (at least) 3 things here:
>
> 1) How do you discover cookbooks you want to use
> 2) How do you track them over time, and potentially make site-specific
> changes, and track *those* over time.
> 3) How do you track and resolve the dependencies one cookbook has on another
>
> I think I've got the 'knife cookbook site vendor' stuff far enough
> that it does all 3 of these based on the cookbook site information
> today, and we can focus on adding in the functionality we need to
> cover the remaining use cases.
>
> You?
>

Sounds good.
Regular work can be a real PITA - give me 24hrs and I should have an
example alternative to discuss/evaluate :)

Regards

> Adam
>
> --
> Opscode, Inc.
> Adam Jacob, CTO
> T: (206) 508-7449 E: 
> 
 
>



-- 
πόλλ' οἶδ ἀλώπηξ, ἀλλ' ἐχῖνος ἓν μέγα
[The fox knows many things, but the hedgehog knows one big thing.]
  Archilochus, Greek poet (c. 680 BC – c. 645 BC)
http://wiki.hedgehogshiatus.com