[chef] Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Centralized cookbook-library repos vs distributed cookbook repos

[dreamcat four < >]

If I may, I would like to offer some discussion about dependency
resolution and cookbook sources.

Over on cookbooks.opscode.com, we make people register an account
username (or organisational account name, such as '37signals'). Isnt
the account username what we are already registering as the upstream
source / cookbooks namespace?

Then over on github.com, that same username string translates to our
github account name. Eg http://github.com/37signals/cookbooks. Other
SCMs like launchpad and bitbucket should be very easily compatible
with some similar naming convention too. If the chef tools can know to
check at cookbooks.opscode.com for the authority of the registered
account names.

Then it seems to follow that (in some version of chef) a local
cookbooks folder might look like this:

  /srv/chef/cookbooks/opscode
  /srv/chef/cookbooks/37signals
  /srv/chef/cookbooks/dreamcat4
  /srv/chef/site-cookbooks/opscode
  /srv/chef/site-cookbooks/37signals
  /srv/chef/site-cookbooks/dreamcat4

To explicitly reference the 37signals' apache cookbook, I might write:

  include_recipe '37signals/apache:recipe_name'

And the same way for roles etc.

When no prefix is specified like for now, the implicit namespace could
default to "opscode". Or we might reference the locally bound
namespace (at some kind of locally bound level).

Im not trying to state that Opscode here can or should really do it
like that. But it seemed kindda worth mentioning here in the
discussion.

On Wed, Apr 14, 2010 at 10:04 PM, Adam Jacob 
<
 >
 wrote:
> On Wed, Apr 14, 2010 at 1:08 PM, Alex Soto 
> <
 >
>  wrote:
>> My comment was triggered by you mentioning that:
>>
>> The assumption that everyone in the chain is using Git, for example,
>> is a false one.  Most of us are - but not all of us, and if someone
>> wants to use Launchpad and Mercurial for source control, I want to
>> include them, not force them to use a different source control system.
>>
>> I read too much into that to think you meant to support all these other 
>> scm's which is not necessarily what you meant.  My bad, I should have 
>> re-read things.  My off the cuff remark, was because it just raised the 
>> flag in my head to point out that we don't want to over engineer this to 
>> make things super flexible.  keep it simple :)
>
> Absolutely.  The pattern I proposed in an earlier message was using
> 'vendor branches', which exist in every VCS since CVS, to deal with
> tracking your local changes against an up-stream release.  The
> integration with various source code control systems is easy here,
> since it's just 'implement that VCS systems vendor branch pattern'.
>
> Check out the source here for an example:
>
> http://github.com/adamhjk/chef/blob/master/chef/lib/chef/knife/cookbook_site_vendor.rb
>
> You can imagine how that could get re-factored to support multiple VCS
> systems without much pain.
>
>> With respect to the developer workflow:
>>
>> Here's my specific problem.
>>
>> My previous chef deployment used my own chef server, which was cool 
>> because it allowed me to have multiple cookbook repos that the server 
>> pulled from.  I maintained an internal repo that had my specific cookbooks 
>> for my infrastructure and I had a fork of the opscode repo that I could 
>> declare dependencies on from my 'internal repo'.  My fork of the opscode 
>> repo also allowed me to update those cookbooks when I found problems.
>>
>> Now I'm trying to use the platform (I'm still just barely scratched the 
>> surface of it, so I may be mistaken with how I'm using it), but it looks 
>> like I have effectively one cookbook repo which is the 'platform' where I 
>> have to push cookbooks into.  So if I want to use some opscode cookbooks, 
>> I have to copy them and push them up individually.  Because the 
>> granularity of the repo is not at the cookbook level, I don't see how I 
>> could have just a copy of an individual opscode cookbook and still track 
>> upstream changes easily.
>
> The message I posted earlier is my answer to this - check it out and
> see if it works.  The idea is that you would still have your own git
> repository full of all your own cookbooks (you want it this way,
> because it lets you rebuild everything from scratch with only a copy
> of what you care about,) that you're publishing to the platform or a
> chef server.
>
> It lets you track the upstream easily, and provides a more flexible
> way for you to have custom modifications that travel forward in your
> repository.  I think it's pretty close.
>
>> I 'think' I like the one repo per cookbook strategy because those are the 
>> logical modules I think of when developing cookbooks/recipes.  Then I 
>> would be able to fork and submodule them into my repo.  However there is a 
>> nagging feeling that makes me feel that's a bit too granular and may get 
>> cumbersome to manage.
>
> It would be significantly harder to manage for the maintainers of a
> collection of complex, interdependent cookbooks.
>
> Adam
>
> --
> Opscode, Inc.
> Adam Jacob, CTO
> T: (206) 508-7449 E: 
> 
 
>