[chef] Re: Re: Re: Re: Re: Re: Re: Centralized cookbook-library repos vs distributed cookbook repos

[Hedge Hog < >]

On Wed, Apr 14, 2010 at 3:30 PM, Adam Jacob 
<
 >
 wrote:
> Okay, in my head, I see two different workflows here:
>
> 1) The Consumer Workflow
>
> This is the workflow where you find a cookbook that works for you, and
> you want to use it in your infrastructure, and track it in your own
> source control (probably a chef-repo.)  You may then want to make
> changes to the upstream cookbook, and you then want to be able to
> easily see the diff between the changes you have made and any new
> versions the upstream has published.
>
> The important thing here is that you care about the cookbook having
> it's changes tracked in your local repository (since you use it to
> build your infrastructure), but you don't care so much about having
> the entirety of the upstream revision history (indeed, you might not
> be able to, in the cases where you want to use a different source
> control system from the upstream.)  So you want to easily track the
> upstream, and apply your changes.
>
> 2) The Developer/Collaborator Workflow
>
> In this case, you want to collaborate with the upstream on development
> of the cookbook directly - you want your changes to be situated in the
> upstream.  To me, this workflow is actually defined by whomever is
> managing the upstream - for Opscode, it's tickets and pull requests to
> the opscode/chef repo, for 37signals it's different, and for someone
> else it may require using mercurial or svn.
>
> I've been thinking about the consumer workflow for a while now, and I
> spent some of today working on getting a good pass at getting it down
> to essentially a single command.  For the developer/collaborator
> workflow, we may also want to make a best practice, but I'm less
> concerned about it right now.
>
> The pattern I'm following is called "vendor branching", and it's been
> around forever.  The basic method is that you are tracking an upstream
> source release, and you want to keep a certain set of patches in sync,
> and be able to move between local versions easily.  It was pretty
> popular in the CVS days, if any of you can get into the way-back
> machine with me.  Here is a good description of doing this with Git:
>
> http://happygiraffe.net/blog/2008/02/07/vendor-branches-in-git/
>
> I've adapted this pattern to be integrated with knife and the
> cookbooks.opscode.com site.  If you already have knife configured to
> work with your local chef repo, from the current opscode/chef, you can
> do:
>
> $ knife cookbook site vendor apache2
>
> And you'll get the latest version of the apache2 cookbook in a local
> vendor branch.  You can then make changes directly to the cookbook in
> your local repository - you don't need to build a site-cookbook, or do
> anything else.  Just make changes like normal.  When the upstream
> releases a new version, just repeat the command:
>
> $ knife cookbook site vendor apache2
>
> And the new version will be downloaded, the vendor branch updated, and
> then merged into your master branch.  The resulting diff can be easily
> viewed, and if there are any merge conflicts, you get a chance to
> review them.  If you don't like the changes, you can just 'git reset
> --hard HEAD~1'.
>
> You can always get a diff between your current cookbook and the upstream 
> with:
>
> 
 
>  git diff chef-vendor-apache2-0.10.1 HEAD
> diff --git a/cookbooks/apache2/attributes/apache.rb
> b/cookbooks/apache2/attributes/apache.rb
> index c733d5e..d6a8f76 100644
> --- a/cookbooks/apache2/attributes/apache.rb
> +++ b/cookbooks/apache2/attributes/apache.rb
> @@ -75,4 +75,3 @@ set_unless[:apache][:worker][:minsparethreads] = 64
>  set_unless[:apache][:worker][:maxsparethreads] = 192
>  set_unless[:apache][:worker][:threadsperchild] = 64
>  set_unless[:apache][:worker][:maxrequestsperchild] = 0
> -# whee, vendor merge.
>
> Check this picture of my local git history out - it shows going from
> the 0.9.1 version of the apache2 cookbook to 0.10.1, with a local
> change in the middle:
>
> http://skitch.com/adamjacob/n6246/chef-repo-branch-master
>
> Check out this gist for some in-action shots:
>
> http://gist.github.com/365452
>
> In order to make this complete, a few things still need to be done:
>
> 1) We need to enable the API for uploading to the cookbook site
> easily, to make it simple for cookbook authors to publish their
> cookbooks.
> 2) We need to enable tracking of the upstream source repository in the
> cookbook site, along with an optional path, which should enable you to
> both vendor HEAD and easily collaborate on development.
>
> What do you think?
>

That is one way of carving cookbooks out of a cookbook-library repo.
Perhaps I have misunderstood:
If I want to organize several libraries I have to download any common
cookbooks to all of them - correct?
When working on cookbook A in library 1, I make some vital changes,
which upstream loves.
How do those changes propogate to cookbook A in library 2?

Still not sure why the aversion to submodules.

I prefer using existing tools and think git solves 1) and 2).  Inter
cookbook repo dependencies is an issue.
My preferred workflow is (currently):
A) See if using Git submodules to track individual cookbook repos works
B) See if the Bundler can be "massaged" to resolve inter-cookbook 
dependencies.
C) See if I can deploy from cookbook-libraries contsructed from the above :)

Still thinking...

Again this doesn't change anything for anyone else, just exporing how
I can organize my cookbooks and libraries of them with maximum
flexibility.

Regards

> Adam
>
> --
> Opscode, Inc.
> Adam Jacob, CTO
> T: (206) 508-7449 E: 
> 
 
>



-- 
πόλλ' οἶδ ἀλώπηξ, ἀλλ' ἐχῖνος ἓν μέγα
[The fox knows many things, but the hedgehog knows one big thing.]
  Archilochus, Greek poet (c. 680 BC – c. 645 BC)
http://wiki.hedgehogshiatus.com