[chef] Re: Re: Re: Re: Re: Re: Re: Re: Centralized cookbook-library repos vs distributed cookbook repos

[Adam Jacob < >]

I want to make sure I'm really clear about what Opscode actually
thinks here: our job is to provide you guys the primitives you need to
be able to do things the way you want to do them.  So I <3 whatever
workflow anyone wants to use that works for them, and that they enjoy
using - and I want to make it as easy as possible for as many of those
workflows to be as painless as they can be.

So don't mistake my answering questions about a workflow, or giving an
opinion about it, to be a final answer - it's just me trying to
suss-out where the primitives are, so we can make it easy to get the
data you need.

On Tue, Apr 13, 2010 at 11:34 PM, Hedge Hog 
<
 >
 wrote:
> That is one way of carving cookbooks out of a cookbook-library repo.
> Perhaps I have misunderstood:
> If I want to organize several libraries I have to download any common
> cookbooks to all of them - correct?
> When working on cookbook A in library 1, I make some vital changes,
> which upstream loves.
> How do those changes propogate to cookbook A in library 2?

That's going to be up to the developer of the cookbook to tell you how
they want to be collaborated with.  I think it's smart to come up with
a common consensus about what the best practice is for that, and
encourage people to set up their repositories that way.  But we need
to be flexible enough to let people go a different way, because in the
end, it's up to the developer how they want to work with others to
incorporate changes to the code they have written.

The assumption that everyone in the chain is using Git, for example,
is a false one.  Most of us are - but not all of us, and if someone
wants to use Launchpad and Mercurial for source control, I want to
include them, not force them to use a different source control system.

For getting changes from your local vendor branch back upstream, in
the example I cooked up , you could do a couple of things.  Assuming
you wanted all your changes from a specific version, you could easily
just do this:


 (141)%
 git diff chef-vendor-apache2-0.10.1 HEAD
diff --git a/cookbooks/apache2/attributes/apache.rb
b/cookbooks/apache2/attributes/apache.rb
index d6a8f76..b889de2 100644
--- a/cookbooks/apache2/attributes/apache.rb
+++ b/cookbooks/apache2/attributes/apache.rb
@@ -17,6 +17,8 @@
 # limitations under the License.
 #

+### Moar comments!
+
 # Where the various parts of apache are
 case platform
 when "redhat","centos","fedora","suse"

If you wanted that to apply directly to the upstream, which in this
case is Opscode, your workflow would look like this:

$ cd ~/opscode-cookbooks
$ (cd ~/local-chef-repo; git diff --no-color
chef-vendor-apache2-0.10.1 HEAD) | patch -p2

Then you could commit your changes, push to your fork on github, send
them to the upstream, whatever.  Again, it would depend on what the
upstream wants - in our case, we would want it on a branch with a
ticket in the cookbooks project on tickets.opscode.com, but someone
else may want a pull request from github. YMMV. :)

> Still not sure why the aversion to submodules.

My personal aversion to submodules is that, as an upstream maintainer,
it's easier for me to maintain a single project, rather than publish a
separate git repo for all 80+ cookbooks we maintain.  I'm not saying
you can't use submodules - just that I don't want to, personally.

> I prefer using existing tools and think git solves 1) and 2).  Inter
> cookbook repo dependencies is an issue.

It solves 1 and 2 if everyone is using git, which they won't be.

> My preferred workflow is (currently):
> A) See if using Git submodules to track individual cookbook repos works
> B) See if the Bundler can be "massaged" to resolve inter-cookbook 
> dependencies.
> C) See if I can deploy from cookbook-libraries contsructed from the above :)

I see almost an identical workflow developing from the hacking I did
yesterday.  The difference would be that A and B are both provided
through discovery of published cookbooks on cookbooks.opscode.com.  We
add the functionality to let you publish a cookbook with references to
the upstream source repository, and extend things to allow you to get
the cookbook artifact essentially from HEAD of the upstream.  We can
do dependency resolution easily from there, since the deps in the
cookbooks will reflect the names of the cookbooks on the cookbook
site.

C will then be trivial.

> Still thinking...

Ditto. :)

> Again this doesn't change anything for anyone else, just exporing how
> I can organize my cookbooks and libraries of them with maximum
> flexibility.

Me too, and to me, that also means not being tied to a particular
source control system or repository layout, if possible.

Adam

-- 
Opscode, Inc.
Adam Jacob, CTO
T: (206) 508-7449 E: