[chef] Re: Re: Re: Re: Re: Re: Re: [chef-dev] Re: ideas for chef-hackday at Chefconf, Tuesday afternoon, May 15

[Adam Jacob < >]

I have no problem with the directory structure of a rubygem. The
comment was more about what it means to distribute and share cookbooks
- rubygems are the wrong model, and the wrong ecosystem, to use inside
Chef. Doesn't mean there isn't a bunch of good stuff to crib. (Just
like there is good stuff to crib from CPAN - who doesn't want CPAN
Testers for cookbooks? I totally do.)

Adam

On Wed, Apr 25, 2012 at 12:47 PM, Bryan Berry 
<
 >
 wrote:
> dammit as usual, when I disagree w/ Jay, it's only because I misunderstand
> him ;)
>
> you deserve the Golden Spatula for sure!
>
> this also begs the question of why we don't package our cookbooks as ruby
> gems. I am not familiar enough with gems to know their weaknesses. Adam
> Jacob told me on Twitter once that "we can do better" than gems, and I hope
> he is right.
>
>
> On Wed, Apr 25, 2012 at 9:34 PM, Jay Feldblum 
> <
 >
>  wrote:
>>
>> Bryan,
>>
>> The original structure I outlined was a proposed structure for a single
>> cookbook repo - a repo whose purpose is the development and testing of a
>> single cookbook -, and was not a structure for a repo hosting multiple
>> cookbooks or for a full infrastructure.
>>
>> In that proposed structure, the single "core" cookbook lives in a
>> subdirectory, while the surrounding boilerplate/tests/caches live either at
>> the top-level (e.g. the Rakefile) or in different subdirectories (e.g. the
>> test cases). That structure means you can pull out and tarball a single
>> subdirectory of the cookbook's repo in order to capture the self-contained
>> cookbook, while skipping the boilerplate/tests/caches for the cookbook 
>> which
>> are outside of that single subdirectory.
>>
>> There would of course be no roles, environments, data-bags, etc., in that
>> proposed structure, since it's for developing & testing a single cookbook 
>> in
>> isolation rather than developing & testing an infrastructure.
>>
>> The proposed structure mirrors the usual structure for most Ruby gems, but
>> replacing lib/GEM-NAME/ with cookbook/COOKBOOK-NAME/ (and replacing Gemfile
>> with Cheffile for those who are so inclined).
>>
>> Cheers,
>> Jay
>>
>> On Wed, Apr 25, 2012 at 2:55 PM, Bryan Berry 
>> <
 >
>> wrote:
>>>
>>> I feel pretty strongly that everything required to test a cookbook
>>> live within the cookbook's git repository, that each cookbook has to
>>> be its own project. Isn't every gem its own project?
>>>
>>> Once something is common to various parts of your infrastructure, the
>>> chance that it will be shared to the broader Chef community rapidly
>>> approaches zero. This is just my experience. I have shared multiple
>>> cookbooks but yet to share a single role, data_bag, environment or
>>> even knife script w/ the broader Chef community.
>>>
>>> Let me know if I have misunderstood you.
>>>
>>> On Wed, Apr 25, 2012 at 8:47 PM, Jay Feldblum 
>>> <
 >
>>> wrote:
>>> > Bryan,
>>> >
>>> > Thanks for the comments.
>>> >
>>> > The way I was thinking of it is that the individual cookbook is
>>> > embedded in
>>> > a larger surrounding project, so that the "core" cookbook directory
>>> > could be
>>> > pulled out and zipped up without much logic concerning which
>>> > files/directories belong to the "core" cookbook and which
>>> > files/directories
>>> > belong to the larger surrounding project. This would make the work of a
>>> > "rake upload" type task much easier.
>>> >
>>> > If, instead, everything is at the toplevel, that will require extra
>>> > logic
>>> > when zipping and pushing a release. But it will make the project
>>> > structure
>>> > simpler.
>>> >
>>> > I wouldn't embed everything deep down in cookbook/COOKBOOK-NAME; unless
>>> > there's something I'm missing, I would either embed just the "core"
>>> > cookbook
>>> > there or keep everything at the top-level. This is for repos containing
>>> > exactly one cookbook, following the repo-per-cookbook pattern.
>>> >
>>> > Either way, let me know how easy/hard it is to integrate Vagrant & Toft
>>> > with
>>> > the Cheffile (in case you or others plan to have one), and based on
>>> > your
>>> > experimental results what can be done to improve that integration.
>>> >
>>> > One thing I am considering is a metadata DSL method that would read the
>>> > metadata.rb from the same directory, so you wouldn't have to spell out
>>> > all
>>> > the dependencies in the metadata.rb and then do so again in the
>>> > Cheffile.
>>> > This would follow the pattern of Bundler's gemspec DSL method. I can
>>> > see
>>> > this being useful for the Cheffile belonging to an individual cookbook
>>> > repo.
>>> >
>>> >     site "http://community.opscode.com/api/v1";
>>> >
>>> >     # apply all the dependencies listed in ./metadata.rb
>>> >     # these all use the default source listed above
>>> >     metadata
>>> >
>>> >     # override the source for one of the cookbooks listed in
>>> > ./metadata.rb
>>> >     # e.g. in case we need a specific fix when developing & testing the
>>> > cookbook
>>> >     cookbook "apache2",
>>> >         :git => "https://github.com/opscode-cookbooks/apache2";,
>>> >         :ref => "be12fede2e99aff3029360df38366a3b8a34171f"
>>> >
>>> > Cheers,
>>> > Jay
>>> >
>>> > On Wed, Apr 25, 2012 at 2:14 PM, Bryan Berry 
>>> > <
 >
>>> > wrote:
>>> >>
>>> >> hmm, I have something different in mind
>>> >>
>>> >> Jay, like we talked on the podcast, I think cookbooks need to be
>>> >> self-contained. Here is how I think it should be laid out
>>> >>
>>> >> I definitely think that the Vagrantfile, Toftfile (my own invention),
>>> >> and Cheffile should live at the cookbook/COOKBOOK-NAME/  level
>>> >>
>>> >> cookbook/COOKBOOK-NAME/
>>> >>                                              attributes/
>>> >>                                              definitions/
>>> >>                                              files/
>>> >>                                              libraries/
>>> >>                                              providers/
>>> >>                                              recipes/
>>> >>                                              resources/
>>> >>                                              temp/      # cache
>>> >> cookbooks sourced by Cheffile
>>> >>                                              tests/              #
>>> >> tests or test? don't know the convention
>>> >>                                                        minitest/
>>> >>
>>> >> recipe1/
>>> >>
>>> >>          some_test.rb
>>> >>
>>> >>          some_test2.rb
>>> >>
>>> >>  recipe2/
>>> >>
>>> >>         . . .
>>> >>                                                        cucumber/
>>> >>                                                        rspec/
>>> >>                                              Cheffile
>>> >>                                              Cheffile.lock
>>> >>                                              Rakefile
>>> >>                                              Toftfile
>>> >> # same as Vagrantfile but for toft
>>> >>                                              Vagrantfile
>>> >>
>>> >> The Cheffile at the at COOKBOOK-NAME level would be used by
>>> >> Vagrantfile or Toftfile to source the other cookbooks needed to run
>>> >> integration tests.
>>> >>
>>> >> that's my euro 0,02
>>> >>
>>> >> On Wed, Apr 25, 2012 at 7:53 PM, Jay Feldblum 
>>> >> <
 >
>>> >> wrote:
>>> >> > It may be wise to structure the Git repository of tested cookbooks
>>> >> > according
>>> >> > to the following convention:
>>> >> >
>>> >> > /
>>> >> >     cookbooks/
>>> >> >     #   .gitignore'd
>>> >> >     #   for cookbooks installed via librarian-chef
>>> >> >     #   this is an assumption librarian-chef currently makes
>>> >> >     cookbook/
>>> >> >         COOKBOOK-NAME/
>>> >> >             attributes/
>>> >> >             definitions/
>>> >> >             files/
>>> >> >             libraries/
>>> >> >             providers/
>>> >> >             recipes/
>>> >> >             resources/
>>> >> >             templates/
>>> >> >             metadata.rb
>>> >> >     test/
>>> >> >         integration/
>>> >> >             some_integration_test.rb
>>> >> >             some_other_integration_test.rb
>>> >> >     .gitignore
>>> >> >     #   includes:
>>> >> >     #       cookbooks/
>>> >> >     #       Cheffile.lock
>>> >> >     Cheffile
>>> >> >     #   to support Vagrantfile
>>> >> >     #   the cookbook under test may depend on other cookbooks
>>> >> >     Cheffile.lock
>>> >> >     #   .gitignore'd
>>> >> >     #   don't add to git in a cookbook repo
>>> >> >     #   only add to git in an infrastructure repo
>>> >> >     Rakefile
>>> >> >     #   has a task for zipping up the content of
>>> >> > cookbooks/COOKBOOK-NAME
>>> >> >     #   and optionally for releasing to the Opscode Community Site
>>> >> >     Vagrantfile
>>> >> >     #   if using Vagrant for testing
>>> >> >     #   references cookbooks located in cookbooks/ (dependencies)
>>> >> > and
>>> >> > cookbook/ (under test)
>>> >> >
>>> >> > If people think this is a good idea, I may add intelligence to
>>> >> > Librarian-Chef's git source to try to find the cookbook inside
>>> >> > cookbook/COOKBOOK-NAME/ if it can't find the cookbook at the
>>> >> > top-level
>>> >> > of
>>> >> > the git repo. However, for now, people can always use the format:
>>> >> >
>>> >> > cookbook "COOKBOOK-NAME",
>>> >> >     :git => "git://github.com/user/COOKBOOK-NAME",
>>> >> >     :path => "cookbook/COOKBOOK-NAME"
>>> >> >
>>> >> > Cheers,
>>> >> > Jay
>>> >> >
>>> >
>>> >
>>
>>
>



-- 
Opscode, Inc.
Adam Jacob, Chief Customer Officer
T: (206) 619-7151 E: