[chef] Re: Re: Wrapper cookbooks


Chronological Thread 
  • From: Douglas Garstang < >
  • To:
  • Subject: [chef] Re: Re: Wrapper cookbooks
  • Date: Thu, 27 Mar 2014 09:42:23 -0700

Gregory,

I suppose the only way to make this clearer is to paste the relevant portions. See below. For each cookbook I have showed the relevant portions of metadata, Berksfile and recipes/default.rb from the top of the 'stack' down. Starting from the bottom of the stack, each cookbook independently tests ok with vagrant. It's not until reaching the top of the stack (bc-role-downloader) that a failure occurs.

The bottom line is that the top level cookbook, bc-role-downloader should not need to know anything about the internal details of the cookbooks it sits on top of. All bc-role-downloader should have to do is reference bc-app-downloader (which independently tests ok), and bc-role-base (which also independently tests ok), and that's it. However, it fails. In the bc-apt cookbook, i check to see if the bc-apt attribute has been set, and if not, abort. The bc-apt attribute is set in the bc-location cookbook. However, as I said, this all tests fine in all these cookbooks right up to before the top level bc-role-downloader one.

It should also be noted that if I add an include_recipe 'bc-location' to the recipes/default.rb file in the bc-role-downloader cookbook, the error does not occur. How that can happen _without_ putting a corresponding entry in metadata.rb and/or the Berksfile, I cannot fathom.

bc-role-downloader:
-------------------
metadata:
depends "bc-role-base"
depends "bc-foocom-downloader"

Berksfile:
cookbook 'bc-role-base', git: " :chef/bc-role-base.git"
cookbook 'bc-foocom-downloader', git: " :chef/bc-foocom-downloader.git"

recipes/default.rb
include_recipe "bc-foocom-downloader"


bc-foocom-downloader:
----------------------
metadata:
depends          'bc-location'
depends          'bc-apt'

Berksfile:
cookbook 'bc-location', git: " :chef/bc-location.git"
cookbook 'bc-apt', git: " :chef/bc-apt.git"

recipes/default.rb
include_recipe "bc-apt"


bc-role-base:
-------------
metadata:
depends "bc-location"
depends "bc-apt"
depends "bc-ohai"
depends "bc-chef-statuslog"
depends "bc-yali-client"
depends "bc-users"
depends "bc-rsyslog"
depends "chef-solo-search"
depends "bc-resolv"
depends "postfix"
depends "sysctl"
depends "bc-motd"
depends "sudo"
depends "ntp"
depends "bc-nrpe"
depends "bc-statsd"
depends "bc-chef-client"

Berksfile:

kbook "bc-location", git: " :chef/bc-location.git"
cookbook "bc-apt", git: " :chef/bc-apt.git"
cookbook "bc-ohai", git: " :chef/bc-ohai.git"
cookbook "bc-chef-statuslog", git: " :chef/bc-chef-statuslog.git"
cookbook "bc-yali-client", git: " :chef/bc-yali-client.git"
cookbook "bc-users", git: " :chef/bc-users.git"
cookbook "bc-rsyslog", git: " :chef/bc-rsyslog.git"
cookbook "bc-resolv", git: " :chef/bc-resolv.git"
cookbook "bc-motd", git: " :chef/bc-motd.git"
cookbook "bc-nrpe", git: " :chef/bc-nrpe.git"
cookbook "bc-statsd", git: " :chef/bc-statsd.git"
cookbook "bc-chef-client", git: " :chef/bc-chef-client.git"

recipes/default.rb:
include_recipe "bc-location"
include_recipe "bc-apt"
include_recipe "bc-ohai"
include_recipe "bc-chef-statuslog"
include_recipe "bc-yali-client"
include_recipe "bc-users"
include_recipe "bc-rsyslog"
include_recipe "chef-solo-search"
include_recipe "bc-resolv"
include_recipe "postfix"
include_recipe "sysctl"
include_recipe "bc-motd"
include_recipe "sudo"
include_recipe "ntp"
include_recipe "bc-nrpe"
include_recipe "bc-statsd"
include_recipe "bc-chef-client"

bc-location:
------------
metadata:
depends 'bc-dnsmasq'

Berksfile:
(nothing)

recipes/default.rb
(no includes)

bc-apt:
------
metadata:
depends "bc-location"
depends "apt",                  "2.1.1"

Berksfile:
cookbook 'bc-location', git: ' :chef/bc-location.git'

recipes/default.rb
(no includes)


ERROR:

[2014-03-27T16:38:18+00:00] INFO: Forking chef instance to converge...
[2014-03-27T16:38:18+00:00] INFO: *** Chef 11.6.0 ***
[2014-03-27T16:38:18+00:00] INFO: Setting the run_list to ["recipe[bc-role-downloader]"] from JSON
[2014-03-27T16:38:18+00:00] INFO: Run List is [recipe[bc-role-downloader]]
[2014-03-27T16:38:18+00:00] INFO: Run List expands to [bc-role-downloader]
[2014-03-27T16:38:18+00:00] INFO: Starting Chef Run for test-co.foocom.com
[2014-03-27T16:38:18+00:00] INFO: Running start handlers
[2014-03-27T16:38:18+00:00] INFO: Start handlers complete.
+++++ HERE I AM

================================================================================
Recipe Compile Error in /tmp/vagrant-chef-1/chef-solo-1/cookbooks/bc-role-downloader/recipes/default.rb
================================================================================


RuntimeError
------------
The bc-apt attribute is missing. Aborting.


Cookbook Trace:
---------------
  /tmp/vagrant-chef-1/chef-solo-1/cookbooks/bc-apt/recipes/default.rb:4:in `from_file'
  /tmp/vagrant-chef-1/chef-solo-1/cookbooks/bc-foocom-downloader/recipes/default.rb:9:in `from_file'
  /tmp/vagrant-chef-1/chef-solo-1/cookbooks/bc-role-downloader/recipes/default.rb:15:in `from_file'


Relevant File Content:
----------------------
/tmp/vagrant-chef-1/chef-solo-1/cookbooks/bc-apt/recipes/default.rb:

  1:  log "@@@ This is cookbook '#{cookbook_name}', version '#{run_context.cookbook_collection[cookbook_name].version}', environment '#{node.chef_environment}'"
  2:
  3:  if not node.has_key? 'bc-apt' then
  4>>     raise "The bc-apt attribute is missing. Aborting. #{node['bc-location']}"
  5:      #raise "The bc-apt attribute is missing for location #{node['bc-location']['location']}. Aborting."
  6:  end
  7:
  8:  template "/etc/apt/apt.conf.d/02allow_unsigned_pkgs" do
  9:      source "etc/apt/apt.conf.d/02allow_unsigned_pkgs.erb"
 10:      owner "root"
 11:      group "root"
 12:  end
 13:



[2014-03-27T16:38:23+00:00] ERROR: Running exception handlers
[2014-03-27T16:38:23+00:00] ERROR: Exception handlers complete
[2014-03-27T16:38:23+00:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
[2014-03-27T16:38:23+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
Chef never successfully completed! Any errors should be visible in the
output above. Please fix your recipes so that they properly complete.


On Thu, Mar 27, 2014 at 8:18 AM, Gregory Patmore < " target="_blank"> > wrote:
a side note, maybe you know this already, but include_recipe runs a recipe and it should not be confused with dependencies. they are related, but quite different in behavior.

regarding the original question, I guess I'm not clear on what you mean by 'wrapping' a cookbook. I am assuming you mean 'resource' when you say 'cookbook' as I can't think of a reason to adjust everything within a cookbook. I apologize if this is erroneous.

Our squad implements layers of abstraction all the time on top of 3rd party cookbook resources as well as core chef resources. It allows us to adjust the behavior of the resources for specific, custom requirements without having to change the base resource that is likely acceptable 99% of the time.

However, what we do would be more accurately described as a proxy or composition pattern vs inheritance as the term 'wrapping' could suggest. Assuming adherence to standard OOP best practices, extending objects within chef land usually seems to lead to increased maintenance overhead and less flexibility over the long term, whereas defining accessor type resources that use the 'wrapped' resources and introduce controlled changes to behavior or more commonly, to abstract additional conventions established by your project and outside the scope of the original objects consideration, leads to higher agility and flexibility when those conventions change and/or a new underlying resource becomes desirable.

We strictly avoid the use of include_recipe unless exhaustively justified.  Cookbook dependencies are necessary, but should only be established via a cookbook's metadata file. If we proxy (wrap) a resource we try to wrap the entirety of the resource's interface, even if it just passes along the call to the proxied resource. this alleviates any need to use both, and allows us to implement additional log feedback and/or adjusted behavior.

The application cookbook is likely the hardest one to deal with as it was intended as what one might know as an 'abstract' class/object/resource; intended to require extending to be of any actual use. Normally one doesn't see this resource being used directly but rather used via a more specific application derivative such as application_python or application_java. So some quick research on how these objects are written would likely be the best source for how to write a new wrapper for the application, assuming you can't find an extension that more closely suits your requirements.

I hope this helps you sort it out some
Greg



> On Mar 26, 2014, at 6:10 PM, Douglas Garstang < "> > wrote:
>
> I'm having issues with wrapper cookbooks. The setup is pretty complex, so I'm not even quite sure how to describe it here in a way that makes sense.
>
> Generally speaking though, I have a wrapper cookbook that wraps an application cookbook. That application cookbook includes other cookbooks. Something along the way, a location cookbook is included that calculates and sets some attributes.
>
> Everything up to and including the application cookbook tests fine with vagrant. However, when I try and wrap that, it fails to find attributes previously set in the location cookbook. All I should need in the application wrapper is a single line for the application cookbook in the metadata, the Berksfile and and include_recipe, right?
>
> The application wrapper cookbook should not need to reference anything except the cookbook it is wrapping. If I put an include_recipe "location" in there, it works. I don't understand this. I'm also very confused about the fact that if I don't mention the location cookbook in the metadata or the Berksfile, a vagrant up still shows it as grabbing it from git. Not sure if that's from a dependancy, or directly.
>
> Help appreciated.
>
> Doug.



--
Regards,

Douglas Garstang
http://www.linkedin.com/in/garstang
Email: ">
Cell: +1-805-340-5627



Archive powered by MHonArc 2.6.16.

§