[chef-dev] Re: Best practice for cross platform cookbook dependencies?

[Joshua Timberman < >]

Ohai,

I agree that it is annoying, and frustrating for the feedback cycle to take 
longer than is really necessary. However, I think having cookbooks' default 
use pattern broken is even more annoying.

In the ease of use department, Chef already has a fairly steep learning 
curve. New users to Chef often have to learn:

* A new paradigm for managing systems (declarative, convergent resources).
* New languages for defining their infrastructure (Ruby, JSON).
* Manage things through an API instead of text files.
* A new authentication system that is similar, but not exactly the same as 
one(s) they know.
* Several new command line tools with their own configuration files.
* A dependency management system to distribute automation components to their 
systems.

The simple example or demonstration is often magical and "just works." But a 
single change can cause things to break in spectacular ways. Sometimes you 
get a nice clear error, like "this resource wasn't found (because you typed 
tumplate instead of template)." But errors like, "Undefined method `[]' on 
NilClass" because you didn't have a dependency on a cookbook that had an 
attribute is really confusing.

By adding stronger dependencies, we can often remove these confusing errors 
for the default use case. Documentation on cookbook usage should be up to 
date with how you actually use the thing, but let's face it, in our busy 
schedules not everyone is going to have the time to read 5 paragraphs of new 
jargon to get something working.

For Opscode's cookbooks in particular, we try to remove as many dependencies 
as possible for the *default use case* for a particular cookbook. This means 
for example, we have statements like this:

    package "perl"

In a recipe, rather than incuding another cookbook as a dependency. We also 
try to document dependencies when the default use case doesn't require them. 
For example, if you're using chef-client::service, it can set up services 
using a variety of init styles. You have to know which ones it supports (like 
bluepill or runit), and then change the appropriate attribute. The reasoning 
to not have these as dependencies is because you'd want those cookbooks 
elsewhere in your node's run list, such as in a "base"/"common" role or 
similar.

Unfortunately we haven't been consistent across all our own cookbooks. I know 
it's annoying and I'm sorry about that.

This is an unsolved problem until one or all of the following are solved:

* suggests/recommends have a behavior (configurable, ideally) which allows 
them to be downloaded. Right now they do *nothing* and only serve as loose 
documentation.
* per-platform dependency mapping, so CentOS doesn't get "windows" or Ubuntu 
doesn't get "yum", for example.

I was unable to find a ticket on these topics, so I opened these:

* http://tickets.opscode.com/browse/CHEF-3870
* http://tickets.opscode.com/browse/CHEF-3871

Cheers,
Joshua


On Feb 11, 2013, at 2:00 PM, Cassiano Leal 
<
 >
 wrote:

> I'd have to say yes.
> 
> My workflow involves running Berkshelf and Vagrant VMs to develop and test 
> cookbooks.
> 
> The more dependencies a cookbook I use have, the longer it takes for me to 
> iterate. Everytime I "vagrant up" or "vagrant provision", every single 
> cookbook that'll be used for that session gets loaded on the system, 
> uploaded to the Chef Server and downloaded in the VM.
> 
> I like the way the apache2 cookbook [0] deals with that: no direct 
> dependencies, default recipe doesn't need any. If you load a recipe with an 
> external dependency, make sure to include whatever recipe you need from 
> that on the node (like it does with logrotate, for example).
> 
> I'm not sure that applies to the nodejs cookbook, but it surely is an 
> annoyance that should be fixed somehow.
> 
> I feel the same about having to load up apt cookbooks onto CentOS/RHEL or 
> Windows nodes.
> 
> Even though the inclusion of unused cookbooks brings no harm in terms of 
> cuntionality, it does slow down both cookbook development and node 
> conversion.
> 
> - cassiano
> 
> On Monday, February 11, 2013 at 16:57, Mike wrote:
> 
>> The dependency manifests as files located on the filesystem.
>> I don't think there's any other problems, are there?
>> 
>> On Mon, Feb 11, 2013 at 1:53 PM, Peter Loron 
>> <
 >
>>  wrote:
>>> I'm working on extending the existing nodejs cookbook to support installs 
>>> on Windows. This brings up a bit of a conundrum…if I do this, I'm going 
>>> to take a dependency hit on the "windows" cookbook. This will cause 
>>> non-windows shops to now have another cookbook they don't need. On the 
>>> flip side, windows-only shops will have the useless to them "apt" 
>>> cookbook.
>>> 
>>> Is there a solution that the community prefers? I could roll this out as 
>>> a separate cookbook, but I would prefer to keep it all in one. Just eat 
>>> the extra dependency overhead?
>>> 
>>> Thanks.
>>> 
>>> -Pete
> 

-- 
Joshua Timberman
Technical Community Manager, Opscode, Inc
IRC, Skype, Twitter, Github: jtimberman