[chef-dev] Re: dependency spaghetti

[Lamont Granquist < >]

The 'suggests' and 'recommends' features of metadata are NOPs, they 
don't do anything.  And the java cookbook is broken on windows without 
the windows cookbook -- so in fixing your pain you would break all the 
windows people.  The problem is that the list of cookbooks to download 
is computed server-side by the dep-solver and it doesn't have visibility 
into the platform the client is running on.  The client could send that 
information to the server and the server could use it to filter the 
results, or server could return enough metadata in response to 
cookbook_versions that the client could filter the cookbooks itself, or 
the client could do additional requests for the cookbook metadata and do 
the filtering (but that means the additional round trips that you don't 
want), etc.  But the problem is deeper than just switching the metadata 
field.

We should also be doing downloads in parallel 
(https://tickets.opscode.com/browse/CHEF-4423).   That patch badly needs 
some careful tests written for it, though, because we need to prove that 
typhoeus works across our whole test matrix - and if it fails somewhere 
like ruby 1.8 or AIX then it would need to get rewritten with net-http 
and threads like dan suggested...

We also have issues where due to eventual consistency issues between S3 
regions we force everything to talk to the US-East S3 region which 
causes large delays in Europe and Asia.

On 10/2/13 4:17 PM, Blake Irvin wrote:
> (Please redirect me if this is the wrong list)
>
> We seem to be in a place as a Chef community where the 'depends' 
> feature of cookbook metadata is getting used a little bit too freely.
>
> For example, the 'java' cookbook includes a 'depends' for the 
> 'windows' cookbook - this means that I'm forced to download the 
> 'windows' cookbook to all of my *nix machines.  Messy at best.
>
> Shouldn't we be using 'suggests'?  Or better yet, shouldn't we reserve 
> dependency for cases where a cookbook really *must* have another 
> cookbook available to it to compile (and, shouldn't we avoid 
> tightly-coupled dependencies whenever possible because we don't wan to 
> be brittle)?
>
> This is especially painful for us because we are currently seeing very 
> slow download rates from hosted Chef (about 60kbps), so downloading 
> addt'l cookbooks makes things noticeably slower.
>
> (Another great example of dependency hell is installing the 'nagios' 
> cookbook, which requires 'apache', which requires 'php', which in turn 
> requires 'mysql' - yet it's not really true that all of these things 
> are intrinsically tied together - we are losing the things we've 
> learned from the Unix model and service-oriented architectures when we 
> staple things to each other this way.)
>
> Okay, so that got a little bit ranty, but that's because I really like 
> Chef and I hate to see a slim, sexy, powerful tool get bloated and 
> brittle for no good reason.
>
> Discuss!
>
>
> Blake