[chef-dev] Re: dependency spaghetti


Chronological Thread 
  • From: Lamont Granquist < >
  • To:
  • Subject: [chef-dev] Re: dependency spaghetti
  • Date: Wed, 02 Oct 2013 17:30:09 -0700


The 'suggests' and 'recommends' features of metadata are NOPs, they don't do anything. And the java cookbook is broken on windows without the windows cookbook -- so in fixing your pain you would break all the windows people. The problem is that the list of cookbooks to download is computed server-side by the dep-solver and it doesn't have visibility into the platform the client is running on. The client could send that information to the server and the server could use it to filter the results, or server could return enough metadata in response to cookbook_versions that the client could filter the cookbooks itself, or the client could do additional requests for the cookbook metadata and do the filtering (but that means the additional round trips that you don't want), etc. But the problem is deeper than just switching the metadata field.

We should also be doing downloads in parallel (https://tickets.opscode.com/browse/CHEF-4423). That patch badly needs some careful tests written for it, though, because we need to prove that typhoeus works across our whole test matrix - and if it fails somewhere like ruby 1.8 or AIX then it would need to get rewritten with net-http and threads like dan suggested...

We also have issues where due to eventual consistency issues between S3 regions we force everything to talk to the US-East S3 region which causes large delays in Europe and Asia.

On 10/2/13 4:17 PM, Blake Irvin wrote:
(Please redirect me if this is the wrong list)

We seem to be in a place as a Chef community where the 'depends' feature of cookbook metadata is getting used a little bit too freely.

For example, the 'java' cookbook includes a 'depends' for the 'windows' cookbook - this means that I'm forced to download the 'windows' cookbook to all of my *nix machines. Messy at best.

Shouldn't we be using 'suggests'? Or better yet, shouldn't we reserve dependency for cases where a cookbook really *must* have another cookbook available to it to compile (and, shouldn't we avoid tightly-coupled dependencies whenever possible because we don't wan to be brittle)?

This is especially painful for us because we are currently seeing very slow download rates from hosted Chef (about 60kbps), so downloading addt'l cookbooks makes things noticeably slower.

(Another great example of dependency hell is installing the 'nagios' cookbook, which requires 'apache', which requires 'php', which in turn requires 'mysql' - yet it's not really true that all of these things are intrinsically tied together - we are losing the things we've learned from the Unix model and service-oriented architectures when we staple things to each other this way.)

Okay, so that got a little bit ranty, but that's because I really like Chef and I hate to see a slim, sexy, powerful tool get bloated and brittle for no good reason.

Discuss!


Blake




Archive powered by MHonArc 2.6.16.

§