[chef] Re: Cookbook Management for Complex Infrastructure


Chronological Thread 
  • From: Jeff Byrnes < >
  • To:
  • Subject: [chef] Re: Cookbook Management for Complex Infrastructure
  • Date: Tue, 21 Jul 2015 16:47:05 -0400

Ugh, for whatever reason, did not see all of the replies to this already. Sorry for the doubling up!

-- 
Jeff Byrnes
@thejeffbyrnes
Lead DevOps Engineer
704.516.4628

On July 21, 2015 at 4:46:19 PM, Jeff Byrnes ( "> ) wrote:

While we solved this with separate repositories/projects per cookbook, I don’t see why you can’t store them in a single, monolithic, repo, together.

I would suggest that your path to solve this is to use your Chef Server, combined with Berkshelf and a Berkshelf API Server (which can run alongside the Chef Server), to handle the dependency management.

Here’s an example Berksfile that, in addition to using the community Berkshelf endpoint (i.e., Supermarket), also uses another one:

source 'https://supermarket.chef.io'
source 'https://berks.compant.com'

metadata

In this way you could have a private cookbook that, using Berks, you upload to your Chef Server. From there, you can then, in another cookbook, have that one as a dependency and, again using Berks, resolve that dependency via your private Berkshelf API.

Thankfully, there is a cookbook to set up the Berkshelf API server: https://supermarket.chef.io/cookbooks/berkshelf-api-server (there are two others, but this is the one we’ve been using successfully, and is also the “official” one that’s part of the Berkshelf API project).

-- 
Jeff Byrnes
@thejeffbyrnes
Lead DevOps Engineer
704.516.4628

On July 20, 2015 at 4:48:03 PM, Erik Ogan ( "> ) wrote:

Background:
We currently have several hundred nodes managed by Chef 0.10(.4) on a private Chef server. We are using librarian-chef to manage off-the-shelf cookbooks (~30 or so), and have a single repository for the remaining custom cookbooks (>60) [*]

I am looking to modernize this setup. It is likely[**] to be a clean-slate rebuild with Chef 12, pulling in cookbooks (more likely portions of recipes) as needed. This is also a chance to align our processes with the state-of-the-art thinking on how to manage these resources.

Berkshelf is the current, prescribed tool for managing cookbooks. Berkshelf (appears to) require each cookbook in its own repository. This is great for modularity, it forces a separation of concerns, but while I plan to reduce the number of custom, private cookbooks (removing cruft, replacing some with off-the-shelf cookbooks configured with attributes), I still expect to have enough cookbooks that this would consume all of our allotment of private repositories.

In light of that, I have tried to set up a repository with cookbooks as peers inside, but the more hoops I jump through to make that work, the more sure I become that I’m missing something obvious in the way to manage cookbooks. (Especially right now where I cannot test a custom cookbook with chefspec if it depends on another custom/private cookbook).

So, how do larger installations manage their private cookbooks? What have I missed?

-e

[*]  I don’t know if that constitutes “large” in the community, but I’m comfortable saying there’s a fair bit of complexity to it.

[**] For reasons that are way out of scope of this message.

--



Archive powered by MHonArc 2.6.16.

§