[chef] Run lists and dependencies (Was: chef-client , apply role)

[Josiah Kiehl < >]

(changed the subject to fork the conversation)

In the described circumstance, I still want things that must exist on the box for the application to run to be on the dependency list, not the run list.

Why not just use an LWRP in each cookbook to add file system mounts as opposed to building attributes and praying someone else mounts what you need? Alternately you can build a list of attributes across all your cookbooks during compile time and use those attributes during convergence when you perform a mount. Separating requirements like you describe adds another possibility of failure if the run list changes between versions, that being forgetting to set the run list during an upgrade. When managing multiple data centers with many engineers touching deployments, forgetting that is a very real possibility, even with run books. Further, run lists are not versioned so if you ever make a mistake, it's up to you to remember what the run list needed to look like, more than just remembering what component lives on that box.

In my opinion, multiple entries on a run list means multiple applications are running on that node. A run list entry should mean that the entirety of what is required for that app to start and function in a default state will be deployed. Any change to that default state should be set in attributes in the environment or on the node.

On Fri, Jan 3, 2014 at 6:01 AM, Dylan Northrup < " target="_blank"> > wrote:

Not to digress too much, but I use explicit dependencies from metadata.rb in individual cookbooks (and, less often, roles and environments... at least until they are able to be versioned) when they're required by the app/service.  But for other, more general "things that should be on a host" cookbooks, those may gather information from other cookbooks in the run list to determine how to proceed.  There's an implicit dependency in those more general cookbooks that guides my suggestion to run a full chef-client run including every cookbook in the run_list.

As an example, let's say I've got multiple apps with multiple development groups working on those individual apps.  Apps are set up using the individual cookbooks, but I want to get user accounts, sudoers, etc set up using a single 'users' cookbook.  The app cookbooks can set node attributes that will be read by the users cookbook to determine what user accounts to put on the node, who to put in sudoers, etc.  If I have a single cookbook for "mount external file systems", it's conceivable individual cookbooks would be registering remote file systems to mount and a separate cookbook in the run list (either explicitly or included) would do the actual mounting of all remote file systems.

If you flip this around and say each app cookbook should manage file system mounts (something reasonable, I'll admit, given the typically small number of file system mounts) or adding users (something that quickly gets out of hand given any reasonably sized shop) that quickly becomes a maintenance/ops nightmare and possible security issue since there's no way to insure users are being added correctly, file system mount options are standardized, etc.  Being able to specify requests from multiple different locations, but have those requests acted upon in a single location allows for flexibility while maintaining ease of updates and increasing ease of debugging.

Digression regarding dependencies aside, I think the idea of only running a single cookbook on a node is something that was not intended.  If you only want a single cookbook to be run on a node, then that's the only thing that should be in the run_list, because next time chef-client runs, it'll run everything in the run_list anyway.  If you want to make sure your results after adding a cookbook to the run_list, you'll need to have the chef-client run execute every cookbook if you want to make sure your test results will be consistent with what will eventually happen with production.

But that's just my opinion.  I could be wrong.

On Fri, Jan 3, 2014 at 2:14 AM, Josiah Kiehl < " target="_blank"> > wrote:

If you have dependencies on other entries on your run list, you're using run lists in a way they are not intended to be used.

Dependencies should be pulled in via "depends" not via run list entries.

On Thu, Jan 2, 2014 at 6:18 AM, Dylan Northrup < " target="_blank"> > wrote:

If you don't run the entire run_list, you may be missing attributes that should be needed during the chef-client run.  As an example, if your run_list were 'wrapper' cookbooks (that contain attributes, and then call other cookbooks), if you had any dependency on those attributes (such as "what data center is this node in?", "is this node dev, ref or prod?", "what service is this node providing?", etc) you wouldn't have access to them.

I'd suggest that there's likely something interesting you're trying to do, but the way you're trying to go about it will cause more headaches down the road than you think and there is likely a better way to go about what you're wanting to do.  If you want to simply update node attributes, knife node edit is one way (though not the easiest to automate).  A quick google search shows 'knife-set-attribute' at https://github.com/amian84/knife-set-attribute may be another way (though I've not used it, so YMMV).

But these are still methods of changing the model on the chef server and you'll still need to do a full chef-client run on the client to render the change(s) on the node.  And, in the end, doing the full chef-client run is the best practice from a reliability standpoint.  When making a change you want to make sure it's fully and completely rendered onto a node the way it will be every time chef-client will run in the future.  If it doesn't, there's a pretty large opportunity for you to be surprised after the next automated chef-client run when something goes boom because of that attribute change 'that simply updated one thing that was ephemeral and should have little effect'.

But that's just my opinion.  I could be wrong.

On Thu, Jan 2, 2014 at 5:28 AM, Sam Darwin < " target="_blank"> > wrote:

is there a way to run:

chef-client -o "role[abc]"

that is, to apply a role without running the entire run_list ?

this should not be ephemeral and have little effect... it should cause all the
results that having the role in the real run_list would have, update the
chef-server with attributes in a persistent way and everything else.     I
would also add the role to the real run_list .     But I don't want to run all
recipes on the client at that moment.