[chef] Re: Re: Best practices: Search dynamically using templates in attributes

[Maxime Brugidou < >]

I understand that it looks ugly to put the search string in node attributes. However I dislike having a separate environment per DC, we have many DCs and I don't want to manage them separately. Prod, preprod, test and dev are logical environments and not related to the physical location (which is a simple ohai attribute to me), and I don't want to add 3 or 4 environments every other month when we open a new DC or update a dozen environments just because we change a value in a logical environment.

I'll check out the discovery cookbook, this looks like a great addition but it requires every cookbook we use to have a dependency on discovery, and we use a log of opscode or other third-party cookbooks that we don't plan to fork at the moment. This is why I am trying to settle on best practices that people would use when they share cookbooks, this would help a lot and maybe - if widely used and agreed upon - could get integrated in Chef itself.

In addition, the discovery cookbook is just a DSL over the search functionality but does not permit any "dynamic" decision. Even if every cookbook I use was using the discovery search, I would have to define a role per DC or something like that.

Right now the only solutions i have is making a search template or adding a site-specific cookbook that will create all the search strings.

On Thu, Jan 31, 2013 at 11:10 AM, Jesse Nelson < " target="_blank"> > wrote:

I prefer convention in this case. I.e. all nodes in an environment, and environments are datacenter specific. So all I have to do is search for "role:my_app AND chef_environment:#{node[:chef_environment]}"  I don't like the idea of putting search strings in node attributes. Just smells bad to me. Alternatively I prefer to write helper libraries for the resolution of where things are.  In fact there is an excellent cookbook/library by the folks @heavywater just for this that i recommend you check out: https://github.com/hw-cookbooks/discovery 

On Thu, Jan 31, 2013 at 1:26 AM, Maxime Brugidou < " target="_blank"> > wrote:

Hi chefs,

For our infrastructure automation, we are currently investigating various Chef design patterns to properly (and maybe dynamically) manage multiple "clusters" (like zookeeper, hadoop, rsyslog...). We are wondering how to define a pattern to search other nodes.

The current state of things in the cookbooks we encountered (from Opscode and others, like OpenStack), is:

1. Specify a role name to search on and optionally set an attribute to search within an environment.

2. Sometimes (for example rsyslog) you can directly specify a search query in the node attributes. This is slightly more flexible but the search is still defined by a static attribute.

All this works fine but sometimes you want to dynamically allocate clusters using various logic (from ohai data like datacenter, or by environment or a combination of attributes). The classic example is if you have N datacenters and don't want to create a role for each datacenter.

One solution would be to create a "site-specific" (or "glue") cookbook that will define dynamically the search string at compile time. This is perfectly OK but adds cookbooks.

Another solution would be to use some sort of template for the search string. Ruby string interpolation, or ERB or whatever would work like:

In attributes/default.rb:

node.default[:my_app][:nodes_search] = 'role:my_app AND location_datacenter:#{node[:location][:datacenter]} AND environment:#{node.chef_environment}'

In recipes/default.rb:

nodes = search(:node, eval("\"#{node[:my_app][:nodes_search]}\""))

Is this pattern reasonable? Or should we stick to the "glue" cookbook way? Or is there anything else we can use?

Thanks,

Maxime