[chef] Re: Re: Re: Re: Re: Re: Re: deep merge - !merge users

[Daniel DeLeo < >]

On Thursday, December 8, 2011 at 9:20 AM, Matt Ray wrote:

> You just described how the firewall and ufw cookbooks work, they
> depend on merging of attributes.
> 
> Thanks,
> Matt Ray
> Senior Technical Evangelist | Opscode Inc.
> 
 
>  
> (mailto:
 )
>  | (512) 731-2218
> Twitter, IRC, GitHub: mattray
> 
> 
> 
> On Thu, Dec 8, 2011 at 4:22 PM, Jay Feldblum 
> <
 
>  
> (mailto:
 )>
>  wrote:
> > Imagine a cookbook for implementing firewall rules, which inspected the 
> > node
> > attributes to discover the rules, and was expected to be configured via
> > roles. One role might want to append a rule, where each rule can include
> > multiple source and destination interfaces, ip-addresses, protocols, port
> > ranges, etc. Another node might want to append another rule, and a third
> > role might want to update the first rule by appending e.g. an additional
> > port range.
> > 
> > Imagine the application or database cookbooks were implemented 
> > differently,
> > where they inspected node attributes rather than searched data bags. The
> > configurations for these are rather complex. Then one role might want to 
> > add
> > a some data, while another role might want to update it; the first role
> > providing blanket/default data, the second role being 
> > application-specific.
> > 
> > Imagine fnichol's chef-rvm cookbook had default attributes that you don't
> > like. You want your role to add the cookbook to the run-list but delete
> > those default attributes, for example, delete the nested attribute that 
> > says
> > ruby vX should be installed by default or gem Y should be installed by
> > default in each ruby.
> > 
> > - Jay Feldblum
First, a little background on attributes and !merge:

There are no plans to remove attribute merging. While it's unfortunate that 
the user interface for attributes is quite complex, the ability to customize 
inputs to recipes based on combinations of roles and (very infrequently, I 
hope) node-specific attributes is required for infrastructures of even modest 
scale.

Attribute merging, as implied by the name, is a generally additive process: 
when merging two Hashes (dicts, JSON "Objects") the keys in the result are 
the set union of the keys in the original Hashes. Likewise, when merging two 
Arrays, you get the set union. 

It's easy to imagine a case where you'd want to *subtract* values, say, 
removing port 80 from an Array specifying which ports httpd should listen on. 
And in fact, Chef's deep merging has some support for this, by inserting 
special "!merge:" values in your attributes.

While one can imagine uses for this "subtractive merging," in practice no one 
uses it, and simply by its existence, the computational and memory 
requirements of the merging algorithm are increased substantially. This is a 
big deal when you're, say, iterating over a large set of nodes returned from 
a search result--a much more common use case.

The alternative existing mechanisms you could use to subtract values from 
attributes (modify the cookbook, use a "proxy" cookbook to forcibly set the 
attributes correctly) aren't very pretty either, but I think they'd at least 
lead you to a more declarative result, and be easier for everyone on your 
team to follow.

-- 
Dan DeLeo