[chef] Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Deep merge in 0.10.10 and 10.12.0

[AJ Christensen < >]

Hello,

On 5 September 2012 11:36, Michael Glenney 
<
 >
 wrote:
> Thanks for all the replies.  I'll have a read through all that but it sounds
> like I'm going to have to reengineer how we supply node data for our haproxy
> configs.  Or stay on 0.10.8 :)   Unfortunately, we don't use roles in our
> organization because they can't be versioned so that's not an option.

I'd suggest using Search for this.

I developed a cookbook some time ago to assist with using Search
directly from recipes (to reduce dire repetition):
- http://ckbk.it/discovery
- https://github.com/hw-cookbooks/discovery/tree/develop

>
> It does seem like a weird thing to get rid off though.  Is there another way
> to clear an attribute and put in new data without that?  Or once a node has
> data that data must live on forever unless you use roles?

You can interact with the node data from the Recipe DSL or Node
Attribute DSL, or well; mostly anywhere before the final node.save is
called during convergence -- I guess technically you could modify the
node in the report handler, and then save again there.. but that's
gross!

You could use Node#delete(key) to delete a key from any of the
attribute classes for the (local) node: normal, default and override,
e.g.:

old_nodes = search(..) # could use discovery_search here
old_nodes.each do |remote_node,params|
  node['haproxy']['some_hash_of_server_data'].delete(remote_node) if
node['haproxy']['some_hash_of_server_data'].key? remote_node
end
node.save

> I suppose the problem ultimately is that we don't use roles.  I just need to
> find a way around this.  Putting this data in data_bags isn't an option
> since nodes can't edit data_bags unless they have admin rights so we only
> use data bags for data that doesn't get updated often or at runtime.

Using this combination of technologies, you may be able to build a
plan-and-commit HAProxy back-end management system with Chef Node data
-- In an Infrastructure I built long ago, we made use of Knife Exec to
perform Search queries to extract the information from the
Infrastructure; this in turn generated a data-bag in JSON form which
was uploaded by a Human control-gate; allowing congruence at the
load-balancer layer dependent upon multiple systems convergence while
tolerating a fluid number of back-ends.

I'm increasingly less fond of Human-gates for this kind of stuff
(updating the source of truth) but sometimes we are useful!

Cheers,

--AJ

>
> MG
>
>
> On Tue, Sep 4, 2012 at 4:18 PM, Daniel DeLeo 
> <
 >
>  wrote:
>>
>> Note that this behavior is intentional:
>>
>> http://tickets.opscode.com/browse/CHEF-2737
>> http://tickets.opscode.com/browse/CHEF-2818
>>
>> Also, knockouts will most likely be removed from Chef 11.0. See:
>>
>> http://lists.opscode.com/sympa/arc/chef/2011-12/msg00079.html
>>
>> --
>> Daniel DeLeo
>>
>> On Tuesday, September 4, 2012 at 4:14 PM, Juanje Ojeda Croissier wrote:
>>
>> Yes, I'm running 10.12.0, but I did it with a role. Maybe that's it.
>>
>> On Wed, Sep 5, 2012 at 12:02 AM, Michael Glenney 
>> <
 >
>> wrote:
>>
>> Hmm. You're running 10.12.0? I just tested again. Started with the
>> following:
>>
>> Node data (with 'knife node show <nodename> -Fj'):
>>
>> "haproxy": {
>> "config": "classroom.cfg.erb",
>> "nodes": {
>> "portal_api": [
>> "2.2.2.2",
>> "3.3.3.3"
>> ]
>> }
>>
>> haproxy config:
>>
>> server portal_api-2.2.2.2 2.2.2.2:443 check port 443 inter 10s rise 4 fall
>> 4
>> server portal_api-3.3.3.3 3.3.3.3:443 check port 443 inter 10s rise 4 fall
>> 4
>>
>> I ran 'chef-client -j /etc/chef/routing_rules.json' with the following in
>> "routing_rules.json":
>>
>> { "haproxy": { "nodes": { "portal_api": [ "!merge:", "10.10.10.10",
>> "11.11.11.11" ] } } }
>>
>>
>> This is what I ended up with:
>>
>> Node data:
>>
>> "haproxy": {
>> "config": "classroom.cfg.erb",
>> "nodes": {
>> "portal_api": [
>> "2.2.2.2",
>> "3.3.3.3",
>> "!merge:",
>> "10.10.10.10",
>> "11.11.11.11"
>> ]
>> }
>>
>> haproxy config:
>>
>> server portal_api-2.2.2.2 2.2.2.2:443 check port 443 inter 10s rise 4 fall
>> 4
>> server portal_api-3.3.3.3 3.3.3.3:443 check port 443 inter 10s rise 4 fall
>> 4
>>
>> server portal_api-!merge: !merge::443 check port 443 inter 10s rise 4 fall
>> 4
>> server portal_api-10.10.10.10 10.10.10.10:443 check port 443 inter 10s
>> rise 4 fall 4
>> server portal_api-11.11.11.11 11.11.11.11:443 check port 443 inter 10s
>> rise 4 fall 4
>>
>> The docs say you can run without the colon now but I'm not seeing a
>> difference. Doesn't work either way.
>>
>> MG
>>
>>
>> On Tue, Sep 4, 2012 at 3:41 PM, Juanje Ojeda Croissier
>> <
 >
>>  wrote:
>>
>>
>> I believe the issue here is the " (without the colon) ". I had some
>> troubles here as well, but it's now working with the colon
>> ("!merge:").
>>
>> Could you test it with colon, please?
>>
>> On Tue, Sep 4, 2012 at 8:27 PM, Michael Glenney 
>> <
 >
>> wrote:
>>
>> I have now verified that putting !merge does not work in 10.12.0 I'll
>> go
>> back an read over the link AJ sent again but it appears as if deep merge
>> is
>> gone or broken.
>>
>> Here's the json I passed to the chef-client run:
>>
>> { "haproxy": { "nodes": { "portal_api": [ "!merge", "4.4.4.4", "5.5.5.5"
>> ] }
>> } }
>>
>> Here's the resulting haproxy config:
>>
>> server portal_api-!merge !merge:443 check port 443 inter 10s rise 4
>> fall 4
>> server portal_api-4.4.4.4 4.4.4.4:443 check port 443 inter 10s rise 4
>> fall
>> 4
>> server portal_api-5.5.5.5 5.5.5.5:443 check port 443 inter 10s rise 4
>> fall
>> 4
>>
>> Not what I was going for obviously.
>>
>> MG
>>
>>
>> On Sun, Sep 2, 2012 at 6:46 PM, Michael Glenney 
>> <
 >
>> wrote:
>>
>>
>> Yeah. Looking at your link it looks like using "!merge" (without the
>> colon) will probably work. just need to test it. That goes along with
>> what
>> it says in the wiki except for the fact that the wiki says it "can" be
>> used
>> instead of saying it must.
>>
>> MG
>>
>> On Sun, Sep 2, 2012 at 3:11 PM, AJ Christensen 
>> <
 >
>> wrote:
>>
>>
>> From DeepMerge mixin class:
>>
>> # Inherited roles use the knockout_prefix array subtraction
>> functionality
>> # This is likely to go away in Chef >= 0.11
>> def role_merge(first, second)
>> first = Mash.new(first) unless first.kind_of?(Mash)
>> second = Mash.new(second) unless second.kind_of?(Mash)
>>
>> DeepMerge.deep_merge(second, first, {:knockout_prefix =>
>> "!merge", :preserve_unmergeables => false})
>> end
>>
>>
>>
>>
>> https://github.com/opscode/chef/blob/master/chef/lib/chef/mixin/deep_merge.rb#L44
>>
>> Maybe there's something useful for you here.
>>
>> I've only once used the attribute knockout functionality -- what is
>> your use case? There may be another simple solution.
>>
>> --AJ
>>
>> On 3 September 2012 09:08, Michael Glenney 
>> <
 >
>> wrote:
>>
>> Weird. For some reason it's not sending me my own. Oh well. Have
>> there
>> been any responses to my question? Because I dont see any of those
>> either
>>
>> Michael Glenney
>> Sent from my iPhone
>>
>> On Sep 2, 2012, at 11:43 AM, Adam Jacob 
>> <
 >
>>  wrote:
>>
>> Nope - I saw it two days ago.
>>
>> Adam
>>
>> On Sun, Sep 2, 2012 at 10:04 AM, Michael Glenney
>> <
 >
>>  wrote:
>>
>> I sent this on friday. Still hasn't made it to my inbox. Am I
>> getting
>> moderated?
>>
>> Michael Glenney
>> Sent from my iPhone
>>
>> On Aug 31, 2012, at 4:08 PM, Michael Glenney
>> <
 >
>> wrote:
>>
>> I had a problem with deep merge not working properly in 10.12.0.
>> Tested in
>> 0.10.10 and it didn't work there either. Instead of replacing my
>> array it
>> was actually putting "!merge:" into my array.
>>
>> After I reverted to 0.10.4 to get my deployment completed I
>> stumbled
>> on this
>> line from http://wiki.opscode.com/display/chef/Deep+Merge:
>>
>> "!merge (without the colon) can be used in Chef >= 0.10.10"
>>
>> I haven't had a chance to go back and test. Will have to wait
>> until
>> Monday.
>> But should that line read:
>>
>> "!merge (without the colon) MUST be used in Chef >= 0.10.10"
>>
>> Thanks,
>>
>> MG
>>
>>
>>
>>
>> --
>> Opscode, Inc.
>> Adam Jacob, Chief Customer Officer
>> T: (206) 619-7151 E: 
>> 
 
>>
>>
>>
>>
>> --
>> Juanje
>>
>> http://about.me/juanje
>>
>>
>>
>>
>> --
>> Juanje
>>
>> http://about.me/juanje
>>
>>
>