[chef] Re: Re: Re: Re: Re: Re: Re: Introducing the httpd cookbook

[Sean OMeara < >]

<edit>.... "lead madness" =)
-s

On Sat, Sep 6, 2014 at 11:15 PM, Sean OMeara 
<
 >
 wrote:
> We're in violent agreement here.
>
> The trick to re-usability is to separate what the consumer needs
> control over (usually a configuration file) from what they don't care
> about (directory permissions, package installations, etc).
>
> The http_service provider(s) handle getting the service up and
> listening, then completely offloads the meaningful configuration
> details to the user. The resource *does* write a small configuration
> file, which tweaks a very minimal amount of things (listen ports,
> etc), but its main purpose in life is to include files from the conf.d
> directory.
>
> In the examples I found on the FB Github, the Phil has created an
> "open ended" API that lets users encode a data structure in node
> attributes. Then they're all consumed by a template resource. This is
> a configuration-file-centric view of the world.
>
> https://github.com/facebook/chef-utils/blob/master/cookbooks/fb_cron/recipes/default.rb#L60-L65
>
> The other resource in the recipes are "hard wired" by the cookbook,
> since they're not being driven by node attributes. The minimal amount
> of cross-platform-ness represented in that example is already starting
> to sprawl. There's a giant case statement at the top, and various
> control flow logic around platform_family.
>
> Most of the community cookbooks out there start out looking simple and
> clean... when they're written for a single version of a single
> platform. Then, over time, they descend into the pit of despair that
> is case statements for package names, service names, init systems, and
> filesystem paths. Driving all those details "open ended" will lead to
>
> A Chef resource (implemented with the LWRP DSL or otherwise), creates
> a "closure" around its subject.
>
> This lets you everything the example recipe *other than the template
> resource*, and shove it behind a wall of implementation detail. This
> lets an "httpd_service" become cross platform while hiding all the
> crazy platform details.
>
> The user can then can drop off configuration via httpd_config, which
> is just a thin wrapper around the template resource that calculates a
> file path based on the instance name. The same attribute tricks can be
> used around the contents of the template.
>
> -s
>
>
> On Sat, Sep 6, 2014 at 9:37 PM, Phil Dibowitz 
> <
 >
>  wrote:
>> On 09/06/2014 06:17 AM, Bráulio Bhavamitra wrote:
>>> Thanks Phil for excelent discussion and examples of how to use node
>>> attributes.
>>>
>>> When LWRP are coded, how do you think both ways to configure can the
>>> connected?
>>
>> Well I think that LWRPs are often the wrong way to provide an API. Not 
>> because
>> LWRPs are themselves bad - they're awesome - but for the same reason we 
>> don't
>> use most of the standard built-in providers. We don't use 'cron' for 
>> example,
>> we built a template and provide a data-api as I described in the last 
>> email.
>> You can see this here:
>>
>>   https://github.com/facebook/chef-utils/tree/master/cookbooks/fb_cron
>>
>> But there are times when an LWRP is what you need. The most common thing we
>> use LWRPs for is when the data we're pulling from the node object cannot be
>> put in a single file, and represents some dynamic set of files - and we 
>> won't
>> know what that set of files is until after everyone has munged the node
>> object. We avoid this wherever possible, because it makes system-level
>> idempotency more difficult, but sometimes you don't have a choice. So the
>> example here might be if the example software I was using *required* every 
>> vip
>> to be in a different file. Now again - often times it's how they set it up,
>> but it's not required. But lets just say it was absolutely required.
>>
>> In this modified example the software requires /etc/foo/foo.conf and then 
>> each
>> site goes in a file like /etc/foo/vips/www.sample.com.conf. And lets 
>> assume,
>> just to make it more interesting that the main configuration doesn't take 
>> that
>> "[global]" header but the other ones do so they know the site name.
>>
>> In this case I'd tweak the attributes slightly and put all the VIPs under
>> their own section:
>>
>>   default['foo']['config_file'] = '/etc/foo/foo.conf'
>>   default['foo']['config']['global'] => {
>>     'logdir' => '/var/log/foo',
>>     'loglevel' => 'DEBUG',
>>   }
>>   default['foo']['config']['vips']['www.sample.com'] => {
>>     'root' => '/var/foo/sample.com',
>>   }
>>
>> Then I'd the recipe to do:
>>
>>   template '/etc/foo/foo.conf' do
>>     source 'foo.conf.erb'
>>     owner 'root'
>>     group 'root'
>>     mode '0644'
>>   end
>>   foo_gen_site_files 'doit'
>>
>> And that would call a 'gen_site_files' LWRP. Notice I'm not passing 
>> anything
>> to it and notice it's not meant for anyone else to call. Now inside the
>> provider I'd have:
>>
>>   action :run do
>>     node['foo']['config']['vips'].keys do |url|
>>       template "/etc/foo/vips/#{url}.conf" do
>>         source 'vip.conf.erb'
>>         user 'root'
>>         group 'root'
>>         mode '0644'
>>         variables({'site' => url})
>>       end
>>     end
>>
>>     # Be idempotent - clean up crap we no longer own
>>     # We have to do this because Foo is too dumb to allow
>>     # all sites in a single file. I hate Foo.
>>     Dir.glob('/etc/foo/vips/*.conf').each do |cfile|
>>       site = File.basename(cfile, '.conf')
>>       next if node['foo']['config']['vips'][site]
>>       file cfile do
>>         action :delete
>>       end
>>     end
>>   end
>>
>> And actually since all this is doing is using built-in resources I'd 
>> actually
>> use notifying_action so I didn't have to deal with notifications myself... 
>> but
>> that's orthogonal to this point.
>>
>> The vip.conf.erb template would now look like:
>>
>>   <% config = 
>> node['foo']['config']['vips'
 
>>  %>
>>   [<%= @site %>]
>>   <% config.each do |key, val| %>
>>   <%=  key %> = <%= val %>
>>   <% end %>
>>
>> And the foo.conf.erb template would now look like:
>>
>>   <% config = node['foo']'config']['global'].to_hash %>
>>   <% config.each do |key, val| %>
>>   <%=  key %> = <%= val %>
>>   <% end %>
>>
>> There's a a few things to note here: variables to templates are evil and
>> should be avoided at all costs... rather use the node object from within 
>> your
>> template. In this case, I passed in static data that told the template 
>> where
>> to look in the node object and that's it.
>>
>> The other use-case we have for LWRPs is in the relatively-rare case where 
>> we
>> have cookbooks that allow you to both enable *or* disable something. We do
>> this with autofs, so our fb_autofs cookbook's default recipe is included
>> everywhere and looks like this:
>>
>>   fb_autofs 'enable it' do
>>     only_if { node['fb']['fb_autofs']['enable'] }
>>     action :enable
>>   end
>>   fb_autofs 'disable it' do
>>     not_if { node['fb']['fb_autofs']['enable'] }
>>     action :disable
>>   end
>>
>> Since 'lazy' doesn't work on actions, we need two invocations of the LWRP
>> using only_if and not_if which are always lazily-evaluated.
>>
>> The final use-case at FB for LWRPs is basically just a modification of the
>> first one in this email. When we want to deal with things like hardware or
>> mounts or things that require a complicated set of running command, 
>> looking at
>> output, running more commands. But even in these cases it works like the 
>> first
>> example above - the LWRP is just scheduled by it's own cookbook and it's
>> *purely* configured by tweaking the node object and it looks at that at
>> runtime inside the LWRP to decide what to do. This is basically just
>> abstracting what would otherwise be a long ruby_block or some such with
>> bunches of Mixlib::ShellOut calls.
>>
>>
>> --
>> Phil Dibowitz                             
>> 
 
>> Open Source software and tech docs        Insanity Palace of Metallica
>> http://www.phildev.net/                   http://www.ipom.com/
>>
>> "Be who you are and say what you feel, because those who mind don't matter
>>  and those who matter don't mind."
>>  - Dr. Seuss
>>
>>