[chef] Re: Re: Re: Re: Re: Re: Re: Introducing the httpd cookbook

[Sean OMeara < >]

We're in violent agreement here.

The trick to re-usability is to separate what the consumer needs
control over (usually a configuration file) from what they don't care
about (directory permissions, package installations, etc).

The http_service provider(s) handle getting the service up and
listening, then completely offloads the meaningful configuration
details to the user. The resource *does* write a small configuration
file, which tweaks a very minimal amount of things (listen ports,
etc), but its main purpose in life is to include files from the conf.d
directory.

In the examples I found on the FB Github, the Phil has created an
"open ended" API that lets users encode a data structure in node
attributes. Then they're all consumed by a template resource. This is
a configuration-file-centric view of the world.

https://github.com/facebook/chef-utils/blob/master/cookbooks/fb_cron/recipes/default.rb#L60-L65

The other resource in the recipes are "hard wired" by the cookbook,
since they're not being driven by node attributes. The minimal amount
of cross-platform-ness represented in that example is already starting
to sprawl. There's a giant case statement at the top, and various
control flow logic around platform_family.

Most of the community cookbooks out there start out looking simple and
clean... when they're written for a single version of a single
platform. Then, over time, they descend into the pit of despair that
is case statements for package names, service names, init systems, and
filesystem paths. Driving all those details "open ended" will lead to

A Chef resource (implemented with the LWRP DSL or otherwise), creates
a "closure" around its subject.

This lets you everything the example recipe *other than the template
resource*, and shove it behind a wall of implementation detail. This
lets an "httpd_service" become cross platform while hiding all the
crazy platform details.

The user can then can drop off configuration via httpd_config, which
is just a thin wrapper around the template resource that calculates a
file path based on the instance name. The same attribute tricks can be
used around the contents of the template.

-s


On Sat, Sep 6, 2014 at 9:37 PM, Phil Dibowitz 
<
 >
 wrote:
> On 09/06/2014 06:17 AM, Bráulio Bhavamitra wrote:
>> Thanks Phil for excelent discussion and examples of how to use node
>> attributes.
>>
>> When LWRP are coded, how do you think both ways to configure can the
>> connected?
>
> Well I think that LWRPs are often the wrong way to provide an API. Not 
> because
> LWRPs are themselves bad - they're awesome - but for the same reason we 
> don't
> use most of the standard built-in providers. We don't use 'cron' for 
> example,
> we built a template and provide a data-api as I described in the last email.
> You can see this here:
>
>   https://github.com/facebook/chef-utils/tree/master/cookbooks/fb_cron
>
> But there are times when an LWRP is what you need. The most common thing we
> use LWRPs for is when the data we're pulling from the node object cannot be
> put in a single file, and represents some dynamic set of files - and we 
> won't
> know what that set of files is until after everyone has munged the node
> object. We avoid this wherever possible, because it makes system-level
> idempotency more difficult, but sometimes you don't have a choice. So the
> example here might be if the example software I was using *required* every 
> vip
> to be in a different file. Now again - often times it's how they set it up,
> but it's not required. But lets just say it was absolutely required.
>
> In this modified example the software requires /etc/foo/foo.conf and then 
> each
> site goes in a file like /etc/foo/vips/www.sample.com.conf. And lets assume,
> just to make it more interesting that the main configuration doesn't take 
> that
> "[global]" header but the other ones do so they know the site name.
>
> In this case I'd tweak the attributes slightly and put all the VIPs under
> their own section:
>
>   default['foo']['config_file'] = '/etc/foo/foo.conf'
>   default['foo']['config']['global'] => {
>     'logdir' => '/var/log/foo',
>     'loglevel' => 'DEBUG',
>   }
>   default['foo']['config']['vips']['www.sample.com'] => {
>     'root' => '/var/foo/sample.com',
>   }
>
> Then I'd the recipe to do:
>
>   template '/etc/foo/foo.conf' do
>     source 'foo.conf.erb'
>     owner 'root'
>     group 'root'
>     mode '0644'
>   end
>   foo_gen_site_files 'doit'
>
> And that would call a 'gen_site_files' LWRP. Notice I'm not passing anything
> to it and notice it's not meant for anyone else to call. Now inside the
> provider I'd have:
>
>   action :run do
>     node['foo']['config']['vips'].keys do |url|
>       template "/etc/foo/vips/#{url}.conf" do
>         source 'vip.conf.erb'
>         user 'root'
>         group 'root'
>         mode '0644'
>         variables({'site' => url})
>       end
>     end
>
>     # Be idempotent - clean up crap we no longer own
>     # We have to do this because Foo is too dumb to allow
>     # all sites in a single file. I hate Foo.
>     Dir.glob('/etc/foo/vips/*.conf').each do |cfile|
>       site = File.basename(cfile, '.conf')
>       next if node['foo']['config']['vips'][site]
>       file cfile do
>         action :delete
>       end
>     end
>   end
>
> And actually since all this is doing is using built-in resources I'd 
> actually
> use notifying_action so I didn't have to deal with notifications myself... 
> but
> that's orthogonal to this point.
>
> The vip.conf.erb template would now look like:
>
>   <% config = 
> node['foo']['config']['vips'
 
>  %>
>   [<%= @site %>]
>   <% config.each do |key, val| %>
>   <%=  key %> = <%= val %>
>   <% end %>
>
> And the foo.conf.erb template would now look like:
>
>   <% config = node['foo']'config']['global'].to_hash %>
>   <% config.each do |key, val| %>
>   <%=  key %> = <%= val %>
>   <% end %>
>
> There's a a few things to note here: variables to templates are evil and
> should be avoided at all costs... rather use the node object from within 
> your
> template. In this case, I passed in static data that told the template where
> to look in the node object and that's it.
>
> The other use-case we have for LWRPs is in the relatively-rare case where we
> have cookbooks that allow you to both enable *or* disable something. We do
> this with autofs, so our fb_autofs cookbook's default recipe is included
> everywhere and looks like this:
>
>   fb_autofs 'enable it' do
>     only_if { node['fb']['fb_autofs']['enable'] }
>     action :enable
>   end
>   fb_autofs 'disable it' do
>     not_if { node['fb']['fb_autofs']['enable'] }
>     action :disable
>   end
>
> Since 'lazy' doesn't work on actions, we need two invocations of the LWRP
> using only_if and not_if which are always lazily-evaluated.
>
> The final use-case at FB for LWRPs is basically just a modification of the
> first one in this email. When we want to deal with things like hardware or
> mounts or things that require a complicated set of running command, looking 
> at
> output, running more commands. But even in these cases it works like the 
> first
> example above - the LWRP is just scheduled by it's own cookbook and it's
> *purely* configured by tweaking the node object and it looks at that at
> runtime inside the LWRP to decide what to do. This is basically just
> abstracting what would otherwise be a long ruby_block or some such with
> bunches of Mixlib::ShellOut calls.
>
>
> --
> Phil Dibowitz                             
> 
 
> Open Source software and tech docs        Insanity Palace of Metallica
> http://www.phildev.net/                   http://www.ipom.com/
>
> "Be who you are and say what you feel, because those who mind don't matter
>  and those who matter don't mind."
>  - Dr. Seuss
>
>