On 08/22/2014 06:54 PM, Adam Jacob wrote:
> I look at a little differently. I think, for most people, the promise of
> shareable recipes that encode policy has failed. If you make them flexible
> enough to matter (apaxhe2, say) you have a very complicated beast, when you
> probably only needed 20 lines of it.
>
> I think it may well be that the having resources be the prime unit of re-use,
> rather than recipes, may well be the right abstraction in the end.
So I think that Adam's right in that sharable recipes have failed - but I
don't actually think the concept is broken.
I believe one of the core patterns common in the community yields cookbooks
that end up _either_ not being flexible enough to be general, or being so
cumbersome as to be unusable. But it doesn't have to be this way.
I'm actually planning to have a session on this at summit, but here's the gist.
When we set out to build the base of cookbooks for Facebook, I had confidence
the overall approach would be useful to other organizations, but I didn't know
how re-usable the cookbooks themselves would be - and I didn't have any clue
how they'd interact with the concept of community cookbooks.
But after spending over a year traveling the world and talking about Chef to
Chef users - and after finally releasing two cookbooks (more to come), I
actually now believe what we did was make a new way to write community
cookbooks. Our goal in our "core cookbooks," as we call them, was to build
generic simple data-driven APIs that would be flexible enough for tons of
different teams across the company to use to do anything with a given
service/config/thing, without us having to know all the combinations ahead of
time. And that's pretty much, I now believe, what you want in a community
cookbook.
At the moment community cookbooks tend to build APIs that look like this
default['foo']['config_file'] = '/etc/foo'
default['foo']['logdir'] = '/var/log/foo'
default['foo']['loglevel'] = 'DEBUG'
default['foo']['primary_vip']['url'] = 'www.sample.com'
default['foo']['primary_vip']['root'] = '/var/foo/sample.com'
default['foo']['primary_vip']['allow_index'] = false
default['foo']['secondary_vip']['url'] = 'www.bobsyouruncle.com'
default['foo']['secondary_vip']['root'] = '/var/foo/bobsyouruncle.com'
default['foo']['secondary_vip']['allow_index'] = false
This seems like a simple API for this software called 'foo' (which I assume is
vaguely apache-esque in it's config concept, but uses an INI syntax).
Now, each of these is templatized in an otherwise-untemplatized config. This
looks like mostly text, and not code:
[global]
logdir = <%= node['foo']['logdir'] %>
loglevel = <%= node['foo']['loglevel'] %>
[<%= node['foo']['primary_vip']['url'] %>]
root = <%= node['foo']['root'] %>
...
And so on. Now this is a simple cookbook. It only has a few options. But
there's a problem. If Foo 2.0 adds a bunch of options, your cookbook
inherently doesn't support them until you change it. And if someone wants to
set something you didn't care about, now you have to add that into the
template, the attributes, the docs, etc.
So if someone wants to add alias support to each VIP now you gotta add:
node['foo']['primary_vip']['alias'] = 'awesome.sample.com'
node['foo']['secondary_vip']['alias'] = ''
And then modify the template and bla bla bla.
What if we did things differently. We give our attributes slightly more
structure - just enough so the template can care only about the format of the
config and not the config itself:
default['foo']['config_file'] = '/etc/foo/foo.conf'
default['foo']['config']['global'] => {
'logdir' => '/var/log/foo',
'loglevel' => 'DEBUG',
}
default['foo']['config']['www.sample.com'] => {
'root' => '/var/foo/sample.com',
}
And then we change our template to be way more flexible:
<% node['foo']['config'].to_hash.each do |section, config| %>
[<%= section %>]
<% config.each do |key, val| %>
<%= key %> = <%= val %>
<% end %>
<% end %>
That tiny template will write out the 'global' section and every VIP section.
It's complete, shorter than the other one, and supports every current and
future option - assuming a consistent format. And if you want have 99999 VIPs
in your config, just keep adding them, there's no max that the cookbook cares
about anymore.
If someone wants to use the allow_index feature, even though you didn't
populate it by default, they just add
node['foo']['config']['www.sample.com']['allow_index'] to a later cookbook, or
to their role. No problem.
That template is far more simple and flexible than the other version. And it's
easy to do things like support arrays on the left hand side of a sectional config:
...
<% config.each do |key, val| %>
<% val = val.join(', ') if val.is_a?(Array)
<%= key %> = <%= val %>
<% end %>
...
Which, given a user can just pass in 'foo, bar', is probably unnecessary, but
you could do it in a generic way like that.
To which you may ask "but how do I document all the options that you can put
under node['foo']['config'][VIP] ?" And the answer is.. Foo already has
documentation and your users already know how to configure Foo. And if they
don't, your cookbook isn't there to teach them. Your cookbook is to help them
automate that configuration.
You no longer have to decide between supporting everything and having a
grokable config.
Occasionally you'll run into something whose config is so asinine that coming
up with a generalized data structure to represent it doesn't seem feasible -
but the vast vast majority of the time there's an obvious and straight forward
mapping.
We do this for tons of stuff - and at the moment we've open-sourced cron and
sysctl.
The other thing that you end up doing when you go down this road is taking
applications that by default split up their config into file-sized chunks so
that RPMs or DEBs can drop off random files, and making one file that holds
all of the Chef-controlled parts. So let's say this is apache instead of
making one file per VirtualHost definition, you'd make a apache_chef.conf
which had all of your VirtualHosts. Now when someone removes one from their
Chef config, it's also removed from the servers - your entire config is
idempotent. Plus, you don't have to fire off tons of resources for every piece
of your config.
--
Phil Dibowitz ">
Open Source software and tech docs Insanity Palace of Metallica
http://www.phildev.net/ http://www.ipom.com/
"Be who you are and say what you feel, because those who mind don't matter
and those who matter don't mind."
- Dr. Seuss
Archive powered by MHonArc 2.6.16.