[chef] Re: Fwd: Resource ordering in chef v. puppet

[Bryan McLellan <btm@loftninjas.org>]

IRT using Ruby for the coniguration language, I've had so many "oh my
god that makes it so easy" moments that come when I think about how to
do something that was irritatingly difficult before in other
languages.

This is definitely a benefit that becomes much clearer once you've
gone around the block a few times.

Bryan

On 7/23/09, Andrew Willis <andrewwillis@yahoo.com> wrote:
> In addition to the complications with accurately modeling dependencies
> (which I found added notable development time for my configurations),
> I like how the machines themselves are not part of the "code" in chef.
> In puppet, the nodes are in the configuration files, so adding a new
> node means updating a configuration file. This by itself isn't a big
> deal, but when you then check this into your SCM, and then check it
> out on the puppetmaster server, it seems like a lot of overhead just
> to add a single machine. I wouldn't be surprised if I was doing it
> wrong, but I found it created extra steps for little value. It seems
> like there are some straight forward ways around this (like including
> node information outside of source control), but I didn't want to have
> to work around a problem that I didn't want to have in the first
> place. With chef, this isn't a big deal. I just validate the node in
> the chef server and assign it a role. In practice, this is proving to
> be much lighter weight.
>
> Chef advocates also note that it uses Ruby, not a custom configuration
> language. I didn't think this was a huge deal at first, but it's great
> not have to learn a syntax that's only used in one place. I'm writing
> cookbooks much faster than puppet class files. Most of the savings are
> from not having to track down missed dependencies, but using Ruby is
> comfortable for me and allows me to refer to the documentation less
> frequently.
>
> I'm in the process of migrating my puppet setup to chef, and I'm
> really enjoying chef. Getting complex configurations right is straight
> forward, primarily because things in chef execute in the order you
> expect them to.
>
>
> -Andrew
>
> On Tue, Jul 21, 2009 at 5:05 PM, Adam Jacob<adam@opscode.com> wrote:
>> On Tue, Jul 21, 2009 at 12:49 PM, Peter Burkholder<pburkholder@gmail.com>
>> wrote:
>>> Anyhow, the gist is that Puppet gets it right if one has been thorough
>>> in modelling the dependencies.
>>
>> That is correct.  The sneaky bit here is that you may not realize that
>> you got it wrong until it fails unpredictably.  So where is the bug -
>> in your manifest, or in Puppet?  Clearly it's in your manifest - but
>> that doesn't make you feel any better when you don't have the system
>> you expect working the way you expected it (in its entirety - half a
>> webserver is a broken webserver, not half-correct.)
>>
>>> Would anyone on this list care to offer an alternative view on
>>> resource ordering vs. recipe ordering (which is my understanding of
>>> how Chef handles this)?
>>
>> Chef handles ordering inside a recipe by applying the resources you
>> specify in the order you specify them.  So:
>>
>> package "apache2"
>>
>> remote_file "/etc/apache2/httpd.conf" do
>>  source "httpd.conf.erb"
>>  mode "0644"
>>  owner "root"
>>  group "root"
>> end
>>
>> Will install Apache first, then render the config file.  In puppet:
>>
>> package { "apache2":
>>  ensure => "installed"
>> }
>>
>> remotefile { "/etc/apache2/httpd.conf":
>>  path => "/etc/apache2/httpd.conf",
>>  source => "httpd.conf",
>>  module => apache2,
>>  mode => 0644,
>>  owner => root,
>>  group => root,
>> }
>>
>> That will fail 50%ish of the time.  You would have a two element
>> directed graph, where each node is not attached to any other.  So when
>> you topologically sort it (to get the Array you need for runtime),
>> you'll wind up with the config file being applied first some of the
>> time, and the package being installed the rest.  You fix this with:
>>
>> remotefile { "/etc/apache2/httpd.conf":
>>  path => "/etc/apache2/httpd.conf",
>>  source => "httpd.conf",
>>  module => apache2,
>>  mode => 0644,
>>  owner => root,
>>  group => root,
>>  require => Package["apache2"]
>> }
>>
>> Which now gives you a two-element directed graph, where the elements
>> are connected to each other, and the package always comes up first.
>> This straw-man example is the simplest case - now imagine you have
>> somewhere on the order of a thousand resources under management on a
>> given system... and if you miss any of those require calls, you will
>> have bugs lurking in your configuration somewhere that won't be
>> surfaced until that dice roll comes up.  That was why I say I hate
>> that feature - I know it's my fault, and I would love to fix it, but I
>> can only see it when it happens.. and when it happens is transient,
>> and it's irritation factor increases the more I relied on the tool.
>>
>> The reason for that feature in puppet is that it can look at the tree
>> build from the sorted graph and cut branches off when a resource fails
>> - it won't even attempt to apply the dependent ones.  That means you
>> can have a system that gets 3/4 configured - perhaps the
>> authentication portion works, but the application portion does not.
>> Often the response to this sort of failure is simply to run puppet
>> again - to get the system to converge.  In my straw-man above, the
>> system will always work if you run puppet twice.
>>
>> Chef assumes that any time it cannot run the entire set of
>> recipes/roles, it has had a critical, human surface-able failure.  You
>> wanted me to be a webserver, and I did not complete that task.  Chef
>> is idempotent at the resource level, which means that if you run it
>> again, we're smart enough to to re-do any work that does not need to
>> be done.. which means that some failure scenarios can also be solved
>> by convergence (in particular, transient failures.)  But in the
>> situation where you reverse the two resources:
>>
>> remote_file "/etc/apache2/httpd.conf" do
>>  source "httpd.conf.erb"
>>  mode "0644"
>>  owner "root"
>>  group "root"
>> end
>>
>> package "apache2"
>>
>> You will always fail if the apache2 package creates the /etc/apache2
>> directory.  That means you can always debug - did the recipe work or
>> not?  And you can rely on it - if it works once, it'll work again, as
>> long as no transient changes creep into the system (entropy is a
>> bitch!).  You also apply the logic you are trained to use already - we
>> all know how to debug a program by looking at the order you wrote it
>> in.
>>
>> Chef does allow you one recipe to include another, and we only apply
>> them once.  But even in this case, we rely on the order we process
>> things to determine what order you wanted it done in - if you stick an
>> include_recipe statement in the middle of a recipe, we will go and run
>> that recipe and return.  This is the prime mechanism of real
>> 'ordering' in Chef - make sure Apache2 is ready before you install
>> Passenger, or that Postfix is ready before you start your e-mail
>> application.
>>
>> With both Puppet and Chef, the end result is an Array of resources
>> that have actions they are going to perform on the system (should they
>> need to be taken.)  The difference between Puppet and Chef is that
>> Chef makes the Array explicit - it is put together by you when you
>> write the recipes and roles for your infrastructure.  Puppet makes it
>> implicit - it attempts to determine the right order on your behalf,
>> based on the data you gave it.
>>
>> I strongly prefer the way Chef does it, because I can hold the idea of
>> a multi-thousand element array in my head (at the very least, I can
>> mentally zoom in on a portion of it) - but I can't hold a
>> multi-thousand element directed graph, and all the permutations of a
>> valid topological sort in my head.
>>
>> We've also never had a question from a newcomer to Chef about what
>> order things are done in, if that's any indication. :)
>>
>> Adam
>>
>> --
>> Opscode, Inc.
>> Adam Jacob, CTO
>> T: (206) 508-7449 E: adam@opscode.com
>>
>

-- 
Sent from my mobile device