[chef] Re: Re: Re: --dry-run?

[Bryan McLellan < >]

On Fri, Feb 17, 2012 at 10:55 AM, mark bradley 
<
 >
 wrote:
> Yes, I can see the paradox, but perhaps you're over thinking this? I'd
> rather have some  reporting and *know* that it might be lying than botch up
> my environment because of a mistake. At least I'll know that my scripts have
> a likelihood of being right.

That is the exactly the problem though, you want noop ensure that you
do not "botch up my environment because of a mistake" but because of
the reasons that I outlined, it won't. Thus when you use it and it
tells you everything will be fine, but then you run it in production
and your mysql server is unexpectedly restarted, what value did you
get? Not only did it lie to you, it lied about the very thing you were
trusting it to do.

I may be rambling here, but we run into a similar problem when writing
unit tests for Ohai. Let us say we're testing a plugin that reads the
current version of FoobarOS from /etc/foobar-release which contains
"Foobar 1.3". We want to ensure our code does the right thing, but
unless you run it on FoobarOS, there isn't going to be a
/etc/foobar-release file, so for the test you create a fake version of
the file. This is all well and good until FoobarOS 2.0 comes out and
suddenly /etc/foobar-release contains "FoobarOS version 2.0" The
solution (which I'm frustrated because I haven't gotten time to do
this yet) is that you need a library of virtual machines where you can
run the tests automatically without mocking the files.

As James chimed in, the real test for your cookbooks is running them
in a real environment. As long as I've used Chef, we've had multiple
environments not only for testing our chef recipes, but for the
software itself. Most commonly, a multiple development environments, a
testing/pre-prod environment, and a production environment.

In summary, we're writing recipes that say:

set A to true if it is false
set B to true only if A is true

Noop would say:

I would set A to true because it is false
I would not set B to true because A is false

Then when you run this:

A is set to true
B is set to true because A is true

But wait, that is different than what noop

> Puppet seems to have solved this problem, somehow,

Puppet is not up front with you about the lie. (Or maybe they are
somewhere in their docs, I don't spend much time there.)

Here is a basic example:

node 'localhost' {

  package {'apache2':
    ensure => present
  }

  service {'apache2':
    ensure => stopped,
    enable => false,
    require => Package['apache2']
  }

  file {'/etc/apache2/fake_config.conf':
    ensure => present,
    content => "This could do something. Maybe",
    backup => false,
    notify => Service['apache2']
  }
}

I've stripped out some of the noise from verbose, and added line numbers:

~$ sudo puppet agent -t --noop
1: info: Caching catalog for localhost
2: info: Applying configuration version '1329519751'
3: notice: 
/Stage[main]//Node[localhost]/File[/etc/apache2/fake_config.conf]/ensure:
current_value absent, should be present (noop)
4: info: /Stage[main]//Node[localhost]/File[/etc/apache2/fake_config.conf]:
Scheduling refresh of Service[apache2]
5: notice: /Stage[main]//Node[localhost]/Package[apache2]/ensure:
current_value purged, should be present (noop)
6: err: /Stage[main]//Node[localhost]/Service[apache2]: Could not
evaluate: Could not find init script for 'apache2'

So it would have restarted apache2 on line 4 because it changed a file
on line 3, and it would have installed the service on line 5, but why
the error on line 6? Clearly the package is not installed yet, so we
cannot check its status. But we were just told that it would have
restarted the service, but it can't tell us if could start the
service? There is a lie in there somewhere. Fortunately this example
is simple and we can deduce what would have happened anyway. But if we
can do that, why do we need noop?

~$ sudo puppet agent -t
1: info: Caching catalog for localhost
2: info: Applying configuration version '1329519751'
3: err: 
/Stage[main]//Node[localhost]/File[/etc/apache2/fake_config.conf]/ensure:
change from absent to present failed: Could not set 'present on
ensure: No such file or directory -
/etc/apache2/fake_config.conf.puppettmp_9021 at
/etc/puppet/manifests/site.pp:18
4: notice: /Stage[main]//Node[localhost]/Package[apache2]/ensure:
ensure changed 'purged' to 'present'
5: notice: /Stage[main]//Node[localhost]/Service[apache2]: Dependency
File[/etc/apache2/fake_config.conf] has failures: true
6: warning: /Stage[main]//Node[localhost]/Service[apache2]: Skipping
because of failed dependencies

Wait, didn't noop tell us that it would create a file and restart
apache2? It didn't even create the file! Clearly I've forgotten how
Puppet dependencies work, because it created an automatic dependency
between the service and the configuration file in the wrong direction,
but noop wasn't honest with me about what was going to do here either.

Further, the Puppet DSL that most people use is quite limited. The
notify syntax only has a single action, in the above case notifying a
service resource means restarting it. You still can't notify a service
to reload [1] without a workaround like declaring an execute resource
to run the reload and notifying that resource instead. The onlyif
attribute in Puppet can only be used with the exec (and augeas)
resource and can only be string containing a command to run, where
with Chef only_if and not_if are meta-attributes; can be used with all
resources, and can take a Ruby block as well as a string to run.

For instance this code:

  exec { "ls /tmp":
    path => ["/bin", "/usr/bin", "/usr/bin", "/usr/sbin"],
    onlyif => "true"
  }

Under noop returns: notice: /Stage[main]//Node[localhost]/Exec[ls
/tmp]/returns: current_value notrun, should be 0 (noop)

But if you change the onlyif from true to false, you don't see
anything. This is because even under noop Puppet still runs the
commands in onlyif. This is a compromise of course, as without running
the commands you can't tell what you would do, but if the user expects
the run to have no side-affects they may be surprised. I think it is
also a bug that it says nothing at all.

> and of course make -n from years gone by.

Having put a bunch of time into testing out the above example to show
you, I'll leave the exercise of make to the reader, but I think this
comparison is unreasonable anyway as long as make is a build system
and not a configuration management system.

> Not that I'm an expert on Chef, by any means. All I was getting at was that
> there is utility in this feature even though it may not be perfect. You guys
> are the experts, I leave it in your capable hands. Just think of this as a
> +1 vote for the feature. Maybe one day I'll know enough to contribute code
> too :)

We look forward to you becoming an expert and contributing. :)

Bryan

[1] http://projects.puppetlabs.com/issues/1014