[chef] Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Continuous Integration

[Chad Woolley < >]

On Fri, Jul 9, 2010 at 9:30 AM, Andrew Shafer 
<
 >
 wrote:
> Here's my thinking at this point... which could be wrong on every level.
> There is really no good way to TDD/BDD configuration management for several
> reasons:
> The recipes are already relatively declarative
> Mocking is useless because it may not reflect 'ground truth'
> The cycle times to really test convergence are relatively long
> Trying to test if a package is installed or not is testing the framework,
> not the recipe IMHO.
> I agree with the general sentiment that the functional service is the true
> test.
> I'm leaning towards 'testing' at that level, ideally with (a superset of?)
> what should be used for the production monitoring system.
> So the CI builds services, runs all the checks in test, green can go live
> and that's that.

Agreed on most points, except I'm not sure about the assertion that
testing package installation is testing the framework.

Your recipe could refer to an incorrect package name, or version, or
the system could not be correctly configured with the source to
download that package.

Those are bugs in your recipe, not the framework.

However, this should be caught by the chef run blowing up.  And, you
shouldn't even be attempting to test the converged system if the chef
run failed.  Which means that package installation is not worth
testing after all...

So, I guess this is leading to the conclusion that the only things
worth testing are the things which could go wrong WITHOUT causing the
actual chef run to fail?