[chef] Re: Re: Re: Re: Re: Re: Re: Continuous Integration

[Thom May < >]

We had some good conversations in this area at Velocity. What we came
down to is that possibly the path of least resistance approach is to
leverage the fact that chef recipes are (mostly) idempotent, so that
Chef itself knows if a cookbook has actually worked successfully.
Clearly there are a few resource types that's not true for - execute
(without not_if/only_if etc) is the major one - but in general a chef
run immediately after a successful chef run should result in zero
actions.
So a very quick Red/Green test would be just to run the compile stage,
not converge it, and check that nothing was expected to happen.
BDD - as was pointed out again at LdnDevOps last week - for cookbooks
could end up being just restating your cookbook in a different
language - there needs to be a compelling way of doing this that
doesn't result in (excuse stupid example):

recipe.rb

package "foo" do
 action :install
end

file "/etc/foo" do
 action :remove
end

bdd.rb

the package "foo" is installed
the file "/etc/foo" is not present

Cheers,
-T

On Thu, Jul 8, 2010 at 09:26, Daniel Pittman 
<
 >
 wrote:
> AJ Christensen 
> <
 >
>  writes:
>
> FWIW, R.I.Pienaar has done some work on this in his mcollective tools:
>
> http://www.devco.net/archives/2010/03/27/infrastructure_testing_with_mcollective_and_cucumber.php
>
> Regards,
>        Daniel
>
>> It seems what we're all really talking about is some kind of BDD framework
>> for Chef cookbooks - unit testing (also related to --noop) has proven
>> unreliable for this kind of thing: you can't mock UNIX.
>>
>> I know there was work done on this in the past - mikehale's chef-bdd
>> project http:// github.com/mikehale/chef-bdd for example.
>>
>> On 8 July 2010 15:55, Chad Woolley 
>> <
 >
>>  wrote:
>>
>>> On Wed, Jul 7, 2010 at 8:09 PM, Erik Kastner 
>>> <
 >
>>>  wrote: > >
>>> > I wonder if something like this could be useful (at least for file, dir,
>>> template, etc) > http://github.com/defunkt/fakefs
>>>
>>> Ah, you are thinking unit testing, where I was thinking integration 
>>> testing.
>>>
>>> In my experience, unit testing Ops/OS/Deploy code is an exercise in
>>> futility.  You can faithfully test drive the implementation as you THINK 
>>> it
>>> should be, but you invariably find that it doesn't work as you expected,
>>> because of some unexpected behavior/interaction of the OS or system.  So,
>>> you change the code to actually work on the real system, then change your
>>> tests to match the code.  That is pretty pointless, it doesn't even buy 
>>> you
>>> the regression safety net of normal unit tests (because upgrades to the
>>> OS/System could break you at any time, even if your chef code doesn't
>>> change).  I suppose it does provide syntax checking, but that's not worth
>>> the considerable effort, in my opinion.
>>>
>>> On the other hand, saying "after I run Chef, assert that a real remote
>>> system should have the package X installed and service Y running" is 
>>> pretty
>>> useful.
>>>
>>> -- Chad
>>>
>
> --
> ✣ Daniel Pittman            ✉ 
> 
 
>             ☎ +61 401 155 707
>               ♽ made with 100 percent post-consumer electrons
>