[chef] Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Continuous Integration

[Trotter Cashion < >]

Yea, getting at your step definitions would be sweeeeeet.

On Fri, Jul 9, 2010 at 3:26 PM, Chad Woolley < "> > wrote:

On Fri, Jul 9, 2010 at 12:20 PM, Jeremy Deininger < "> > wrote:
> Hey guys,
> Thought I'd chime in with my experience testing system configuration code @
> RightScale so far.  What we've been building are integration style cucumber
> tests to run a cookbook through it's paces on all platforms and OSs that we
> support.
>
> First we use our API to spin up 'fresh' server clusters in EC2, one for
> every platform/OS (variation) that the cookbook will be supporting.  The
> same could be done using other cloud APIs (anyone else doing this with
> VMware or etc?)  Starting from scratch is important because of chef's
> idempotent nature.
>
> Then a cucumber test is run against every variation in parallel.  The
> cucumber test runs a series of recipes on the cluster then uses what we call
> 'spot checks' to ensure the cluster is configured and functional.  The spot
> checks are updated when we find a bug, to cover the bug.  An example spot
> check would be, sshing to every server and checking the mysql.err file for
> bad strings.
>
> These high level integration tests are long running but have been very
> useful flushing out bugs.
> -J

This sounds like exactly the right approach.  Is it open source (yet)?