[chef] Re: Re: Automated test-kitchen runs?

[Christopher Armstrong < >]

Thanks guys!! Appreciate all the feedback. Going to take a stab at this tomorrow.

On Thu, Mar 27, 2014 at 5:21 PM, Eric G. Wolfe < " target="_blank"> > wrote:

I've been running a few of our cookbooks through Jenkins with Kitchen and Docker, since first of January.  I had to use either Docker, or LXC, because my Jenkins build host was already a VMWare VM and getting nested virtualization working with something like Vagrant/Virtualbox wasn't all that straightforward.

I broke down the build jobs up into 3 stages, quick tests, integration tests, and deploy.

Stage 1: Unit and linting tests, with Chefspec, Rubocop, and Foodcritic, which all takes about 1 minute or less.  First stage is triggered by our Gitlab server, or upstream dependencies.

Stage 2: Use kitchen.ci and kitchen-docker to spin up baseboxes generated with rinse and then run BATS integration tests against those machines.  Some of the cookbooks are EL5 and EL6 which take about 6-8 minutes to test both.  Some just need to be tested with EL6 which take about half the time, 3-4 minutes.

Stage 3: do a berks upload, and berks apply production.  If there are data bags in the cookbook, then I test with chef-zero in Stage 2, and do a rake databag:upload_all on Stage 3.

If there are downstream dependencies like a role cookbook which depends on an upstream baseline role cookbook.  Then the upstream cookbook triggers a 3-stage build on downstream cookbooks.

If you want to validate branching behaviour such as in your org-java cookbook, then a good way to test a lot of corner cases is probably Chefspec.  Kitchen works great doing just a few branching platforms and automatically integration testing those cases.  As you add more platforms to Kitchen, two things tend to happen.  Your integration tests become more complex, and your build time grows.  If your integration tests for all your combined platforms for a given cookbook don't fit in a ten to fifteen minute window, then unit tests with Chefspec is probably going to provide more value and quicker feedback.

Eric G. Wolfe
Senior Linux Administrator,
IT Infrastructure Systems
--------------------------------------
Marshall University Computing Services
Drinko Library 428-K
One John Marshall Dr.
Huntington, WV 25755
Phone: 304.942.3970
Email: 
 
 " target="_blank">
 

The worst sin towards our fellow creatures is not to hate them,
but to be indifferent to them; that's the essence of inhumanity.
		-- G.B. Shaw

On 03/27/2014 07:18 PM, Christopher Armstrong wrote:

Ohai Chefs,

We have rubocop and foodcritic configured to run on Travis CI to automatically check our pull requests, which is awesome. I verify each build with test-kitchen before I cut a new cookbook release, but this is getting very time consuming to run manually each time. For the java cookbook, testing different java versions on various platforms results in a ton of individual runs. I'd love to set up a system to automatically run test-kitchen and then give me the red/green light.

Has anyone tried doing this and could recommend some tools/workflows?

Thanks,

Chris