[chef] Re: status of test-kitchen?

[Joshua Timberman < >]

Hi Bryan,

On Sep 26, 2012, at 8:13 AM, Bryan Berry 
<
 >
 wrote:

> I really like test-kitchen and I think it has tremendous potential however 
> I find that in its current state it is not a productive tool for me.  There 
> are many outstanding bugs, as one can see here: 
> http://tickets.opscode.com/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+KITCHEN+AND+fixVersion+%3D+EMPTY
>  but the one that I find particularly challenging is that I cannot get 
> test-kitchen to test only one virtualbox image rather than all 4 (centos 
> 5,6, ubuntu 10,11). Further, I can't figure out how to run `librarian-chef 
> update` if I make any changes to my Cheffile.

Our current code review meetings cover tickets that are resolved. That is, we 
review pull requests/patches. Most of the tickets in Test Kitchen aren't 
resolved, so there isn't much to review (yet).

That said, I see four tickets in resolved state. They're all things I'd like 
to get into the next release (0.5.6), but I don't have an ETA.

- KITCHEN-4
- KITCHEN-22
- KITCHEN-35
- KITCHEN-37

Much of our general code review effort recently has been focused on getting 
10.14 regressions fixed (mainly in the deploy resource)[0]. This week we had 
a couple people out of the office, too.

Bringing up the topic on the mailing list(s) is good though, and this is 
exactly the kind of topic that the chef-dev list and #chef-hacking IRC 
channel are intended to facilitate.

> I have gotten the distinct impression that test-kitchen is the preferred 
> testing tool for the opscode-cookbooks/ set of cookbooks. However, at this 
> stage I find myself just spending to much time fighting w/ test-kitchen. I 
> would like to know if anyone plans to put in the time and effort to bring 
> test-kitchen to a more stable and usable form. I would love to pitch in, 
> but right now my priority is to improve the state of the 
> database/postgresql and java-related cookbooks.

Your contributions to the cookbooks are awesome, thank you. The 1.6.0 release 
of the Java cookbook passed a test-kitchen run, too :-).

As you've noticed, one of the things we're doing is add Test Kitchen support 
to our cookbooks, and that is revealing issues that need to be fixed. For 
example, we can't really do much useful testing of heavily search based 
cookbooks (like nagios or munin) in test kitchen because it only supports 
Chef Solo[1]. Our internal CI system uses OpenStack, so a runner for that in 
Test Kitchen would be awesome[2].

> In the meantime I will refocus my workflow, which I call "The Riot Way" 
> based on Jamie Winsor's cookbook authoring guide 
> http://vialstudios.com/guide-authoring-cookbooks.html

I've only skimmed the guide, but it looks great. Jamie and Riot are awesome.

[0]: 10.14.4 was just tagged for release while I wrote this post.
[1]: https://tickets.opscode.com/browse/KITCHEN-31
[2]: http://tickets.opscode.com/browse/KITCHEN-5
-- 
Opscode, Inc
Joshua Timberman, Technical Community Manager
IRC, Skype, Twitter, Github: jtimberman