[chef] Chef CI Pipeline

[Dan Adams < >]

Hi

I have been looking into options for creating a CI (continuous 
integration) pipeline for my Chef configuration. My CI server of choice 
is Jenkins, and I currently have a pipeline that looks like:

ruby syntax check -> ruby lint check -> chef syntax check -> chef lint 
check

using the toolchain:

"ruby -c" -> "nitpick" -> "knife cookbook test" -> "foodcritic"

This gives me a pipeline that runs in under 1 minute for a relatively 
large repo, which is fast enough not to make people want to skip the 
process, and catches a lot of silly problems early.

However, there's still a lot of problems of one sort or another that 
pass this chain but fail in one way or another in production. I'm 
looking for a programmatic way to prevent these additional problems by 
tagging something on to the end of my existing pipeline. Something where 
unit or integration testing might sit in a traditional CI pipeline.

[I know that a lot of people use vagrant, VMTH or toft or some other 
system to spin up a temporary VM or set of VMs at the end of their CI 
chain in order to prove the recipes in the role that traditionally would 
be filled by integration testing. However, for environmental reasons, 
this is not an option for me and I'd like to ignore that option as a 
solution for the purposes of this discussion to save getting sidetracked 
please.]

What I'm looking for basically is something that:
1) provides near to or the same level of understanding as the chef 
server API as to whether your chef config is sane
2) Ideally something drop-in that doesn't require writing individual 
tests for each new cookbook/recipe (a la cucumber-chef)
3) It must also be fast, under 1 minute to test a chef repo containing 
hundreds of nodes and hundreds of cookbooks

I have no experience with it but it looks as though chefspec 
(http://acrmp.github.com/chefspec/) might match at least requirement 
(1). Any other suggestions for investigation?

Cheers

Dan