[chef] Re: Re: Meet test-kitchen

[Torben Knerr < >]

Hi Joshua,

thanks for the quick reply!

>
>> If these dependencies are indeed automatically resolved then that
>> would be worth mentioning in the README.md
>
> Would you please open a ticket in the KITCHEN project for that?
>

Cool, having automatic dependency resolution is great!

I have created
http://tickets.opscode.com/browse/KITCHEN-8 for the missing README ;
documentation
and
http://tickets.opscode.com/browse/KITCHEN-9 for Berksfile support

>
> I'd like to see Travis.CI support in each of our cookbooks, whether they're 
> just doing the foodcritic/cookbook test, or running test-kitchen, that'd be 
> great.
>

Fully support that :-)

Concerning the provisioning of a Vagrant VM on Travis, I will give it
a try next week. If someone has already done that or a working example
please let me know!

>>
>> Oh wait, isn't that already supported by the cookbook / script
>> parameter which defaults to 'rspec spec'? Or would I do that within
>> the cookbook / preflight command to make sure this runs _before_ the
>> vagrant VM is spun up?
>
> Yes, that should be done by the "script" parameter. You should be able to 
> just run that in your cookbook directory without test-kitchen if you don't 
> want the full provisioning cycle.
>

Ok, great!

>>
>> Also, is the "integration_test" block in the Kitchenfile really
>> necessary to run the features?
>
> The cookbook features will automatically get run by test-kitchen if the 
> directory exists.
>

Perfect, good to know,

>
> The integration_test block is for non-cookbook projects (like 
> mixlib-shellout) that have unit or integration testing. While 
> mixlib-shellout is kind of a contrived example, a better one would be a 
> Rails application that has its own test suite that should be run in a fully 
> deployed scenario with its database, etc.
>

Now that I read it again it's clear :-)

>
> Rvm was used as a relatively simple way to get specific versions of Ruby 
> installed for performing the tests. Think of a non-cookbook project, like a 
> Ruby library that you want to test on different versions of Ruby (like 
> 1.8.7 vs 1.9.3, or even JRuby). My personal preference for cookbook testing 
> is that we make the RVM part optional, and use the Ruby that is included in 
> Chef on the base boxes already.
>
> I created a ticket to track this:
>
> http://tickets.opscode.com/browse/KITCHEN-4
>

Yes, for non-cookbook projects using "integration_test" that makes sense.

For cookbook projects using "cookbook" the rvm option still does not
(generally) make sense to me. Okay, if I know I have a basebox with
rvm and some Rubies installed, then it would be fair to state that you
want to test against several different rubies. Yes, but this makes
some heavy assumptions about rvm and rubies being installed on the
basebox, so I would rather see "runtimes" as an option within
"platform", but not within "cookbooks", e.g.

"""
platform :centos do
  version "5.8" do
    box "opscode-centos-5.8"
    box_url 
"https://opscode-vm.s3.amazonaws.com/vagrant/boxes/opscode-centos-5.8.box";
    runtimes ['1.9.2', '1.9.3']
  end
end
"""

The default of [] would mean that the system ruby that is installed
along with Chef of basebox is used. If "runtimes" is specified it
would explicitly mean rvm rubies.

For sure, you could then also exclude specific runtimes:

"""
cookbook "apache2" do
  configuration "default"
  exclude :platform => 'centos', :runtimes => ['1.9.2']
  run_list_extras ['apache2_test::setup']
end
"""

Does that make sense?


Finally, this part of the current "cookbook" / "runtimes"
documentation confuses me most: "The default for cookbook projects is
[], which effectively disables spec/feature tests"

Why should spec/feature tests be disabled in this case?


Cheers,
Torben

> --
> Opscode, Inc
> Joshua Timberman, Technical Program Manager
> IRC, Skype, Twitter, Github: jtimberman
>