[chef] Re: Serverspec vs Mintest Feedback

[Joshua Timberman < >]

Ohai!

On Sat, Jan 3, 2015 at 1:00 PM, David Petzel < " target="_blank"> > wrote:

TLDR - I'm interesting in knowing why folks are moving away from Minitest, toward ServerSpec.

As Julian said, there's an element of not wanting to "contaminate" the tests. However, there's a couple other reasons.

 

I've been noticing a lot of cookbooks recently have been migrating their testings from minitest to ServerSpec. I've got nothing against ServerSpec, but I'm curious why/where this trend is coming from.

Back in the before times, that is, before `test kitchen` and the notion of `bussers`, the best (only?) way of reasonably testing nodes post convergence was through the minitest chef handler. The early version of test kitchen (pre 1.0) did run those, but the rewritten version, with the bussers made it more flexible.

I think Serverspec is popular because Rspec is popular. It's not a value judgment, just more widely used - Rspec is what CHEF uses for chef itself, Ruby and Ruby on Rails programmers are familiar with Rspec. While minitest is in the Ruby stdlib (and I like it more than Rspec for that reason), it's simply not widespread. Also, Serverspec does an "outside in" kind of test approach. I have more confidence that the tests are doing the right thing. I can defer configurable options to attributes, and then set those attributes on a per-suite basis.

Maybe my question would be better asked in the context of an example... Lets assume I am consuming a community cookbook which sets up some application that has an HTTP port. There is an attribute that controls what that HTTP port is.

In ServerSpec land, I think you need to hard code the HTTP port to test?

In Minitest, I have access to the node object, so my test is not hard coded to a port, but instead uses the value of the node attribute. 

For the sake of the discussion, lets say I run that application on port 78 in dev, 79 in qa, and 80 in production. While using minitests I'm able to assert the application is working properly, because the minitest test would be reading the node attribute. However from what I've seen with ServerSpec, I don't have that same ability to adapt the test to use the right port, without manual intervention?

This is why I like, and use test kitchen. I can set up two suites.

suites:

  - name: default

    run_list:

      - recipe[webapp]

    attributes:

  - name: port78

    run_list:

      - recipe[webapp]

    attributes:

      www:

        port: 78

Then I can have tests for each suite, one that checks that my web app responds on port 80 (presuming its the default), and responds on port 78. I don't actually care about the different ports in different environments, I want to test that the non-default port does the right thing. Adding more suites that only change this attribute just tests that Chef works, which we've already proved in the first case. In fact, if the port is an attribute, I don't think the `default` suite is really *really* necessary.

Cheers,

Joshua