[chef] Re: Re: Re: Re: Re: Re: Testing Cookbooks vs Testing Infrastructure

[Tensibai Zhaoying < >]

Oops, forgot to say we're doing this integration tests with gatling to test 
the service from Jenkins but targeting the app server directly and not 
through all the bricks a user will 

Le 22 avr. 2015 20:28, Torben Knerr 
<
 >
 a écrit :
>
> Sorry, I was probably not specific enough :-) 
>
> Serverspec is asserting state by ssh'ing into the VM, then checking 
> for processes, files being present, services being started, etc... (I 
> call that "from the inside") 
>
> However, if all that works, it doesn't necessarily mean that it works 
> for the end user (lets say the webapp is served on port 80 and 
> accessible "from outside" of the VM. That's probably a grey zone where 
> infrastructure testing ends and you the acceptance testing for a 
> specific application starts. 
>
> A more concrete example: say I have a cookbook for setting up 
> JenkinsCI with Slaves on multiple nodes, with nginx in front, etc.. 
> I'd like to have a test suite that: 
> - sets up the multi-node scenario 
> - then verifies that the front page is served... 
> - ...and that you can create a basic job that is executed on a slave 
> - ...and so on (some basic acceptance level smoke tests) 
>
> That's what I'm up to... did it get clearer? 
>
> Cheers, 
> Torben 
>
>
>
>
>
> On Wed, Apr 22, 2015 at 3:54 PM, Tensibai 
> <
 >
>  wrote: 
> > We wish to have integration tests launched from jenkins too, that's not 
> > really outside, but the idea is to test the functional state in IC before 
> > promotion to QA with human testers behind load balancer and reverse 
> > proxy. 
> > 
> > I may misunderstand what you think of under acceptance term, sorry if 
> > it's 
> > the case :) 
> > 
> > Le 2015-04-22 15:37, Torben Knerr a écrit : 
> > 
> > Are you planning to (acceptance level) test it from the outside as well? 
> > 
> > Or just for convergence to succeed? 
> > 
> > Am 22.04.2015 14:46 schrieb "Tensibai" 
> > <
 >:
> >  
> >> 
> >> I forgot the link to terraform: http://www.terraform.io/intro/index.html ;
> >> 
> >> Le 2015-04-22 14:41, Tensibai a écrit : 
> >> 
> >> I had a discussion with a colleague this morning about this and pointed 
> >> to 
> >> Terraform[1]. 
> >> 
> >> As we're already using vagrant with a custom provider (for vsphere) 
> >> we're 
> >> planning to move toward this path. for now it's just an idea with no 
> >> feedback but the described way Terraform works sounds interesting to 
> >> write 
> >> "plans" to ensure idem-potency for cookbooks (spin up a machine, 
> >> converge it 
> >> once, re-converge it, change a parameter somewhere, converge again, 
> >> ensure 
> >> expectation are there). 
> >> 
> >> That's what we're aiming to do, launched from jenkins on cookbook commit 
> >> after usual linting/chefspec test. 
> >> 
> >> I'll try to write something about it if we end up with something 
> >> functional at end. 
> >> 
> >> Le 2015-04-22 07:40, Torben Knerr a écrit : 
> >> 
> >> Hey everybody, 
> >> 
> >> @jtimberman's recent blog post about test-driven infrastructure with 
> >> Chef made me thinking about this again. 
> >> 
> >> All the tools we currently have are focused on cookbook testing, but 
> >> we have nothing established for acceptance level infrastructure 
> >> testing yet, do we? 
> >> 
> >> I mean specifically: 
> >> 
> >>  * testing from the outside vs from the inside 
> >>  * testing multi-vm setups 
> >> 
> >> See also my comment on the original post: 
> >> 
> >> https://www.chef.io/blog/2015/04/21/overview-of-test-driven-infrastructure-with-chef/
> >>  
> >> 
> >> 
> >> What do we have in this space? 
> >> 
> >> Cheers, 
> >> Torben 
> >> 
> >> 
> >> 
> >> 
> >> 
> >> 
> >> 
> >> 
> > 
> > 
> > 
> >