[chef] Re: chef server load testing

[Daniel DeLeo < >]

I wrote a thing like this quite a while ago. Unfortunately the code is not 
public. What I did was analyze our logs to find the approximate mix of 
different API calls we were getting and then made a couple of “stories” for 
simulated chef-client runs doing a mix of different calls (initial run with 
client registration vs. re-run of an existing client/node pair, search, data 
bag fetches and so on). The API calls were done using Chef::Rest directly 
rather than running chef client so I could skip the stuff that wasn’t 
relevant. Then I prepopulated some required data and ran the simulated 
chef-client runs in threads using JRuby.  

Even if it was open source, I don’t know that the code would actually be 
usable for most people as there’s a lot of undocumented manual steps required 
in the setup. I also am not sure if the code has been kept up to date with 
API changes, as more recently I started using basho_bench with the erlang 
HTTP client for chef to benchmark server features (this benchmarks a 
proprietary add-on for chef-server so it is non-public as well).

Anyway, you may be able to approximate my approach by writing ruby scripts 
that just do the necessary API calls and skip the rest of the client code. 
One tricky aspect of that is for self-hosted chef servers, you need to put 
some load on bookshelf (the s3 clone that stores the cookbooks’ files). 
You’ll probably have to read through the Chef source to see how this works.

Good luck,  

--  
Daniel DeLeo


On Thursday, March 12, 2015 at 9:11 AM, Marcus Simonsen wrote:

> Hi,
>  
> I'd like to load test the chef-server (11 or 12) with simulated node 
> registrations - ie. simulating chef-client process.
> I'm wondering if anyone is doing anything like this?
>  
> My hack thus far is that I have created a jmeter plugin that creates a 
> chef-client process per thread. I had to modify the run_lock.rb code to 
> allow concurrent runs of chef-client on a single node. What I really need 
> is the ability to simulate http payload(s) that ultimately simulates an 
> entire chef-client process - and makes my plugin much lighter and more 
> flexible.
>  
> All suggestions welcome!
>  
> -marcus