[chef] Re: RE: Re: RE: RE: Re: chef server load testing

[AJ Christensen < >]

The cost of ruby processes/classes is expensive > I know I'll run it
in JRuby; hehe, I love it - interesting approach.

I do not know that any work has been done to get Chef running on
JRuby. It is not likely to work, as mentioned, requiring the fork
syscall (this will spill over to affecting Ohai as well which relies
on it); you may be able to patch mixlib::shellout::unix to use Spoon
or a process_spawn type implementation which will be available to you.
Similar work would have to be undertaken for ohai. That's probably
only touching the surface.

https://rubygems.org/gems/spoon
https://github.com/rtomayko/posix-spawn

I would not recommend this path to someone inexperienced with Ruby,
Chef and deeply familiar with implementation-specific challenges
regarding a JRuby application.

cheers,

--aj

On Sat, Mar 14, 2015 at 8:19 AM, Marcus Simonsen
<
 >
 wrote:
> Many Thanks Daniel.
> I've been able to do basic create nodes and clients, with the rest api. And 
> I'm sure with more digging that I can load cookbooks etc...
> It appears that the cost of starting the ruby process/associated ruby 
> classes is expensive, so I'm now wondering about the possibility of running 
> jruby/chef inside of the JMeter JVM.
> Results aren't promising:
> ./jruby my_sample_rest_client.rb
> =>
> NotImplementedError: fork is not available on this platform
>                             fork at org/jruby/RubyKernel.java:1735
>                  fork_subprocess at 
> /opt/jruby-9.0.0.0.pre1/lib/ruby/gems/shared/gems/mixlib-shellout-2.0.1/lib/mixlib/shellout/unix.rb:284
>                      run_command at 
> /opt/jruby-9.0.0.0.pre1/lib/ruby/gems/shared/gems/mixlib-shellout-2.0.1/lib/mixlib/shellout/unix.rb:52
>                      run_command at 
> /opt/jruby-9.0.0.0.pre1/lib/ruby/gems/shared/gems/mixlib-shellout-2.0.1/lib/mixlib/shellout.rb:248
>                shell_out_command at 
> /opt/jruby-9.0.0.0.pre1/lib/ruby/gems/shared/gems/chef-12.1.1/lib/chef/mixin/shell_out.rb:97
>    shell_out_with_systems_locale at 
> /opt/jruby-9.0.0.0.pre1/lib/ruby/gems/shared/gems/chef-12.1.1/lib/chef/mixin/shell_out.rb:61
>   shell_out_with_systems_locale! at 
> /opt/jruby-9.0.0.0.pre1/lib/ruby/gems/shared/gems/chef-12.1.1/lib/chef/mixin/shell_out.rb:65
>            guess_internal_locale at 
> /opt/jruby-9.0.0.0.pre1/lib/ruby/gems/shared/gems/chef-12.1.1/lib/chef/config.rb:673
>                           (root) at 
> /opt/jruby-9.0.0.0.pre1/lib/ruby/gems/shared/gems/chef-12.1.1/lib/chef/config.rb:702
>                           (root) at 
> /opt/jruby-9.0.0.0.pre1/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:1
>                          require at 
> /opt/jruby-9.0.0.0.pre1/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:54
>                          require at org/jruby/RubyKernel.java:954
>                           (root) at 
> /opt/jruby-9.0.0.0.pre1/lib/ruby/gems/shared/gems/chef-12.1.1/lib/chef.rb:24
>                           (root) at 
> /opt/jruby-9.0.0.0.pre1/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:1
>                          require at org/jruby/RubyKernel.java:954
>                          require at 
> /opt/jruby-9.0.0.0.pre1/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:128
> <=
> Unless you know how to suppress fork?
>
>
>
>
> ________________________________________
> From: Daniel DeLeo 
> 
 
>  on behalf of Daniel DeLeo 
> 
 
> Sent: 12 March 2015 22:03
> To: 
> 
 
> Subject: [chef] Re: RE: RE: Re: chef server load testing
>
> On Thursday, March 12, 2015 at 1:52 PM, Marcus Simonsen wrote:
>> Chef::ApiClient::Registration appears promising, and your name on the code!
>> Am I heading in the right direction?
>
> Generally the “model” objects in Chef (node, role, etc.) follow an Active 
> Record type pattern, where each object/class has methods for 
> save/load/update/etc. So you can look though all those classes for how they 
> use Chef::REST (doing `git grep ‘def save’` might narrow it down a bit). In 
> a few cases, the logic for running a request is a bit more complicated, and 
> these things tend to have their own classes to encapsulate the behavior for 
> just that one request. `Chef::ApiClient::Registration` is one of those, as 
> it handles making a request with the validator key to create a new 
> ApiClient object and store the private key on disk, which is what usually 
> happens the very first time you run chef-client on a host. Likewise, the 
> process of syncing cookbooks from the server to a host is a bit 
> complicated, so that has it’s own CookbookSynchronizer class, and the 
> process where chef-client queries the dependency solver to get the list of 
> cookbooks to run is done by PolicyBuilder::ExpandNodeObject.
>
>
>>
>>
>> ________________________________________
>> From: Marcus Simonsen 
>> 
 
>>  
>> (mailto:
 )]
>> Sent: 12 March 2015 15:26
>> To: 
>> 
 
>>  
>> (mailto:
 )
>> Subject: [chef] RE: Re: chef server load testing
>>
>> Thanks Daniel.
>>
>> This is more in line with what I want to achieve. A very granular approach 
>> to simulating chef client operations against the server.
> Cool. Just remember to validate the proportion of different API calls 
> against your real-world usage. There’s a lot of ways that your data and the 
> way you use the API can cause the Chef Server to have varying performance 
> characteristics.
>
>> I've been taking a look into the chef code to understand a bit more what 
>> these Chef::Rest calls would look like. If I simulate payload then I won't 
>> even have to write any ruby at all, I'll just need to be able to template 
>> the http headers and body. My objective is to write as little code as 
>> possible.
>
> The tricky part is that the HTTP headers include a timestamp, and that 
> timestamp is part of the request signature. The server only allows a small 
> deviation in the current time and the timestamp in the request as a 
> mitigation against replay attacks. It’s possible that the amount of 
> deviation allowed might be configurable, but there’s no documented config 
> option for this. So you need to generate your request headers on-demand or 
> at least a short time before your tests run.
>
>> On the other hand leveraging already working client code would be easier. 
>> If you have some samples of how you would simulate this via the Chef api 
>> that would be appreciated.
>> Cheffish gem seems to be very helpful in this respect - but documentation 
>> lacking I do have to dig into source.
>
> If you are more familiar with some other language than ruby, then you might 
> look around to see if there are any language bindings for whatever language 
> you want to use. I know that at least python, go, php, and java have been 
> written at some point, though I’m not sure if they’re well maintained.
>
> --
> Daniel DeLeo
>
>
>