[chef] Re: debugging slow knife?

[Daniel DeLeo < >]

On Tuesday, May 3, 2011 at 9:54 AM, Michael Schueler wrote:

Hi Chefs,

I'm using the opscode platform.  Before 0.10 the knife command was unbearably slow.  It's definately at least twice as fast, but it's still kind of sluggish.  Even issuing the help command to knife takes 2 seconds.

Some simple stuff:

Off my MacBook Pro..
% time knife node show e02
<snip>
knife node show e02  3.17s user 0.23s system 74% cpu 4.544 total

% time knife help
<snip>
knife help  2.12s user 0.17s system 99% cpu 2.296 total

% ruby -v
ruby 1.9.2p180 (2011-02-18 revision 30909) [x86_64-darwin10.7.0]


My results are similar from an ec2 c1.xlarge...

# time knife ec2 --help
<snip>
real    0m1.843s
user    0m1.300s
sys    0m0.550s

# ruby -v
ruby 1.8.7 (2010-01-10 patchlevel 249) [x86_64-linux]


Is this expected?  It might not seem like much but it's enough to really slow me down and annoy me..

This is something I've optimized as much as possible. Given knife's current architecture, it has to load all of the files to generate the help output, so whether or not the files are cached makes a huge difference (as shown by the numbers you've all posted).

There are some things you can look in to:

* Since knife has to load all commands including plugins when it runs, it's important that any plugins you have installed make use of lazy loading.

* Running `gem clean` can improve your runtimes dramatically. It can also break things that depend on those gems, so be careful.

After running gem clean, I'm seeing times around 0.6 seconds to run `knife -h` and `knife help` when the disk cache is hot.

I'd certainly like to improve this further, especially for the case that the disk cache is cold, but this would require fairly substantial architectural changes to knife--and the gains from doing so could be fairly small compared to the factors that are our of our control (e.g., factors affected by `gem clean`).

-- 
Dan DeLeo

 

Thanks
Mike