[chef] Re: Re: Re: Re: Re: debugging slow knife?

[Michael Schueler < >]

On Tue, May 3, 2011 at 9:43 PM, Daniel DeLeo < "> > wrote:

On Tuesday, May 3, 2011 at 9:31 PM, Michael Schueler wrote:

On Tue, May 3, 2011 at 6:44 PM, AJ Christensen < " target="_blank"> > wrote:

Yo,

On 4 May 2011 13:39, Daniel DeLeo < " target="_blank"> > wrote:
> On Tuesday, May 3, 2011 at 9:54 AM, Michael Schueler wrote:
>
> Hi Chefs,
>
> I'm using the opscode platform.  Before 0.10 the knife command was
> unbearably slow.  It's definately at least twice as fast, but it's still
> kind of sluggish.  Even issuing the help command to knife takes 2 seconds.
>
> Some simple stuff:
>
> Off my MacBook Pro..
> % time knife node show e02
> <snip>
> knife node show e02  3.17s user 0.23s system 74% cpu 4.544 total
>
> % time knife help
> <snip>
> knife help  2.12s user 0.17s system 99% cpu 2.296 total
>
> % ruby -v
> ruby 1.9.2p180 (2011-02-18 revision 30909) [x86_64-darwin10.7.0]
>
>
> My results are similar from an ec2 c1.xlarge...
>
> # time knife ec2 --help
> <snip>
> real    0m1.843s
> user    0m1.300s
> sys    0m0.550s
>
> # ruby -v
> ruby 1.8.7 (2010-01-10 patchlevel 249) [x86_64-linux]
>
>
> Is this expected?  It might not seem like much but it's enough to really
> slow me down and annoy me..
>
> This is something I've optimized as much as possible. Given knife's current
> architecture, it has to load all of the files to generate the help output,
> so whether or not the files are cached makes a huge difference (as shown by
> the numbers you've all posted).
> There are some things you can look in to:
> * Since knife has to load all commands including plugins when it runs, it's
> important that any plugins you have installed make use of lazy loading.
> * Running `gem clean` can improve your runtimes dramatically. It can also
> break things that depend on those gems, so be careful.
> After running gem clean, I'm seeing times around 0.6 seconds to run `knife
> -h` and `knife help` when the disk cache is hot.
> I'd certainly like to improve this further, especially for the case that the
> disk cache is cold, but this would require fairly substantial architectural
> changes to knife--and the gains from doing so could be fairly small compared
> to the factors that are our of our control (e.g., factors affected by `gem
> clean`).

After 'gem clean':

knife -h  0.30s user 0.08s system 98% cpu 0.391 total
knife help  0.49s user 0.16s system 99% cpu 0.663 total

Optimizing more than this seems like a waste of time! =D

Is that knife-ec2 or just plain knife?

With knife-ec2 installed, I'm getting:

knife -h  0.41s user 0.13s system 99% cpu 0.543 total

 

I made a new RVM environment for knife and installed Chef and knife-ec2 along with their dependencies.

It's now down to:

knife -h  1.69s user 0.16s system 99% cpu 1.847 total
knife help  1.73s user 0.17s system 99% cpu 1.905 total

Is this with the disk cache hot? 

And what version of rubygems? I'm on 1.6.2 ATM.

Yes it was with the disk cache hot.  I was using the latest, 1.7.2

I think we found the primary variable: Rubygems version.

With 1.3.7 I am now getting these numbers:

Ok, wow. I think I found the problem. I didn't update my rubygems to the latest this time and kept it at 1.3.7.

knife node list 0.76s user 0.17s system 53% cpu 1.754 total
knife --help 0.62s user 0.14s system 99% cpu 0.761 total
knife -h 0.62s user 0.15s system 99% cpu 0.766 total
knife node show e02 0.85s user 0.19s system 63% cpu 1.647 total

Now to go thru the Rubygems versions and see which is quickest and switch to it using RVM when I'm working with knife.

Thanks,
Mike
 

-- 
Dan DeLeo

 

Also this is what my .gemrc looks like:

---
:benchmark: false
gem: --no-ri --no-rdoc --no-wrappers
:update_sources: true
:bulk_threshold: 1000
:verbose: true
:sources:
- http://rubygems.org
:backtrace: false




Thanks
Mike
 

>
> --
> Dan DeLeo
>
>
> Thanks
> Mike
>
>