[chef] Re: Re: Re: knife command very slow


Chronological Thread 
  • From: Cassiano Leal < >
  • To: < >, Lamont Granquist < >
  • Cc:
  • Subject: [chef] Re: Re: Re: knife command very slow
  • Date: Fri, 16 Aug 2013 09:47:17 -0300

Thanks for the tips. I don’t use a chef-repo per se, and I issue knife command from anywhere so the Gemfile approach is not an option.

I also don’t use RVM or rbenv (I ditched both in favour of chruby, which is much leaner), and I had been using 1.9.3-p429 all along. Tried with 2.0 and had similar results (~1s with a clean gem environment).

I’m happy for now. Knife’s slowness (when it happens again) will serve as a reminder to clean up gems that are only occupying disk space and taking away my time and patience. :)

On August 15, 2013 at 17:44:44, Lamont Granquist ( ) wrote:


Try using an up-to-date rvm that has bundle exec integration (rvm >= 1.1.0) or use rbenv with the rbenv-bundler installed and then setup a Gemfile in your chef-repo and bundle install it.  Run your knife commands under that bundle and there's no longer any need to bundle install --binstubs or bundle exec every command.  With ruby-1.9.3-p429 or ruby-2.0 run with that bundle it should be fairly fast.

Note, if you go down this path that you'll wind up wanting to keep recent versions of chef, berkshelf and other command line tools in your bundle which gets interesting when they have common deps like the json gem.  Right now there may be a bug in the bundler depsolving code where it somehow solves to berkshelf pinning json to >= 1.8.0 where all I can see is the ridley restriction of json >= 1.7.7 which is still compatible with chef's json <= 1.7.7 restriction.  If you let the berkshelf version float you wind up with an old version of it, if you try to pin berkshelf to ~> 2.0.8 and chef to 11.6.0 then it blows up.  But if you also pin the json gem to 1.7.7 explicitly then its perfectly happy.  I've been using rvm with ruby-2.0.0-p247 and this Gemfile:

% cat Gemfile
source 'https://rubygems.org'

ruby '2.0.0'

gem 'rake'
gem 'thor'
gem 'chef', '11.6.0'
gem 'json', '1.7.7'
gem 'chefspec'
gem 'foodcritic'
gem 'knife-spork'
gem 'berkshelf', '~> 2.0.8'
gem 'knife-ec2'
gem 'knife-hp'

Going down this road speeds up knife and is convenient.  You can check in your Gemfile.lock and get your whole team using the same versions of your commandline toolchain, which is cool.  But, you will probably get intimately familiar with transitive deps between all these tools and the json gem (which probably makes for a really good argument to check in specific versions of Gemfile.lock that are found to work and bump versions for the team only when the universe is aligned...)

Even if you don't go the whole way with setting up a Gemfile to limit the number of gems that knife needs to search for plugins, ruby-1.9.3-p429 or ruby-2.0 is pretty mandatory to speed up knife.  Unless you're using a very recent distro you will not have a recent enough ruby in /usr/bin/ruby and must use either rvm or rbenv.

(Also note that ruby 2.0 requires chef-11.6 and may still have the odd bug)

On 8/15/13 1:10 PM, John Dewey wrote:
" type="cite">
I have run into this before.  Was due to many gems installed.
Does this help?

On Thursday, August 15, 2013 at 1:01 PM, Cassiano Leal wrote:

Hi,

I’m having a bit of a hard time here.

Any knife command I issue takes approximately 15s to run. I’ve ruled out network issues by running knife by itself (http://pastie.org/8240023).

If I ‘top’ during the knife execution I can see a ruby process taking up a whole CPU core.

How can I profile that to find out what’s taking so long to process?



--
Cassiano Leal
http://cassianoleal.com
http://twitter.com/cassianoleal



Archive powered by MHonArc 2.6.16.

§