Just to clear things up in case anybody was interested, I finally got to the bottom of this. I thought you might like to know why although it's not really chef-related.
The problem was I installed my chef infrastructure on a 'example.local' domain.
By ruby-debugging most of chef to find the extremely slow code, it turned out to be Net::HTTP. I then realised that I was using a Mac (OS X Lion), and Net::HTTP uses the mDNSResolver BSD tools which were preferred over /etc/resolv.conf, /etc/hosts, and anything else since OS X Snow Leopard and above use it to do its resolving. (Tricky to debug as standard tools like 'telnet', 'dig', 'ping', and 'host' use completely different resolver pathways on Mac OS X)
Turning on lots of debug on mDNSResolver, it became clear that requests to example.local were sending out ipv6 dns multicast packets which weren't being returned as the domain was on ipv4. I came across this article: http://serverfault.com/questions/286185/multicast-hostname-lookups-on-osx which explains the rest
Basically, when running spiceweasal and/or knife commands, each request is treated separately, each file (e.g. in cookbooks) is a separate upload request to the chef server, and each request had a +3-5seconds delay DNS lookup, just because its on the .local domain.
When I switched my infrastructure over to an 'example.other' domain, knife ran just as fast as a rake install on the chef server itself.
I guess chef could help this problem (and speed up generally) by uploading multiple assets in one request.. so there is not so much http wrapping to do , but i fixed the real problem anyhu
On 12 Sep 2011, at 17:48, Stephen Delano wrote:
Archive powered by MHonArc 2.6.16.