[chef] Re: Re: Re: Berks upload seems very slow

[Steven Danna < >]

Hi Jeff,

Here are some debugging ideas:

1) use berks debug logging (-d).  It should print timestamps, use
those to figure out what parts of the upload are slow.
2) Point berks at a local chef-zero instance.  This will be uploading
the cookbooks to memory, it should be fast, if it is slow, you can
rule out your chef server.
3) Take a look at the server logs (start with
/var/log/opscode/nginx/access.log or /var/log/chef-server/access.log)
and see how long your chef-server thinks each request is taking.
Cookbook uploads involve a number of different HTTP requests, however,
they all flow through nginx in your chef-server.   The nginx logs
lines look something like

1.2.3.4 - - [11/Aug/2014:11:06:18 +0100]  "PUT
/bookshelf/organization-baed8c... HTTP/1.1" 204 "0.066" 0 "-" "Chef
Knife/11.14.2 (ruby-1.9.3-p484; ohai-7.2.0; x86_64-linux;
+http://opscode.com)" "127.0.0.1:4321" "204" "0.064" "11.14.2"
"algorithm=sha1;version=1.0;" "myuser" "2014-08-11T10:06:17Z"
"6NQz7YnDiWTasa1jtj0CCLAECMU=" 4315

The bit after HTTP/1.1 is the response times in miliseconds.

I would start with (1).  If you post the full debug output to a gist
or something similar, I'm sure many on this list would be happy to
take a look.

Cheers,

Steven

On Fri, Dec 12, 2014 at 12:46 PM, Jeff MacDonald
<
 >
 wrote:
> Good call but my .kitchen folder is 1 meg, so I don't think that is it. All
> of terida-base is 1.5 megs.
>
>
> On Thu Dec 11 2014 at 2:45:32 PM Peter Burkholder 
> <
 >
> wrote:
>>
>> Make sure you don't have any 'huge' files that are being uploaded. You may
>> need a 'chefignore' file to ignore .kitchen/ if you're doing KitchenCI.
>> E.g. set 'terrida-base/chefignore' to include a line with '.kitchen'.
>>
>> --Peter
>>
>>
>>
>> On Thu, Dec 11, 2014 at 1:07 PM, Jeff MacDonald
>> <
 >
>>  wrote:
>>>
>>> Hi,
>>>
>>> I'm uploading from a MacBook with 16 gigs of ram, to a VM running chef
>>> server with 4 gigs of ram. 30mbit/s upload speed.
>>>
>>> To do a berks upload of about 6 cookbooks takes over 3 minutes. Seems
>>> kind of slow to me, but I don't understand all of its inner workings.
>>>
>>> Just looking for some feedback to understand this or make it better..
>>>
>>> time berks upload --force
>>> Uploaded apt (2.6.0) to: 'https://chef:443/organizations/terida'
>>> Uploaded build-essential (2.1.3) to:
>>> 'https://chef:443/organizations/terida'
>>> Uploaded mongodb (0.16.2) to: 'https://chef:443/organizations/terida'
>>> Uploaded python (1.4.6) to: 'https://chef:443/organizations/terida'
>>> Uploaded runit (1.5.10) to: 'https://chef:443/organizations/terida'
>>> Uploaded terida-base (0.1.0) to: 'https://chef:443/organizations/terida'
>>> Uploaded yum (3.5.1) to: 'https://chef:443/organizations/terida'
>>> Uploaded yum-epel (0.5.3) to: 'https://chef:443/organizations/terida'
>>>
>>> real        2m58.414s
>>> user        0m7.241s
>>> sys 0m1.433s
>>>
>>>
>>>
>>
>