[chef] Re: RE: Re: Re: Berks upload seems very slow

[Jeff MacDonald < >]

I just brought my chef server up to 8 gigs of ram and 4 cpus, and ran 3 tests of a mongodb upload in a row. They all took the same time, right around 20 seconds. So its not server resources.

On Mon Dec 15 2014 at 9:10:56 AM Jeff MacDonald < "> > wrote:

I tried this with 4 or 5 of the recipes in my cook book, they all averaged ab out 20 seconds each. Didn't matter if it was a super market one, or one of my own. Max was 24 seconds, min was 18 after about 6 tests.

Its not network for me. I have 30 megs up to a 100meg down data centre. Latency isn't horrible either.

I ran the upload with the -d flag, they authentication portion of the upload happens pretty quick. Looks like uploads take about 4 put requests, each of those took about 4 to 5 seconds to complete.

I confirmed this by looking at the logs on the server it self, and they agree. To upload mongodb it was a GET POST PUT PUT, and took about 18 seconds. 

So far, what I'm seeing seems to jive with what others are saying. 

Steve Danna suggested i point berks at a local chef-zero instance. I have no idea how to do that yet, so I'll have to research it :)

Jeff

On Fri Dec 12 2014 at 6:03:44 PM Kevin Keane Subscription < " target="_blank"> > wrote:

You may be able to verify this by just uploading one of the supermarket cookbooks by itself; that will take your own cookbook out of the equation.

time berks upload yum --force

I also found the uploads very, very slow even for small cookbooks, but it was never a big enough nuisance for me to investigate.

Two things that I suspect in my case:

- It may be a network connectivity issue. In my case, the workstation is on a T-1, the chef server is behind a DSL line.

- More likely, my chef server may be underpowered; it's running on a relatively small virtual machine (2 GB). Maybe some database operation or so needs a beefier VM and/or need some tuning.

Kevin Keane

The NetTech

http://www.4nettech.com

Our values: Privacy, Liberty, Justice

See https://www.4nettech.com/corp/the-nettech-values.html

-----Original message-----
From: Jeff MacDonald < " target="_blank">mr.jeffmacdonald@gmail.com>
Sent: Friday 12th December 2014 5:05
To: " target="_blank">
Subject: [chef] Re: Re: Berks upload seems very slow

Good call but my .kitchen folder is 1 meg, so I don't think that is it. All of terida-base is 1.5 megs.

On Thu Dec 11 2014 at 2:45:32 PM Peter Burkholder < " title="This external link will open in a new window" target="_blank"> > wrote:

Make sure you don't have any 'huge' files that are being uploaded. You may need a 'chefignore' file to ignore .kitchen/ if you're doing KitchenCI.  E.g. set 'terrida-base/chefignore' to include a line with '.kitchen'.

--Peter

On Thu, Dec 11, 2014 at 1:07 PM, Jeff MacDonald < " title="This external link will open in a new window" target="_blank"> > wrote:

Hi,

I'm uploading from a MacBook with 16 gigs of ram, to a VM running chef server with 4 gigs of ram. 30mbit/s upload speed.

To do a berks upload of about 6 cookbooks takes over 3 minutes. Seems kind of slow to me, but I don't understand all of its inner workings.

Just looking for some feedback to understand this or make it better..

time berks upload --force

Uploaded apt (2.6.0) to: 'https://chef:443/organizations/terida'

Uploaded build-essential (2.1.3) to: 'https://chef:443/organizations/terida'

Uploaded mongodb (0.16.2) to: 'https://chef:443/organizations/terida'

Uploaded python (1.4.6) to: 'https://chef:443/organizations/terida'

Uploaded runit (1.5.10) to: 'https://chef:443/organizations/terida'

Uploaded terida-base (0.1.0) to: 'https://chef:443/organizations/terida'

Uploaded yum (3.5.1) to: 'https://chef:443/organizations/terida'

Uploaded yum-epel (0.5.3) to: 'https://chef:443/organizations/terida'

 

real 2m58.414s

user 0m7.241s

sys 0m1.433s