[chef] RE: Re: Error uploading Cookbooks

[Baruch Shpirer < >]

We found the problem, we changed IP and the older one was still in /etc/hosts 
and nginx didn’t like it
Thanks for the help

-----Original Message-----
From: Seth Falcon 
[mailto:
 
 
Sent: Wednesday, March 06/03/2013 18:51
To: 
<
 >
Subject: [chef] Re: Error uploading Cookbooks

Hi there,

In the logs I see this bit, which seems to be getting closer to the cause of 
the issues you are seeing:

> 2013-03-06_21:41:03.84638 =ERROR REPORT==== 6-Mar-2013::16:41:03 ===
> 2013-03-06_21:41:03.84639 Checking presence of checksum: 
> <<"960131de20bb72d55d52df914d73a5e2">> for org 
> <<"00000000000000000000000000000000">> from bucket "bookshelf" has 
> taken longer than 5000 ms
>  

It would be interesting to see if you always see this error leading up to the 
cookbook upload failure and if it is for the same checksum or different 
checksums.

The timeout is tunable, but unless your server is under considerable load, I 
would have expected that you'd run into this -- so there may be something 
else going on or a bug we will identify :)

As an aside, it would be worth upgrading to 11.0.6 as there are some security 
fixes that are worth having.


On Mar 6, 2013, at 1:44 PM, Baruch Shpirer wrote:
> 192.168.75.74 - - [06/Mar/2013:16:40:58 -0500]  "POST /sandboxes 
> HTTP/1.1" 201 "0.070" 752 "-" "Chef Knife/0.10.10 (ruby-1.9.2-p180; 
> ohai-0.6.12; x86_64-linux; +http://opscode.com)" "127.0.0.1:8000" 
> "201" "0.023" "0.10.10" "version=1.0" "baruchs" "2013-03-06T21:40:58Z" 
> "17WVwc7QAnAtCnOlTAlgc5YUZM8=" 1232

So a sandbox was created.

> 192.168.75.74 - - [06/Mar/2013:16:40:58 -0500]  "PUT 
> /bookshelf/organization-00000000000000000000000000000000/checksum-9601
> 31de20bb72d55d52df914d73a5e2?AWSAccessKeyId=e849c1918a22eabefc7e7832b2
> 7173ab5b93016d&Expires=1362606958&Signature=G5O2Ex3Gz7l0aWmmwyiHRPQFmI
> 0%3D HTTP/1.1" 204 "0.014" 0 "-" "Ruby" "127.0.0.1:4321" "204" "0.007" 
> "-" "version=1.0" "baruchs" "2013-03-06T21:40:58Z" 
> "hZyLgj7RPtRg/pSvY7LOq2Fotrc=" 3573

The problem checksum appears to have been successfully uploaded to bookshelf. 
You could verify that it is on disk by looking in 
/var/opt/chef-server/bookshelf/data for a file with the 
960131de20bb72d55d52df914d73a5e2 in its name.

>  ==> /var/log/chef-server/erchef/erchef.log.1 <== 2013-03-06T21:40:58Z 
> 
 
>  INFO req_id=rXyk1ks5Taz5qtOTGtPtuQ==; status=200; 
> method=GET; path=/cookbooks?num_versions=all; user=baruchs; msg=[]; 
> req_time=9; rdbms_time=3; rdbms_count=2 2013-03-06T21:40:58Z 
> 
 
>  INFO req_id=2Qzog7D0fQQx3g3z1KUPZQ==; status=201; 
> method=POST; path=/sandboxes; user=baruchs; msg=[]; req_time=20; 
> rdbms_time=16; rdbms_count=3
>  
> ==> /var/log/chef-server/erchef/current <==
> 2013-03-06_21:41:03.84624
> 2013-03-06_21:41:03.84638 =ERROR REPORT==== 6-Mar-2013::16:41:03 ===
> 2013-03-06_21:41:03.84639 Checking presence of checksum: 
> <<"960131de20bb72d55d52df914d73a5e2">> for org 
> <<"00000000000000000000000000000000">> from bucket "bookshelf" has 
> taken longer than 5000 ms
>  
> ==> /var/log/chef-server/nginx/access.log <==
> 192.168.75.74 - - [06/Mar/2013:16:41:03 -0500]  "PUT 
> /sandboxes/000000000000be13430442db7a83ab5f HTTP/1.1" 500 "5.059" 36 
> "-" "Chef Knife/0.10.10 (ruby-1.9.2-p180; ohai-0.6.12; x86_64-linux; 
> +http://opscode.com)" "127.0.0.1:8000" "500" "5.010" "0.10.10" 
> "version=1.0" "baruchs" "2013-03-06T21:40:58Z" 
> "oMRtV6loUDnbKJuGcW6nqBbF8ww=" 1029

Here we see the error report for the failure to verify the existance of the 
checksum in bookshelf and then the 500 for the request trying to commit the 
sandbox.

Does this happen for any cookbook upload? E.g. can you see what happens if 
you try to upload one of the community cookbooks?

+ seth