[chef] Re: Re: Re: Re: Re: Re: berks upload error


Chronological Thread 
  • From: Michael Lindsay < >
  • To:
  • Subject: [chef] Re: Re: Re: Re: Re: Re: berks upload error
  • Date: Thu, 14 May 2015 13:54:14 -0700

I seem to have now disabled my ability to upload cookbooks to either of my chef servers with berkshelf...

I've put the backup of my old knife.rb file back, moved everything out of .chef that was intended for the configuration of the new server, copied my .berkshelf file out of my home directory to re-initalize.  I can do berks update, but berks install gives me the following error:

Ridley::Errors::FromFileParserError Could not parse `/var/folders/w5/1k5n_41n3fg_ph5nc4k3xwn80000gp/T/d20150514-31290-1w5vk7n/metadata.rb': undefined method `source_url' for #<Ridley::Chef::Cookbook::Metadata:0x007fca2d03bf70>

I've just officially screwed myself, and I don't know how, and I don't know how to fix it.  Please Help!

TIA,

 - Michael



On Thu, May 14, 2015 at 11:46 AM, Michael Lindsay < " target="_blank"> > wrote:
So I'm tailing the logs during restart.  Could this be anything:
==> /var/log/opscode/opscode-expander/current <==
2015-05-14_18:21:22.65634 log writing failed. can't be called from trap context
2015-05-14_18:21:22.97614 [Thu, 14 May 2015 18:21:22 +0000] INFO: Stopping worker 1 (PID: 9232)
2015-05-14_18:21:22.97617 [Thu, 14 May 2015 18:21:22 +0000] INFO: Stopping worker 2 (PID: 9235)
2015-05-14_18:21:22.97617 log writing failed. can't be called from trap context
2015-05-14_18:21:22.97617 /opt/opscode/embedded/service/gem/ruby/2.1.0/gems/amqp-0.6.7/lib/mq.rb:246:in `synchronize': can't be called from trap context (ThreadError)


Looking through the logs at anything I might not be able to access locally doesn't seem to be yielding any fruit either:

$ curl http://0.0.0.0:9090
<html><body>You are being <a href=""http://0.0.0.0:9090/id/signin">redirected</a>.</body></html>

$ telnet 127.0.0.1 4321
Trying 127.0.0.1...
Connected to 127.0.0.1.

$ telnet 127.0.0.1 16379
Trying 127.0.0.1...
Connected to 127.0.0.1.

$ telnet 127.0.0.1 5672
Trying 127.0.0.1...
Connected to 127.0.0.1.

Does anything external have to talk to any of these ports?

My best guess about where things might be starting to go wrong is this request, or the one after it: 

2015-05-14T18:41:52Z " target="_blank"> method=GET; path=/containers/0e8a6fe4d394239f11394f32cd67065e/acl/create/actors/29b471de57bab8c221569918c41056e9; status=200; requestor_id=29b471de57bab8c221569918c41056e9; req_time=4; rdbms.bifrost_db.has_permission_time=2; rdbms.bifrost_db.has_permission_count=1; rdbms.bifrost_db.exists_time=0; rdbms.bifrost_db.exists_count=1; 

or this request:

PUT /organizations/metrodigi/sandboxes/d81dca36ec7445c7773a76ce4362f2a6 HTTP/1.1" 500

Is it failing to create an organization sandbox?  If so, any ideas what might cause that?



On Thu, May 14, 2015 at 11:05 AM, Michael Lindsay < " target="_blank"> > wrote:
Hey Daniel,

Thanks for your input...
With chef-server-ctl i'm now seeing entries for nginx and oc_bifrost as well, but still challenging to find the root cause of the problem.  As far as I can tell, it looks like chef is sending back adequate responses to the client.

==> /var/log/opscode/nginx/access.log <==
50.193.20.201 - - [14/May/2015:17:42:27 +0000]  "GET /organizations/metrodigi/cookbooks/apt/2.6.1 HTTP/1.1" 404 "0.019" 65 "-" "Ridley v4.0.0" "127.0.0.1:8000" "404" "0.019" "11.4.0" "algorithm=sha1;version=1.0;" "mlindsay" "2015-05-14T17:42:28Z" "2jmj7l5rSw0yVb/vlWAYkK/YBwk=" 949
50.193.20.201 - - [14/May/2015:17:42:29 +0000]  "POST /organizations/metrodigi/sandboxes HTTP/1.1" 201 "0.199" 7391 "-" "Ridley v4.0.0" "127.0.0.1:8000" "201" "0.199" "11.4.0" "algorithm=sha1;version=1.0;" "mlindsay" "2015-05-14T17:42:30Z" "UiN/Daaqc12o4YUmmZuhSKOSn60=" 1896
50.193.20.201 - - [14/May/2015:17:42:29 +0000]  "PUT /bookshelf/organization-f03b7e8dadaf3bb09705d81dca36ec74/checksum-c4dea3dbb103ce5203726dbf60289da4?AWSAccessKeyId=8c0a17c966f9c0df8783049623b56f39cfcfbdc4&Expires=1431626249&Signature=%2FHIhr0cDXbNH00NS9RtNcaP1ygM%3D HTTP/1.1" 204 "0.041" 0 "-" "Faraday v0.9.0" "127.0.0.1:4321" "204" "0.006" "11.4.0" "algorithm=sha1;version=1.0;" "mlindsay" "2015-05-14T17:42:30Z" "zJPbL3MjEPno/dkNXQ9xNIXGWwk=" 2275

As far as the network is concerned, We're moving into and Amazon VPC so, it is the thing most likely to be broken,  Aside from allowing inbound access to 443, which is clearly happening and outbound access to everything, is there anything else I should look for specifically?  And by 'normal network debugging techniques' are you talking about something like, wireshark?  

TIA,
 - Michael

On Thu, May 14, 2015 at 10:24 AM, Daniel DeLeo < " target="_blank"> > wrote:


On Thursday, May 14, 2015 at 10:19 AM, Michael Lindsay wrote:

> I have also tried setting my new configuration to be the default knife.rb and renamed config.json in my .berkshelf directory. Same error:
>
> ==> /var/log/opscode/opscode-erchef/current <==
> 2015-05-14_17:12:38.73280 [error] Checking presence of checksum: <<"5b259a23bcccc6be76de01d811162d36">> for org <<"f03b7e8dadaf3bb09705d81dca36ec74">> from bucket "bookshelf" has taken longer than 5000 ms
> ...
> 2015-05-14_17:12:43.74133 [error] Checking presence of checksum: <<"471f01912a5109386225389563db4159">> for org <<"f03b7e8dadaf3bb09705d81dca36ec74">> from bucket "bookshelf" has taken longer than 5000 ms
> 2015-05-14_17:12:43.74134


This is definitely your problem, it could be any number of things. If you run `chef-server-ctl tail`, it will tail the logs of every component. You should see a request to the bookshelf application, which is one possible source of the issue. If you have some super aggressive firewall or something, it’s possible that the same behavior could be caused by the TCP packets from the erchef service to the bookshelf service simply getting dropped, which would hang for a long time attempting (and failing) to set up the TCP connection. Your normal network debugging techniques could help determine if this is your issue.

--
Daniel DeLeo









Archive powered by MHonArc 2.6.16.

§