- From: Daniel DeLeo <
>
- To:
- Subject: [chef] Re: Error after crash of Chef server
- Date: Mon, 11 Jun 2012 13:36:18 -0700
On Monday, June 11, 2012 at 2:07 AM, Jesai Langenbach wrote:
>
>
Hi,
>
>
We have a Problem since a server crash with chef.
>
>
The chef server is working but I cant rebuild the index /upload new data
>
bags/modify data bags/upload cookbooks . I can't figure out where the
>
problem is.
>
>
The Error Massage on the chef server (server.log) is:
>
merb : chef-server (api) : worker (port 4001) ~ end of file reached -
>
(EOFError)
>
/usr/lib/ruby/1.8/net/protocol.rb:135:in `sysread'
>
/usr/lib/ruby/1.8/net/protocol.rb:135:in `rbuf_fill'
>
/usr/lib/ruby/1.8/timeout.rb:67:in `timeout'
>
/usr/lib/ruby/1.8/timeout.rb:101:in `timeout'
>
/usr/lib/ruby/1.8/net/protocol.rb:134:in `rbuf_fill'
>
/usr/lib/ruby/1.8/net/protocol.rb:116:in `readuntil'
>
/usr/lib/ruby/1.8/net/protocol.rb:126:in `readline'
>
/usr/lib/ruby/1.8/net/http.rb:2028:in `read_status_line'
>
/usr/lib/ruby/1.8/net/http.rb:2017:in `read_new'
>
/usr/lib/ruby/1.8/net/http.rb:1051:in `request'
>
/usr/lib/ruby/1.8/net/http.rb:1037:in `request'
>
/usr/lib/ruby/1.8/net/http.rb:543:in `start'
>
/usr/lib/ruby/1.8/net/http.rb:1035:in `request'
>
/usr/lib/ruby/vendor_ruby/chef/rest/rest_request.rb:99:in `call'
>
/usr/lib/ruby/vendor_ruby/chef/rest/rest_request.rb:114:in
>
`hide_net_http_bug'
>
/usr/lib/ruby/vendor_ruby/chef/rest/rest_request.rb:98:in `call'
>
/usr/lib/ruby/vendor_ruby/chef/rest.rb:241:in `api_request'
>
/usr/lib/ruby/vendor_ruby/chef/rest.rb:328:in `retriable_rest_request'
>
/usr/lib/ruby/vendor_ruby/chef/rest.rb:240:in `api_request'
>
/usr/lib/ruby/vendor_ruby/chef/rest.rb:144:in `put_rest'
>
/usr/lib/ruby/vendor_ruby/chef/couchdb.rb:114:in `store'
>
/usr/lib/ruby/vendor_ruby/chef/node.rb:619:in `cdb_save'
>
/usr/share/chef-server-api/app/controllers/nodes.rb:69:in `update'
>
/usr/lib/ruby/1.8/merb-core/controller/abstract_controller.rb:315:in `send'
>
/usr/lib/ruby/1.8/merb-core/controller/abstract_controller.rb:315:in
>
`_call_action'
>
/usr/lib/ruby/1.8/merb-core/controller/abstract_controller.rb:289:in
>
`_dispatch'
>
/usr/lib/ruby/1.8/merb-core/controller/merb_controller.rb:252:in `_dispatch'
>
/usr/lib/ruby/1.8/merb-core/dispatch/dispatcher.rb:102:in `dispatch_action'
>
/usr/lib/ruby/1.8/merb-core/dispatch/dispatcher.rb:74:in `handle'
>
/usr/lib/ruby/1.8/merb-core/dispatch/dispatcher.rb:36:in `handle'
>
/usr/lib/ruby/1.8/merb-core/rack/application.rb:17:in `call'
>
/usr/lib/ruby/1.8/merb-core/rack/middleware/static.rb:28:in `call'
>
/usr/lib/ruby/1.8/rack/content_length.rb:13:in `call'
>
/usr/lib/ruby/1.8/thin/connection.rb:76:in `pre_process'
>
/usr/lib/ruby/1.8/thin/connection.rb:74:in `catch'
>
/usr/lib/ruby/1.8/thin/connection.rb:74:in `pre_process'
>
/usr/lib/ruby/1.8/thin/connection.rb:57:in `process'
>
/usr/lib/ruby/1.8/thin/connection.rb:42:in `receive_data'
>
/usr/lib/ruby/1.8/eventmachine.rb:257:in `run_machine'
>
/usr/lib/ruby/1.8/eventmachine.rb:257:in `run'
>
/usr/lib/ruby/1.8/thin/backends/base.rb:57:in `start'
>
/usr/lib/ruby/1.8/thin/server.rb:156:in `start'
>
/usr/lib/ruby/1.8/merb-core/rack/adapter/thin.rb:30:in `start_server'
>
/usr/lib/ruby/1.8/merb-core/rack/adapter/abstract.rb:298:in `start_at_port'
>
/usr/lib/ruby/1.8/merb-core/rack/adapter/abstract.rb:128:in `start'
>
/usr/lib/ruby/1.8/merb-core/server.rb:174:in `bootup'
>
/usr/lib/ruby/1.8/merb-core/server.rb:42:in `start'
>
/usr/lib/ruby/1.8/merb-core.rb:170:in `start'
>
/usr/sbin/chef-server:86
>
>
>
>
I increased the log level but still do not find anything.
>
>
The Full Log is:
>
chef-server -p 4001 -e production -a thin
>
Loading init file from /usr/share/chef-server-api/config/init.rb
>
[Mon, 11 Jun 2012 09:49:54 +0200] DEBUG: Sending HTTP Request via GET to
>
localhost:5984/_all_dbs
>
[Mon, 11 Jun 2012 09:49:54 +0200] DEBUG: Sending HTTP Request via GET to
>
localhost:5984/chef/_design/id_map
>
[Mon, 11 Jun 2012 09:49:54 +0200] DEBUG: Sending HTTP Request via GET to
>
localhost:5984/chef/_design/nodes
>
[Mon, 11 Jun 2012 09:49:54 +0200] DEBUG: Sending HTTP Request via GET to
>
localhost:5984/chef/_design/roles
>
[Mon, 11 Jun 2012 09:49:54 +0200] DEBUG: Sending HTTP Request via GET to
>
localhost:5984/chef/_design/data_bags
>
[Mon, 11 Jun 2012 09:49:54 +0200] DEBUG: Sending HTTP Request via GET to
>
localhost:5984/chef/_design/clients
>
[Mon, 11 Jun 2012 09:49:54 +0200] DEBUG: Sending HTTP Request via GET to
>
localhost:5984/chef/_design/users
>
[Mon, 11 Jun 2012 09:49:54 +0200] DEBUG: Sending HTTP Request via GET to
>
localhost:5984/chef/_design/cookbooks
>
[Mon, 11 Jun 2012 09:49:54 +0200] DEBUG: Sending HTTP Request via GET to
>
localhost:5984/chef/_design/sandboxes
>
[Mon, 11 Jun 2012 09:49:54 +0200] DEBUG: Sending HTTP Request via GET to
>
localhost:5984/chef/_design/checksums
>
[Mon, 11 Jun 2012 09:49:54 +0200] DEBUG: Sending HTTP Request via GET to
>
localhost:5984/chef/_design/environments
>
[Mon, 11 Jun 2012 09:49:54 +0200] DEBUG: Sending HTTP Request via GET to
>
localhost:5984/chef/_design/id_map/_view/name_to_id
>
[Mon, 11 Jun 2012 09:49:54 +0200] DEBUG: Sending HTTP Request via GET to
>
localhost:5984/chef/_design/id_map/_view/name_to_id
>
[Mon, 11 Jun 2012 09:49:54 +0200] DEBUG: Sending HTTP Request via GET to
>
localhost:5984/chef/_design/id_map/_view/name_to_id
>
[Mon, 11 Jun 2012 09:49:54 +0200] INFO: Loading roles
>
/usr/lib/ruby/1.8/mixlib/authentication/http_authentication_request.rb:25:
>
warning: already initialized constant MANDATORY_HEADERS
>
/usr/lib/ruby/1.8/mixlib/authentication/signatureverification.rb:28:
>
warning: already initialized constant SignatureResponse
>
~ Could not find resource model User
>
~ Could not find resource model Node
>
~ Could not find resource model Role
>
~ Could not find resource model Environment
>
~ Could not find resource model Datum
>
merb : chef-server (api) : worker (port 4001) ~ Starting Thin at port 4001
>
merb : chef-server (api) : worker (port 4001) ~ Using Thin adapter on host
>
0.0.0.0 and port 4001.
>
merb : chef-server (api) : worker (port 4001) ~ Successfully bound to port
>
4001
>
[Mon, 11 Jun 2012 09:50:29 +0200] INFO: Authenticating client j_langenbach
>
[Mon, 11 Jun 2012 09:50:29 +0200] DEBUG: Sending HTTP Request via GET to
>
localhost:5984/chef/_design/id_map/_view/name_to_id
>
[Mon, 11 Jun 2012 09:50:29 +0200] DEBUG: Authenticating Client:
>
#<Chef::ApiClient:0x7f573c1d54a8 @couchdb=#<Chef::CouchDB:0x7f573c1db498
>
@db="chef", @rest=#<Chef::REST:0x7f573c1db470 @sign_request=true,
>
@default_headers={}, @url="http://localhost:5984", @sign_on_redirect=true, ;
>
@redirect_limit=10, @redirects_followed=0,
>
@auth_credentials=#<Chef::REST::AuthCredentials:0x7f573c1db218
>
@key_file=nil, @client_name=nil>, @cookies={}>>, @public_key="-----BEGIN
>
RSA PUBLIC
>
KEY-----\nMIIBCgKCAQEAskYoWJdzJF+jzGJuyJC9P4CRKUTIBdEthsRmitw2A+I5eoIVhJub\nkdWYGLjbzvy/WuVE8rfsAmJlolvRkcaXWYn+CxFopS3SUZ0EXouliTF2GLtE0jJO\n3L3OoEc+l2jIPfpcfvoLr01W/djk2Ap+4TWOiKbtphUf4ExEzr1eJ5pT2XMoPRT7\ncYkXSqpec3v1Zbusg+xzeEVK66JtBwfwKSW0NX0L96PdRbGrvrSe+RYEJyFzhyYJ\ny3NwkeCk97OyO8KfYoE8H5wkQ11jifhP38hvwnyHDFybW0XKyhEZC2CSP2GEwrik\nMCVzKX1MuHhFMXTncYlVwiWZ3X2VrDbicwIDAQAB\n-----END
>
RSA PUBLIC KEY-----\n",
>
@couchdb_id="6c2d5608-4c7e-4163-a57a-a2b3f795af57", @admin=true,
>
@couchdb_rev="1-788bfa953a0d2c07f0dddaf7fbb12214", @private_key=nil,
>
@name="j_langenbach", @index_id="6c2d5608-4
c7e-4163-a57a-a2b3f795af57">
>
>
[Mon, 11 Jun 2012 09:50:29 +0200] DEBUG: Initializing header auth :
>
#<Merb::Request:0x7f573c1dfbd8 @route=/search/reindex,
>
@route_params={:action=>"reindex", :controller=>"search"},
>
@multipart_params={}, @env={"HTTP_HOST"=>"chef.kdz-ws.intra.net:4001
>
(http://chef.kdz-ws.intra.net:4001)", "HTTP_ACCEPT"=>"application/json",
>
"SERVER_NAME"=>"chef.kdz-ws.intra.net (http://chef.kdz-ws.intra.net)",
>
"rack.url_scheme"=>"http", "CONTENT_LENGTH"=>"2", "HTTP_USER_AGENT"=>"Chef
>
Knife/0.10.8 (ruby-1.9.3-p125; ohai-0.6.12; x86_64-linux;
>
+http://opscode.com)", "REQUEST_PATH"=>"//search/reindex",
>
"SERVER_PROTOCOL"=>"HTTP/1.1", "CONTENT_TYPE"=>"application/json",
>
"HTTP_X_CHEF_VERSION"=>"0.10.8",
>
"HTTP_X_OPS_CONTENT_HASH"=>"vyGp6PvFo4RvsFtPoIWeCReyIC8=",
>
"HTTP_X_OPS_TIMESTAMP"=>"2012-06-11T07:50:30Z",
>
"rack.errors"=>#<IO:0x7f57437c0aa0>, "async.callback"=>#<Method:
>
Thin::Connection#post_process>, "REMOTE_ADDR"=>"172.16.81.180",
>
"PATH_INFO"=>"//search/reindex", "rack.run_once"=>false, "rack.version"=>
[1, 0], "SERVER_SOFTWARE"=>"thin 1.2.4 codename Flaming Astroboy",
"SCRIPT_NAME"=>"",
"HTTP_X_OPS_AUTHORIZATION_1"=>"JnNPHNDCMxAxNRebYy5RFuTEvaT5a9W+xHNi1Za+EqH93lPm8TSNgpQr0pfk",
"HTTP_X_OPS_AUTHORIZATION_2"=>"1mjiotOfWfBbz6dAclzGrUc55gv/J35vfQydABF2YYXc2oUIlvEDmHguPm6Q",
"HTTP_VERSION"=>"HTTP/1.1", "rack.multithread"=>false,
"HTTP_X_OPS_AUTHORIZATION_3"=>"z0bdvrgFSnXJ9JJLqjfBdIgYfZX5VkCqSP9SlUdKnWQ3GEPV3n11z3PItltH",
"REQUEST_URI"=>"//search/reindex", "rack.multiprocess"=>false,
"SERVER_PORT"=>"4001",
"HTTP_X_OPS_AUTHORIZATION_4"=>"6WqNv5GD7dZICyMIpTyU/NnivtpHyb50bgl+ZvnKMppDUignv2QuTQUn46oY",
"async.close"=>#<EventMachine::DefaultDeferrable:0x7f573c1dfed0>,
"HTTP_X_OPS_AUTHORIZATION_5"=>"cMy06RqkqBHUHD94T3jYJCZRM4kSUig617Z9YisMMSpm42Jf7nVokn2Mp7dJ",
"HTTP_X_OPS_USERID"=>"j_langenbach", "REQUEST_METHOD"=>"POST",
"HTTP_X_OPS_AUTHORIZATION_6"=>"DQRs4OXc/OUhTr4y/iZ+HJQyhCaXNAu26c9a7wwkXQ==",
"QUERY_STRING"=>"", "GATEWAY_INTERFACE"=>"CGI/1.2",
"HTTP_CONNECTION"=>"close", "HTTP_X_OPS_SI
GN"=>"version=1.0", "rack.input"=>#<StringIO:0x7f573c1e75b8>},
@json_params={}, @raw_post="{}", @params={"action"=>"reindex",
"controller"=>"search"}, @body_params=nil, @method=:post, @query_params={},
@xml_params=nil, @body_and_query_params={}, @body=#<StringIO:0x7f573c1e75b8>>
>
[Mon, 11 Jun 2012 09:50:29 +0200] DEBUG: Parsed signing description:
>
{:version=>"1.0"}
>
[Mon, 11 Jun 2012 09:50:29 +0200] DEBUG: Digesting body: '{}'
>
[Mon, 11 Jun 2012 09:50:29 +0200] DEBUG: Reconstituted (user-supplied)
>
request signature:
>
JnNPHNDCMxAxNRebYy5RFuTEvaT5a9W+xHNi1Za+EqH93lPm8TSNgpQr0pfk
>
1mjiotOfWfBbz6dAclzGrUc55gv/J35vfQydABF2YYXc2oUIlvEDmHguPm6Q
>
z0bdvrgFSnXJ9JJLqjfBdIgYfZX5VkCqSP9SlUdKnWQ3GEPV3n11z3PItltH
>
6WqNv5GD7dZICyMIpTyU/NnivtpHyb50bgl+ZvnKMppDUignv2QuTQUn46oY
>
cMy06RqkqBHUHD94T3jYJCZRM4kSUig617Z9YisMMSpm42Jf7nVokn2Mp7dJ
>
DQRs4OXc/OUhTr4y/iZ+HJQyhCaXNAu26c9a7wwkXQ==
>
[Mon, 11 Jun 2012 09:50:29 +0200] DEBUG: Verifying request signature:
>
[Mon, 11 Jun 2012 09:50:29 +0200] DEBUG: Expected Block is: 'Method:POST
>
Hashed Path:ZKpVFjmsFMEHqnjeztcib+NRIgo=
>
X-Ops-Content-Hash:vyGp6PvFo4RvsFtPoIWeCReyIC8=
>
X-Ops-Timestamp:2012-06-11T07:50:30Z
>
X-Ops-UserId:j_langenbach'
>
[Mon, 11 Jun 2012 09:50:29 +0200] DEBUG: Decrypted block is: 'Method:POST
>
Hashed Path:ZKpVFjmsFMEHqnjeztcib+NRIgo=
>
X-Ops-Content-Hash:vyGp6PvFo4RvsFtPoIWeCReyIC8=
>
X-Ops-Timestamp:2012-06-11T07:50:30Z
>
X-Ops-UserId:j_langenbach'
>
[Mon, 11 Jun 2012 09:50:29 +0200] DEBUG: Signatures match? : 'true'
>
[Mon, 11 Jun 2012 09:50:29 +0200] DEBUG: Request time difference: 0.155941,
>
within 900 seconds? : true
>
[Mon, 11 Jun 2012 09:50:29 +0200] DEBUG: Expected content hash is:
>
'vyGp6PvFo4RvsFtPoIWeCReyIC8='
>
[Mon, 11 Jun 2012 09:50:29 +0200] DEBUG: Request Content Hash is:
>
'vyGp6PvFo4RvsFtPoIWeCReyIC8='
>
[Mon, 11 Jun 2012 09:50:29 +0200] DEBUG: Hashes match?: true
>
[Mon, 11 Jun 2012 09:50:29 +0200] DEBUG: POSTing document to SOLR:
>
<?xml version="1.0" encoding="UTF-8"?>
>
<delete><query>X_CHEF_database_CHEF_X:chef</query></delete>
>
>
[Mon, 11 Jun 2012 09:50:30 +0200] DEBUG: POSTing document to SOLR:
>
<?xml version="1.0" encoding="UTF-8"?>
>
<commit/>
>
>
[Mon, 11 Jun 2012 09:50:30 +0200] DEBUG: Sending HTTP Request via GET to
>
localhost:5984/chef/_design/clients/_view/all
>
[Mon, 11 Jun 2012 09:50:30 +0200] FATAL: Chef encountered an error while
>
attempting to load Chef::ApiClient objects back into the index
>
merb : chef-server (api) : worker (port 4001) ~ Started request handling:
>
Mon Jun 11 09:50:29 +0200 2012
>
merb : chef-server (api) : worker (port 4001) ~ Routed to:
>
{"action"=>"reindex", "controller"=>"search"}
>
merb : chef-server (api) : worker (port 4001) ~ Params:
>
{"action"=>"reindex", "controller"=>"search"}
>
merb : chef-server (api) : worker (port 4001) ~ end of file reached -
>
(EOFError)
>
/usr/lib/ruby/1.8/net/protocol.rb:135:in `sysread'chef-server -p 4001 -e
>
production -a thin
>
Loading init file from /usr/share/chef-server-api/config/init.rb
>
[Mon, 11 Jun 2012 09:49:54 +0200] DEBUG: Sending HTTP Request via GET to
>
localhost:5984/_all_dbs
>
[Mon, 11 Jun 2012 09:49:54 +0200] DEBUG: Sending HTTP Request via GET to
>
localhost:5984/chef/_design/id_map
>
[Mon, 11 Jun 2012 09:49:54 +0200] DEBUG: Sending HTTP Request via GET to
>
localhost:5984/chef/_design/nodes
>
[Mon, 11 Jun 2012 09:49:54 +0200] DEBUG: Sending HTTP Request via GET to
>
localhost:5984/chef/_design/roles
>
[Mon, 11 Jun 2012 09:49:54 +0200] DEBUG: Sending HTTP Request via GET to
>
localhost:5984/chef/_design/data_bags
>
[Mon, 11 Jun 2012 09:49:54 +0200] DEBUG: Sending HTTP Request via GET to
>
localhost:5984/chef/_design/clients
>
[Mon, 11 Jun 2012 09:49:54 +0200] DEBUG: Sending HTTP Request via GET to
>
localhost:5984/chef/_design/users
>
[Mon, 11 Jun 2012 09:49:54 +0200] DEBUG: Sending HTTP Request via GET to
>
localhost:5984/chef/_design/cookbooks
>
[Mon, 11 Jun 2012 09:49:54 +0200] DEBUG: Sending HTTP Request via GET to
>
localhost:5984/chef/_design/sandboxes
>
[Mon, 11 Jun 2012 09:49:54 +0200] DEBUG: Sending HTTP Request via GET to
>
localhost:5984/chef/_design/checksums
>
[Mon, 11 Jun 2012 09:49:54 +0200] DEBUG: Sending HTTP Request via GET to
>
localhost:5984/chef/_design/environments
>
[Mon, 11 Jun 2012 09:49:54 +0200] DEBUG: Sending HTTP Request via GET to
>
localhost:5984/chef/_design/id_map/_view/name_to_id
>
[Mon, 11 Jun 2012 09:49:54 +0200] DEBUG: Sending HTTP Request via GET to
>
localhost:5984/chef/_design/id_map/_view/name_to_id
>
[Mon, 11 Jun 2012 09:49:54 +0200] DEBUG: Sending HTTP Request via GET to
>
localhost:5984/chef/_design/id_map/_view/name_to_id
>
[Mon, 11 Jun 2012 09:49:54 +0200] INFO: Loading roles
>
/usr/lib/ruby/1.8/mixlib/authentication/http_authentication_request.rb:25:
>
warning: already initialized constant MANDATORY_HEADERS
>
/usr/lib/ruby/1.8/mixlib/authentication/signatureverification.rb:28:
>
warning: already initialized constant SignatureResponse
>
~ Could not find resource model User
>
~ Could not find resource model Node
>
~ Could not find resource model Role
>
~ Could not find resource model Environment
>
~ Could not find resource model Datum
>
merb : chef-server (api) : worker (port 4001) ~ Starting Thin at port 4001
>
merb : chef-server (api) : worker (port 4001) ~ Using Thin adapter on host
>
0.0.0.0 and port 4001.
>
merb : chef-server (api) : worker (port 4001) ~ Successfully bound to port
>
4001
>
[Mon, 11 Jun 2012 09:50:29 +0200] INFO: Authenticating client j_langenbach
>
[Mon, 11 Jun 2012 09:50:29 +0200] DEBUG: Sending HTTP Request via GET to
>
localhost:5984/chef/_design/id_map/_view/name_to_id
>
[Mon, 11 Jun 2012 09:50:29 +0200] DEBUG: Authenticating Client:
>
#<Chef::ApiClient:0x7f573c1d54a8 @couchdb=#<Chef::CouchDB:0x7f573c1db498
>
@db="chef", @rest=#<Chef::REST:0x7f573c1db470 @sign_request=true,
>
@default_headers={}, @url="http://localhost:5984", @sign_on_redirect=true, ;
>
@redirect_limit=10, @redirects_followed=0,
>
@auth_credentials=#<Chef::REST::AuthCredentials:0x7f573c1db218
>
@key_file=nil, @client_name=nil>, @cookies={}>>, @public_key="-----BEGIN
>
RSA PUBLIC
>
KEY-----\nMIIBCgKCAQEAskYoWJdzJF+jzGJuyJC9P4CRKUTIBdEthsRmitw2A+I5eoIVhJub\nkdWYGLjbzvy/WuVE8rfsAmJlolvRkcaXWYn+CxFopS3SUZ0EXouliTF2GLtE0jJO\n3L3OoEc+l2jIPfpcfvoLr01W/djk2Ap+4TWOiKbtphUf4ExEzr1eJ5pT2XMoPRT7\ncYkXSqpec3v1Zbusg+xzeEVK66JtBwfwKSW0NX0L96PdRbGrvrSe+RYEJyFzhyYJ\ny3NwkeCk97OyO8KfYoE8H5wkQ11jifhP38hvwnyHDFybW0XKyhEZC2CSP2GEwrik\nMCVzKX1MuHhFMXTncYlVwiWZ3X2VrDbicwIDAQAB\n-----END
>
RSA PUBLIC KEY-----\n",
>
@couchdb_id="6c2d5608-4c7e-4163-a57a-a2b3f795af57", @admin=true,
>
@couchdb_rev="1-788bfa953a0d2c07f0dddaf7fbb12214", @private_key=nil,
>
@name="j_langenbach", @index_id="6c2d5608-4
c7e-4163-a57a-a2b3f795af57">
>
>
[Mon, 11 Jun 2012 09:50:29 +0200] DEBUG: Initializing header auth :
>
#<Merb::Request:0x7f573c1dfbd8 @route=/search/reindex,
>
@route_params={:action=>"reindex", :controller=>"search"},
>
@multipart_params={}, @env={"HTTP_HOST"=>"chef.kdz-ws.intra.net:4001
>
(http://chef.kdz-ws.intra.net:4001)", "HTTP_ACCEPT"=>"application/json",
>
"SERVER_NAME"=>"chef.kdz-ws.intra.net (http://chef.kdz-ws.intra.net)",
>
"rack.url_scheme"=>"http", "CONTENT_LENGTH"=>"2", "HTTP_USER_AGENT"=>"Chef
>
Knife/0.10.8 (ruby-1.9.3-p125; ohai-0.6.12; x86_64-linux;
>
+http://opscode.com)", "REQUEST_PATH"=>"//search/reindex",
>
"SERVER_PROTOCOL"=>"HTTP/1.1", "CONTENT_TYPE"=>"application/json",
>
"HTTP_X_CHEF_VERSION"=>"0.10.8",
>
"HTTP_X_OPS_CONTENT_HASH"=>"vyGp6PvFo4RvsFtPoIWeCReyIC8=",
>
"HTTP_X_OPS_TIMESTAMP"=>"2012-06-11T07:50:30Z",
>
"rack.errors"=>#<IO:0x7f57437c0aa0>, "async.callback"=>#<Method:
>
Thin::Connection#post_process>, "REMOTE_ADDR"=>"172.16.81.180",
>
"PATH_INFO"=>"//search/reindex", "rack.run_once"=>false, "rack.version"=>
[1, 0], "SERVER_SOFTWARE"=>"thin 1.2.4 codename Flaming Astroboy",
"SCRIPT_NAME"=>"",
"HTTP_X_OPS_AUTHORIZATION_1"=>"JnNPHNDCMxAxNRebYy5RFuTEvaT5a9W+xHNi1Za+EqH93lPm8TSNgpQr0pfk",
"HTTP_X_OPS_AUTHORIZATION_2"=>"1mjiotOfWfBbz6dAclzGrUc55gv/J35vfQydABF2YYXc2oUIlvEDmHguPm6Q",
"HTTP_VERSION"=>"HTTP/1.1", "rack.multithread"=>false,
"HTTP_X_OPS_AUTHORIZATION_3"=>"z0bdvrgFSnXJ9JJLqjfBdIgYfZX5VkCqSP9SlUdKnWQ3GEPV3n11z3PItltH",
"REQUEST_URI"=>"//search/reindex", "rack.multiprocess"=>false,
"SERVER_PORT"=>"4001",
"HTTP_X_OPS_AUTHORIZATION_4"=>"6WqNv5GD7dZICyMIpTyU/NnivtpHyb50bgl+ZvnKMppDUignv2QuTQUn46oY",
"async.close"=>#<EventMachine::DefaultDeferrable:0x7f573c1dfed0>,
"HTTP_X_OPS_AUTHORIZATION_5"=>"cMy06RqkqBHUHD94T3jYJCZRM4kSUig617Z9YisMMSpm42Jf7nVokn2Mp7dJ",
"HTTP_X_OPS_USERID"=>"j_langenbach", "REQUEST_METHOD"=>"POST",
"HTTP_X_OPS_AUTHORIZATION_6"=>"DQRs4OXc/OUhTr4y/iZ+HJQyhCaXNAu26c9a7wwkXQ==",
"QUERY_STRING"=>"", "GATEWAY_INTERFACE"=>"CGI/1.2",
"HTTP_CONNECTION"=>"close", "HTTP_X_OPS_SI
GN"=>"version=1.0", "rack.input"=>#<StringIO:0x7f573c1e75b8>},
@json_params={}, @raw_post="{}", @params={"action"=>"reindex",
"controller"=>"search"}, @body_params=nil, @method=:post, @query_params={},
@xml_params=nil, @body_and_query_params={}, @body=#<StringIO:0x7f573c1e75b8>>
>
[Mon, 11 Jun 2012 09:50:29 +0200] DEBUG: Parsed signing description:
>
{:version=>"1.0"}
>
[Mon, 11 Jun 2012 09:50:29 +0200] DEBUG: Digesting body: '{}'
>
[Mon, 11 Jun 2012 09:50:29 +0200] DEBUG: Reconstituted (user-supplied)
>
request signature:
>
JnNPHNDCMxAxNRebYy5RFuTEvaT5a9W+xHNi1Za+EqH93lPm8TSNgpQr0pfk
>
1mjiotOfWfBbz6dAclzGrUc55gv/J35vfQydABF2YYXc2oUIlvEDmHguPm6Q
>
z0bdvrgFSnXJ9JJLqjfBdIgYfZX5VkCqSP9SlUdKnWQ3GEPV3n11z3PItltH
>
6WqNv5GD7dZICyMIpTyU/NnivtpHyb50bgl+ZvnKMppDUignv2QuTQUn46oY
>
cMy06RqkqBHUHD94T3jYJCZRM4kSUig617Z9YisMMSpm42Jf7nVokn2Mp7dJ
>
DQRs4OXc/OUhTr4y/iZ+HJQyhCaXNAu26c9a7wwkXQ==
>
[Mon, 11 Jun 2012 09:50:29 +0200] DEBUG: Verifying request signature:
>
[Mon, 11 Jun 2012 09:50:29 +0200] DEBUG: Expected Block is: 'Method:POST
>
Hashed Path:ZKpVFjmsFMEHqnjeztcib+NRIgo=
>
X-Ops-Content-Hash:vyGp6PvFo4RvsFtPoIWeCReyIC8=
>
X-Ops-Timestamp:2012-06-11T07:50:30Z
>
X-Ops-UserId:j_langenbach'
>
[Mon, 11 Jun 2012 09:50:29 +0200] DEBUG: Decrypted block is: 'Method:POST
>
Hashed Path:ZKpVFjmsFMEHqnjeztcib+NRIgo=
>
X-Ops-Content-Hash:vyGp6PvFo4RvsFtPoIWeCReyIC8=
>
X-Ops-Timestamp:2012-06-11T07:50:30Z
>
X-Ops-UserId:j_langenbach'
>
[Mon, 11 Jun 2012 09:50:29 +0200] DEBUG: Signatures match? : 'true'
>
[Mon, 11 Jun 2012 09:50:29 +0200] DEBUG: Request time difference: 0.155941,
>
within 900 seconds? : true
>
[Mon, 11 Jun 2012 09:50:29 +0200] DEBUG: Expected content hash is:
>
'vyGp6PvFo4RvsFtPoIWeCReyIC8='
>
[Mon, 11 Jun 2012 09:50:29 +0200] DEBUG: Request Content Hash is:
>
'vyGp6PvFo4RvsFtPoIWeCReyIC8='
>
[Mon, 11 Jun 2012 09:50:29 +0200] DEBUG: Hashes match?: true
>
[Mon, 11 Jun 2012 09:50:29 +0200] DEBUG: POSTing document to SOLR:
>
<?xml version="1.0" encoding="UTF-8"?>
>
<delete><query>X_CHEF_database_CHEF_X:chef</query></delete>
>
>
[Mon, 11 Jun 2012 09:50:30 +0200] DEBUG: POSTing document to SOLR:
>
<?xml version="1.0" encoding="UTF-8"?>
>
<commit/>
>
>
[Mon, 11 Jun 2012 09:50:30 +0200] DEBUG: Sending HTTP Request via GET to
>
localhost:5984/chef/_design/clients/_view/all
>
[Mon, 11 Jun 2012 09:50:30 +0200] FATAL: Chef encountered an error while
>
attempting to load Chef::ApiClient objects back into the index
>
merb : chef-server (api) : worker (port 4001) ~ Started request handling:
>
Mon Jun 11 09:50:29 +0200 2012
>
merb : chef-server (api) : worker (port 4001) ~ Routed to:
>
{"action"=>"reindex", "controller"=>"search"}
>
merb : chef-server (api) : worker (port 4001) ~ Params:
>
{"action"=>"reindex", "controller"=>"search"}
>
merb : chef-server (api) : worker (port 4001) ~ end of file reached -
>
(EOFError)
>
/usr/lib/ruby/1.8/net/protocol.rb:135:in `sysread'
>
/usr/lib/ruby/1.8/net/protocol.rb:135:in `rbuf_fill'
>
/usr/lib/ruby/1.8/timeout.rb:67:in `timeout'
>
/usr/lib/ruby/1.8/timeout.rb:101:in `timeout'
>
/usr/lib/ruby/1.8/net/protocol.rb:134:in `rbuf_fill'
>
/usr/lib/ruby/1.8/net/protocol.rb:116:in `readuntil'
>
/usr/lib/ruby/1.8/net/protocol.rb:126:in `readline'
>
/usr/lib/ruby/1.8/net/http.rb:2028:in `read_status_line'
>
/usr/lib/ruby/1.8/net/http.rb:2017:in `read_new'
>
/usr/lib/ruby/1.8/net/http.rb:1051:in `request'
>
/usr/lib/ruby/1.8/net/http.rb:1037:in `request'
>
/usr/lib/ruby/1.8/net/http.rb:543:in `start'
>
/usr/lib/ruby/1.8/net/http.rb:1035:in `request'
>
/usr/lib/ruby/vendor_ruby/chef/rest/rest_request.rb:99:in `call'
>
/usr/lib/ruby/vendor_ruby/chef/rest/rest_request.rb:114:in
>
`hide_net_http_bug'
>
/usr/lib/ruby/vendor_ruby/chef/rest/rest_request.rb:98:in `call'
>
/usr/lib/ruby/vendor_ruby/chef/rest.rb:241:in `api_request'
>
/usr/lib/ruby/vendor_ruby/chef/rest.rb:328:in `retriable_rest_request'
>
/usr/lib/ruby/vendor_ruby/chef/rest.rb:240:in `api_request'
>
/usr/lib/ruby/vendor_ruby/chef/rest.rb:128:in `get_rest'
>
/usr/lib/ruby/vendor_ruby/chef/couchdb.rb:181:in `list'
>
/usr/lib/ruby/vendor_ruby/chef/api_client.rb:178:in `cdb_list'
>
/usr/lib/ruby/vendor_ruby/chef/solr_query.rb:167:in `reindex_all'
>
/usr/lib/ruby/vendor_ruby/chef/solr_query.rb:156:in `rebuild_index'
>
/usr/lib/ruby/vendor_ruby/chef/solr_query.rb:155:in `each'
>
/usr/lib/ruby/vendor_ruby/chef/solr_query.rb:155:in `rebuild_index'
>
/usr/share/chef-server-api/app/controllers/search.rb:55:in `reindex'
>
/usr/lib/ruby/1.8/merb-core/controller/abstract_controller.rb:315:in `send'
>
/usr/lib/ruby/1.8/merb-core/controller/abstract_controller.rb:315:in
>
`_call_action'
>
/usr/lib/ruby/1.8/merb-core/controller/abstract_controller.rb:289:in
>
`_dispatch'
>
/usr/lib/ruby/1.8/merb-core/controller/merb_controller.rb:252:in `_dispatch'
>
/usr/lib/ruby/1.8/merb-core/dispatch/dispatcher.rb:102:in `dispatch_action'
>
/usr/lib/ruby/1.8/merb-core/dispatch/dispatcher.rb:74:in `handle'
>
/usr/lib/ruby/1.8/merb-core/dispatch/dispatcher.rb:36:in `handle'
>
/usr/lib/ruby/1.8/merb-core/rack/application.rb:17:in `call'
>
/usr/lib/ruby/1.8/merb-core/rack/middleware/static.rb:28:in `call'
>
/usr/lib/ruby/1.8/rack/content_length.rb:13:in `call'
>
/usr/lib/ruby/1.8/thin/connection.rb:76:in `pre_process'
>
/usr/lib/ruby/1.8/thin/connection.rb:74:in `catch'
>
/usr/lib/ruby/1.8/thin/connection.rb:74:in `pre_process'
>
/usr/lib/ruby/1.8/thin/connection.rb:57:in `process'
>
/usr/lib/ruby/1.8/thin/connection.rb:42:in `receive_data'
>
/usr/lib/ruby/1.8/eventmachine.rb:257:in `run_machine'
>
/usr/lib/ruby/1.8/eventmachine.rb:257:in `run'
>
/usr/lib/ruby/1.8/thin/backends/base.rb:57:in `start'
>
/usr/lib/ruby/1.8/thin/server.rb:156:in `start'
>
/usr/lib/ruby/1.8/merb-core/rack/adapter/thin.rb:30:in `start_server'
>
/usr/lib/ruby/1.8/merb-core/rack/adapter/abstract.rb:298:in `start_at_port'
>
/usr/lib/ruby/1.8/merb-core/rack/adapter/abstract.rb:128:in `start'
>
/usr/lib/ruby/1.8/merb-core/server.rb:174:in `bootup'
>
/usr/lib/ruby/1.8/merb-core/server.rb:42:in `start'
>
/usr/lib/ruby/1.8/merb-core.rb:170:in `start'
>
/usr/sbin/chef-server:86
>
merb : chef-server (api) : worker (port 4001) ~ Params:
>
{"action"=>"reindex", "controller"=>"search"}
>
merb : chef-server (api) : worker (port 4001) ~
>
/usr/lib/ruby/1.8/net/protocol.rb:135:in `rbuf_fill'
>
/usr/lib/ruby/1.8/timeout.rb:67:in `timeout'
>
/usr/lib/ruby/1.8/timeout.rb:101:in `timeout'
>
/usr/lib/ruby/1.8/net/protocol.rb:134:in `rbuf_fill'
>
/usr/lib/ruby/1.8/net/protocol.rb:116:in `readuntil'
>
/usr/lib/ruby/1.8/net/protocol.rb:126:in `readline'
>
/usr/lib/ruby/1.8/net/http.rb:2028:in `read_status_line'
>
/usr/lib/ruby/1.8/net/http.rb:2017:in `read_new'
>
/usr/lib/ruby/1.8/net/http.rb:1051:in `request'
>
/usr/lib/ruby/1.8/net/http.rb:1037:in `request'
>
/usr/lib/ruby/1.8/net/http.rb:543:in `start'
>
/usr/lib/ruby/1.8/net/http.rb:1035:in `request'
>
/usr/lib/ruby/vendor_ruby/chef/rest/rest_request.rb:99:in `call'
>
/usr/lib/ruby/vendor_ruby/chef/rest/rest_request.rb:114:in
>
`hide_net_http_bug'
>
/usr/lib/ruby/vendor_ruby/chef/rest/rest_request.rb:98:in `call'
>
/usr/lib/ruby/vendor_ruby/chef/rest.rb:241:in `api_request'
>
/usr/lib/ruby/vendor_ruby/chef/rest.rb:328:in `retriable_rest_request'
>
/usr/lib/ruby/vendor_ruby/chef/rest.rb:240:in `api_request'
>
/usr/lib/ruby/vendor_ruby/chef/rest.rb:128:in `get_rest'
>
/usr/lib/ruby/vendor_ruby/chef/couchdb.rb:181:in `list'
>
/usr/lib/ruby/vendor_ruby/chef/api_client.rb:178:in `cdb_list'
>
/usr/lib/ruby/vendor_ruby/chef/solr_query.rb:167:in `reindex_all'
>
/usr/lib/ruby/vendor_ruby/chef/solr_query.rb:156:in `rebuild_index'
>
/usr/lib/ruby/vendor_ruby/chef/solr_query.rb:155:in `each'
>
/usr/lib/ruby/vendor_ruby/chef/solr_query.rb:155:in `rebuild_index'
>
/usr/share/chef-server-api/app/controllers/search.rb:55:in `reindex'
>
/usr/lib/ruby/1.8/merb-core/controller/abstract_controller.rb:315:in `send'
>
/usr/lib/ruby/1.8/merb-core/controller/abstract_controller.rb:315:in
>
`_call_action'
>
/usr/lib/ruby/1.8/merb-core/controller/abstract_controller.rb:289:in
>
`_dispatch'
>
/usr/lib/ruby/1.8/merb-core/controller/merb_controller.rb:252:in `_dispatch'
>
/usr/lib/ruby/1.8/merb-core/dispatch/dispatcher.rb:102:in `dispatch_action'
>
/usr/lib/ruby/1.8/merb-core/dispatch/dispatcher.rb:74:in `handle'
>
/usr/lib/ruby/1.8/merb-core/dispatch/dispatcher.rb:36:in `handle'
>
/usr/lib/ruby/1.8/merb-core/rack/application.rb:17:in `call'
>
/usr/lib/ruby/1.8/merb-core/rack/middleware/static.rb:28:in `call'
>
/usr/lib/ruby/1.8/rack/content_length.rb:13:in `call'
>
/usr/lib/ruby/1.8/thin/connection.rb:76:in `pre_process'
>
/usr/lib/ruby/1.8/thin/connection.rb:74:in `catch'
>
/usr/lib/ruby/1.8/thin/connection.rb:74:in `pre_process'
>
/usr/lib/ruby/1.8/thin/connection.rb:57:in `process'
>
/usr/lib/ruby/1.8/thin/connection.rb:42:in `receive_data'
>
/usr/lib/ruby/1.8/eventmachine.rb:257:in `run_machine'
>
/usr/lib/ruby/1.8/eventmachine.rb:257:in `run'
>
/usr/lib/ruby/1.8/thin/backends/base.rb:57:in `start'
>
/usr/lib/ruby/1.8/thin/server.rb:156:in `start'
>
/usr/lib/ruby/1.8/merb-core/rack/adapter/thin.rb:30:in `start_server'
>
/usr/lib/ruby/1.8/merb-core/rack/adapter/abstract.rb:298:in `start_at_port'
>
/usr/lib/ruby/1.8/merb-core/rack/adapter/abstract.rb:128:in `start'
>
/usr/lib/ruby/1.8/merb-core/server.rb:174:in `bootup'
>
/usr/lib/ruby/1.8/merb-core/server.rb:42:in `start'
>
/usr/lib/ruby/1.8/merb-core.rb:170:in `start'
>
/usr/sbin/chef-server:86
>
merb : chef-server (api) : worker (port 4001) ~ Params:
>
{"action"=>"reindex", "controller"=>"search"}
>
merb : chef-server (api) : worker (port 4001) ~
>
>
>
I also started chef-server with strace to see if it is a real file or the
>
tcp stream, I couldn't find any relevant file which chef tries to access
>
before the error occurs.
>
>
Any Ideas?
>
>
Thank you
Both of those stack traces point to issues fetching documents from CouchDB.
What version of CouchDB are you using, and is there any relevant info in the
logs?
--
Dan DeLeo
Archive powered by MHonArc 2.6.16.