[chef] Re: Re: RE: Re: odd errors after restoring couchdb onto new chef server

more verbosely, here's what i did. and i think this verbosity will
help you not get tripped up on some things, like i did while working
thru this:

curl -XDELETE http://127.0.0.1:5984/chef
curl -XPUT http://127.0.0.1:5984/chef

the above assumes you have a fresh chef-server install and that it
started. when it starts, it creates a new keypair for the chef-validator
client. i want to retain my original keypair (chef-validator pubkey
present in couchdb and the validation.pem private key) so that i don't
have to rekey my entire infrastructure.

couchdb-load --input=chef_couchdb.latest http://127.0.0.1:5984/chef ;
--ignore-errors 1>couchdb-load.stdout 2>couchdb-load.stderr

why --ignore-errors? i dunno :\  it didn't load without that. but it
seeeeeems to be ok.

after this loads, i fire up a working knife client and make sure my
chef-validator pubkey is what i want.

and then i delete all cookbooks, and reload them all from my chef-repo
in git.

and i have the best of intentions to add my experience with restoring
a chef server from backup to the opscode wiki .. :>


On Wed, 07 Mar 2012, Chris wrote:

> couchdb-load http://<server>:5984/chef < filenameofdatabasedump
> 
> Assuming you've installed that tool from here:
> http://code.google.com/p/couchdb-python/
> 
> 
> 
> On Wed, Mar 7, 2012 at 9:35 AM, Van Fossan,Randy 
> <
 >
>  wrote:
> 
> > Kallen,
> >
> > Do you have the steps you used to backup and then restore the couchdb to
> > a newly built server?   If you could post them, I would be extremely
> > grateful..
> >
> > Thanks
> >
> > -----Original Message-----
> > From: 
> > 
 
> >  
> > [mailto:
 
> > Sent: Tuesday, February 28, 2012 2:09 PM
> > To: 
> > 
 
> > Subject: [chef] Re: Re: Re: Re: Re: odd errors after restoring couchdb
> > onto new chef server
> >
> >
> > just to close out this thread:
> >
> > to solve this on the newly built server, i purged all cookbooks, then
> > reuploaded all from my chef-repo in git. i didn't attempt to backup and
> > restore the checksum_path dir.
> >
> > the new restored server seems to be working ok.
> >
> > thanks for the help!
> >
> > kallen
> >
> >
> > On Fri, 24 Feb 2012, Daniel DeLeo wrote:
> >
> > >
> > >
> > > On Friday, February 24, 2012 at 6:18 PM, 
> > > 
 
> > >  wrote:
> > >
> > > > On Fri, 24 Feb 2012, 
> > > > 
 
> > (mailto:
 )
> >  wrote:
> > > >
> > > > > On Sat, 25 Feb 2012, AJ Christensen wrote:
> > > > >
> > > > > > > 2) wierd errors running a chef-client against new server,
> > like:
> > > > > > > * ERROR: Server returned error for
> > > > > > > https://chef.venus.spergacula.com/cookbooks/users/1.0.0/files/
> > > > > > > [snip]
> > > > > > > * cookbook_file EOFError: end of file reached
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > > did you copy the upload/copy the cookbooks to the new server?
> > > > > > are they in the file cache? Kind of looks like couchdb has
> > > > > > information about the files, but the files aren't on the
> > > > > > chef-server file system (hazard
> > > > > > guess)
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > i hadn't done an upload/copy of the cookbooks to the new server
> > > > > because i thought the cookbooks were loaded in the couchdb-load.
> > > > > when i run knife cookbook list, it looks like all my cookbooks are
> > already there.
> > > > >
> > > > > regardless, i just reuploaded all cookbooks from my chef-repo to
> > > > > the new server. i saw no upload errors. now, running chef-client
> > > > > on a non-chef-server node, pointing chef_server_url directly at
> > > > > port 4000 rather than hitting nginx proxy over https, is erroring
> > > > > out on "EOFError - end of file reached", which is similar to error
> > shown in gist.
> > > > >
> > > > >
> > > > > [Fri, 24 Feb 2012 23:13:08 +0000] DEBUG: Sending HTTP Request via
> > > > > GET to
> > > > > chef.venus.spergacula.com:4000/cookbooks/mrepo/0.0.1/files/c0cffe5
> > > > > 0c70191353d8bd9a8bd568ce5
> > > > > (http://chef.venus.spergacula.com:4000/cookbooks/mrepo/0.0.1/files
> > > > > /c0cffe50c70191353d8bd9a8bd568ce5)
> > > > > [Fri, 24 Feb 2012 23:13:08 +0000] ERROR: Running exception
> > > > > handlers [Fri, 24 Feb 2012 23:13:08 +0000] FATAL: Saving node
> > > > > information to /var/cache/chef/failed-run-data.json
> > > > > [Fri, 24 Feb 2012 23:13:08 +0000] ERROR: Exception handlers
> > > > > complete [Fri, 24 Feb 2012 23:13:08 +0000] DEBUG: Re-raising
> > > > > exception: EOFError - end of file reached
> > /usr/lib/ruby/1.8/net/protocol.rb:135:in `sysread'
> > > > > /usr/lib/ruby/1.8/net/protocol.rb:135:in `rbuf_fill'
> > > > > /usr/lib/ruby/1.8/timeout.rb:67:in `timeout'
> > > > > /usr/lib/ruby/1.8/timeout.rb:101:in `timeout'
> > > > >
> > > > >
> > > > > i'm looking at chef-server debug log and couchdb log. but nothing
> > > > > is jumping out at me as relevant. but i may not recognize
> > relevancy..
> > > > >
> > > > >
> > > > >
> > > > > last 3 lines of couchdb log fwiw:
> > > > >
> > > > > [Fri, 24 Feb 2012 23:18:13 GMT] [info] [<0.17029.0>] 127.0.0.1 - -
> >
> > > > > 'POST' /chef/_all_docs?include_docs=true 200 [Fri, 24 Feb 2012
> > > > > 23:18:14 GMT] [info] [<0.17030.0>] 127.0.0.1 - - 'GET'
> > > > > /chef/_design/id_map/_view/name_to_id?key=[%22client%22,%22admin2.
> > > > > venus.spergacula.com
> > > > > (http://22admin2.venus.spergacula.com)%22]&include_docs=true 200
> > > > > [Fri, 24 Feb 2012 23:18:14 GMT] [info] [<0.17031.0>] 127.0.0.1 - -
> >
> > > > > 'GET'
> > > > > /chef/_design/id_map/_view/name_to_id?key=[%22cookbook_version%22,
> > > > > %22mrepo-0.0.1%22]&include_docs=true 200
> > > > >
> > > > >
> > > > > thoughts?
> > > >
> > > >
> > > > trying various things, flailing. for the heck of it:
> > > >
> > > > * deleted possible problematic cookbook (mrepo) and reuploaded. same
> > errors.
> > > > * deleted client node which i'm testing on (admin2), reran, same
> > errors.
> > > > * reloaded couchdb dump again
> > > >
> > > > and now a different cookbook download claims "EOFError - end of file
> > reached".
> > > >
> > > >
> > > > [Sat, 25 Feb 2012 02:14:22 +0000] DEBUG: Sending HTTP Request via
> > > > GET to
> > > > cheftain04.venus.spergacula.com:4000/cookbooks/begin/0.0.1/files/e44
> > > > 7bc0fcbef6dafe72007fe70507767
> > > > (http://cheftain04.venus.spergacula.com:4000/cookbooks/begin/0.0.1/f
> > > > iles/e447bc0fcbef6dafe72007fe70507767)
> > > > [Sat, 25 Feb 2012 02:14:22 +0000] ERROR: Running exception handlers
> > > > [Sat, 25 Feb 2012 02:14:22 +0000] FATAL: Saving node information to
> > > > /var/cache/chef/failed-run-data.json
> > > > [Sat, 25 Feb 2012 02:14:22 +0000] ERROR: Exception handlers complete
> >
> > > > [Sat, 25 Feb 2012 02:14:22 +0000] DEBUG: Re-raising exception:
> > > > EOFError - end of file reached
> > /usr/lib/ruby/1.8/net/protocol.rb:135:in `sysread'
> > > > /usr/lib/ruby/1.8/net/protocol.rb:135:in `rbuf_fill'
> > > >
> > > >
> > > > dunno what to do here..
> > > The files that comprise your cookbooks aren't stored in couch, they're
> > stored according to their MD5 under whatever you've configured for
> > `checksum_path` in your server.rb. When you backup/restore your chef
> > server, you need to restore these files also.
> > >
> > > The error you're seeing occurs because the file transfer happens
> > outside of the normal request cycle, so server side errors aren't
> > converted to 500s[1].
> > >
> > > To fix it, you can either bulk delete all cookbooks using the
> > `--purge` option, or copy the files.
> > >
> > > HTH,
> > >
> > > --
> > > Dan DeLeo
> > >
> > >
> > > 1. Merb is trying to hand off the work to another thread (in the case
> > of threaded servers) or the event loop (in case of evented servers like
> > thin) so that the transfer can happen while other requests are
> > processed. I'm not sure if this even works any more. In any case, I
> > believe a user has contributed a fix for this problem.
> >
> >
> >
> 
> 
> -- 
> Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
> permitted by applicable law.