[chef] opscode nagios quick start

[Tim Dunphy < >]

Hey guys,

 Coming from long familiarity with puppet I am completely jazzed by what I've seen so far of the capabilities of chef. Knife is simply the most amazing server automation tool I've ever seen. Period. Especially for could instances.

 Ok, now that I'm done gushing let me describe the issue I'm having. The nagios quickstart tutorial on the opscode wiki is astounding and works amazingly well. Except, it's not completely flawless in my case as you might have gathered. For some reason my validation.pem never makes it to the new ec2 server and therefore my chef (community) chef server cannot validate the client, and apply the roles to the new instance. So I have to log into the new server (which in it self is cool) to make sure I can. Then I trey to run chef-client and have it complain about it not being able to validate against the chef server. I then scp up my validation.pem into place and add the roles to the new server on the command line. Then I run chef-client again on the new instance and have my new nagios server and can log into the web interface. Still amazing, but I want it to be as seemless as the how-to implies it can be. :D Here's the link to the how-to for quick reference:

http://wiki.opscode.com/display/chef/Nagios+Quick+Start



 I followed all the steps of the tutorial including cloning the git repo and especially these particular steps:

mkdir ~/nagios-quick-start/.chef

cp ~/chef-repo/.chef/knife.rb ~/nagios-quick-start/.chef

cp ~/chef-repo/.chef/USERNAME.pem ~/nagios-quick-start/.chef

cp ~/chef-repo/.chef/validation.pem ~/nagios-quick-start/.chef

Just to be clear I am using my own open source community chef server 10.16. For the validation.pem I copied the one generated by my chef server into my git repo. Just to be double sure I scp'd it down from my chef server to ~/nagios-quick-start/.chef and also of course copied over my knife.rb and username.pem.

Here's a quick look at my knife.rb

---------------------------------------------------------------------------------

 :~/chef-repo] #cat ~/.chef/knife.rb
log_level                :info
log_location             STDOUT
node_name                'dunphy'
client_key               '/Users/dunphy/.chef/dunphy.pem'
validation_client_name   'chef-validator'
validation_key           '/etc/chef/validation.pem'
chef_server_url          'http://chef.mydomain.com:4000'
cache_type               'BasicFile'
cache_options( :path => '/Users/dunphy/.chef/checksums' )
cookbook_path            ["/Users/dunphy/chef-repo/cookbooks"]

## EC2 Authentication
knife[:aws_access_key_id]  = "aws-key-here"
knife[:aws_secret_access_key] = "aws-secret-here"

# Rackspace:
knife[:rackspace_api_key]      = "rackspace-key-here"
knife[:rackspace_api_username] = "myuser"

# Linode
knife[:linode_api_key] = "linode-key-here"

----------------------------------------------------------------

This is the exact command I used:


----------------------------------------------------------------

 :~] #knife ec2 server create -G default -I ami-7000f019 -f m1.small -S mykeypair-aws -i ~/.ssh/id_rsa -x ubuntu -r 'role[production],role[base],role[monitoring]'


----------------------------------------------------------------

When I log into the new instance, this is what the client.rb looks like

 :~# cat /etc/chef/client.rb
log_level        :info
log_location     STDOUT
chef_server_url  "http://chef.mydomain.com:4000"
validation_client_name "chef-validator"
node_name "i-23d72052"

------------------------------------------

This is what the chef-validator looks like on my chef command line:

 :~] #knife client show chef-validator
_rev:        1-ff7b4f7168c42a35431f815bd48ddbf2
admin:       false
chef_type:   client
json_class:  Chef::ApiClient
name:        chef-validator
public_key:  -----BEGIN RSA PUBLIC KEY-----
             MIIBCgKCAQEA6KepnhFvTGDXyhwFFc0gxO7exMgMqOcs5BPKa+0vo5ruC0jihz5I
             CZoblwHTxzVoSryQY6kJzwJvD/S6csmDGu1Wr7wuY4hMr9vaAWv9t6ODfAX59VLT
             dUlkas6KyXQdGWcYqMNaV0BSqd6/IqAOiEPdVx3TfGLMa9zc+odJ0tuqmIx7Line
             Y4WtWYIctAp76RdyLLO78Vv06Mwd4CL8VSk+mT2eMZGiQL5zYf20S3zejsNFBHQo
             0aA92RwmWm0x9zslPTBXBtQKM98KCR7tXDtTtkJUYD/5ne+Gl1Vzu/OHej4e3RpM
             pz7TwsTrAu4SXXcUy22peVGpGivMMf61/QIDAQAB
             -----END RSA PUBLIC KEY-----

------------------------------------------

Back on the new instance, if I cat the firstboot.json all looks well. It looks as if the roles I applied to the server are listed:

------

 :~# cat /etc/chef/first-boot.json 
{"run_list":["role[production]","role[base]","role[monitoring]"]}


------

But when I run chef-client this is what I see:

 :~$ sudo chef-client
[2012-12-31T04:54:32+00:00] INFO: *** Chef 10.16.2 ***
[2012-12-31T04:54:33+00:00] INFO: Client key /etc/chef/client.pem is not present - registering
[2012-12-31T04:54:33+00:00] INFO: HTTP Request Returned 401 Unauthorized: Failed to authenticate. Ensure that your client key is valid.

================================================================================
Chef encountered an error attempting to create the client "i-53ae5822"
================================================================================

Authentication Error:
---------------------
Failed to authenticate to the chef server (http 401).

Server Response:
----------------
Failed to authenticate. Ensure that your client key is valid.

Relevant Config Settings:
-------------------------
chef_server_url         "http://chef.mydomain.com:4000"
validation_client_name  "chef-validator"
validation_key          "/etc/chef/validation.pem"

If these settings are correct, your validation_key may be invalid.

[2012-12-31T04:54:33+00:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
[2012-12-31T04:54:33+00:00] FATAL: Net::HTTPServerException: 401 "Unauthorized"

When I do a knife client list and a knife node list I see the new ec2 instance. But the roles have not been applied. So i scp my validation.pem up to the new instance, do a 'diff' between that one and the one at /etc/chef/validation.pem and they are _completely_ different. 

And as mentioned I copy over the validation.pem to the right place and bang! I have a new nagios server. I'm more or less willing to settle for this level of coolness, but man it would be amazing if I could make it seemless. Not to mention impress my coworkers at the big website where i work who (believe it or not) are TOTALLY re-inventing the wheel by invention their own 'in-house' version of chef. Or a very chef like command line tool built in ruby that ties together puppet and mcollective. I kid you not!

So the problem is that for some reason the only step that's not working is that the new instance is using a newly generated validation.pem from the fresh chef install and not the one specified in my knife.rb. At least this is what I believe should happen.

Can anyone be kind enough and perspicacious enough to point out where I'm going wrong?

Thanks!
Tim

Thanks
Tim

-- 
GPG me!!

gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B