[chef] Re: Re: nodes not showing up in `knife status`

[Sam Pointer < >]

As has previously been said, "knife node show" hits couch, "knife status" hits the index. If nodes are missing from the index, and they are identical to your other nodes and have failed to appear within a given time-frame, it is likely that the expander is not properly forwarding couch updates to the index.

Try: http://wiki.opscode.com/display/chef/Chef+Indexer#ChefIndexer-ChefExpanderOperationandTroubleshooting for steps to examine the expander state, and also consider interrogating the length of your queues.

If you don't want to or can't go through the exercise of monitoring all of the individual back-end components then a a nice basic litmus check is to check your couchdb ohai_time attribute against the same attribute stored in solr, and ensure that they match. ohai_time is the time the attributes were last persisted to the back-end by the client.

Here is a simple example:

#!/usr/bin/env ruby

require 'chef/node'

# Assumes Chef::Config is populated with correct values.

hostname = 'host.corp.com'

begin

  couch_node = Chef::Node.cdb_load(hostname)

rescue Chef::Exceptions::CouchDBNotFound

  # I'm probably not managed by Chef.

rescue Exception => e

  # Something else went terribly wrong

end

begin

  index_node = Chef::Search::Query.new.search(:node, "name:#{hostname}")

rescue Exception => e

  # Something went terribly wrong

end

couchdb_time = couch_node.ohai_time.to_i

begin

  index_time = index_node[0][0].ohai_time.to_i

rescue NoMethodError

  # The node index data is missing. Alert!

end

# Now check that neither couchdb_time or index_time are nil, and that they aren't too

# old, and they're identical (or within some acceptable boundary), etc, etc.

--

Sam.

On Tue, Jan 29, 2013 at 2:22 AM, Chirag Jog < " target="_blank"> > wrote:

Hello Spike,

Knife status uses the solr based search mechanism to find all the nodes . It seems the nodes that are missing don't meet that search criteria . 

Can you share the entire knife status command ?

--

Regards, Chirag 

On Tuesday, January 29, 2013, Spike Grobstein wrote:

Hi Chefs,

I've recently added some new nodes to our chef-server, but when I run `knife status`, I don't see these nodes in the list.

`knife node edit $NODE` works
`knife client edit $NODE` works

the nodes show up in the webui.

the name in /etc/chef/client.rb is correct on the nodes.

Is there anything else I should look at?

75 total nodes show up when I run `knife status`. We're missing about 6 nodes from the list that I can see so far. I have a full range of last-run-at times; from minutes to over 6000 hours (some nodes I haven't upgraded the client on for quite a while), so I don't think it's just truncating the list.

Thanks in advance!


...spike

--

Regards,

Chirag Jog

Chief Technology Officer, 

Clogeny Technologies | http://clogeny.com 

(M) 0091-9766619440 | Skype: chirag.jog