[chef] Re: Re: Re: Re: Re: Re: node search does not return all hosts


Chronological Thread 
  • From: Brian Smith < >
  • To:
  • Subject: [chef] Re: Re: Re: Re: Re: Re: node search does not return all hosts
  • Date: Tue, 18 Oct 2011 14:11:29 -0400

If I run chef-expander -n1 manually, I can see the logging output.  It shows:

[Tue, 18 Oct 2011 14:01:31 -0400] INFO: Chef Expander 0.10.4 starting cluster with 1 nodes
[Tue, 18 Oct 2011 14:01:31 -0400] INFO: Starting cluster worker 1
[Tue, 18 Oct 2011 14:01:31 -0400] INFO: Starting Consumers for vnodes 0-1023
[Tue, 18 Oct 2011 14:01:33 -0400] INFO: indexed node[7652cbe5-4601-430b-bc99-e6e91718534d] database[chef] transit,xml,solr-post | 1,0.083899974822998,0.205962896347046 |
[Tue, 18 Oct 2011 14:02:39 -0400] INFO: indexed node[3b87d23a-75ea-4615-8402-172b0207d288] database[chef] transit,xml,solr-post | 0,0.259773015975952,0.30780291557312 |
[Tue, 18 Oct 2011 14:03:27 -0400] INFO: indexed node[3b87d23a-75ea-4615-8402-172b0207d288] database[chef] transit,xml,solr-post | 0,0.242645025253296,0.275618076324463 |
[Tue, 18 Oct 2011 14:03:58 -0400] INFO: indexed node[70325296-64d2-49f8-8cbe-fa2349da0728] database[chef] transit,xml,solr-post | 0,0.136043071746826,0.168022155761719 |
[Tue, 18 Oct 2011 14:03:59 -0400] INFO: indexed node[70325296-64d2-49f8-8cbe-fa2349da0728] database[chef] transit,xml,solr-post | 0,0.106320858001709,0.124919891357422 |
[Tue, 18 Oct 2011 14:04:37 -0400] INFO: indexed client[6b219262-9796-427f-98ab-6009a70bb1c9] database[chef] transit,xml,solr-post | 0,0.000603914260864258,0.0189318656921387 |
[Tue, 18 Oct 2011 14:04:37 -0400] INFO: indexed client[62024cd2-e660-417b-8107-2013cd3e22e8] database[chef] transit,xml,solr-post | 0,0.0004119873046875,0.00909209251403809 |
[Tue, 18 Oct 2011 14:04:37 -0400] INFO: indexed client[5e309b37-e3f7-475a-9c78-5fcc1ed43b27] database[chef] transit,xml,solr-post | 0,0.000396013259887695,0.00794005393981934 |
[Tue, 18 Oct 2011 14:04:37 -0400] INFO: indexed client[117b3ffa-3872-4efc-b288-a597b1d8d4aa] database[chef] transit,xml,solr-post | 0,0.000380992889404297,0.00734901428222656 |
[Tue, 18 Oct 2011 14:04:37 -0400] INFO: indexed client[2fbd1661-7dcb-4d76-b305-f448778f3bf8] database[chef] transit,xml,solr-post | 0,0.000608921051025391,0.0103509426116943 |
[Tue, 18 Oct 2011 14:04:37 -0400] INFO: indexed client[edfc3802-a65d-4042-91a4-4a7f251e060f] database[chef] transit,xml,solr-post | 0,0.000377178192138672,0.00926113128662109 |
[Tue, 18 Oct 2011 14:04:37 -0400] INFO: indexed client[3bf6326e-c59d-4420-9a40-4fdc92b84482] database[chef] transit,xml,solr-post | 0,0.000382184982299805,0.00770020484924316 |
[Tue, 18 Oct 2011 14:04:37 -0400] INFO: indexed client[9b1caa3c-7dad-4329-bd82-c8106635cea8] database[chef] transit,xml,solr-post | 0,0.00041508674621582,0.00790596008300781 |
[Tue, 18 Oct 2011 14:04:37 -0400] INFO: indexed client[8d5656e3-2244-4c94-939c-9abccbe4b9f9] database[chef] transit,xml,solr-post | 0,0.000387907028198242,0.00792384147644043 |
[Tue, 18 Oct 2011 14:04:37 -0400] INFO: indexed client[ef3e8783-f5b5-47b5-bf20-faf92528c32c] database[chef] transit,xml,solr-post | 0,0.000397920608520508,0.00805187225341797 |
[Tue, 18 Oct 2011 14:04:37 -0400] INFO: indexed client[f0defc29-0f7f-47fa-877f-9ea8b58fad38] database[chef] transit,xml,solr-post | 0,0.000381231307983398,0.00814223289489746 |
[Tue, 18 Oct 2011 14:04:37 -0400] INFO: indexed client[753026ad-db27-496e-97e9-92a8eb379ae9] database[chef] transit,xml,solr-post | 0,0.000420093536376953,0.00829505920410156 |
[Tue, 18 Oct 2011 14:04:37 -0400] INFO: indexed client[9aa15ee8-a2e8-4a3b-9c9b-4d55ac96e226] database[chef] transit,xml,solr-post | 0,0.000396966934204102,0.00800204277038574 |
[Tue, 18 Oct 2011 14:04:37 -0400] INFO: indexed client[a413bf05-0414-44fe-a93d-8ad0dca36e93] database[chef] transit,xml,solr-post | 0,0.000396013259887695,0.00772786140441895 |
[Tue, 18 Oct 2011 14:04:37 -0400] INFO: indexed client[6255831e-1928-4802-94bf-9873a07c3f55] database[chef] transit,xml,solr-post | 0,0.000390052795410156,0.00789713859558105 |
[Tue, 18 Oct 2011 14:04:37 -0400] INFO: indexed client[951de01b-c8df-41fa-856b-45a1e1e71988] database[chef] transit,xml,solr-post | 0,0.000364065170288086,0.00783300399780273 |
[Tue, 18 Oct 2011 14:04:38 -0400] INFO: indexed node[4711372a-163c-4d9a-b8dd-fc374b5e21d5] database[chef] transit,xml,solr-post | 0,0.000235795974731445,0.00694584846496582 |
[Tue, 18 Oct 2011 14:04:38 -0400] INFO: indexed node[0735f918-3ceb-426a-aa54-0dbd6444cb8f] database[chef] transit,xml,solr-post | 0,0.0516819953918457,0.0682859420776367 |
[Tue, 18 Oct 2011 14:04:38 -0400] INFO: indexed node[a820136d-d704-4d56-bc20-36c979b5d27d] database[chef] transit,xml,solr-post | 0,0.0955410003662109,0.108435869216919 |
[Tue, 18 Oct 2011 14:04:38 -0400] INFO: indexed node[7652cbe5-4601-430b-bc99-e6e91718534d] database[chef] transit,xml,solr-post | 0,0.043848991394043,0.0569930076599121 |
[Tue, 18 Oct 2011 14:04:39 -0400] INFO: indexed node[47c3e5b3-ea08-43fb-86ba-66caebde43a4] database[chef] transit,xml,solr-post | 1,0.110408067703247,0.126874208450317 |
[Tue, 18 Oct 2011 14:04:39 -0400] INFO: indexed node[70325296-64d2-49f8-8cbe-fa2349da0728] database[chef] transit,xml,solr-post | 0,0.0638837814331055,0.0798130035400391 |
[Tue, 18 Oct 2011 14:04:39 -0400] INFO: indexed node[3b87d23a-75ea-4615-8402-172b0207d288] database[chef] transit,xml,solr-post | 0,0.206537961959839,0.236276865005493 |
[Tue, 18 Oct 2011 14:04:39 -0400] INFO: indexed node[897aa03d-0804-41ea-8cd6-1265cd6e764c] database[chef] transit,xml,solr-post | 0,0.0518558025360107,0.0679068565368652 |
[Tue, 18 Oct 2011 14:04:39 -0400] INFO: indexed node[fc6ac59c-cdf0-4d16-a172-22cbd35591eb] database[chef] transit,xml,solr-post | 0,0.0623269081115723,0.080096960067749 |
[Tue, 18 Oct 2011 14:04:40 -0400] INFO: indexed node[89e7ae14-21f1-4546-8cdb-986a846e9c8f] database[chef] transit,xml,solr-post | 0,0.180221080780029,0.208502054214478 |
[Tue, 18 Oct 2011 14:04:40 -0400] INFO: indexed node[bd97525b-704a-4601-a313-1e7cb9c85893] database[chef] transit,xml,solr-post | 0,0.0465750694274902,0.117887020111084 |
[Tue, 18 Oct 2011 14:04:40 -0400] INFO: indexed node[87e6263c-cba0-4a6f-b214-9c20451e84d9] database[chef] transit,xml,solr-post | 0,0.0579969882965088,0.0754320621490479 |
[Tue, 18 Oct 2011 14:04:40 -0400] INFO: indexed role[d9c69c5a-9ad9-46e5-bda2-99fc740c82fc] database[chef] transit,xml,solr-post | 0,0.00243496894836426,0.0112459659576416 |
[Tue, 18 Oct 2011 14:04:40 -0400] INFO: indexed role[00c31554-fcf2-4074-8678-ac47e8a99314] database[chef] transit,xml,solr-post | 0,0.000580787658691406,0.00935697555541992 |
[Tue, 18 Oct 2011 14:04:40 -0400] INFO: indexed environment[6d2d835e-2eb6-41a0-9fc6-3cc59f406a8c] database[chef] transit,xml,solr-post | 0,0.000301122665405273,0.00833916664123535 |
[Tue, 18 Oct 2011 14:04:40 -0400] INFO: indexed role[d996a12e-5b09-4ad8-99ac-f7ee98ee37e5] database[chef] transit,xml,solr-post | 0,0.00143790245056152,0.00992584228515625 |
[Tue, 18 Oct 2011 14:04:40 -0400] INFO: indexed role[c1873ee1-d5e7-41c1-8b98-32972dc2285e] database[chef] transit,xml,solr-post | 0,0.00037693977355957,0.00780487060546875 |
[Tue, 18 Oct 2011 14:04:40 -0400] INFO: indexed role[b1882ad8-3de1-4cb1-b497-c158a4b55ffa] database[chef] transit,xml,solr-post | 0,0.000663995742797852,0.00769305229187012 |
[Tue, 18 Oct 2011 14:04:40 -0400] INFO: indexed role[1cca950d-e319-4b5d-bec6-c2366c4a5752] database[chef] transit,xml,solr-post | 0,0.000308990478515625,0.00720810890197754 |
[Tue, 18 Oct 2011 14:04:40 -0400] INFO: indexed role[db9357fb-231a-432a-870c-df4854dbec0f] database[chef] transit,xml,solr-post | 0,0.000368118286132812,0.00716805458068848 |
[Tue, 18 Oct 2011 14:04:40 -0400] INFO: indexed data_bag_item[71233651-9fab-4d6b-9d22-98f1e86ed5de] database[chef] transit,xml,solr-post | 0,0.000286102294921875,0.00728702545166016 |
[Tue, 18 Oct 2011 14:04:40 -0400] INFO: indexed data_bag_item[b852aa46-50c6-46dc-a919-3acc13b0eda1] database[chef] transit,xml,solr-post | 0,0.000451087951660156,0.00730395317077637 |
[Tue, 18 Oct 2011 14:04:40 -0400] INFO: indexed environment[bf81d04b-781c-424a-9abf-ff57c59971ab] database[chef] transit,xml,solr-post | 0,0.000319004058837891,0.00710606575012207 |
[Tue, 18 Oct 2011 14:04:40 -0400] INFO: indexed environment[149177ba-d93a-4c01-b1b6-24ba62de1b86] database[chef] transit,xml,solr-post | 0,0.000303983688354492,0.00760889053344727 |
[Tue, 18 Oct 2011 14:04:40 -0400] INFO: indexed role[d57ef5df-6d71-44b5-b89e-a141ecc37a15] database[chef] transit,xml,solr-post | 0,0.000612020492553711,0.00822687149047852 |
[Tue, 18 Oct 2011 14:04:40 -0400] INFO: indexed data_bag[60b02a5d-85f6-4632-a74e-634abcba33c8] database[chef] transit,xml,solr-post | 0,0.000259876251220703,0.00741982460021973 |
[Tue, 18 Oct 2011 14:04:40 -0400] INFO: indexed role[37305494-ecd9-47a9-ad28-73e61841e01c] database[chef] transit,xml,solr-post | 0,0.000364065170288086,0.00799798965454102 |
[Tue, 18 Oct 2011 14:04:40 -0400] INFO: indexed data_bag[1e0589f2-6a50-41a8-a940-41649c1ce649] database[chef] transit,xml,solr-post | 0,0.00029301643371582,0.00872611999511719 |
[Tue, 18 Oct 2011 14:04:40 -0400] INFO: indexed environment[68d20a83-7fb0-43f7-b310-0458e8d6f6b3] database[chef] transit,xml,solr-post | 0,0.000298023223876953,0.00695300102233887 |
[Tue, 18 Oct 2011 14:04:41 -0400] INFO: indexed role[731b07f7-b78c-4ec6-9838-e737b0c78df4] database[chef] transit,xml,solr-post | 1,0.00034785270690918,0.00690293312072754 |
[Tue, 18 Oct 2011 14:04:41 -0400] INFO: indexed environment[77b1e46f-4784-419c-afb1-d9e3427a5d8a] database[chef] transit,xml,solr-post | 1,0.000473976135253906,0.00699400901794434 |


chef-expanderctl queue-depth
  total messages:       0
  average queue depth:  0.0
  max queue depth:      0
  min queue depth:      0



chef-expanderctl node-status
Collecting status info from the cluster...done

Host                                  PID                                   GUID                                  Vnodes                              
chef.bonniercorp.local                18713                                 2fcf4dff-6499-47d5-9504-6a2861557324  0-1023       





knife index rebuild
This operation is destructive. Rebuilding the index may take some time. You sure? (yes/no): yes
Chef::ApiClient:    success
Chef::DataBag:      success
Chef::Environment:  success
Chef::Node:         success
Chef::Role:         success



But Still the node will not return when searching hostname:[* TO *]



Brian Smith
Senior Network Engineer . Bonnier Corporation
">
407.212.7090


On Tue, Oct 18, 2011 at 11:58 AM, Daniel DeLeo < "> > wrote:

On Tuesday, October 18, 2011 at 8:40 AM, Brian Smith wrote:
> The chef server is ubuntu 11.04. The node with the hostname issue is running 10.04. My expander log is empty. I ran knife index rebuild and the node still does not show up, the output was
>
> Chef::ApiClient: success
> Chef::DataBag: success
> Chef::Environment: success
> Chef::Node: success
> Chef::Role: success
>
>
> Brian Smith
> Senior Network Engineer . Bonnier Corporation
> "> (mailto: "> )
> 407.212.7090
Could be this bug: http://tickets.opscode.com/browse/CHEF-2346

Also, if your chef-expander logs are empty, that could be symptomatic of a different problem, namely that the node data isn't making it to chef-expander (via rabbitmq). You should definitely see one log entry per searchable object when you run `knife index rebuild`. You can adjust the log level of chef-expander on the fly with `chef-expanderctl log-level debug` and check the queue for a backlog with `chef-expanderctl queue-depth`. You can also ping chef-expander's workers with `chef-expanderctl node-status`.

HTH,

--
Dan DeLeo




Archive powered by MHonArc 2.6.16.

§