[chef] Re: Re: Re: Re: Re: Node attributes aren't searchable for some reason?

[Chris < >]

Thanks Ian

I figured out that 50k is the minimum for my setup, I just wish i
could keep the system stable. Maybe moving to Ruby 1.9.x will finally
fix this problem. I've been able to work around this by using Adam
Jacobs whitelist cookbook. But it would be nice not to have to worry
about updating the cookbook if i want to add a new attribute.

On Fri, Jun 1, 2012 at 11:53 AM, Ian Marlier 
<
 >
 wrote:
> I've made two changes:
>
> * maxFieldLength from 10k to 120k (sort of a random number, mostly just
> going for "biggerer").  There are two places that this appears in
> /var/lib/chef/solr/conf/solrconfig.xml, and it needs to be changed in both
> of them.
> * -Xms and -Xmx changed to 2GB.  This was done in /etc/chef/solr.rb, by
> changing the variable "solr_heap_size".  I didn't want to max out on it,
> because I was afraid of GC behavior, but I suspect that it's not really
> relevant since the resident set size doesn't seem to go above 750MB or
> thereabouts.  As the number of machines that we have on this Chef instance
> rises, I suspect that I'll need to spend some time looking into that GC
> behavior and maybe doing some tuning.
>
> Other than that, everything is stock.
>
> OS is Ubuntu 11.04, Ruby version 1.9.2p290.  Running in EC2 on an m1.xlarge
> instance dedicated to the application and to Solr (Couch is running on a
> separate box).
>
>
> On Fri, Jun 1, 2012 at 10:10 AM, Chris 
> <
 >
>  wrote:
>>
>> I would love to know your configuration, I've never been able to make this
>> work. Every time I try solr stops responding and chef server eventually
>> crashes. I'd like to be able to blame it on centos 5.5 and ruby 1.8.7, but 
>> I
>> dont anything to compare to
>>
>> Sent from a phone
>>
>>
>> On May 31, 2012, at 2:10 PM, Ian Marlier 
>> <
 >
>>  wrote:
>>
>> So, it turned out that this DID fix my problem, but there's a trick:
>> /var/lib/chef/solr/conf/solrconfig.xml has the maxFieldLength field set
>> _twice_, and you need to change the value in both places for it to take.
>>
>> On Tue, May 29, 2012 at 11:14 PM, Ian Marlier 
>> <
 >
>> wrote:
>>>
>>> Sadly, changing that parameter and restarting solr doesn't seem to have
>>> helped.  I changed it in both locations (/etc/solr/conf/solrconfig.xml and
>>> /var/lib/chef/solr/conf/solrconfig.xml) and restarted chef-solr after 
>>> doing
>>> so.  I initially changed it from the default value of 10000 to 20000, and
>>> then tried again raising it to 120000
>>>
>>> In order to try to force reindexing I've done a number of things,
>>> including editing the nodes and deleting the cluster/uuid parameter
>>> entirely, and then editing again and adding it back.  I've even tried
>>> dumping the entire node out to a JSON file, deleting it entirely from 
>>> Chef,
>>> and then adding it back in again.  Neither of those worked.
>>>
>>> Querying solr directly, it's clear that the issue is that this particular
>>> attribute isn't making it in there, but I really can't comprehend why.
>>>
>>> On Tue, May 29, 2012 at 10:39 PM, Ian Marlier 
>>> <
 >
>>> wrote:
>>>>
>>>> I'll take a look at that setting, Peter.  Thanks for the pointer.  I'll
>>>> report back once I've got something.
>>>>
>>>>
>>>> On Tue, May 29, 2012 at 9:15 PM, Peter Donald 
>>>> <
 >
>>>> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> Do the nodes that are not being returned from search have a lot of
>>>>> attribute data? If so the problem may be that they are failing to be
>>>>> indexed by solr as you are exceeding the setting maxFieldLength. If
>>>>> this is the case look for the solr config file on your system and
>>>>> increase the size of this. (Just be warned the config file actually
>>>>> appears twice on ubuntu systems for some reason).
>>>>>
>>>>> HTH
>>>>>
>>>>> On Wed, May 30, 2012 at 3:00 AM, Ian Marlier 
>>>>> <
 >
>>>>> wrote:
>>>>> > I'm working with the following two nodes, which are supposed to be
>>>>> > part of a
>>>>> > pacemaker cluster:
>>>>> > 
 
>>>>> >  chef-repo (hc_production)]$ k node show
>>>>> > login01.fal
>>>>> > --attribute cluster --format json
>>>>> > {
>>>>> >   "cluster": {
>>>>> >     "heartbeat_coredumps": "true",
>>>>> >     "uuid": "dff82156-9145-4a89-b92a-ba3bb238442b",
>>>>> >     "udpport": "759",
>>>>> >     "heartbeat_compression": "bz2",
>>>>> >     "pacemaker": {
>>>>> >       "resource_dir": "/usr/lib/ocf/resource.d"
>>>>> >     },
>>>>> >     "deadtime": 60,
>>>>> >     "initdead": 90,
>>>>> >     "warntime": 30,
>>>>> >     "keepalive": 5
>>>>> >   }
>>>>> > }
>>>>> > 
 
>>>>> >  chef-repo (hc_production)]$ k node show
>>>>> > login02.fal
>>>>> > --attribute cluster --format json
>>>>> > {
>>>>> >   "cluster": {
>>>>> >     "heartbeat_coredumps": "true",
>>>>> >     "uuid": "dff82156-9145-4a89-b92a-ba3bb238442b",
>>>>> >     "udpport": "759",
>>>>> >     "heartbeat_compression": "bz2",
>>>>> >     "deadtime": 60,
>>>>> >     "pacemaker": {
>>>>> >       "resource_dir": "/usr/lib/ocf/resource.d"
>>>>> >     },
>>>>> >     "initdead": 90,
>>>>> >     "keepalive": 5,
>>>>> >     "warntime": 30
>>>>> >   }
>>>>> > }
>>>>> > 
 
>>>>> >  chef-repo (hc_production)]$
>>>>> >
>>>>> >
>>>>> > However, for reasons that I can't explain, searching based on cluster
>>>>> > UUID
>>>>> > doesn't return these nodes:
>>>>> >
>>>>> > 
 
>>>>> >  chef-repo (hc_production)]$ k search node
>>>>> > 'cluster_uuid:dff82156-9145-4a89-b92a-ba3bb238442b'
>>>>> > 0 items found
>>>>> >
>>>>> > 
 
>>>>> >  chef-repo (hc_production)]$
>>>>> >
>>>>> >
>>>>> > Anyone have any idea why this would be?  Is there a way to check the
>>>>> > index
>>>>> > status for a given node or something like that, or to force
>>>>> > re-indexing of
>>>>> > the node?  (Note: I did try editing the node, removing the UUID
>>>>> > parameter,
>>>>> > saving the node, then editing again and adding the UUID parameter
>>>>> > back in.)
>>>>> >
>>>>> > Thanks,
>>>>> >
>>>>> > Ian
>>>>> >
>>>>> > --
>>>>> > Ian Marlier | Senior Systems Engineer
>>>>> > Brightcove, Inc.
>>>>> > 290 Congress Street, 4th Floor, Boston, MA 02110
>>>>> > 
 
>>>>> >
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Cheers,
>>>>>
>>>>> Peter Donald
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Ian Marlier | Senior Systems Engineer
>>>> Brightcove, Inc.
>>>> 290 Congress Street, 4th Floor, Boston, MA 02110
>>>> 
 
>>>>
>>>
>>>
>>>
>>> --
>>> Ian Marlier | Senior Systems Engineer
>>> Brightcove, Inc.
>>> 290 Congress Street, 4th Floor, Boston, MA 02110
>>> 
 
>>>
>>
>>
>>
>> --
>> Ian Marlier | Senior Systems Engineer
>> Brightcove, Inc.
>> 290 Congress Street, 4th Floor, Boston, MA 02110
>> 
 
>>
>
>
>
> --
> Ian Marlier | Senior Systems Engineer
> Brightcove, Inc.
> 290 Congress Street, 4th Floor, Boston, MA 02110
> 
 
>



-- 
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.