[chef] Re: Re: Using unary NOT in knife search

[Christine Draper < >]

The more things I try, the more complex it gets. What I'm trying to do is take a query _expression_ and restrict it to either:
1) nodes that don't have a specific attribute i.e.  "NOT topo_name:*"

or 2) nodes that have a specific attribute i.e.  "AND topo_name:some_name"

Doing something like:

knife search "(QUERY) NOT topo_name:*"

works in many cases, but when QUERY consists of one or more NOTs it fails as an invalid query; so in that case I have to do:

knife search "QUERY_WITH_NOTS NOT topo_name:*"

Appending an AND to QUERY_WITH_NOTS doesn't work, so I have to prepend instead:

knife search "topo_name:some_name AND (QUERY)"
knife search "topo_name:some_name AND QUERY_WITH_NOTS"

But this is all empirical, and gives some wrong answers with 12.0.3 due to the known (and possibly other) bugs. 

Anyone know more about the query transformer, or can give me some pointers?

Regards,

Christine

On Wed, Mar 18, 2015 at 11:15 AM, Daniel DeLeo < " target="_blank"> > wrote:

On Tuesday, March 17, 2015 at 2:26 PM, Christine Draper wrote:

> I am looking for some clarification on the knife search syntax, particularly related to unary NOT.
>
> I understand that knife search uses a modified Lucene query. In Lucene, unary NOT is invalid. With knife it seems to sometimes work.
>
> For example, this works with both hosted Chef and knife localmode 12.0.3 (and something like it is in the docs):
>
> knife search node "NOT name:appserver" -i
>
> However, the following works with knife localmode 12.0.3 but does not work with hosted Chef:
>
> knife search node "(NOT name:appserver) AND name:dbserver" -i
>
> so, is unary NOT actually valid in either of these cases?
>
> I realize I can turn the second example into a valid Lucene query by reversing it:
>
> knife search node "name:dbserver (NOT name:appserver)" -i
>
> But currently there's a bug that means this isnt working in knife local mode:
> https://github.com/chef/chef/issues/3073
>
> and it would be great to know more about what "modified" actually means.

I’m not sure about the rest of your question, but I can answer this part. In the past, we used to convert documents for search such that every field we indexed was a top-level key in lucene/solr. This worked for a while, but it’s actually really bad to use lucene this way, as it’s designed to be used with maybe 40 or so top-level keys but we had millions in Hosted Chef. So we changed the documents we send to Solr such that all the data to be indexed is in a single key (a single field in the XML document). Since we previously sent users search queries directly to Solr (using the filter query to restrict search to a single organization) we ended up writing a query transformer that rewrites your incoming query to match the new storage format. This allows things like unary NOT and queries beginning with wildcards to work (e.g., name:*foo isn’t valid in lucene but you can use it in Chef)..

As for your other questions, it could be a bug in the query transformer, or maybe it’s not possible to transform it into a valid query, I’m not sure.

>
> Thanks!
> Christine

--
Daniel DeLeo