[chef] Re: Re: Re: Overriding Search

[Kevin Nuckolls < >]

Both of these are interesting. I'm not sure how we'd like to supply the mock / override data to the system. I get the feeling that it might be simplest if we just institutionalize using Vagrant and having some local yaml file that Vagrant picks up and supplies to the nodes within it.

Mostly I was just looking for any prior art as far as this is concerned, so thanks guys.

-Kevin

On Tue, Dec 18, 2012 at 2:26 PM, AJ Christensen < " target="_blank"> > wrote:

This is quite similar to my Discovery library cookbook of yore: https://github.com/hw-cookbooks/discovery -- this one is not specific to ElasticSearch, but doesn't directly support mocked results.

It surely wouldn't be too hard to add. How would the mock response data be supplied?

--AJ

On 19 December 2012 09:22, Daniel Condomitti < " target="_blank"> > wrote:

I wrote a quick library (lives in elasticsearch/libraries) to wrap search for discovering nodes in our applications and haproxy cookbooks that rely on it.

module ElasticSearch

  class Discovery

    attr_accessor :additional_conditions, :dc, :datacenter_restriction, :logger

    attr_reader :environment

    def initialize(recipe_context, opts = {})

      # We need to access chef search methods and the easiest way to get them is through

      # passing self from the chef recipe referencing this class

      @recipe_context = recipe_context

      @environment = opts.fetch(:environment)

      @dc = opts.fetch(:dc, 'default-datacenter-name')

      # @conditions are non-overridable attributes used in the search like environment

      # and es-cluster role

      @conditions = opts.fetch(:conditions, {

        chef_environment:           self.environment,

        role:                       opts.fetch(:backend_role, 'elasticsearch-cluster'),

        elasticsearch_cluster_name: opts.fetch(:cluster_name)

      })

      @additional_conditions = opts.fetch(:additional_conditions, Hash.new)

      @logger = opts.fetch(:logger, Chef::Log)

      @logger.info "Initialized discovery in #{self.environment}/#{self.dc} for #{@conditions[:role]}"

    end

    def conditions

      merged_conditions = @conditions.merge(self.additional_conditions)

      @logger.info "Search conditions are #{merged_conditions.flatten}"

      merged_conditions

    end

    def query

      @conditions.inject([]) do |query,condition|

        query << "(#{condition.first}:#{condition.last})"

      end.join(' AND ')

    end

    def search(opts = {})

      sorted = opts.fetch(:sorted, false)

      @logger.info "Searching with #{self.query}"

      @results = @recipe_context.search(:node, self.query)

      (sorted) ? sorted_results : @results

    end

    def restrict_to_datacenter!

      @logger.info "Restricting search to #{@dc}"

      @conditions[:dc] = @dc

    end

    def restricted_to_datacenter?

      @conditions[:dc] == @dc

    end

    def sorted_results

      return [] unless @results

      @results.sort { |node1, node2| node1[:fqdn] <=> node2[:fqdn] }

    end

    def to_a

      @results.collect { |node| { :ip => node[:ipaddress], :fqdn => node[:fqdn], :datacenter => node[:dc] } }

    end

  end

end

It's used like the following:

es_discovery = ElasticSearch::Discovery.new(self, 

  environment: node.chef_environment, 

  backend_role: node[:haproxy][:edge_backend_role],

  cluster_name: node[:elasticsearch][:cluster_name]

)

es_discovery.restrict_to_datacenter! if node[:haproxy][:edge_datacenter_restriction]

es_discovery.additional_conditions = node[:haproxy][:additional_conditions] if node[:haproxy][:additional_conditions].is_a?(Hash)

results = es_discovery.search(sorted: true)

This way it'll find all the nodes in a specific cluster but I can also add extra conditions for restricting the search if needed on a per-node/role basis. It needs some work and should probably be generalized for discovering other services besides ES but it could be a start for you.

Dan

On Tuesday, December 18, 2012 at 2:59 PM, Kevin Nuckolls wrote:

We had a long discussion today about a pain point we've had with Chef thus far. We'd like to rely on the search functionality that chef provides so things will automatically wire up with each other in a given environment. But during cookbook development we'd like to have more flexibility with the search functionality.

We decided that we'd like to have a bit more control over the search functionality and be able to provide overrides while we're working on something. Has anyone already written a library for doing something like that?

Thanks,

-Kevin