[chef] Re: Re: Overriding Search

[AJ Christensen < >]

This is quite similar to my Discovery library cookbook of yore: https://github.com/hw-cookbooks/discovery -- this one is not specific to ElasticSearch, but doesn't directly support mocked results.

It surely wouldn't be too hard to add. How would the mock response data be supplied?

--AJ

On 19 December 2012 09:22, Daniel Condomitti < " target="_blank"> > wrote:

I wrote a quick library (lives in elasticsearch/libraries) to wrap search for discovering nodes in our applications and haproxy cookbooks that rely on it.

module ElasticSearch

  class Discovery

    attr_accessor :additional_conditions, :dc, :datacenter_restriction, :logger

    attr_reader :environment

    def initialize(recipe_context, opts = {})

      # We need to access chef search methods and the easiest way to get them is through

      # passing self from the chef recipe referencing this class

      @recipe_context = recipe_context

      @environment = opts.fetch(:environment)

      @dc = opts.fetch(:dc, 'default-datacenter-name')

      # @conditions are non-overridable attributes used in the search like environment

      # and es-cluster role

      @conditions = opts.fetch(:conditions, {

        chef_environment:           self.environment,

        role:                       opts.fetch(:backend_role, 'elasticsearch-cluster'),

        elasticsearch_cluster_name: opts.fetch(:cluster_name)

      })

      @additional_conditions = opts.fetch(:additional_conditions, Hash.new)

      @logger = opts.fetch(:logger, Chef::Log)

      @logger.info "Initialized discovery in #{self.environment}/#{self.dc} for #{@conditions[:role]}"

    end

    def conditions

      merged_conditions = @conditions.merge(self.additional_conditions)

      @logger.info "Search conditions are #{merged_conditions.flatten}"

      merged_conditions

    end

    def query

      @conditions.inject([]) do |query,condition|

        query << "(#{condition.first}:#{condition.last})"

      end.join(' AND ')

    end

    def search(opts = {})

      sorted = opts.fetch(:sorted, false)

      @logger.info "Searching with #{self.query}"

      @results = @recipe_context.search(:node, self.query)

      (sorted) ? sorted_results : @results

    end

    def restrict_to_datacenter!

      @logger.info "Restricting search to #{@dc}"

      @conditions[:dc] = @dc

    end

    def restricted_to_datacenter?

      @conditions[:dc] == @dc

    end

    def sorted_results

      return [] unless @results

      @results.sort { |node1, node2| node1[:fqdn] <=> node2[:fqdn] }

    end

    def to_a

      @results.collect { |node| { :ip => node[:ipaddress], :fqdn => node[:fqdn], :datacenter => node[:dc] } }

    end

  end

end

It's used like the following:

es_discovery = ElasticSearch::Discovery.new(self, 

  environment: node.chef_environment, 

  backend_role: node[:haproxy][:edge_backend_role],

  cluster_name: node[:elasticsearch][:cluster_name]

)

es_discovery.restrict_to_datacenter! if node[:haproxy][:edge_datacenter_restriction]

es_discovery.additional_conditions = node[:haproxy][:additional_conditions] if node[:haproxy][:additional_conditions].is_a?(Hash)
results = es_discovery.search(sorted: true)

This way it'll find all the nodes in a specific cluster but I can also add extra conditions for restricting the search if needed on a per-node/role basis. It needs some work and should probably be generalized for discovering other services besides ES but it could be a start for you.

Dan

On Tuesday, December 18, 2012 at 2:59 PM, Kevin Nuckolls wrote:

We had a long discussion today about a pain point we've had with Chef thus far. We'd like to rely on the search functionality that chef provides so things will automatically wire up with each other in a given environment. But during cookbook development we'd like to have more flexibility with the search functionality.

We decided that we'd like to have a bit more control over the search functionality and be able to provide overrides while we're working on something. Has anyone already written a library for doing something like that?

Thanks,

-Kevin