[chef] Re: How do I know if my application has really been "provisioned"? a suggestion

[Cassiano Leal < >]

I had to deal with a similar problem in some capistrano recipes for deploying JBoss applications.

The approach I used was to monitor the "deployments" directory in JBoss and check for an "artifact.deployed" file with a timestamp greater than that of "artifact.ear". It's been working beautifully, but your solution of checking the tiestamp in the logs might be more robust.

-- 

Cassiano Leal

On Sunday, December 9, 2012 at 10:22, Bryan Berry wrote:

I have been struggling lately w/ the problem of how to know whether or

not my servers have been fully provisioned. This is a problem that

affects both cookbook testing and orchestration.

I have numerous java applications that have long startup times. I

believe Rails 3 apps also have long startup times. For example,

elasticsearch takes > 5s to start up fully. However Chef does not

natively have a way to determine when elasticsearch has fully started

up. All Chef can know is the exit code returned by `service

elasticsearch start` and its various platform equivalents. Why is this

an issue? Well, I don't want to run my minitest-handler integration

tests until elasticsearch is actually functional. Also, if I am

testing a multiple VM setup where another VM depends on elasticsearch,

I need to know when elasticsearch has completed startup.

Erik Hollensbe is doing some freaking awesome work on workflow

orchestration w/ chef-workflow and I think it illustrates the problem

here

require 'chef-workflow/helper'

class MyTest < MiniTest::Unit::VagrantTestCase

def before

@json_msg = '{ 'id': "dumb message json msg"}'

end

def setup

provision('elasticsearch')

provision('logstash')

wait_for('elasticsearch')

wait_for('logstash')

inject_logstash_message(@json_msg)

end

def test_message_indexed_elasticsearch

assert es_has_message?(@json_msg)

end

end

If I understand Erik's code correctly, the `wait_for('elasticsearch')`

only waits for the vagrant provisioner to return. The vagrant

provisioner in turn only waits for `service elasticsearch start` to

return a non-zero exit-code.

We need an optional way to determine whether an server has been

complete provisioned, or that all the resources have entered a "done"

state. The only way I know that elasticsearch has started

successfully is if I see in the log "Elasticsearch has started" w/ a

timestamp more recent than when I started the service.

Here is my idea, pls tell me if it is crazy.

I think that the service resource should have an additional attribute

"done" that takes two ruby blocks, named before and after. The first

block would return a hash with the results of the block.

# Warning much of the Ruby code here maybe horribly incorrect.

def get_current_offset(file_name)

{ :offset => ::File.seek(file_name, EOF).get_offset }

end

def es_started?(params)

!::File.open(params[:log_file], params[:offset],

'r').grep(params[:started_text]).empty?

end

log_file = '/usr/local/var/log/elasticsearch.log'

started_test = "elasticsearch has started successfully"

timeout = 60

service "elasticsearch" do

done(

:before => { get_current_offset(logfile) },

:after => { es_started?(:log_file => log_file, :offset

=> @before_results[:offset]

:timeout => 60

)

end

The before block would run before the service is actually actioned.

Now Chef would need some additional machinery to collect all the done

:after blocks and the related @before_results. This could be done by

chef_handler but may be better as part of chef itself. Let's call it

the done_handler for now. This done_handler would mark the time before

it starts handling any done_after blocks, then loop through the

collected done_after blocks for the specified timeout. Once all blocks

are complete it would continue onto other handlers, such as the

minitest_handler.

I think this done_handler could be part of something called "wait_mode"