[chef] Using ruby_block


Chronological Thread 
  • From: Sachin Sagar Rai < >
  • To: " " < >
  • Subject: [chef] Using ruby_block
  • Date: Sat, 26 Jan 2013 17:41:42 +0545

Ohai! chefs!

My Recipe has to import from S3, extract and seed the data from the mysqldump.

To do this, I want it using ruby and the aws-sdk gem.
AFAIK, the ruby code in the recipe file is evaluated at compil time and the ones that are inside `ruby_block` is evaluated at convergence time.
So if I do `require 'aws'`, it won't since its not installed. When I add it inside the ruby_block, it errors out.
So to fix it, I used the `chef_gem 'aws-sdk'`, but this will run at the compile time which is fine.
But to install it, it depends on packages like lib-xml2 to be installed too at the compile time.

What I want is, what is a better way to do this using the ruby_block way?

.....
if app['db_import']

  gem_package 'aws-sdk'

  ruby_block "import_backed-up_files_from_S3" do
    block do
      # Getting the files from S3 backup
      require 'aws'
.....


[2013-01-26T09:12:25+00:00] INFO: Processing gem_package[aws-sdk] action install (empowerme::extract_files_from_s3_backup line 6)
[2013-01-26T09:12:25+00:00] DEBUG: gem_package[aws-sdk] detected omnibus installation in /opt/chef/embedded/bin
[2013-01-26T09:12:25+00:00] DEBUG: gem_package[aws-sdk] searching for 'gem' binary in path: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/chef/embedded/bin:/opt/chef/embedded/bin
[2013-01-26T09:12:25+00:00] DEBUG: gem_package[aws-sdk] using gem '/usr/bin/gem'
[2013-01-26T09:12:26+00:00] DEBUG: gem_package[aws-sdk] no installed version found for aws-sdk (>= 0)
RubyGems Environment:
  - RUBYGEMS VERSION: 1.8.23
  - RUBY VERSION: 1.9.3 (2012-11-10 patchlevel 327) [i686-linux]
  - INSTALLATION DIRECTORY: /var/lib/gems/1.9.1
  - RUBY EXECUTABLE: /usr/bin/ruby1.9.1
  - EXECUTABLE DIRECTORY: /usr/local/bin
  - RUBYGEMS PLATFORMS:
    - ruby
    - x86-linux
  - GEM PATHS:
     - /var/lib/gems/1.9.1
     - /root/.gem/ruby/1.9.1
  - GEM CONFIGURATION:
     - :update_sources => true
     - :verbose => true
     - :benchmark => false
     - :backtrace => false
     - :bulk_threshold => 1000
  - REMOTE SOURCES:
     - http://rubygems.org/
[2013-01-26T09:12:27+00:00] DEBUG:  found gem aws-sdk version 1.8.0 for platform ruby from http://rubygems.org/
When you HTTParty, you must party hard!
Building native extensions.  This could take a while...
Successfully installed uuidtools-2.1.3
Successfully installed multi_xml-0.5.2
Successfully installed httparty-0.10.0
Successfully installed json-1.7.6
Successfully installed aws-sdk-1.8.0
5 gems installed
[2013-01-26T09:12:42+00:00] DEBUG: gem_package[aws-sdk] resetting gem environment to default
[2013-01-26T09:12:42+00:00] INFO: Processing ruby_block[import_backed-up_files_from_S3] action create (empowerme::extract_files_from_s3_backup line 8)

================================================================================
Error executing action `create` on resource 'ruby_block[import_backed-up_files_from_S3]'
================================================================================

LoadError
---------
cannot load such file -- aws

Cookbook Trace:
---------------
/var/chef/cache/cookbooks/empowerme/recipes/extract_files_from_s3_backup.rb:11:in `block (2 levels) in from_file'

Resource Declaration:
---------------------
# In /var/chef/cache/cookbooks/empowerme/recipes/extract_files_from_s3_backup.rb

  8:   ruby_block "import_backed-up_files_from_S3" do
  9:     block do
 10:       # Getting the files from S3 backup
 11:       require 'aws'
 12:       AWS.config(access_key_id: node['aws']['access_key_id'], secret_access_key: node['aws']['secret_access_key'])
 13:       s3 = AWS::S3.new
 14: 
 15:       bucket = s3.buckets[node['empowerme']['s3_bucket']]
 16: 
 17:       file_to_download = bucket.objects.map{|ob| ob.key}.sort.last # e.g. "empowerme/2012.10.14.20.40.02/empowerme.tar"
 18: 
 19:       Chef::Log.info("\n\nBackup Import from S3\n---------- \n #{file_to_download}\n-------------\n")
 20: 
 21:       obj = bucket.objects[file_to_download]
 22: 
 23:       local_file = Pathname.new("/tmp/#{node['empowerme']['backup_model_name']}.tar")
 24: 
 25:       File.open(local_file, "w") do |f|
 26:         obj.read do |chunk|
 27:           f.write chunk
 28:         end
 29:       end
 30: 
 31:       execute "extracting downloaded file" do
 32:         cwd local_file.dirname.to_s
 33:         command "tar -xf ./#{local_file.basename.to_path}"
 34:         user "root"
 35:       end
 36: 
 37:       # execute "change permission on #{app['deploy_to']}" do
 38:       #   command "chmod g+rw #{app['deploy_to']}; chown -R #{app['owner']}:#{app['group']} #{app['deploy_to']}"
 39:       # end
 40: 
 41:       # execute "change permission on #{node['empowerme']['ebs_mount_dir']}" do
 42:       #   command "chmod g+rw #{node['empowerme']['ebs_mount_dir']} && chown -R #{app['owner']}:#{app['group']} #{node['empowerme']['ebs_mount_dir']}"
 43:       # end
 44:     end
 45:     action :create
 46:   end
 47: end

Compiled Resource:
------------------
# Declared in /var/chef/cache/cookbooks/empowerme/recipes/extract_files_from_s3_backup.rb:8:in `from_file'

ruby_block("import_backed-up_files_from_S3") do
  action [:create]
  retries 0
  retry_delay 2
  block_name "import_backed-up_files_from_S3"
  cookbook_name "empowerme"
  recipe_name "extract_files_from_s3_backup"
  block #<Proc:0x85c7300@/var/chef/cache/cookbooks/empowerme/recipes/extract_files_from_s3_backup.rb:9>
end

[2013-01-26T09:12:43+00:00] DEBUG: Re-raising exception: LoadError - ruby_block[import_backed-up_files_from_S3] (empowerme::extract_files_from_s3_backup line 8) had an error: LoadError: cannot load such file -- aws
/opt/chef/embedded/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
  /opt/chef/embedded/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
  /var/chef/cache/cookbooks/empowerme/recipes/extract_files_from_s3_backup.rb:11:in `block (2 levels) in from_file'
  /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/provider/ruby_block.rb:33:in `call'
  /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/provider/ruby_block.rb:33:in `block in action_create'
  /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/mixin/why_run.rb:52:in `call'
  /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/mixin/why_run.rb:52:in `add_action'
  /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/provider.rb:152:in `converge_by'
  /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/provider/ruby_block.rb:32:in `action_create'
  /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/provider.rb:115:in `run_action'
  /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/resource.rb:597:in `run_action'
  /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/runner.rb:49:in `run_action'
  /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/runner.rb:81:in `block (2 levels) in converge'
  /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/runner.rb:81:in `each'
  /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/runner.rb:81:in `block in converge'
  /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/resource_collection.rb:94:in `block in execute_each_resource'
  /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/resource_collection/stepable_iterator.rb:116:in `call'
  /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block'
  /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
  /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate'
  /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
  /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/resource_collection.rb:92:in `execute_each_resource'
  /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/runner.rb:80:in `converge'
  /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/client.rb:378:in `converge'
  /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/client.rb:420:in `do_run'
  /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/client.rb:176:in `run'
  /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/application.rb:140:in `run_chef_client'
  /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/application/client.rb:274:in `block in run_application'
  /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/application/client.rb:267:in `loop'
  /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/application/client.rb:267:in `run_application'
  /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/application.rb:72:in `run'
  /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/bin/chef-client:26:in `<top (required)>'
  /usr/bin/chef-client:23:in `load'
  /usr/bin/chef-client:23:in `<main>'
[2013-01-26T09:12:43+00:00] ERROR: Running exception handlers
[2013-01-26T09:12:43+00:00] ERROR: Report handler Chef::Handler::ErrorReport raised #<Encoding::UndefinedConversionError: "\xE2" from ASCII-8BIT to UTF-8>
[2013-01-26T09:12:43+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/resource.rb:500:in `encode'
[2013-01-26T09:12:43+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/resource.rb:500:in `to_json'
[2013-01-26T09:12:43+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/resource.rb:500:in `to_json'
[2013-01-26T09:12:43+00:00] ERROR: /opt/chef/embedded/lib/ruby/1.9.1/json/common.rb:278:in `generate'
[2013-01-26T09:12:43+00:00] ERROR: /opt/chef/embedded/lib/ruby/1.9.1/json/common.rb:278:in `pretty_generate'
[2013-01-26T09:12:43+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/json_compat.rb:49:in `to_json_pretty'
[2013-01-26T09:12:43+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/handler/error_report.rb:27:in `report'
[2013-01-26T09:12:43+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/handler.rb:226:in `run_report_unsafe'
[2013-01-26T09:12:43+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/handler.rb:214:in `run_report_safely'
[2013-01-26T09:12:43+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/handler.rb:118:in `block in run_exception_handlers'
[2013-01-26T09:12:43+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/handler.rb:117:in `each'
[2013-01-26T09:12:43+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/handler.rb:117:in `run_exception_handlers'
[2013-01-26T09:12:43+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/handler.rb:127:in `block in <class:Handler>'
[2013-01-26T09:12:43+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/client.rb:118:in `call'
[2013-01-26T09:12:43+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/client.rb:118:in `block in run_failed'
[2013-01-26T09:12:43+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/client.rb:117:in `each'
[2013-01-26T09:12:43+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/client.rb:117:in `run_failed'
[2013-01-26T09:12:43+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/client.rb:436:in `rescue in do_run'
[2013-01-26T09:12:43+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/client.rb:444:in `do_run'
[2013-01-26T09:12:43+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/client.rb:176:in `run'
[2013-01-26T09:12:43+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/application.rb:140:in `run_chef_client'
[2013-01-26T09:12:43+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/application/client.rb:274:in `block in run_application'
[2013-01-26T09:12:43+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/application/client.rb:267:in `loop'
[2013-01-26T09:12:43+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/application/client.rb:267:in `run_application'
[2013-01-26T09:12:43+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/lib/chef/application.rb:72:in `run'
[2013-01-26T09:12:43+00:00] ERROR: /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-10.18.1/bin/chef-client:26:in `<top (required)>'
[2013-01-26T09:12:43+00:00] ERROR: /usr/bin/chef-client:23:in `load'
[2013-01-26T09:12:43+00:00] ERROR: /usr/bin/chef-client:23:in `<main>'
[2013-01-26T09:12:43+00:00] ERROR: Exception handlers complete
[2013-01-26T09:12:43+00:00] DEBUG: Server doesn't support resource history, skipping resource report.
[2013-01-26T09:12:43+00:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
[2013-01-26T09:12:43+00:00] DEBUG: LoadError: ruby_block[import_backed-up_files_from_S3] (empowerme::extract_files_from_s3_backup line 8) had an error: LoadError: cannot load such file -- aws

-------------------------------------------
@millisami
~ Sachin Sagar Rai
Ruby on Rails Developer
http://tfm.com.np
http://nepalonrails.com
Sent with Sparrow




Archive powered by MHonArc 2.6.16.

§