[chef] Re: setting up a continuous integration system with Chef... (not going so well so far)

[Zac Stevens < >]

Hi Phil,

On Fri, Mar 14, 2014 at 11:24 PM, phil swenson < " target="_blank"> > wrote:

I posted about this yesterday, I’m going to post again - but this time with links to code.

Here is the code:

https://github.com/pswenson/JenkinsChefRecipe

Code always helps, so thanks for linking that up...

 

My goal is to be able to install java, oracle DB, jenkins server, install jenkins slaves, create jobs, update jobs with Chef Solo on a system of 15 servers.  1 server will be the jenkins master and the other 14 will be slaves.

I’m starting my recipe with java and jenkins installs.

I’m guessing I’m doing several things wrong:

I ended up configuring the java attributes in nodes/redstone.json as I couldn’t get them to work inside my recipe.  It seems to me it would be better to do this in my recipe as this will not be node specific, I want it global.

Here is what happens with the current code (even with include_recipe "java::default”

as suggested by Christopher Armstrong)

Any help, suggestions, critiques would be greatly appreciated.  Links to real-life projects would be great, almost everything I see is completely trivial or just doc on attributes for recipes.  I’d love to see a real life project I could download and execute.  

Good cookbooks typically use the README file to document the attributes you may want to manipulate - that's certainly true for the java coobkook (see https://github.com/socrata-cookbooks/java/blob/master/README.md ).  As an example of putting it all together, you might want to look at this cookbook ( https://github.com/zts/cooking-with-jenkins ) I wrote to experiment with Jenkins configuration.  It hasn't been updated to use the latest version of the Jenkins cookbook, but it will show you how to make use of some community cookbooks to achieve your goal.

With that said, let's look at the problem you're having...

here is the output for running the current code:

:~/dev/chef/myfirstcookbook (master)$ knife solo cook

WARNING: solo.rb found, but since knife-solo v0.3.0 it is not used any more

WARNING: Please read the upgrade instructions: https://github.com/matschaffer/knife-solo/wiki/Upgrading-to-0.3.0

Running Chef on redstone...

Checking Chef version...

Installing Berkshelf cookbooks to '/Users/philswenson/.berkshelf/knife-solo/35c024dd80a9ab482eadf86b98a86fb90c9521d7'...

Using myfirstcookbook (0.1.0)

Using jenkins (2.0.2)

Using git (3.1.0)

Using java (1.20.0)

Using apt (2.3.8)

Using runit (1.5.10)

Using build-essential (2.0.0)

Using yum (3.1.4)

Using yum-epel (0.3.4)

Using dmg (2.2.0)

Using windows (1.30.0)

Using chef_handler (1.1.5)

Uploading the kitchen...

cannot delete non-empty directory: main

WARNING: Local role_path './roles' does not exist

WARNING: Local data_bag_path './data_bags' does not exist

WARNING: Local environment_path './environments' does not exist

Generating solo config...

Running Chef...

Starting Chef Client, version 11.10.4

[2014-03-14T19:07:47-04:00] WARN: unable to detect ip6address

Compiling Cookbooks...

HI PHIL HI PHIL

================================================================================

Recipe Compile Error in /root/chef-solo/cookbooks-3/main/recipes/default.rb

================================================================================

NoMethodError

-------------

undefined method `[]' for nil:NilClass

Cookbook Trace:

---------------

  /root/chef-solo/cookbooks-2/java/recipes/oracle.rb:38:in `from_file'

  /root/chef-solo/cookbooks-2/java/recipes/default.rb:22:in `from_file'

  /root/chef-solo/cookbooks-3/main/recipes/default.rb:8:in `from_file'

Relevant File Content:

----------------------

/root/chef-solo/cookbooks-2/java/recipes/oracle.rb:

 31:

 32:  case node['java']['jdk_version'].to_s

 33:  when "6"

 34:    tarball_url = node['java']['jdk']['6'][arch]['url']

 35:    tarball_checksum = node['java']['jdk']['6'][arch]['checksum']

 36:    bin_cmds = node['java']['jdk']['6']['bin_cmds']

 37:  when "7"

 38>>   tarball_url = node['java']['jdk']['7'][arch]['url']

 39:    tarball_checksum = node['java']['jdk']['7'][arch]['checksum']

 40:    bin_cmds = node['java']['jdk']['7']['bin_cmds']

 41:  end

 42:

 43:  if tarball_url =~ /example.com/

 44:    Chef::Application.fatal!("You must change the download link to your private repository. You can no longer download java directly from http://download.oracle.com without a web broswer")

 45:  end

 46:

 47:  include_recipe "java::set_java_home" 

The error "undefined method `[]' for nil:NilClass" means that some code has tried to look up a key - "item['key']" on an item that is set to nil (which doesn't know how to look up keys.  The error doesn't tell us exactly which item was nil - it technically could be "node", but it almost certainly isn't, so we can guess that it's one of the chain of attributes -  'java' -> 'jdk' -> '7' -> 'arch'.

If the 'java' attribute hasn't been set, node['java'] will equal nil.  That means that node['java']['jdk'] would equal nil['jdk'], which would cause the error you saw.  You do set the 'java' attribute in your node file, so that's probably not the problem here.  Let's look at the next one.  If node['java']['jdk'] is unset, that will equal nil - and node['java']['jdk']['7'] will cause the error.

The java cookbook's attributes file should be setting that attribute, so why isn't this working?  Daniel DeLeo explained this in your previous thread - most likely, you haven't put "depends 'java'" in your cookbook's metadata.rb.  If you were using chef-client/server, this would cause an error as soon as you tried to include_recipe "java" - because you're using chef-solo, the include_recipe succeeds while the cookbook hasn't been properly loaded.

As Daniel noted, the solution is to add the dependency to your metadata.rb

Checking your code to confirm that, I see a metadata.rb in the root of your git repository - but that repository doesn't look like a cookbook (I guess it's your chef-repo for use with "knife solo").  You don't need a metadata.rb in your chef-repo (it won't do anything), but you do need one in every cookbook.

I'd suggest installing the Foodcritic tool ( http://www.foodcritic.io/ ), which you can do with "gem install foodcritic".  It checks your Chef code for a bunch of different things, suggesting improvements for any problems it finds.  Let's check yours:

$ pwd

/Users/zts/code/JenkinsChefRecipe

$ foodcritic cookbooks

FC011: Missing README in markdown format: cookbooks/main/README.md:1

FC031: Cookbook without metadata file: cookbooks/main/metadata.rb:1

FC045: Consider setting cookbook name in metadata: cookbooks/main/metadata.rb:1

You don't need to fix everything it tells you, but you should make sure you understand each issue before choosing to ignore it.  The main problem it reports here is that the cookbook "main" didn't have a metadata file.  Let's move the one from the root of your repository into cookbooks/main, and try again:

$ mv metadata.rb cookbooks/main/

$ foodcritic cookbooks

FC007: Ensure recipe dependencies are reflected in cookbook metadata: cookbooks/main/recipes/default.rb:8

FC007: Ensure recipe dependencies are reflected in cookbook metadata: cookbooks/main/recipes/default.rb:16

FC008: Generated cookbook metadata needs updating: cookbooks/main/metadata.rb:3

FC011: Missing README in markdown format: cookbooks/main/README.md:1

Now we see that your recipe "main::default" depends on two cookbooks that haven't been declared in your metadata.rb.  Fix that up, then the attributes from the 'java' and 'jenkins' cookbooks will be properly loaded, and you should get a little bit further.  You should also update the other fields in metadata.rb (especially the cookbook name) to match your cookbook.

Hope that helps,

Zac