[chef] Re: Data bag contents once inside recipe code

[Daniel DeLeo < >]

On Monday, September 23, 2013 at 7:00 PM, Russell Bateman wrote:

I'm a little confused by the examples I see of data bags, in http://docs.opscode.com/essentials_data_bags.html and http://docs.opscode.com/dsl_recipe_method_data_bag.html on using my data bag from Ruby in a recipe.

I keep all my Chef stuff in a filesystem (and Git) thus:

cookbooks
nodes
roles
data_bags
    admins
        charlie.json

having knifed my data bag using:

knife data bag from file admins data_bags/admins/charlie.json

I assume that, when my recipes run, charlie.json contains this, since this is what's in the filesystem:

{
    "id" : "charlie",
    "login" : "chuck",
    "password" : "Test123"
}

(Please ignore, I think, the phenomenon of encrypted data bags which doesn't interest me just yet. I've taken charlie here from the doc only as the example; my own needs do not require storing keys or passwords yet.)

Again I'm looking at http://docs.opscode.com/dsl_recipe_method_data_bag.html. (This page purports to document methods data_bag() and data_bag_item(). It does the latter, but not the former.)

I'm hoping someone can correct my misgivings of how this works. I want to access what's in charlie (assuming only charlie and there happens to be nothing else under admins subdirectory) from recipe code as below.

Should all three of the following paragraphs not print exactly the same thing?

admins = data_bag( :admins )
admins.each do | bag |              # there's only one--charlie
  puts "id",       bag[ :id ]
  puts "login",    bag[ :login ]
  puts "password", bag[ :password ]
end
#------------------------------------
charlie = data_bag( :admins )
puts "id",       charlie[ :id ]
puts "login",    charlie[ :login ]
puts "password", charlie[ :password ]
#------------------------------------
puts "id",       data_bag_item( :admins, "id" )
puts "login",    data_bag_item( :admins, "login" )
puts "password", data_bag_item( :admins, "password" )

The way that `data_bag` works in recipes can be confusing. It's a very thin wrapper over the actual server REST API for data bags. 

In general, the REST API serves content in a way very similar to how a basic (HTML) web site would work. Whenever you do a GET to a "container" URL (e.g., /nodes and _not_ /nodes/a-specific-node), the API returns a list of the items and their URLs. To get the actual content of an item, you need to follow that item's URL. (FWIW, this aspect of the API design is called HATEOAS: http://en.wikipedia.org/wiki/HATEOAS).

Data Bags are just user-definable containers, so when you list them via the API, you get a list of the IDs and their URLs back. Since the `data_bag` method is just a thin wrapper over the api:

    chef:recipe > x = data_bag "users-example"

     => ["charles"] 

So, you have to call `data_bag_item` on each one in order to load the actual item:

    chef:recipe > data_bag_item "users-example", "charles"

     => data_bag_item["users-example", "charles", {"id"=>"charles", "shell"=>"zsh", # more stuff

 In practice most people use search to load everything in a data bag in a single call when they want to do something with all of the items. 

Also note that chef-solo differs from chef-client in that it loads every item when you use `data_bag` in a recipe (I think this is for practical reasons, because chef-solo doesn't have search), so what you posted would work for a chef-solo only environment, but will not work if you switch to client-sever.

Maybe I've misunderstood the relationship between subdirectories under data_bags in my filesystem. They seem to play a role semantically in the examples. What if I create:

data_bags
    admins
        stuff                     # add this additional level of subdirectory
            charlie.json

What effect would that have on code just above? Or is this not provided for/unused/silly/etc.?

Data bags don't support nesting. For example, an "admins" data bag is a container with the URL https://chef.example.org/data/admins and the "charlie" item is located at https://chef.example.org/data/admins/charlie, but the server doesn't support putting one data bag in another, so you can't get something located at https://chef.example.com/data/admins/stuff/charlie. 

Most of the tooling around expects your on-disk layout to resemble the API layout, so you'll probably run into trouble if you try to nest.

 

I appreciate you bearing with me. I keep not freeing myself from being a Chef noob (and I'm new to Ruby too) though  I have high hopes. Thanks for any and all comments.

Russ

HTH,

-- 

Daniel DeLeo