[chef] Re: Re: data bag problems when upgrading from 10.12 to 10.14 and up

[Daniel DeLeo < >]

See this ticket:

http://tickets.opscode.com/browse/CHEF-3393

This is one of the reasons we changed the encrypted data bag item format in Chef 11. 

FYI, Chef 10.18+ can read encrypted data bag items you create with Chef 11's knife command so you could potentially upgrade knife on your workstation without upgrading chef on your servers.

-- 

Daniel DeLeo

On Wednesday, January 23, 2013 at 2:29 AM, Jake Vanderdray wrote:

Hi Kallen,

  I don't know the exact details of the problem, but my colleagues ran into this and it apparently comes down to a change in the default yaml parser.  Ultimately you'll need to re-create your encrypted data bags, but in the short-term you can add this to /etc/chef/client.rb in order to force newer versions of chef to use the old yaml parser.

YAML::ENGINE.yamler = 'syck' 

Hope that helps,

Jake.

On Wed, Jan 23, 2013 at 2:45 AM, < " target="_blank"> > wrote:

hiya.

i'm trying to upgrade my chef clients from chef-full-0.10.10-1 to 10.18, and
i just discovered a problem pulling values out of an encrypted data bag when
upgrading. during my tests, my example recipe (below) works with
chef-full-0.10.10-1 and chef-10.12.0-1, but breaks when i try chef-10.14.4-2
(and 10.16 and 10.18).

is this a known bug? or, do i need to do something differently?


here's a brief recipe that illustrates what's happening:


  aws_creds = Chef::EncryptedDataBagItem.load("secrets","aws-creds-quux")
  grab_cert   = "MEEP_X509_SERVERCERT"
  grab_key    = "MEEP_X509_PRIVATEKEY"
  grab_access = "MEEP_AWS_ACCESS_KEY_ID"
  grab_secret = "MEEP_AWS_SECRET_ACCESS_KEY"

  x509_cert  = aws_creds["#{grab_cert}"]
  x509_key   = aws_creds["#{grab_key}"]
  aws_access = aws_creds["#{grab_access}"]
  aws_secret = aws_creds["#{grab_secret}"]
  Chef::Log.debug("HIGGS-BOSON: AWS CREDS #{aws_creds.class} ")
  Chef::Log.debug("HIGGS-BOSON: AWS CREDS #{aws_creds.inspect} ")
  Chef::Log.debug("HIGGS-BOSON: AWS CERT pulled out of data bag is #{x509_cert}")
  Chef::Log.debug("HIGGS-BOSON: AWS KEY pulled out of data bag is #{x509_key}")
  Chef::Log.debug("HIGGS-BOSON: AWS ACCESS pulled out of data bag is #{aws_access}")
  Chef::Log.debug("HIGGS-BOSON: AWS SECRET pulled out of data bag is #{aws_secret}")



here's the logging results when the client runs 10.14. pardon the ugly data bag
inspection ... snipped for brevity.

[2013-01-23T06:18:32+00:00] DEBUG: HIGGS-BOSON: AWS CREDS Chef::EncryptedDataBagItem
[2013-01-23T06:18:32+00:00] DEBUG: HIGGS-BOSON: AWS CREDS #<Chef::EncryptedDataBagItem:0x00000002f8e358 @enc_hash=data_bag_item["secrets", "aws-creds-quux",
 {"BOOP_X509_SERVERCERT"=>"z8qqqqqqqqqqqqqqqqqqqnr9mWxzzzzzzzzzzzzzz/sPYH7Cyw/\nQ9ftouk8RRRRRRRRRRRRF9Ryl/fmkkkkkkkkkkkkkkkkkkkkk0EFegz\nenmy1K2/VZph1kdE7DXxxxSNIP\n", "MEEP_AWS_SECRET_ACCESS_KEY"=>"OJ9xxxxxxxxxxZFxxxxxxxxxxwMxxxxxxxxxxxux/rdPpxxxxxxxxxxoffff\ne2ec\n",
 "id"=>"aws-creds-quux", "BOOP_AWS_SECRET_ACCESS_KEY"=>"m1xxxxxxxxxxrKpwh/dixxxxxxxxxxZkuxxxxxxxxxxpoxxxxxxxxxx1cr4\nEpmQ\n", "MEEP_AWS_ACCESS_KEY_ID"=>"vtxxxxxxxxxxdCg/0xxxxxxxxxxloZxxxxxxxxxxx5hQ=\n", "BOOP_X509_PRIVATEKEY"=>"NU99999999999999999999A6IEMUUKf7IccccccccccccccccccvtodL\nTy444444444444lQnePuuuuuuuuuuuuuuuuuuuuuuuDbuFB/hws\nSNIPxxxxxCHEXXjDk+oHWWWWWWefXHgQ999999SNIP\n",
   "MEEP_X509_PRIVATEKEY"=>"xa99999999999999999999dI0sfAjoDW999999999999999999998SPWBoAu\nC9qLxxxxxxxxxxxxxxxxxxxK+AN58y9999999999999999999999\n
NDYFZjeNqDY8rAsnnnnnnnnipppppppBA=\n", "MEEP_X509_SERVERCERT"=>"LkuXuuuuuuuuuuuuuuuuuuuuuuEkyjjxxxxxxxxjD0GRCyH\nfZSfffffffffRrZxxxxxxSNIPxx=">
[2013-01-23T06:18:32+00:00] DEBUG: HIGGS-BOSON: AWS CERT pulled out of data bag is
[2013-01-23T06:18:32+00:00] DEBUG: HIGGS-BOSON: AWS KEY pulled out of data bag is
[2013-01-23T06:18:32+00:00] DEBUG: HIGGS-BOSON: AWS ACCESS pulled out of data bag is AKIsuperthankforaskingXX
[2013-01-23T06:18:32+00:00] DEBUG: HIGGS-BOSON: AWS SECRET pulled out of data bag is Z1DbeverlyhillbillieslexnLKzKF4xxxxxxnNQ


above, you can see i'm getting no value for x509_cert and x509_key, but i am for
aws_access and aws_secret. i tossed a couple more lines into the recipe to look
at #{x509_cert.class} and #{x509_cert.inspect}, and i can see it's an empty
String; it's not nil.


when i run chef client 0.10.10 or 10.12, i get this:

[Wed, 23 Jan 2013 06:17:34 +0000] DEBUG: HIGGS-BOSON: AWS CERT pulled out of data bag is MIIDjjCxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
[Wed, 23 Jan 2013 06:17:34 +0000] DEBUG: HIGGS-BOSON: AWS KEY pulled out of data bag is MIICXQyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
[Wed, 23 Jan 2013 06:17:34 +0000] DEBUG: HIGGS-BOSON: AWS ACCESS pulled out of data bag is AKIsuperthankforaskingXX
[Wed, 23 Jan 2013 06:17:34 +0000] DEBUG: HIGGS-BOSON: AWS SECRET pulled out of data bag is Z1DbeverlyhillbillieslexnLKzKF4xxxxxxnNQ

the result i get works because i use x509_cert and x509_key to populate x509
certs via a template. (but i do wonder why only the content of each is printed
in the log up to the first newline.)

thanks!
kallen