[chef] Why an environment's override_attributes are not set until chef-client completes successfully?

[Jay Pipes < >]

Been trying to diagnose why something is happening in our environment,
and can't seem to figure it out.

Here is some commands showing the issue:


 
 16:52:06:/opt/chef-repo# knife node show
c5r3.int.iad1.attcompute.com
Node Name:   c5r3.int.iad1.attcompute.com
Environment: production
FQDN:        c5r3.int.iad1.attcompute.com
IP:          192.168.112.143
Run List:    role[iad1], role[openstack-identity]
Roles:       booted
Recipes:     apt, ohai, chef-client::cron, users::sysadmins, openssh,
sudo, reboot-handler, networking, raid, sol
Platform:    ubuntu 12.04
Tags:

 
 16:52:11:/opt/chef-repo# grep -n2 "admin_user"
environments/production.json
98-        "demo"
99-      ],
100:      "admin_user": "ksadmin",
101-      "users": {
102-        "ksadmin": {

 
 16:52:16:/opt/chef-repo# knife node show
c5r3.int.iad1.attcompute.com -Fj | grep admin_user

 
 16:52:28:/opt/chef-repo#

From above, you can see that:

a) The node in question has the production environment
b) The production environment has the keystone:admin_user attribute set
to "ksadmin", not "admin"

Unfortunately, when running chef-client on the node above, the override
"ksadmin" value set in the environment's override_attributes does not
get used. Instead, the recipe's default value of "admin" gets used,
which results in a failure.

Here is the output of chef-client on the node:

http://paste.openstack.org/show/30418/

and here is the code that is calling the above:

http://paste.openstack.org/show/30419/

Why doesn't a node's environment override_attributes get merged to the
node's attribute collection before chef-client runs? Why would
convergence need to occur before a node's environment attributes are set
in the node's attributes collection?

A more general question would be: data is data, why on Earth do Chef
searches return different data about a node depending on whether
chef-client has run successfully on a node or not? I can understand this
behaviour for automatic attributes from Ohai, but it does not make much
sense for any other attributes, IMHO.

Best,
-jay