Heads up everyone, this morning my "Amazon Linux AMI 2014.09" servers starting failing upon attempting to install any package with chef-client. The failure looks like this for any "package" install you try:
[2015-03-25T14:49:09+00:00] INFO: Processing package[tzdata] action install (timezone-ii::default line 16)
================================================================================
Error executing action `install` on resource 'package[tzdata]'
================================================================================
Mixlib::ShellOut::ShellCommandFailed
------------------------------------
Expected process to exit with [0], but received '1'
---- Begin output of /usr/bin/python /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.8/lib/chef/provider/package/yum-dump.py --options --installed-provides ----
STDOUT:
STDERR: Traceback (most recent call last):
File "/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.8/lib/chef/provider/package/yum-dump.py", line 36, in <module>
import yum
ImportError: No module named yum
---- End output of /usr/bin/python /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.8/lib/chef/provider/package/yum-dump.py --options --installed-provides ----
Ran /usr/bin/python /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.8/lib/chef/provider/package/yum-dump.py --options --installed-provides returned 1
Resource Declaration:
---------------------
# In /root/.chef/local-mode-cache/cache/cookbooks/timezone-ii/recipes/default.rb
16: )
17:
Compiled Resource:
------------------
# Declared in /root/.chef/local-mode-cache/cache/cookbooks/timezone-ii/recipes/default.rb:16:in `from_file'
package("tzdata") do
action :install
retries 0
retry_delay 2
guard_interpreter :default
package_name "tzdata"
cookbook_name "timezone-ii"
recipe_name "default"
end
After investigation, I found that the package which I update in my userdata "aws-cfn-bootstrap" was updated sometime in the last 24 hours, updating its dependencies to install python27. yum-dump.py seems to be calling /usr/bin/python, which is symlinked to python2.6 after the 2.7 install. /usr/bin/yum header was updated to "#!/usr/bin/python2.7". The problem seems to be that the 2.7 install breaks 2.6 interactivity with yum.
~]$ ls -al /usr/bin/python*
lrwxrwxrwx 1 root root 24 Mar 25 14:48 /usr/bin/python -> /etc/alternatives/python
lrwxrwxrwx 1 root root 9 Mar 25 14:48 /usr/bin/python2 -> python2.6
-rwxr-xr-x 1 root root 5120 Mar 18 19:36 /usr/bin/python26
-rwxr-xr-x 1 root root 5120 Mar 18 19:36 /usr/bin/python2.6
-rwxr-xr-x 1 root root 5120 Mar 4 20:43 /usr/bin/python27
-rwxr-xr-x 1 root root 5120 Mar 4 20:43 /usr/bin/python2.7
~]$
~]$
~]$ /usr/bin/python2.6 /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.8/lib/chef/provider/package/yum-dump.py --options --installed-provides
Traceback (most recent call last):
File "/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.8/lib/chef/provider/package/yum-dump.py", line 36, in <module>
import yum
ImportError: No module named yum
~]$
~]$
~]$ /usr/bin/python2.7 /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.8/lib/chef/provider/package/yum-dump.py --options --installed-provides | head
[option installonlypkgs] kernel kernel-devel kernel-source installonlypkg(kernel) installonlypkg(kernel-module) installonlypkg(vm)
389-admin 0 1.1.35 1.11.amzn1 i686 [] a amzn-main
389-admin 0 1.1.35 1.11.amzn1 x86_64 [] a amzn-main
389-admin-console 0 1.1.8 3.1.amzn1 noarch [] a amzn-main
389-admin-console-doc 0 1.1.8 3.1.amzn1 noarch [] a amzn-main
389-adminutil 0 1.1.19 1.4.amzn1 i686 [] a amzn-main
389-adminutil 0 1.1.19 1.4.amzn1 x86_64 [] a amzn-main
389-adminutil-devel 0 1.1.19 1.4.amzn1 x86_64 [] a amzn-main
389-console 0 1.1.7 2.1.amzn1 noarch [] a amzn-main
389-ds 0 1.2.2 2.1.amzn1 noarch [] a amzn-main
My quick solution for this morning was to remove the update of the aws-cfn-bootstrap package from my userdata before chef-client runs.
If anyone has a more appropriate fix for this problem, let me know. If this is a bug, I will gladly file it for chef-client.
I updated to the latest chef-client, and this is still a problem. I suspect this might actually be a bug that needs to be reported?