[chef] Re: Re: How would I prevent python_pip from installing packages that are already installed?

[Fabien Delpierre < >]

Aimelyne,

Thanks for pointing it out, but no, it's all good in the code, I think. In fact the code uses attributes extensively, I just made it plainer just for clarity. I guess I typo'd the 'python33', indeed it should read 'python3.3'. Only the "python" is hardcoded in the code but the digits come from an attribute and should = 3.3, which is what I need since that's the correct path :)

default['python']['version'] = '3.3.5'

default['foo']['python_minor_version'] = node['python']['version'].split(/(^\d+\.\d+)/)[1]

reqs.each do |req|

  python_pip req do

    virtualenv node['foo']['venv']

    action :install

    not_if "ls -1 #{node['foo']['venv']}/lib/python#{node['foo']['python_minor_version']}/site-packages | grep '#{req}$'"

  end

end

On Fri, Mar 6, 2015 at 4:58 PM, Aimelyne Mochiron < " target="_blank"> > wrote:

Fabien,

This may be a copy-paste typo restricted to your message to the list, I realise, but just in case: the path you specify in your guard is not the same as the one you use when running the command manually:

ls -1 /foo/lib/python33/site-packages

ls -1 /foo/lib/python3.3/site-packages

cf. 'python33' in the first one vs 'python3.3' (33 vs 3.3) in the second one. Is the typo present in your code?

am

On 6 Mar 2015, at 20:43, Fabien Delpierre < " target="_blank"> > wrote:

Thanks for the suggestion!

No dice, unfortunately, it still took me for the full round of unneeded "updates".

On Fri, Mar 6, 2015 at 1:38 PM, Michael Weinberg < " target="_blank"> > wrote:

Does it work if you append '$' to the grep like so:

"ls -1 /foo/lib/python33/site-packages | grep '#{req}$'"

--

Michael F. Weinberg | Director of Operations

http://heavywaterops.com | @heavywaterops

On Fri, Mar 6, 2015 at 10:29 AM, Fabien Delpierre < " target="_blank"> > wrote:

Hello folks,

I have the following bit of code:

reqs.each do |req|
  python_pip req do
    virtualenv /foo
    action :upgrade
    not_if "ls -1 /foo/lib/python33/site-packages | grep #{req}"
  end
end

Now, reqs is a pretty large list of pip packages (probably about 30) compiled from a requirements file. Additionally, each package has a specific version requirement.

So whenever Chef runs, it painstakingly attempts to upgrade every package listed in reqs despite my apparently feeble and incompetent not_if. It causes my Chef run to take 5-7 minutes instead of 2-3. This is fine for the first run, but this is a development VM for a team that iterates pretty fast, so 5-7 minute "compile times" are going to be a drag.

One strange thing that might be relevant is that the Chef log reads something like the below line for each pip package:

==> default: [2015-03-06T13:02:05-05:00] INFO: Upgrading python_pip[alembic==0.6.4] version from uninstalled to latest

Note that it says "uninstalled" even though the package is definitely installed.

$ pip list
pip (6.0.8)
setuptools (12.4)
virtualenv (12.0.7)

$ source /foo/bin/activate
(foo)$ pip list
alembic (0.6.4)
...
pip (6.0.8)
...
setuptools (13.0.2)
...

(foo)$

Interestingly though, there is one package in reqs without a version requirement, it's gunicorn. When Chef tries to install it, it says this instead:

==> default: [2015-03-06T13:01:57-05:00] INFO: Upgrading python_pip[gunicorn] version from 19.3.0 to latest

Frankly it shouldn't even be trying to install gunicorn anyway, but it's interesting/weird to me that for this one package, it accurately detects that it's installed and returns the installed version, whereas all the other ones report that they're uninstalled even though that's wrong.

The ls command in my not_if returns the correct code whether the virtual environment is active or not.

$ ls -1 /foo/lib/python3.3/site-packages | grep alem; echo $?
alembic
alembic-0.6.4-py3.3.egg-info
0
$ ls -1 /foo/lib/python3.3/site-packages | grep foo; echo $?
1

So unless I'm off the mark with my logic, the python_pip resource should execute only if grep returns 1... but that's not the case and I'm confused and help please pretty please with powdered sugar and a cherry on top.

Thanks in advance!

Fabien