[[chef-dev]] Re: [[chef-dev]] Introduction and Alternate Rubygems provider

[Daniel DeLeo < >]

Pierre,
This is awesome, and something we've wanted for a long time. Cf.
http://tickets.opscode.com/browse/CHEF-349

On Fri, May 14, 2010 at 7:19 AM, Pierre Baillet 
<
 >
 wrote:
> Hi,
> I'm quickly introducing myself in the list and will then expose an issue we
> have here and a way we found to fix it.
> I'm Pierre Baillet, the lead server administrator at fotonauts. I'm in
> charge of most of the servers used for the fotopedia website. We use Chef on
> all our development, testing and production instances as a way to
> standardise our deployments, keep a state of what's installed on each server
> in case of server failure and automate releases. We are quite happy with
> Chef, especially when it comes to its powerful Search features that allows
> building all our monitoring configuration automatically !
>
> When installing (usually at bootstrap) complex projects, the regular
> Rubygems Package Provider ( Chef::Provider::Package::Rubygems ) does a lot
> of `gem` calls, hence slowing the installation of gem quite lot. Moreover,
> using chef recipes that include  "~>" version checking are run at every chef
> run and because we run chef every 5 minutes here at fotonauts (mainly as a
> deployment helper) we appreciate every spared second during this procedure.
> As a workaround for this, we have developed a Rubygems provider (based on a
> monkey patched version of the regular provider) which no longer calls the
> gem binary every time but:
> * Uses the Gem API to grab the list of available gems,
> * Keeps this list in a cache so that subsequent calls to the Provider are
> much faster
> The only parts that use `gem ..` is the one that actually install the gems
> although there is also a way to programmatically install a gem using the Gem
> API.
> By looking at the original source code of the Provider, I think that the
> fact that it's not using the Gem API is something that's voluntary and I
> guess that it's to prevent any dependency with the internal Gem API. Am I
> wrong ?

Now, the tricky part is that we support managing a different rubygems
installation than the one used by chef. This is what the gem_binary
parameter is for. I was also playing with converting the gem provider
to use rubygems' internal API, and I think I have the solution. If you
shell out to `gem env gemdir` you will get one or more gem
directories, and the gemspecs will be in a subdirectory
$gemdir/specifications. You can then run Gem::SourceIndex.from_gems_in
to get the specs from that gem installation.

> If this is the case, I suppose that our patched version will never make it
> to the official HEAD (although I'm not sure I understand the interest of
> this restriction today as Chef already relies on several external and
> sometimes API-changing libraries :D).
> However, if this is not the case, I'll gladly provide a patched version of
> the Rubygems provider with our enhancement, for testing first, of course :)
> Thank you very much for your time,
> --
> Pierre.
>
>

In summary, this is really cool, and if you can get it to manage
multiple gem installations, we'll accept it into master. Hope my tip
is helpful.

Dan DeLeo