[chef] MySQL and PostgreSQL cookbooks, RubyGems and You!

[Joshua Timberman < >]

Hey Chefs!

I have a topic I'd like to discuss with the community, with regard to
behavior in the "mysql" and "postgresql" (and related, the "database")
cookbooks we publish. First, a couple questions about your
expectations of these cookbooks (whether you use them or not):

1. Do you expect that the "client" recipe for mysql/postgresql will
install Ruby libraries?
2. If so, do you expect that because you want to use them within the
same Chef run? Or because you want to use them with your
application(s)?

The original intention of the package/gem installation of mysql and pg
Ruby libraries in the client recipes was that they should be installed
because one would want them used for an application. This is a 3+ year
old assumption, and probably not the case now. It seems that people
really only want/need those installed when using the Ruby libraries
through a resource/provider elsewhere in the Chef run, such as with
the "database" cookbook.

There are some tickets about this topic in the COOK project[0], and I
would like to get consensus around resolving the issue for the
broadest use case(s). The behavior between the two client recipes is
inconsistent, and we'd like to resolve that, as well.

Current behavior
============

In both cookbooks, the "client packages" are installed, such that the
node can interact with the database through the native C libraries and
client-side programs.

In mysql::client:

* On Red Hat family distributions, the system package "ruby-mysql" is 
installed.
* On Debian family distributions, the system package "libmysql-ruby"
is installed.
* On all other platforms, the RubyGem "mysql" is installed.

Issues with this:

* It is inflexible. If you want to use the Gem for your application,
it won't be installed on RHEL/Debian[1].
* Other platforms might have native packages where the library is
already compiled, and there isn't the option for the native package.
* The gem installation uses the "gem_package" resource. This doesn't
necessarily make the gem available to be used by Chef, especially
under the "omnibus" full stack package.

In postgresql::client:

* The postgresql client libraries and the RubyGem are installed during
the compile phase of the Chef run.
* The "pg" gem is installed on all platforms.

Issues with this:

* COOK-1011 brings up the issue that during the compile phase, the
package cache on Debian/Ubuntu may not be updated, causing the
installation to fail.
* Users who wish to use the native system package don't have an option
without modifying the recipe.

Propsal
======

I have a proposal, generalized for both cookbooks, which I think will
give use more flexibility, and be the least surprising.

1. Move the list of package names for client installation and server
installation to attributes in both cookbooks. They would be:

    node['mysql']['client_packages']
    node['mysql']['server_packages']
    node['postgresql']['client_packages']
    node['postgresql']['server_packages']

Respectively, and platform specific package names would be selected
using a case statement. This is proposed in part for the ticket
COOK-1236[2]. This replaces the current use of a local variable.

2. Remove the RubyGem installation from both cookbooks' client recipes.

3. Add an attribute that indicates the name of the native package that
should be installed, and will be false by default, meaning that the
package wouldn't be installed. Proposed attribute names:

   node['mysql']['ruby_package']
   node['postgresql']['ruby_package']

 Then, a user can set the attribute (e.g., via a role) to the package
name if they want to install it.

4. In the "database" cookbook, create "mysql" and "postgresql" recipes
which handle installing the RubyGems "mysql" and "pg" respectively,
using the chef_gem resource so they can be used by the database
resources.

5. Handle installing build-essential's packages during recipe compile
phase so chef_gem works, and also ensure that the package cache is
updated on Debian family.

I have opened COOK-1441 for tracking the above, and I am working on
migrating related tickets to sub-tasks (or resolving duplicates).

Thoughts? We'd love to hear your feedback here or on the ticket.

-- 
Opscode, Inc
Joshua Timberman, Technical Program Manager
IRC, Skype, Twitter, Github: jtimberman

[0]: COOK-1009, COOK-1011, COOK-1384, and others that have been closed
as duplicates.
[1]: this is the general platform families, including centos, fedora,
amazon, ubuntu, etc.
[2]: this ticket in particular allows the use of percona instead of mysql, 
too.