Re: some questions

[Adam Jacob <adam@opscode.com>]

On Thu, Apr 23, 2009 at 4:48 PM, snacktime <snacktime@gmail.com> wrote:
> I'll throw in my thoughts on this.  For the amount of free formed data being
> stored, I think couchdb is overkill.  It's another moving part that imo is
> not really adding much value.  If you were storing gigabytes of data and
> doing map/reduce over distributed data sets I could see the point, but right
> now couchdb is more a point of frustration for doing lots of stuff that
> would be easier with datamapper/activerecord.

CouchDB really isn't the point of frustration here.  We're storing
fairly large semi-structured JSON data on the server, which we want
clients to be able to query via a RESTful API.   In our case, CouchDB
is providing the back-end storage, and that query API is coming from
Ferret.  The query API is the part that sucks here, not really
CouchDB.

If you were to move this to a relational database, or to something
like sqlite, you'll be essentially creating a trivial table with two
columns: key and a BLOB. (Any other structure is a road to ruin -
check out iClassify for an example) At which point any benefits SQL
bought you are pretty well moot.

> Why not just serialize the node attributes and stick them in a text column?
> Index them, even keep them all in memory if you want, it's not a large
> amount of data even with hundreds of servers.  You could store it as just
> one large hash.    Another thing is that couchdb plus an sql db would end up
> being too much, it really needs to be one or the other.  Which one will add
> the most value overall?  I'm guessing datamapper/activerecord will make chef
> easier to work with, get more people contributing, and result in more people
> using it.

You'll get very little to no benefit from DM/AR, since that won't ever
be the interface you are exposing to the clients themselves, unless
you want every system on your network connecting directly to the
database to handle your queries.  The Chef internals are pretty simple
and clear with CouchDB in place.  I would be open to having switchable
data storage layers, but really, I don't think it will solve the
problem you think it will.

Adam

-- 
Opscode, Inc.
Adam Jacob, CTO
T: (206) 508-4759 E: adam@opscode.com