Re: storing easily queried metadata - was Re: some questions

[David Lee <david.lee@kanji.com.au>]

Actually, before I go offering any more opinions, can I please ask what 
we're actually doing? In reading the responses, I realised I don't know 
enough about what's going on yet to offer very constructive input.

*Please* correct me if i'm wrong, but it seems like we're basically 
wanting to:

1) store node data records, which are a fairly large, arbitrarily 
structured nested hash (ruby / json), with string key/value pairs

2) find node records which match some very simple criteria, and return 
the entire node data structure for matches

3) store some additional metadata about nodes themselves, recipes, and 
other Chef classes / objects; these bits of metadata would be 
lightweight and possibly act like polymorphically associated 
ActiveRecord objects.

Is this a reasonable summary?

If it is, would a decent native Ruby object database be a pretty 
reasonable thing to use as a backend? Does such a thing exist?


David Lee wrote:
> well, my immediate thought is to use eg sqlite3 + activerecord to store 
> metadata for a node like this, keyed on the node/recipe name or other 
> unique ID.
>
> This would be pretty easy, and make implementing these features a snap- 
> not sure if the additional dependencies would be welcomed though. It'd 
> also mean 2 separate data stores, which I don't think I like the sound of.
>
> Otherwise, it seems there are a few couch-backed ORMs turning up. Any of 
> those decent?
>
> Writing complex queries directly is so 2001 ...
>
>
> Adam Jacob wrote:
> > On Thu, Apr 23, 2009 at 12:45 AM, David Lee <david.lee@kanji.com.au> 
> > wrote:
> > > > I'm not sure I understand how the search indexes help solve Miguel's
> > > > problem?
> > > I was just curious about how difficult the implementation is likely 
> > > to be.
> > > An ActiveRecord, with polymorphic joins etc, would likely require 
> > > less work
> > > than one using native database interaction.
> > >
> > > But I gather it would have to be built directly on top of couch; I have
> > > little sense of the difficulty involved at this point, and I'm curious.
> >
> > CouchDB is particularly ill-suited to ad-hoc queries of that sort -
> > while you would be easily able to pull out single objects, you really
> > don't have the ability to string together arbitrary queries in the way
> > you are thinking.  This is a side-effect of being schema-free and
> > document oriented, and it's why something like a full text index for
> > the CouchDB documents is necessary.
> >
> > We're chatting about ways to make this better in the long term - we
> > would love to hear your thoughts on the matter.
> >
> > Adam


--

David Lee

Application Development Coordinator
Kanji Group Pty Ltd

02 8272 9483
david.lee@kanji.com.au


---
This message and any attachment are confidential and may be privileged 
or otherwise protected from disclosure.  If you are not the intended 
recipient, please telephone or email the sender and delete this message 
and any attachment from your system.  If you are not the intended 
recipient you must not copy this message or attachment or disclose the 
contents to any other person.

This email message does not constitute legal, financial or any other 
kind of advice and reliance must not be placed on its contents.  Any 
advice will be prefixed with a notice to that effect - and unless such a 
notice is affixed all liability for the contents of this email is 
disclaimed. The integrity of this email, its contents or any attachments 
is not certified in any way by the sender.

Liability limited by a scheme approved under Professional Standards 
Legislation