[chef] race conditions during resource allocation

[Nick Ohanian < >]

Hi all-

I have a question about possible race conditions during resource allocation.  Let me give a 
simple (but somewhat silly) example.  Imagine I have two types of nodes: web-server nodes and 
database nodes.  As far as Chef is concerned, new nodes can come alive at any random time (in 
reality, they come alive when an auto-scaler determines that more resources are needed in the 
deployment).  I want to make sure that each web-server, once configured and running, is connected 
to exactly one database.  I can accomplish this by storing an attribute on the web-server nodes 
called "using-database".  When a new web-server comes alive, it searches the 
chef-server db for the list of web-server and database nodes, and it looks for a database node 
that is not being used by any other web-server.  Then it sets that database node in its 
"using-database" attribute.

This works fine, but you get the usual race condition problem with shared read-write 
resources.  If two web-servers come alive at the same time, they will search for an 
available database at the same time, and possibly select the same one, and save it to 
their "using-database" attributes.  The one-web-server-per-database 
constraint is then violated.

Has anyone else come across this problem?  Does this situation go against the intent of a Chef system?  Or is there support for it already, or some other Chef best-practice?  

Thanks in advance.

-Nick O