[chef] Best practices for multiple data centers

[John Alberts < >]

I'm wondering how others use Chef for multiple datacenters and if they
use multiple Chef servers.
In my case, we have multiple data centers around the world and each
data center has a private network that all of the servers are on.  I
currently have a single Chef server at one of the data centers that
every client connects to.  This works fine, but it does have
disadvantages.
1. chef-client runs slow at data centers located on the opposite side
of the world because of latency and bandwidth.
2. While the bandwidth usage right now is not much, I'm worried that
it will be significant as our usage of Chef for various things
increases.

One obvious solution is to have a separate Chef server at each data
center; however, I am then managing multiple Chef installations that
use 99.9% of the same code.  The primary disadvantage of this is that
I then have my nodes on different Chef servers and no way to search
for all of the nodes.  Specifically, my monitoring is automated and
needs to be able to get a list of all servers, roles, etc.  Having
separate Chef servers really ends up creating a large barrier to
managing all of my nodes.

I'm sure others have run into something similar, so I'm wondering what
others are doing.

One thought was that it would be great if there was some sort of Chef
proxy server that I could have at each location that cached the files
and node data.  That way my nodes at a dc could contact the proxy for
all of it's needs and the proxy would sync up with the main Chef
server every so often.  Anyone working on anything like that?  :)



-- 
John Alberts