[chef] Re: Re: Re: Re: SCM for node definitions?

[Lamont Granquist < >]

"Rare and release managed as much as possible" is the opposite of agile, and in pretty strict opposition to ideas like continuous delivery as well, unless I'm misunderstanding you.  It also just doesn't scale as the business grows.  Eventually centralized beaurocratic operations is simply overwhelmed and doesn't serve the business.  At that point I prefer lightweight change that happens often, through well engineered channels and is distributed throughout the Enterprise.  If all we've done with devops is move it from "SAs" logging into boxes using their godlike powers to type 'adduser' to "devops" making changes in git and typing 'git push', I think you've just shuffled the responsibilities around without really breaking down the walls to the rest of the company.

I'm also very skeptical that synchronizing across orgs will scale.  As someone who dealt with horizontal SOX and PCI-DSS configuration responsibilities in an enterprise with something like 6,000 different individual roles and hundreds of business units, the idea of having to manage compliance across 100 tenants where its designed around those being compartmentalized and partitioned, instead of sharing
common state, is a little horrifying.  You lose the ability there to have a single base role that contains recipes that are pushed to every single server in your platform, which is very powerful when it comes to compliance.  When you are small and you have some committed team members then you can do a good job at synchronizing across orgs, but when you hit more orgs you'll suffer from rot and you'll wildly varying degrees of compliance across your orgs.

You could address that by trying to engineer better synchronization primitives around orgs, but again the design goal there is for hard partitioning between orgs in hosted chef, so the organization concept starts with being fundamentally hostile to trying to do that.

On 8/14/13 2:46 PM, Maxime Brugidou wrote:

" type="cite">

What we do is a separate chef server per team/org with entirely separate chef repositories. We distribute cookbooks to the whole company and people have to upgrade the common cookbooks every now and then (using librarian internally)

It would be nice to have the "organization" thing available in open source chef so that we don't have a chef server for each team. We try to maintain the number of teams as low as possible because of that.

However within one team the nodes are managed within git (note: it's bare metal we don't have any VM or dynamic node creation) and literally every operation on prod goes through git and then automated release management. This forces people to automate, document and log anything happening on prod. I really think this is the right way. Operations should be rare and release managed as much as possible. Dynamic scaling with VMs could also work but then you have something else automated that manage the nodes for you, not a human.