[chef] Re: Re: Re: Re: Re: Re: SCM for node definitions?

[Benjamin Bytheway < >]

That pretty much sums up the thought process we've been going through.  It is tempting to have everything in one place (the chef repo), with full version control, history, etc.  But it sure puts a huge barrier to entry up to even the most minor of changes. And getting configuration changes into 

We've gone back and forth on driving configuration through lwrp + application cookbooks, or putting configuration in data bags, or creating a separate configuration database with a REST api that chef cookbooks query to get their marching orders in laying down the right components.

All have their strengths and drawbacks.  We've had a hard time seeing a way forward.  One issue is that we have a huge number of apps that currently run on a much smaller number of servers, with many apps stacked deep on each cluster.  This makes nearly every server different in fundamental ways.  There is much that is common as far as the larger components (weblogic, apache, jvm), but after that the variation is enormous.

Seems like there are as many ways to do this as there are organizations using chef.  

-Ben

On Tue, Aug 13, 2013 at 7:09 PM, Lamont Granquist < " target="_blank"> > wrote:

I find that limiting because then you have to be a dev in order to push SCM updates, and you're going to run into SOX and PCI issues sooner or later as you try to expand who has the ability to push config updates.  If your configuration is also represented in a database then you can use RBAC in order to control who can update those roles and modify config.  People could be writing dashboards that write to dynamic databags in order to change server access control (add/remove users), and to affect workflow (approve deployment to prod) and other ways to hook in 'IT Governance' dynamically into Chef, beyond having the people with commit access to the repo being the gateway to all configuration change.

I don't much care for mantras.  I don't think adopting a database-model is in conflict with the intent of that mantra, since the data in the database is being translated into configuration change on the servers by code.  So its still code doing the work, we're not logging into the servers and making with the typey-typey.  But, if its in conflict then someone needs to explain how the hardcore-SCM-centric worldview produces utility and gets the job done better.  I tend to see it as not scaling out well beyond this developer-centric IT model.  Databases and SCM are tools, but they have different purposes.

On 8/13/13 5:28 PM, Benjamin Bytheway wrote:

I'm currently trying to untangle this question for my own organization. If the mantra "configuration as code" is really true, shouldn't _everything_ (that isn't true dynamic/discoverable data, like ohai) be "code" and under SCM?

On Tue, Aug 13, 2013 at 6:10 PM, Lamont Granquist < " target="_blank"> > wrote:

On 8/13/13 12:08 PM, Jeff Blaine wrote:

On 8/13/2013 2:51 PM, Ranjib Dey wrote:

u mean the node object?

Yes. We're a "lab" type of environment with a wide array of node types and configurations. It is a rarity for any of our servers to have the exact same configuration as another. Fun...

I'm having uneasy feelings lately about everything *except* our node code/data being in files under SCM. It's a break from convention, an unnecessary(???) anomaly for other less experienced staff to remember, etc.

I guess one could download all of the current nodes' JSON data into files, check them all in, and somehow disable 'knife node edit' in favor of 'knife node from file'?

Anyone doing this?

That data isn't really code, its a row in a database.  I can see the value of periodically dumping the node data to SCM and storing it as history, but people don't normally try to manage database tables with SCM.  If you start trying to limit what can write to nodes and data bags you start to unnecessarily cripple your ability to use the chef database as a dynamic CMDB.  And IMO there is some unmined potential in treating databags and nodes as simple database tables and pushing data in them from other sources in the enterprise, and to look beyond only pushing to them from SCM.  You can try to push the SCM tooling as far as possible, and maybe I'm wrong about this and if we only took SCM /really/ seriously we'd discover something sublime about how to manage servers (e.g. being able to git push directly to a chef server), but I come from having managed servers using classical CMDBs and found them very powerful and trying to manage CMDBs entirely with SCM seems unnecessarily crippling to me...  You don't typically dump your LDAP or AD databases into SCM. Nobody dumps their customer databases into SCM.  In the networking world rancid pushes Cisco configs into SCM, but its to maintain a history and log of changes, not to manage networking gear with SCM.