[chef-dev] Re: Re: Re: Re: Re: Re: edit a file using chef

[Lamont Granquist < >]

Problem is that while this is always the most correct way to do things, 
that it may not be the correct way for the business that is using chef 
to get things done right now.  Its quite possible that you've got a huge 
mess on your hands and the easiest way to alleviate 90% of the pain is 
to simply enforce one line of state in a config file that has a thousand 
unique snowflakes that have been deployed by hand over the last decade.  
The 'right' thing to do is clean up the whole mess, but that often takes 
10x the effort, since you often need to audit the whole thing and figure 
out what deltas in all of the config options are actual necessary 
business requirements and what is cargo cult and deltas that are 
unnecessary.  Leaving that unmanaged state out there is often messy, but 
I've made that decision a lot in order to move forward in increments.  
We also leave most of the state of the system unmanaged because most of 
the state doesn't matter.

There's also other use cases, such as using bundled chef-solo to manage 
state for an application that needs to ship and be integrated with 
uncontrolled external config management systems.  Such code may need to 
enforce state in a file, but the rest of the state of that file is 
complete out of your control and fundamentally cannot be brought under 
your own management.

Within the same company you can wind up seeing this problem where 
departments may have written tooling that edits files already and config 
management systems may need to integrate with those and chef may need to 
make partial edits to the portion of the file that it 'controls'.  The 
right solution there is always to fix that separation of concerns and 
make chef authoritative and to call a tool in order so that chef could 
render the whole template, but stupid thing like that can make for 
political snarls in large organizations with legacy infrastructure and 
legacy teams, and I don't think chef should be in the business of 
inserting itself into politics.

And this goes to the whole meme of being "opinionated", and I'm already 
highly skeptical and tired of that meme and think its a bad one for this 
kind of product.  Its also why I don't use CFEngine anymore because of 
some of the opinions of that crowd that made it almost deliberately 
difficult to integrate that product with a central database.  I really 
don't like using tools that have opinions that force me into making 
certain cost/benefit decisions in my job, I need to be able to weigh 
those decisions myself.

So, yes, I've run into this decision over and over through the years, 
and have quite often determined that it is *far* from trivial to take 
the whole file and templatize it (its often far from trivial to 
understand the actual requirements from the cargo culting), and have 
chosen to either do the expedient thing for the business and clean up a 
fragment of it first, or simple to ignore the rest of the config.  In 
some cases I've actually written code that does "if the file exists, 
edit it to enforce this one line, if the file does not exist, then use 
my nice standardized template" to gain eventual convergence and reduce 
config deltas over time while at the same time punting the existing 
mess, and allowing me to test my standardized template every time a new 
server goes into prod.

And the whole use of "our model" and such is language that is kind of 
annoying.  Chef didn't invent that, its just common practice that its 
the better way of doing things in the unix sysadmin world, but its not 
the /only/ configuration management design problem out there -- although 
for a small startup with 40 nodes in the cloud that has been using chef 
since day #1 it may be the only one that you need.  When talking about 
walking into thousands of unmanaged servers grown organically over a 
decade and on different major flavors of operating systems....  You will 
need every trick in the book.

And being able to use every trick in the book and be able to easily get 
the job done -- no matter how broken the existing systems and no matter 
how ingrained the departmental politics -- is what is going to make chef 
awesome to work with for practicing/suffering devops/admins out there...

On 6/15/12 7:29 AM, Christopher Brown wrote:
> This exchange (Aaron and Lamont) is almost exactly the conversation
> Adam and I had when I first had Nuo write file_edit and that's why we
> decided to bury it.  It's useful, but cuts against the grain.
> Pragmatically though, there are files over which you may not have full
> control.  We constantly tell people that you can ease into as much
> Chef as you can take, without having to rigidly accept our model.
> Doesn't saying "but you can only manage whole files, rendered by Chef"
> blunt that argument?
>
> -C
>
> On Fri, Jun 15, 2012 at 7:21 AM, Aaron 
> Peterson<
>  >
>   wrote:
> > Lamont "I promise I'll only use it correctly, tactfully, and rarely"
> > Granquist, your argument does not persuade. :). But even if you held
> > true to that...
> >
> > -1, I could never recommend this, and it fails most methodological
> > tests that Chef works under.
> >
> > It should be trivial to take the whole file and templatize it. If you
> > cannot just easily take the whole file under management, that means
> > there is interestingly different content "out there" that represents
> > unmanaged yet important operational state. Now you are making
> > successive error-prone edits to a collection of interestingly
> > different files, and there is no guarantee that you will not have
> > drift unless you have already analyzed the differences across the
> > collection, at which point you might as well manage the whole thing.
> >
> > Even if it is consistent, or the file differences are reproduced
> > normally by an underlying OS or software process, I had better only
> > add but never delete file edits to recipes, because if ever a machine
> > misses a past edit because it is off or whatever, or we launch a new
> > machine, we won't be creating the file correctly.
> >
> > Generally, if you cannot always guarantee the recreation of the file
> > in its current state, you have entombed state and risk and have failed
> > to create an infrastructure you can reason about or rebuild.
> >
> > Making it a full resource is the Chef imprimatur to use it, and this
> > isn't just rope, it's already tied into a noose and permanently
> > attached to a scaffold.  IMO.
> >
> > -a
> >
> > On Jun 14, 2012, at 14:02, Adam 
> > Jacob<
> >  >
> >   wrote:
> >
> > > I wonder if it's time to finally turn that into a resource, even
> > > though I think there is near universal agreement it's not a great
> > > idea?
> > >
> > > Adam
> > >
> > > On Thu, Jun 14, 2012 at 4:54 AM, AJ 
> > > Christensen<
> > >  >
> > >   wrote:
> > > > https://github.com/opscode/chef/blob/master/chef/lib/chef/util/file_edit.rb
> > > >
> > > > On 14 June 2012 23:25, Bryan 
> > > > Berry<
> > > >  >
> > > >   wrote:
> > > > > minimin, u can use the template resource for this purpose
> > > > >
> > > > > questions like this are probably better suited to the main chef mailing list
> > > > >
> > > > >
> > > > > On Thu, Jun 14, 2012 at 1:11 
> > > > > PM,<
> > > > >  >
> > > > >   wrote:
> > > > > > Hi,
> > > > > >
> > > > > > I saw a file resource which can be used to create and delete a file,but i
> > > > > > want
> > > > > > to edit a file with values provided by the user.
> > > > > > Does chef provides such facility??
> > >
> > >
> > > --
> > > Opscode, Inc.
> > > Adam Jacob, Chief Customer Officer
> > > T: (206) 619-7151 E: 
> > >
> > >