[chef] Re: Re: Re: Re: Re: RE: Running chef in multiple environments

[Lamont Granquist < >]

On Thu Jul 31 16:48:19 2014, Sean OMeara wrote:
> Think about if this way: if your application has a security
> vulnerability that allows an attacker to chmod 777 /etc/shadow... do
> you want to wait until your next production change window to repair
> it? Or do you want to fix it as quickly as possible?

Yeah, I'd forgotten about that.  In a SOX/PCI-DSS world you really want 
to be able to show that you're running it frequently and enforcing 
security with it.

One thing I used to do and typically suggest is to run it in a window 
once a day with random splay.  I used to run CFEngine/Chef from 
8pm->8am PST.  That would mean that during business hours I could push 
changes and they wouldn't immediately take effect until you poked the 
box.  Then you could make it part of your rollout plan to test on some 
canary-style boxes in production and if you discovered a serious 
problem you could revert the change without having trashed all of 
production.  Once you were confident that the canarys were all still 
alive then you could manually push it out with knife ssh.

Means you've got a 24h window before you revert issues like the 777 
/etc/shadow file.  Also means that if you accidentally chmod 400 
/etc/resolv.conf you've got 12 hours to catch it as it rolls out though.

Worked great for me (at large scale even), but does not seem to be a 
common pattern anyone else uses.