[chef] Re: Re: Re: Re: Best Practice :: Performing App Deployments with Chef

["steve ." < >]

I've had to solve this problem a few times now at work for different teams with different philosophies, so I've had the chance to try a few different approaches.

The one that seems to have worked the best for everyone concerned is this:

* Build server uploads build artifacts to an artifact repository (Artifactory or Nexus, for us).

* The artifact repository app gets to decide what artifact's appropriate for what environment.

* Chef client runs query the artifact repository (possibly multiple repos) and use the newest accessible version.

I don't much like the idea of a development server that's ssh'ing directly into nodes and running chef-client on them (though I guess if you're limiting that user to only be able to "sudo chef-client" that could be okay from a security perspective).  The security guys here didn't like it either, so we set up a RunDeck server in the data center environment and have the developer-controlled CI servers triggering pre-scoped RunDeck jobs to do the Chef runs (i.e., "trigger the RunDeck job that runs Chef on all the Tomcat servers that run my app in the development environment").

Works pretty well so far.

On Fri, May 31, 2013 at 10:34 AM, Jonathan Mickle < " target="_blank"> > wrote:

Daniel,

Data bags are doable as well and I have used and seen them used in the past. For us, environment files were easiest for everyone to pick up. Also we keep our most secure items in the data bags so that they are not in the environment files where developers have access to in the github repo.

On Fri, May 31, 2013 at 8:52 AM, Daniel DeLeo < " target="_blank"> > wrote:

On Friday, May 31, 2013 at 7:56 AM, Jonathan Mickle wrote:

Hey Luke, 

Thanks for bringing this up, this is a common problem that I personally have seen time and time again needing to be solved in the chef community. I can at least shed some light on how my teams have accomplished this. My team performs deploys at any time, sometimes we can have up to 20-30 deploys a day to any environment (dev, staging, qa, production). We first started out using the environment files as the first indication of the build. Essentially we would all update an attribute override for application-version in the environment. We have about 50 different portions that go into our whole application so we can build a single portion an deploy that portion as needed. The problem with this was, our developers needed to push as well and eventually we ended up having to put information overrides into the environment file as well that included things like production application passwords or keys that would fill in certain configuration files per environment. These attributes we did not want everyone to be able to see or have access to. 

Our second approach was to use jenkins to edit the json file. We attempted a series of regex/perl replacements for strings, this sucked and i will leave it at that :-P

Eventually, we got to where we recently landed within the last couple of weeks. I ended up writing tools for our team, I am contemplating open sourcing it if I can see enough people who would like to use it. But essentially I wrote a web based (django app) that allows users to edit portions of the environment files based on their permissions level. This does sanity checking to ensure that its modified properly, when the edit is done, it pushes to git to track and backup the modification, and it also makes an API call to the local chef server to actually process the change. The tool also allows the developers and admins to launch deploys per server/environment based on their permissions levels as well.

I hope that helps but essentially yes, you should be tracking everything in git. How you want to edit the json files seems to be the #1 disconnect so far of managing chef. I have written this tool and I know some others who have done similar. If interested I could hack together an open source version of my tool you can fit into your environment.

Jonathan 

Why not use data bags? You can have either one big data bag of app versions, or one per app, and have data bag items named after the environment, which makes it easy to look them up in your recipe code. You can then just let Ci own these data bags and not have to worry about partial edits (as long as you make things granular enough). If git is the only workable change tracking solution for you, you could probably put these data bags in their own repo that only Ci commits to, so you won't have to worry about merge conflicts either.

-- 

Daniel DeLeo