[chef] WordPress / Git management with Chef

[Morgan Blackthorne < >]

I'm currently running a WordPress cluster on AWS for livemocha.com. This has been interesting (especially overlapping it with the existing legacy site), but one of the big problems that I've been facing is trying to keep the themes/plugins/etc consistent across the EC2 nodes. WordPress gets unhappy if you activate a new plugin on one node and then can't find it on another node. The plugin activation state is stored in the DB, not the local fs, so when it finds it missing it de-activates it across the board... not helpful.

So what I currently do is put the homepage into maintenance (by shoving all relevant requests over to the login page for the actual site), point my hosts file to a build host (that does double-duty as the cron host), do my updates, build a new AMI, spin down the old instances and spin up new instances. When everything is copacetic, I then take the site out of maintenance. This is, needless to say, time consuming. The timeframe from queuing an AMI build to out of maintenance is about 20 minutes, let alone the time taken to update things and test them.

Updates are becoming frequent, so I want to trim the amount of time this process takes by updating the nodes already in service and saving full AMI rebuilds for things like major OS updates (kernel updates, etc). I've thought of a few different ways to handle this, but the one that I'm currently leaning towards is a mix of Git and Chef. Basically I would put the site into maintenance, do my changes on the build host, and if it passes testing, then I would commit the changes into Git. This would essentially be everything underneath the WordPress wp-contents/ directory (if WordPress core updates become a big enough issue as well, then I can expand it to be the entire web root, but for right now I'll avoid that). Once everything is pushed up in Git, I'll then use Knife SSH to trigger a chef run across the cluster that will do a git pull and ideally bounce Apache (or I can use Knife to do that as well; either way, not a big deal). If it's something like a new plugin being added, I don't even have to put it into maintenance at all, just commit it in Git and push away with Knife.

Has anyone done something along these lines before? Can you think of a better way to handle it? I thought about using s3fs to update files in-place, given that APC would (or at least, should...) cache the PHP files so that the latency on s3fs's io wouldn't (or at least, shouldn't) be a concern. However, I'm not 100% confident that it would perform up to snuff, and if s3fs errors out, then the website goes offline, which my boss and I agree makes it a non-starter approach. I imagine that I could store all the files on the Chef server and do a recursive file pull, but I'm thinking Git is simpler and easier to manage. It'll also make tracking changes from our theme designer a bit easier, too, which is one of the reasons I thought about this in the first place.

Assuming that folks don't point out some idea why this is bad, is the a specific cookbook/LWRP that I should use when setting up this recipe? Also, is there an easy way for me to configure this so that the chef-client service wouldn't update on every git commit (like the theme dev pushing stuff up to us) but allow me to kick it off specifically via Knife? If not, then I'll just use different branches... the dev will push to a testing branch, and then I'll push from there to a release branch that Chef is pulling from. I just want to make sure that he doesn't make a tweak that suddenly appears on the site before it's ready.

Thanks for all the help!

--

~*~ StormeRider ~*~

"Every world needs its heroes [...] They inspire us to be better than we are. And they protect from the darkness that's just around the corner."

(from Smallville Season 6x1: "Zod")

On why I hate the phrase "that's so lame"... http://bit.ly/Ps3uSS