[chef] Re: Re: Re: On Chef patterns - ways to organize a Chef kitchen

[Mat Schaffer < >]

The whole roles vs cookbooks thing is an ongoing discussion to be sure. But last I checked roles don't have any sort of versioning support which can them a bit of a liability when you start sharing them between projects or apps. It was covered pretty well on episode 36 of the foodfight show so I'll point you there: http://foodfightshow.org/2013/01/roles.html

The win for really using chef is partly in the "scaling out" but it also helps when you need to replicate your current stack. For example, imagine that DO machine were somehow lost and you needed to start over again. Any manual changes or updates would now have to get redone but if you've been managing with chef it should be easy to recreate everything again.

Personally, if I needed a basic Rails PaaS, I'd go with heroku. The run-your-own PaaS is interesting, but I'm not sure it's worth the overhead for personal use. I could see doing that if you needed a place for a dev team to run internal tooling. Mesos is also interesting there but I haven't really tried it yet.

As for mixing the kitchen with the project, I don't see any issue with that. Might be a little messy in terms of directory structure though. Another option would be to have a "deployment" or "chef" directory and put everything in there. I've done that in the past too.

-Mat

about.me/matschaffer

On Sun, Feb 2, 2014 at 4:06 PM, Marcelo Serpa < " target="_blank"> > wrote:

Hi Mat,

Thank you for taking the time to answer. I appreciate it.

This also helps avoid loading too much info into roles which is another good practice.

Interesting… could you elaborate on this one? An example would be nice :)

Finally, you should try to get into the practice of using chef to provision as well as manage the servers since it's where configuration "management" really starts to shine.

Excellent point. Now I see that the main reason why I haven’t been using a kitchen per project, is that I’ve been using Chef only to provision servers (hence, we could call that a provision kitchen ;), and not manage them. For me, so far, it was the most painful SCM aspect. Since my apps’ don’t really change a lot when it comes to infrastructure, it has been enough to do those changes by hand. So far, it has been enough to scale vertically (basically expanding the RAM of a single node), which I can do easily on digitalocean (or AWS or any cloud VM provider ).

By the way, how would this work with knife solo? Or do you think that when the infrastructure becomes complex to the point where Chef is useful to manage it, then it’s time to add a Chef server to the equation? 

Also, I’ve been playing with PaaS solutions lately. One of them is Tsuru, written in Go, and Dokku, which is pretty easy to setup on digital ocean. Perhaps the best combo for apps that don’t require a very complex infrastructure, would be to use Chef to provision, say, Dokku and also setup the necessary plugins (for db access, for example), and then just use Dokku from then on. I think this could work for simpler apps, but as soon as you need more server side components, then it becomes a burden to deal with PaaS plugins. What do you think about this?

Lastly, do you think it would be feasible to have the kitchen integrated in the app project itself? Let’s say I have an app built on Rails:

$ rails new test

Then, create the kitchen *inside* the app’s dir"

$ cd test

$ knife solo init . 

Ending up with:

➜   ls

Gemfile        app            config.ru      db             log            roles          tmp

README.rdoc    bin            cookbooks      environments   nodes          site-cookbooks vendor

Rakefile       config         data_bags      lib            public         test

So, we have the app, *and* the code that does all the SCM (potentially adding Capistrano to the mix).

—

Marcelo

On Sunday, February 2, 2014 at 5:13 PM, Mat Schaffer wrote:

"It depends" ;)

When I was doing client work I generally kept one kitchen per client just to make the final delivery as clean as possible. For myself I tend to keep kitchens logically separate by project.

If you need to share stuff between kitchens I recommend pushing all that stuff into cookbooks so that you can use berkshelf or librarian-chef to load in the shared cookbooks from their own dedicated git repositories. This also helps avoid loading too much info into roles which is another good practice.

Finally, you should try to get into the practice of using chef to provision as well as manage the servers since it's where configuration "management" really starts to shine. Of course there are other options but if you're on a lamp stack, generating configuration files and/or data from chef will probably be the easiest to set up.

Hope that helps some. I think this is one of those things that varies a bit from project to project.

Feel free to open a knife-solo issue if you run into trouble there.

-Mat

about.me/matschaffer

On Sun, Feb 2, 2014 at 12:00 AM, Marcelo Serpa < " target="_blank"> > wrote:

Hi list,

I’m wondering Chef kitchen patterns. Right now, I use knife-solo, and I have a general-purpose kitchen. I add nodes as needed (not necessarily for the same app). Since I don’t really deal with a project that requires a more than one (or two) server(s), yet, it’s easy to just do something like:

➜  mykitchenv2 git:(master) ✗ ls

Berksfile      Berksfile.lock Gemfile        Gemfile.lock   Vagrantfile    cookbooks      data_bags      environments   nodes          roles          site-cookbooks

➜  roles git:(master) ✗ ls

lamp.json      rack.json

➜  nodes git:(master) ✗ ls

127.0.0.1.json       anotherhost.json …

So, it’s basically a general-purpose kitchen. It’s not really geared towards a specific application. My workflow looks like this:

1) New project, hack, app ready to be deployed

2) Create new VM @ digital ocean, grab the IP, create a new node json in the kitchen

4) It’s probably a Rails app. Add the rack role[lamp] to the run_list for this node. The rack role basically setups a new rack box using the rackbox cookbook (https://github.com/teohm/rackbox-cookbook)

3) knife solo prepare

4) knife solo cook

5) Use capistrano to deploy the app.

Then, I do the same thing if I need a staging server.

It works pretty well. However, most of the apps I develop right now are apps with a quite simple infrastructure. They don’t require more than 2 servers (prod,staging). I did work with more elaborated apps before with up to 10 different servers (but without using Chef at the time, go figure). 

The problem with this approach is that I end up having several unrelated nodes, and in the end, I don’t really describe the infrastructure of a single app/system, but of several ones, which can become clunky. Basically, for now, I’m just using Chef to get the sever ready, but not really using it to *manage* the server afterwards.

That being said, I’d like some insights on Chef patterns related to this, for example,  when is it time to create a kitchen solely for an app? 

Thanks in advance,

-- 

Marcelo