[chef] Re: Re: Re: Re: Re: Re: Re: Re: Re: Better workflow for actively-developed cookbooks

[Jamie Winsor < >]

Absolutely, I'm finishing up the documentation today which will outline this in a better format.

Scenario 1: developing an application

Imagine you are working on an application called "pvpnet". Pvpnet contains a Berksfile and a Vagrantfile. The Berskfile contains a source with a path location to an application cookbook on disk (:path => "~/code/pvpnet-cookbook"). You run the command "knife berks install --shims". It installs the dependencies of the "pvpnet-cookbook" to a directory on disk at "~/.berkshelf" (by default) and creates a directory at cookbooks/ containing links to the cookbooks you just installed, including the one defined by the :path source.

In vagrant you have a "cookbooks" entry in your cookbooks_path which points to the shims directory made by Berkshelf. Now you are able to startup Vagrant and Vagrant will always read the changes you are making to the "pvpnet-cookbook" and it will pick up the dependencies of the "pvpnet-cookbook" 

Scenario 2: developing a new cookbook

Imagine we aren't working on an application ourselves, but instead we're working on the "nginx" cookbook. The cookbook contains a Berksfile with an entry "metadata". This entry means to look at the current working directory to find a metadata file and use this directory as a cookbook. We also have a Vagrant file here pointing to "cookbooks" and, the same deal as the first time through, you run "knife berks install --shims" to create the cookbooks/ directory of shims. Now when you run vagrant provision you will be using the current contents of the cookbook you are developing.

Scenario 3: building your building your cookbooks

We use Jenkins at Riot, but this would go for any build server. Every cookbook lives in it's own repository. Every repository is hooked up to a Jenkins job that monitors changes. Every cookbook has a build script which does the following thing (you need to do this part). 

1. Runs a bootstrap (Installs dependent gems and software)

2. Using Berkshelf, gather the cookbook's dependencies (knife berks install)

3. Unit test the cookbook (if you have unit tests)

4. Lint test the cookbook with food critic

5. Bump the version number of the Cookbook

6. Upload the cookbook and it's dependencies to your Chef server for later consumption via "knife berks upload"

This will setup your Cookbook with continuous delivery model to allow you to rapidly deliver incremental updates of a Cookbook to your organization.

Scenario 4: building an application

Every piece of software you are building should have a very similar life cycle to what I just outlined in Scenario 3. The difference is that you have a Berksfile containing a source for your application's cookbook. You can then gather it's dependencies and figure out what version of the cookbook this application code should be deployed with. Then you can mark your built artifact with metadata saying you want it deployed with X version of X cookbook and lock your environment to deploy the app code with that cookbook and that artifact version.

I can elaborate on Scenario 4 more, but it's less about Berkshelf at that point and more a blog post about how to do continuous delivery with Chef.

-- 

Jamie Winsor

@resetexistence

https://github.com/reset

On Sunday, June 24, 2012 at 2:24 AM, Torben Knerr wrote:

I'm using librarian-chef and wouldn't know how else I should manage my cookbook dependencies. It is great for resolving / downloading dependencies from various sources (community site, git, local path) if you treat them as "binaries" and I'm pretty happy with it.

But I'm also experiencing some friction when it comes to actively developing on cookbooks which I want to upload to chef server during my test / devlopment cycle. With librarian-chef I see only a few, all not ideal, alternatives:

 - do the full  `git add . && git commit -am 'foo' && git push origin master && librarian-chef update` cycle as mentioned by Bryan

 - do active development using a `:path` dependency, then when you have changes do the `librarian-chef update` cycle. When you are done testing do `git add . && git commit -am 'foo' && git push origin master` and switch to a `:git` dependency

 - do the changes directly in the $CHEF_REPO/cooksbooks directory (dangerous: don't do `librarian-chef update / install` or you will lose everything!!!), and when you are done testing replay the changes to your original cookbook source (might be suitable for tiny changes only)  

@Jamie: can you give us a quick comparison on the differences of librarian-chef and berkshelf? would berkshelf better support the scenario above?

Cheers,

Torben

On Sat, Jun 23, 2012 at 12:13 PM, Jamie Winsor < " target="_blank"> > wrote:

That's the piece that I mentioned earlier today. We'll make a better announcement once we've got the proper documentation and examples in place.

-- 

Jamie Winsor

@resetexistence

https://github.com/reset

On Saturday, June 23, 2012 at 3:06 AM, AJ Christensen wrote:

ohai, just saw this:

https://github.com/RiotGames/berkshelf

On 23 June 2012 20:45, Bryan Berry < " target="_blank"> > wrote:

Jay,

I don't quite understand your point.  Are you are saying that Librarian-chef

isn't meant to meet the particular use case I have described and should be

looking for a separate tool?

On Fri, Jun 22, 2012 at 11:44 PM, Jay Feldblum < " target="_blank"> > wrote:

Bryan,

The various rigorous practices that developers have are often

well-supported by their tools.

The same practices are not as well-supported by the devops tools because

these tools are still being built and because the ideas and practices are

still coming across. Devops as a field is still under construction.

In the meantime: you mix and match, you pick your battles, and you do what

you have to do.

Cheers,

Jay

On Fri, Jun 22, 2012 at 5:28 PM, Bryan Berry < " target="_blank"> >

wrote:

alright, I have a pretty heterodox idea of how I would like to use

librarian-chef so that a team of infrastructure devs can work in sync

I am on a team of 3 infrastructure devs, i am the (relative) expert, the

other 2 guys are smart but n00bs

I want us to have one common Cheffile in our one common chef-repo

the cookbooks we develop independently each have their own git repo,

unfortunately, private ones for the most part

When I create a new git repository for application-foo, I want to add the

git repository link, branch name/tag/commit name, to Cheffile so that when

the other guys are working, they can easily pull in the cookbooks that I am

working on and vice versa

However, I dont want to do the `git add . && git commit -am 'foo' && git

push origin master && librarian-chef update`  dance when I am actively

working on a cookbook that is within an "active" cookbook.

I don't need librarian to resolve any dependencies for my active

cookbooks, I just want a common file w/ the list of all cookbooks we are

working on as a team and i want librarian to download them if they don't

exist already.

Perhaps this is a perversion of all things bundler but this is what I

want. It also would get much more complicated if we didn't have git repos w/

shared commit access.

Is this crazy or a good idea or both?