[chef] Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Better workflow for actively-developed cookbooks

[Hedge Hog < >]

On Mon, Jun 25, 2012 at 5:49 AM, Jamie Winsor 
<
 >
 wrote:
> Absolutely, I'm finishing up the documentation today which will outline this
> in a better format.
>

What Jamie Winsor outlines is very interesting and Berkshelf seems to
address a common use case.

I do wonder if this is not Chef-server centric.
Having setup Chef-server locally, via vagrant, in order to pursue
continuous deployment, I have since moved to use Chef-solo.
In this respect I think Librarian-Chef is still where-it-is-at wrt
cookbook deployment.

As Jay Feldblum notes, devops is still in a state of flux.
I've discussed some issues and possibilities with Jay in terms of the
production deployment of cookbooks, where they are obtained from and
how (protocol/conventions).
I can't say, from memory, that he has agreed, but the ability to
remove '.git' is nice and hopefully over time Librarian-Chef will
fulfill its potential.

As Jay notes the devops area is still very much decorated with
wet-paint notices (witness the opscode repo reorganization), and
developing/patching cookbooks is a universal need.
However, my view is that getting the production deployment nailed down
is the priority.
After that it will be possible to determine how best to handle
development/patched cookbooks.

For what it is worth right now I work on development cookbooks in a
separate local clone of my remote fork.
Push changes to my remote fork, and reference the remote fork in the Cheffile.

I think this is what is described by Bryan.
This is not the most convenient, but it has the substantial benefit
that my development environment, in terms of cookbooks/Cheffile, is
100% what my production environment is.
This is the reason I don't use :path in my Cheffile.

My 2c.


> Scenario 1: developing an application
>
> Imagine you are working on an application called "pvpnet". Pvpnet contains a
> Berksfile and a Vagrantfile. The Berskfile contains a source with a path
> location to an application cookbook on disk (:path =>
> "~/code/pvpnet-cookbook"). You run the command "knife berks install
> --shims". It installs the dependencies of the "pvpnet-cookbook" to a
> directory on disk at "~/.berkshelf" (by default) and creates a directory at
> cookbooks/ containing links to the cookbooks you just installed, including
> the one defined by the :path source.
>
> In vagrant you have a "cookbooks" entry in your cookbooks_path which points
> to the shims directory made by Berkshelf. Now you are able to startup
> Vagrant and Vagrant will always read the changes you are making to the
> "pvpnet-cookbook" and it will pick up the dependencies of the
> "pvpnet-cookbook"
>
> Scenario 2: developing a new cookbook
>
> Imagine we aren't working on an application ourselves, but instead we're
> working on the "nginx" cookbook. The cookbook contains a Berksfile with an
> entry "metadata". This entry means to look at the current working directory
> to find a metadata file and use this directory as a cookbook. We also have a
> Vagrant file here pointing to "cookbooks" and, the same deal as the first
> time through, you run "knife berks install --shims" to create the cookbooks/
> directory of shims. Now when you run vagrant provision you will be using the
> current contents of the cookbook you are developing.
>
> Scenario 3: building your building your cookbooks
>
> We use Jenkins at Riot, but this would go for any build server. Every
> cookbook lives in it's own repository. Every repository is hooked up to a
> Jenkins job that monitors changes. Every cookbook has a build script which
> does the following thing (you need to do this part).
>
> 1. Runs a bootstrap (Installs dependent gems and software)
> 2. Using Berkshelf, gather the cookbook's dependencies (knife berks install)
> 3. Unit test the cookbook (if you have unit tests)
> 4. Lint test the cookbook with food critic
> 5. Bump the version number of the Cookbook
> 6. Upload the cookbook and it's dependencies to your Chef server for later
> consumption via "knife berks upload"
>
> This will setup your Cookbook with continuous delivery model to allow you to
> rapidly deliver incremental updates of a Cookbook to your organization.
>
> Scenario 4: building an application
>
> Every piece of software you are building should have a very similar life
> cycle to what I just outlined in Scenario 3. The difference is that you have
> a Berksfile containing a source for your application's cookbook. You can
> then gather it's dependencies and figure out what version of the cookbook
> this application code should be deployed with. Then you can mark your built
> artifact with metadata saying you want it deployed with X version of X
> cookbook and lock your environment to deploy the app code with that cookbook
> and that artifact version.
>
> I can elaborate on Scenario 4 more, but it's less about Berkshelf at that
> point and more a blog post about how to do continuous delivery with Chef.
>
> --
> Jamie Winsor
> @resetexistence
> https://github.com/reset
>
> On Sunday, June 24, 2012 at 2:24 AM, Torben Knerr wrote:
>
>
> I'm using librarian-chef and wouldn't know how else I should manage my
> cookbook dependencies. It is great for resolving / downloading dependencies
> from various sources (community site, git, local path) if you treat them as
> "binaries" and I'm pretty happy with it.
>
> But I'm also experiencing some friction when it comes to actively developing
> on cookbooks which I want to upload to chef server during my test /
> devlopment cycle. With librarian-chef I see only a few, all not ideal,
> alternatives:
>  - do the full  `git add . && git commit -am 'foo' && git push origin master
> && librarian-chef update` cycle as mentioned by Bryan
>  - do active development using a `:path` dependency, then when you have
> changes do the `librarian-chef update` cycle. When you are done testing
> do `git add . && git commit -am 'foo' && git push origin master` and switch
> to a `:git` dependency
>  - do the changes directly in the $CHEF_REPO/cooksbooks directory
> (dangerous: don't do `librarian-chef update / install` or you will lose
> everything!!!), and when you are done testing replay the changes to your
> original cookbook source (might be suitable for tiny changes only)
>
> @Jamie: can you give us a quick comparison on the differences of
> librarian-chef and berkshelf? would berkshelf better support the scenario
> above?
>
> Cheers,
> Torben
>
>
> On Sat, Jun 23, 2012 at 12:13 PM, Jamie Winsor 
> <
 >
> wrote:
>
> That's the piece that I mentioned earlier today. We'll make a better
> announcement once we've got the proper documentation and examples in place.
>
> --
> Jamie Winsor
> @resetexistence
> https://github.com/reset
>
> On Saturday, June 23, 2012 at 3:06 AM, AJ Christensen wrote:
>
> ohai, just saw this:
>
> https://github.com/RiotGames/berkshelf
>
> On 23 June 2012 20:45, Bryan Berry 
> <
 >
>  wrote:
>
> Jay,
>
> I don't quite understand your point.  Are you are saying that Librarian-chef
> isn't meant to meet the particular use case I have described and should be
> looking for a separate tool?
>
>
> On Fri, Jun 22, 2012 at 11:44 PM, Jay Feldblum 
> <
 >
>  wrote:
>
>
> Bryan,
>
> The various rigorous practices that developers have are often
> well-supported by their tools.
>
> The same practices are not as well-supported by the devops tools because
> these tools are still being built and because the ideas and practices are
> still coming across. Devops as a field is still under construction.
>
> In the meantime: you mix and match, you pick your battles, and you do what
> you have to do.
>
> Cheers,
> Jay
>
>
> On Fri, Jun 22, 2012 at 5:28 PM, Bryan Berry 
> <
 >
> wrote:
>
>
> alright, I have a pretty heterodox idea of how I would like to use
> librarian-chef so that a team of infrastructure devs can work in sync
>
> I am on a team of 3 infrastructure devs, i am the (relative) expert, the
> other 2 guys are smart but n00bs
>
> I want us to have one common Cheffile in our one common chef-repo
>
> the cookbooks we develop independently each have their own git repo,
> unfortunately, private ones for the most part
>
> When I create a new git repository for application-foo, I want to add the
> git repository link, branch name/tag/commit name, to Cheffile so that when
> the other guys are working, they can easily pull in the cookbooks that I am
> working on and vice versa
>
> However, I dont want to do the `git add . && git commit -am 'foo' && git
> push origin master && librarian-chef update`  dance when I am actively
> working on a cookbook that is within an "active" cookbook.
>
> I don't need librarian to resolve any dependencies for my active
> cookbooks, I just want a common file w/ the list of all cookbooks we are
> working on as a team and i want librarian to download them if they don't
> exist already.
>
> Perhaps this is a perversion of all things bundler but this is what I
> want. It also would get much more complicated if we didn't have git repos w/
> shared commit access.
>
> Is this crazy or a good idea or both?
>
>
>
>



-- 
πόλλ' οἶδ ἀλώπηξ, ἀλλ' ἐχῖνος ἓν μέγα
[The fox knows many things, but the hedgehog knows one big thing.]
  Archilochus, Greek poet (c. 680 BC – c. 645 BC)
http://hedgehogshiatus.com