[chef] Re: Re: Nexus for Deployment

["steve ." < >]

Here's an actual use pattern that has worked well so far for a number of teams within our organization:

Builds get uploaded to Nexus (or Artifactory, zoom out enough and we're using both), then the CI server updates the application's development-environment data bag so that the next deployed version to the dev instance will be the version that was just uploaded.  Then it triggers a Chef run via RunDeck and waits for a successful response.  Then it queries the application for its version.  If the returned version matches the version that was deployed, deployment is successful.

Otherwise, email everybody that the deployment failed!

This works especially well if your CI environment can allow only certain users/groups to trigger certain build stages.  You could repeat the "update-data-bag/trigger-Chef-run/validate-environment" stage for non-development environments and assign privileges only to the right people in your organization.

"Query for version" is the tip of the iceberg for me, I want to stuff a bunch of acceptance tests in there, time permitting.

In case it's not obvious, the cookbook is written so that it's pulling a file from Nexus (or Artifactory), but it's pulling the search criteria from the data bag/data bag item that matches its chef_environment.  The CI deployment process modifies the search criteria in the data bag to point to a different artifact.  The cookbook doesn't need to be modified with each new release unless the new version of the application requires new configs or other deployment/operational behavior.

Once we refactor our internal cookbook testing framework to use Librarian, I'd modify this process so that any changes pushed to the application source repository *or* the application cookbook source repository trigger a new deployment.

I'm not calling this Continuous Delivery yet but that's definitely what we're building towards...

On Fri, Dec 7, 2012 at 7:59 AM, Peter Norton < " target="_blank"> > wrote:

If you're using a chef server, it will contain the recipes so you do not want or need the maven/nexus artifact unless you're going to use it to further automate something like environment creating/updating.

If you're using chef-solo, you can use the infrastructure zip since you need to copy your cookbooks to your target system and this is a fine way to do it.

There are a lot of details left to be done in either of the above cases, like how to ensure, with chef server, that you're using the version(s) of cookbooks that you intend to.

Once most of the code design is complete, the recipe shouldn't have to change to support newer versions.  E.g. once you know what your conf file will look like you should be able to deploy your app, make config changes via node attributes or role attributes, etc. without having to change the recipes, while you fix bugs and add features.  Once you hit this point of relative stability (and it'll happen pretty early on - you should only *have* to change a cookbook once in a great while, compared to app code changes) you really won't want to maintain the cookbook together with the code.  This is because after that point you will not know what has changed in the cookbook, you'll just be releasing cookbooks that have version number bumps but no code changes so the constant versioning will make it harder to understand when a change has actually been made to your infrastructure.

-Peter

On Fri, Dec 7, 2012 at 10:39 AM, Nayan Hajratwala < " target="_blank"> > wrote:

Chef Newbie here ... please be gentle.

The team I'm working with is planning to use Chef for deployment, but our approach seems to have some issues and I think it might be partly a conceptual misunderstanding.

The project is Java with artifacts being created by Maven and deployed into Nexus. Each deployed artifact is uniquely identified by an SCM revision number (101 in the example below). No SNAPSHOT builds.

My current thinking is that we create a new Maven module in our existing project that contains all the needed Chef recipes. This way, the correct infrastructure configuration will always be tied to the correct code.

The Maven artifacts would be deployed by our build server to Nexus such as:
  * web-client-1.5.101.war      <-- for web server
  * batch-client-1.5.101.jar    <-- for batch server
  * infrastructure-1.5.101.zip  <-- chef recipes

Now here's where I get confused.

  * Should an external Chef process (chef server?) be responsible for picking up a specific version of infrastructure.zip and running the recipes in it on the target machines?
  * Should the recipes be stored apart from the code? But if so, how does it hang together with the deployable artifacts?
  * Am i just thinking of this all wrong?

Thanks!

---
Nayan Hajratwala - 734.658.6032 - http://agileshrugged.com - @nhajratw