[chef] Re: Re: Re: Environment Cookbook Patterns & Questions

[Matt Juszczak < >]

Agreed!! Thanks for the help! It's worth mentioning that "poise-appenv" seems to try to solve this problem. 

Matt

On Jan 4, 2015, at 7:08 AM, Christine Draper < "> > wrote:

Matt,

Your analysis makes a lot of sense to me. We need a way to separate the inherent variances between infrastructure environments as one dimension; and the through-time release ("code") variances as another. 

Christine

On Sat, Jan 3, 2015 at 10:09 PM, Matt Juszczak < " target="_blank"> > wrote:

Christine,

> It makes sense to me to create a base 'myface' cookbook that knows how to set up the logical components of the application like database servers, application servers, the actual application code etc.  I'd consider this to be an application cookbook, one that you could then reuse in multiple contexts (e.g. the different production data centers).  Perhaps it might even be reused in different lifecycle phases (prod, staging, systest).
>
> The above might become an environment cookbook if you use it to lock down the cookbook versioning for actual deployments.

Gotcha. So in this case, the application cookbook and environment cookbook are *literally* the same thing and you don’t have BOTH. An application cookbook simply becomes an environment cookbook once a Berksfile.lock is checked in. Jamie mentions that application cookbooks and environment cookbooks don’t differ much, except to me it’s been unclear whether having an environment cookbook for an app replaces the functionality of the application cookbook all-together (simply by now having a Berksfile.lock in revision control). I think I got it now.

> In your case it sounds like intend the east and west data centers to be part of one overall production environment and want to keep them in sync from a cookbook (and application software) version perspective. If so, the environment cookbook could be 'myface-prod' and consist of little more than a berksfile.lock plus attributes to customize the application cookbook specifically for prod environment.

I’m thinking we would make our environment cookbooks *code environments* only. In other words, while a prod environment might have static attributes that differ from a staging environment and will *always* differ, it seems the environment cookbook pattern that Jamie describes is about managing an SDLC. You apply version 1.0.1 of a cookbook to your staging environment, and eventually that makes its way to prod. Therefore, to me, it makes sense to have a “myface” environment cookbook that can be versioned, and utilize the berkflow tool to promote different versions of that cookbook through chef server environments.

But that still doesn’t solve the problem of *infrastructure environments* (IE: configuration that applies to servers in the staging environment but never the prod environment and vice versa). For example, if you deploy version 1.0.0 of myface to staging, you’re eventually going to deploy that to production. However, if you deploy changes to “staging environment configuration” (perhaps network addresses?), you likely aren’t going to copy those configuration parameters to production any time soon. That configuration is strictly for the staging infrastructure environment!

So…

> The key question is how you can then add in the 'east' and 'west' data center variances. Environment files for myface-prod-east and myface-prod-west would seem a decent option, if the variances are fairly simple attribute differences, and particularly if the variances were relatively independent/orthogonal to the application (e.g. updating networking information for one data center is relatively independent of changes to the myface application). You'd likely want to version the environment definitions externally in source control.  You may need to coordinate cookbook changes with environment changes, in the case where an application change requires networking changes in the two data centers.

Based on your feedback, and the more I think about this, the more I think it does make sense to create more environment files in chef server and maintain them in revision control. These environment files would contain both application (managed by berkflow) and infrastructure environment configuration options (such as network configuration). Therefore, we would likely create something like:

myface-prod-us-east
myface-prod-us-west
myface-staging-us-east
myface-staging-us-west

and use berkflow to manage version pinning:

blo upgrade myface-prod-useast myface
blo upgrade myface-prod-uswest myface

...and simply build a wrapper tool to upgrade all “prod” environments at once that calls blo upgrade numerous times. Meanwhile, the environments above (myface-prod-useast) have specific configuration for servers in that environment, such as network configuration.

Thanks for your insight. I’m still curious to hear some other opinions but it’s good to know others have been confused on this and I’m not missing something simple.

-Matt