[chef] Re: Re: Re: Re: Re: Environment Cookbook Patterns & Questions

[Matt Juszczak < >]

Thanks for the suggestion!

> On Jan 5, 2015, at 6:02 AM, Torben Knerr 
> <
 >
>  wrote:
> 
> My favorite approach is to use a "top-level cookbook". A top-level
> cookbook roughly represents a VM. It is essentially the same as an
> "environment cookbook" but uses metadata.rb for locking the dependency
> graph rather than environments.
> 
> This lets you use environments for representing your infrastructure
> environments rather than (ab?)using them for locking dependency
> graphs. The only cookbook dependency you have in the environment is
> the one to the top-level cookbook (all others come in transitively via
> metadata.rb).
> 
> Related discussions:
> * http://lists.opscode.com/sympa/arc/chef/2014-01/msg00419.html
> * http://lists.opscode.com/sympa/arc/chef/2014-06/msg00173.html
> 
> Examples:
> * https://github.com/tknerr/sample-toplevel-cookbook
> * https://gist.github.com/tknerr/4e3236d00ceba917abea
> 
> Word of caution: I'm using this with chef-solo and have not checked
> whether this way of keeping metadata.rb and Berksfile.lock in sync
> (https://gist.github.com/tknerr/4e3236d00ceba917abea) works with
> chef-client too. If not, you might want to us a generative approach
> instead.
> 
> HTH,
> Torben
> 
> On Mon, Jan 5, 2015 at 12:04 AM, Matt Juszczak 
> <
 >
>  wrote:
>> Agreed!! Thanks for the help! It's worth mentioning that "poise-appenv"
>> seems to try to solve this problem.
>> 
>> Matt
>> 
>> On Jan 4, 2015, at 7:08 AM, Christine Draper
>> <
 >
>>  wrote:
>> 
>> Matt,
>> 
>> Your analysis makes a lot of sense to me. We need a way to separate the
>> inherent variances between infrastructure environments as one dimension; 
>> and
>> the through-time release ("code") variances as another.
>> 
>> Christine
>> 
>> On Sat, Jan 3, 2015 at 10:09 PM, Matt Juszczak 
>> <
 >
>>  wrote:
>>> 
>>> Christine,
>>> 
>>>> It makes sense to me to create a base 'myface' cookbook that knows how
>>>> to set up the logical components of the application like database 
>>>> servers,
>>>> application servers, the actual application code etc.  I'd consider this 
>>>> to
>>>> be an application cookbook, one that you could then reuse in multiple
>>>> contexts (e.g. the different production data centers).  Perhaps it might
>>>> even be reused in different lifecycle phases (prod, staging, systest).
>>>> 
>>>> The above might become an environment cookbook if you use it to lock
>>>> down the cookbook versioning for actual deployments.
>>> 
>>> Gotcha. So in this case, the application cookbook and environment cookbook
>>> are *literally* the same thing and you don’t have BOTH. An application
>>> cookbook simply becomes an environment cookbook once a Berksfile.lock is
>>> checked in. Jamie mentions that application cookbooks and environment
>>> cookbooks don’t differ much, except to me it’s been unclear whether having
>>> an environment cookbook for an app replaces the functionality of the
>>> application cookbook all-together (simply by now having a Berksfile.lock 
>>> in
>>> revision control). I think I got it now.
>>> 
>>>> In your case it sounds like intend the east and west data centers to be
>>>> part of one overall production environment and want to keep them in sync
>>>> from a cookbook (and application software) version perspective. If so, 
>>>> the
>>>> environment cookbook could be 'myface-prod' and consist of little more 
>>>> than
>>>> a berksfile.lock plus attributes to customize the application cookbook
>>>> specifically for prod environment.
>>> 
>>> I’m thinking we would make our environment cookbooks *code environments*
>>> only. In other words, while a prod environment might have static 
>>> attributes
>>> that differ from a staging environment and will *always* differ, it seems
>>> the environment cookbook pattern that Jamie describes is about managing an
>>> SDLC. You apply version 1.0.1 of a cookbook to your staging environment, 
>>> and
>>> eventually that makes its way to prod. Therefore, to me, it makes sense to
>>> have a “myface” environment cookbook that can be versioned, and utilize 
>>> the
>>> berkflow tool to promote different versions of that cookbook through chef
>>> server environments.
>>> 
>>> But that still doesn’t solve the problem of *infrastructure environments*
>>> (IE: configuration that applies to servers in the staging environment but
>>> never the prod environment and vice versa). For example, if you deploy
>>> version 1.0.0 of myface to staging, you’re eventually going to deploy that
>>> to production. However, if you deploy changes to “staging environment
>>> configuration” (perhaps network addresses?), you likely aren’t going to 
>>> copy
>>> those configuration parameters to production any time soon. That
>>> configuration is strictly for the staging infrastructure environment!
>>> 
>>> So…
>>> 
>>>> The key question is how you can then add in the 'east' and 'west' data
>>>> center variances. Environment files for myface-prod-east and
>>>> myface-prod-west would seem a decent option, if the variances are fairly
>>>> simple attribute differences, and particularly if the variances were
>>>> relatively independent/orthogonal to the application (e.g. updating
>>>> networking information for one data center is relatively independent of
>>>> changes to the myface application). You'd likely want to version the
>>>> environment definitions externally in source control.  You may need to
>>>> coordinate cookbook changes with environment changes, in the case where 
>>>> an
>>>> application change requires networking changes in the two data centers.
>>> 
>>> Based on your feedback, and the more I think about this, the more I think
>>> it does make sense to create more environment files in chef server and
>>> maintain them in revision control. These environment files would contain
>>> both application (managed by berkflow) and infrastructure environment
>>> configuration options (such as network configuration). Therefore, we would
>>> likely create something like:
>>> 
>>> myface-prod-us-east
>>> myface-prod-us-west
>>> myface-staging-us-east
>>> myface-staging-us-west
>>> 
>>> and use berkflow to manage version pinning:
>>> 
>>> blo upgrade myface-prod-useast myface
>>> blo upgrade myface-prod-uswest myface
>>> 
>>> ...and simply build a wrapper tool to upgrade all “prod” environments at
>>> once that calls blo upgrade numerous times. Meanwhile, the environments
>>> above (myface-prod-useast) have specific configuration for servers in that
>>> environment, such as network configuration.
>>> 
>>> Thanks for your insight. I’m still curious to hear some other opinions but
>>> it’s good to know others have been confused on this and I’m not missing
>>> something simple.
>>> 
>>> -Matt
>> 
>>