[chef] RE: Re: Re: Re: conflict between using environments to manage cookbook releases and to manage application environments

[RUSSELL Scott < >]

Hi,
         We are using environments in a way that more matches our business, 
and how we currently think about things.    So we currently have 60+ 
environments.  

These take the form of:-

<project>_prod
<project>_test
<project>_dev

Where <project>, may equal, SAP, or Documentum etc....  

        Each of the projects may/may not have a prod, test or dev 
environment, though strictly speaking that should be a <project>_prod, 
<project>_test, or <project>_dev environment.  This does nice things like 
orders your environments by project, and for each project it is easy to see 
if you have a prod environment or just dev etc. Also you have a nice drop 
down in the chef console, such that you can select your environment and only 
browse the nodes in that environment.  This is a nice easy win for us.  Makes 
the chef web console actually useful, as we use it "read only".

       We plan to use environments in roles, but to be honest, so far we have 
not done so, execept in a few cases.  It is a work in progress, but for now, 
it gives us a nice easy business view of the environments that make sense.  
Of course, we have to handle issues like separation of concerns and SOX 
compliance, so this is a precursor to doing this type of activity.


Almost forgot.    - The above all refers to our "visible" network.    For 
development of cookbooks we use the following process:-

1. each sysadmin has vmware workstation, locally with 16Gb ram on workstation.
2. each sysadmin has their own chef server, chef developer, and a redhat 
instance, all running in separate vm's inside local vmware workstation.
3.  All testing is done here.  Builds, etc.   Allows snapshots, upgrades, 
trial/error, do whatever you like, it is yours and only yours.#
4. We have a vmware domain, where we deploy "new" builds as a final check, 
this onto vmware ESX server.
5. Deploy the new code to the "prodcution" chef server(ie the one at the 
start of the email).

               This keeps development separate from the prod/dev/test 
environments.    A bit of a different viewpoint from most.


                               Hope this helps.    We can expand on naming 
conventions in another thread, if that helps anyone, and maybe cover how we 
maintain install logs if that helps anyone.


                                                              Sc0tt...

 






________________________________________
From: Bryan Berry 

 
Sent: 03 May 2012 08:19
To: 

 
Subject: [chef] Re: Re: Re: conflict between using environments to manage 
cookbook releases and to manage application environments

gotta say I really like Peter's solution, will have to look at it in more 
detail

To answer Adam's, question I use manual upload for my cookbooks

The big cause of the instability in my cookbooks is that I am essentially 
learning these applications at the same time that I write cookbooks for them, 
not the ideal situation. I am also under a lot of pressure to set up a quite 
complicated  architecture in a very short period of time. Consequently, many 
of my initial configuration choices are less than ideal. Going back and 
fixing those choices w/out breaking running apps is a challenge. The obvious 
answer here is to implement automated cookbook testing w/ minitest/cucumber 
and toft/vagrant. Once my cookbooks are more stable and tested, i should be 
able to move many of those attributes that I have in data_bags over to 
environments. The only drawback is that I will have a long duplicated list of 
cookbook version constraints in each environment.

On Wed, May 2, 2012 at 2:02 AM, AJ Christensen 
<
 <mailto:
 >>
 wrote:
Peter,

That's an awesome assertion to have as part of your run_list -- may be
stealing this one for ourselves!

--AJ

On 2 May 2012 11:43, Peter Donald 
<
 <mailto:
 >>
 wrote:
> Hi,
>
> We have a very similar requirements here but the way we represent it
> in chef is with different environments. We do not allow the _default
> environment and instead have the following set
>
> * production
> * staging (still needed as some of the infrastructure is still part of
> a manual non-chef managed release process)
> * uat
> * development
> * ci (aka integration tests)
> * desktop (i.e our desktop managed machines)
> * chef_dev (i.e. when developing cookbooks)
>
> For each of these environments we have flags indicating the
> datacenter/location of the environment and the equivalent of a
> "stable" flag. If the "stable" flag is set then we require that the
> environment explicitly lists a version of a cookbook and that the
> cookbook version is frozen. This workflow is enforced by a recipe that
> included at the start of every chef run. You can see a snippet that
> illustrates the idea at [1].
>
> [1] https://gist.github.com/2048310
>
> On Tue, May 1, 2012 at 6:47 PM, Bryan Berry 
> <
 <mailto:
 >>
>  wrote:
>> Dear Chefs,
>>
>> I am flummoxed by my conflicting needs for Chef environments. I currently
>> have 2 environments:
>>
>> _default
>> QA:   cookbooks that I am testing,  I know exactly which nodes are in here
>> and they can tolerate a service restart or two. No important nodes belong 
>> to
>> this environment
>>
>> PROD: Only stable, tested cookbook versions here, the vast majority of 
>> nodes
>> belong to this environment
>>
>> on the other hand, my organization has 4, count 'em, 4
>> application_environments
>>
>> PROD - production
>> QA   - user-facing tests
>> TS    - integration tests
>> DV    - development servers
>>
>> All of these servers should be using stable, tested cookbooks as my devs
>> don't want to be disrupted by a misbehaving cookbook. Thus all of them
>> belong to PROD chef_environment, besides the couple that I am using to test
>> cookbooks at a given moment.
>>
>> I currently manage application_environments with data bags that look like
>> this
>>
>> apps/esb.json
>> {
>>    "pr": {
>>           "database_schema": "production_schema"
>>      },
>>      "qa": {
>>           "database_schema": "qa_schema"
>>      }
>>     . . . .
>> }
>>
>> additionally, I add a top-level attribute to each node "app_env"  that
>> indicates which application_environment a node belongs to
>>
>> Some cookbooks, such as the haproxy cookbook, treat chef_environments like
>> application_environments
>>
>> I would love to treat chef_environments like application_environments but
>> that would completely break my workflow. How are others dealing w/ the
>> issues I have brought up here?
>>
>>
>
>
>
> --
> Cheers,
>
> Peter Donald