[chef] Re: Re: Re: Re: Re: More on Cookbook Design Patterns

[Tom Duffield < >]

Lamont is right that you want to avoid forking your base role (i.e. having a recipe inside your base cookbook for every single application you manage). Don't have base::tomcat, base::oracle, base::apache because tomcat, oracle and apache have a scope (and a cookbook) of their own. If oracle requires certain pre-requisites, then the oracle cookbook should ensure that those prereqs are met. Use acme_oracle::base, not base::oracle. 

Going back to our example, lets say your 'base::default' recipe is comprised of base::monitoring, base::chef-client and base::dns. However, the oracle server cannot use the base::dns recipe because it includes dns::hosts. The solution would be that the cookbook responsible for the oracle server would have a base recipe (acme_oracle::base) that would just include the recipes it needed from base cookbook (base::monitoring, base::chef-client)  instead of "include_recipe 'base::default'" and include whatever dns cookbooks it needed (dns::resolve.conf and dns::nsupdate) directly. So it would look something like this:

# acme_oracle::base

include_recipe 'base::monitoring'

include_recipe 'base::chef-client'

include_recipe 'dns::resolve.conf'

include_recipe 'dns::nsupdate'

All the other cookbooks can still get away with using "include_recipe 'base::default'" but you got what you needed without having to make one-off exceptions to your base cookbook.

As for your users cookbook, as you guessed, the same rule applies. Have acme_tomcat::users, not users::tomcat. 

You can (and should) also layer the standardization. If you have an Acme, Co way of installing Tomcat, create an acme_tomcat cookbook that wraps the tomcat community cookbook. Enforce your Acme, Co tomcat standards in acme_tomcat and have all your app cookbooks use acme_tomcat cookbook instead of community tomcat cookbook. You aren't changing/forking the tomcat community cookbook, just wrapping it and enforcing your standards. 

As an example, lets say you have a java app being deployed into tomcat called Widget. You would want an application cookbook called 'widget'  that depends on the acme_tomcat and java cookbooks. As an example, lets say you need the acme_tomcat cookbook because you have Acme, Co standards (users, security, etc) specific to Tomcat that you need to enforce, but you can use the community java cookbook because you have no such standards for Java. 

In your acme_tomcat cookbook you could have a recipes that handle Acme, Co specific tomcat user implementation (acme_tomcat::users), security (acme_tomcat::security), or monitoring (acme_tomcat::monitoring). 

• Your acme_tomcat::security recipe may be an include_recipe statement to your acme co security cookbook (acme_security::default) with one or two Acme, Co tomcat specific configs thrown in. 
• Your acme_tomcat::monitoring might have an include_recipe statement to your acme_monitoring cookbook and calls to the acme_monitoring LWRP which sets up Acme, Co-specific monitors. 

Users, security and monitoring don't have to have their own recipes in acme_tomcat though. Its perfectly fine to have a acme_tomcat::base that does all of those things in a single recipe. What is important is that you are enforcing a standard associated with particular scope (tomcat-specific users) in a cookbook that is dedicated for maintaing that scope (acme_tomcat). 

Don't be afraid to add layers of abstraction if you need them, but don't add abstraction just for the sake of abstraction. In our example above we didn't create an acme_java cookbook because we didn't need one. If the day comes where we do need an acme_java cookbook we can add one but don't add one just to have one. Its kind of like the Goldilocks principle for cookbook abstraction - you don't want too little or too much. 

Tom Duffield — Consulting Engineer

651.769.7497 –  " style="color:rgb(105,117,130)" target="_blank">  – my: Linkedin  Twitter

CHEF GETCHEF.COM

TM

getchef.com   Blog   Facebook   Twitter   Youtube

Meet me at #ChefConf 2014

On Thu, Apr 3, 2014 at 10:00 AM, Lamont Granquist < " target="_blank"> > wrote:

On Thu Apr  3 07:29:05 2014, Jay Perry wrote:

Thanks Tom.  So let me make sure I'm following you correctly about the
base cookbook.  It would look something like:

base cookbook
  recipes
    default.rb
    monitoring.rb
    chef-cleint.rb
    etc

One good thing about a monolithic base cookbook the way that you are doing it now is that it provides consistency across all your servers.  Then in emergencies you have standardization across your entire platform and there's no issues arising from dealing with different kinds of servers while you're time crunched trying to solve a problem.  By enforcing standardization you also ensure that you don't accidentally miss critical cookbooks from all your different proliferating 'base' roles.

When you're confronted with a problem like needing different base config for datacenters (EC2 vs internal openstack) that may affect networking and DNS config (one set of servers may be published to dyn, one not), the wrong way to solve that problem is to start forking your base roles/cookbooks.  IMO, that intelligence should get pushed into the networking or dns cookbooks and broken up into different recipes for the different use cases and controlled by attributes at a lower level -- then your base role still only asserts that everything includes the networking::default recipe -- but the conditionals should be moved into that cookbook, not expressed at the base level.  Then you can look at the cookbook and the default recipe and know that all the logic across your entire platform for how you configure your networking is expressed right there.

Chef is flexible enough so that you can start breaking up your base roles and only including pieces of it, as required, but then I think you're missing something important that comes from standardization.  From the perspective of the chef client its all managed well and you can follow the logic and eventually reason about your systems, but the beautiful simplicity of bludgeoning your entire infrastructure into looking absolutely identical to the highest extent possible may be lost.

I would wait until I see that I start really needing this kind of flexibility in what a base server looks like, rather than engineering it from the outset.  I'd almost consider this overflexibility a design flaw of Chef based on my experience.  I've used configuration management systems at a large scale which had hard-coded a single base role (in a 'language' that didn't support any conditionals) and doing that really forced standardization.  Chef will allow you to create a mess where you have different piles of servers that all have different 'base' roles that all look fundamentally different (probably good that it lets you do it, but its like Singletons and global variables -- you need to know the consequences of using them to understand that they should be lightly used, if at all).