I’ve several reasons to want chefdk on individual hosts:
· Chef servers are, from painful personal experience, bulky, expensive to host, and destabilizing to dynamic, mixed environments. A single mistaken update to a data bag for a central service, or a simple cookbook update, can take down an entire environment by screwing up every host. Particular examples include:
o Typos in ‘bind’ configuration files, for which there used to be no sanity checking and would break the ‘named’ service on restart.
o The refactoring and non-backwards-compatible update to yum, splitting off “yum::epel” recipe to a separate “yum-epel” cookbook and breaking all the old cookbooks that relied on yum::epel.
o The refactoring and non-backwards-compatible update to mysql, which went from a single MySQL instance configuration tool to an LWRP for multiple instances, and which deleted /etc/my.cnf.
· Using chef-solo on individual hosts makes them much safer to modify and update.
· chefdk is prefereable to the “chef-client” tool because already contains precompiled copies of the libgecode software, and I can pre-deploy it in a managed fashion to my chef-solo environments to manage their cookbooks individually.
o I consider the components necessary to build libgecode to be inappropriate to install on lightweight, exposed servers of any sort. (gcc and g++, in particular, are begging for trouble to install on a production server.) chefdk is therefore, much, much safer than building libgecode.
· I’d use librarian-chef instead of Berkshelf, but librarian-chef has a nasty tendency to update every depencency for your designated cookbook is unacceptable for daily use. This is unsafe, precisely because of the unexpected ‘yum’ and ‘mysql’ udpates I mentioned above.
· If you have two sysadmins developing changes on the same cookbook pushed to the same chef server, you’re going to have conflicts when you both upload modified cookbooks with the same number. There is no way around this that fits the standard numbering schemes for chef: one of you can take a *much lower* build number and keep it out of the way of the other active developer, but that’s likely to cause confusion
So, I use chef-solo for small environments, I use Berkshelf to manipulate localized chef cookbook selection on each host, and I use chefdk to get Berkshelf installed consistently.
Now, if I could find and shave the yak responsible for making libgecode a requirement for Berrkshelf, I’d do so. But that code is a bit out of my personal reach to rewrite or publish fixes for.
Nico Kadel-Garcia
Lead DevOps Engineer
">
From: Peter Burkholder [mailto:
Sent: Wednesday, March 25, 2015 10:12 PM
To:
Subject: [chef] Re: Re: RE: Re: RE: RE: RE: Re: RE: Re: No chefdk RPM available for RHEL 5?
Hi Nico,
I hope you're not shaving the wrong yak here. ChefDK and Berkshelf are generally only needed on development systems, not on the chef-client systems. If you have RHEL6 or Ubuntu or Windows or OsX then you should be fine developing cookbooks for RHEL5 clients. Sorry if I'm missing something obvious, but are you sure you need ChefDK or Berkshelf for RHEL5, as opposed to just the chef-client?
In an earlier message you mentioned, "I’ll have to use the standard ‘chef’ client, which means that to use Berkshelf on them I’ll have to compile Berkshelf," but Berkshelf is only needed for dependency management on the development side, not on the client end of things.
On Wed, Mar 25, 2015 at 8:54 PM Daniel DeLeo <
">
> wrote:
On Wednesday, March 25, 2015 at 3:12 PM, Nico Kadel-Garcia wrote:
> The reluctance for the chefdk develers to directly support CentOS 5 is understandable, the ruby on it is quite out of date and insufficient for the current build tools.
> I've activated a "ruby-1.9.3" cookbook on a CentOS 5 VM, and encountered adventures trying to use the bundle tool, but the CentOS 5 build chefdk RPM seems to work in my casual testing. I do have some comments.
You can use the built-in test kitchen stuff to do this, or you can reverse engineer it to get the cookbooks it uses and run them.
>
> * Using "bundle" to build up the chefdk toolkit the first time runs into exactly the problems the basic "chef" and "chef-server" packages have building Berkshelf. It requires extensive compilation tools, and a *LOT* of RAM or swap, to successfully build the "dep-selector-gem" requirement. I wound up having to add several gigabytes of swap space just to allow it to complete.
The built-in kitchen stuff again handles this, or you can crib from it to size your build machines: https://github.com/chef/omnibus-chef/blob/master/.kitchen.yml
>
> * The massive resource consumption of libgecode compilation is exacerbated by the hardcoded "make -j5", which I personally consider a premature optimization. The default should, ideally, be "make -j1" which complex tests or manual settings can optimize for powerful environments: being slow on instances that could compile faster is much safer than breaking on lightweight instances. I've actually commented about this before from a different workplace, and it's one of my compelling reasons to want to simply install chefdk in the first place..
It’s definitely not premature, we have to build it as part of the build process for ChefDK and Chef Server; getting the build down to a few minutes saves a ton of time. I’ve sketched out how to use ohai to make the build auto-detect lower resource machines, I just need someone to try it and ensure the build works. You’ve probably seen my comment about it in the pull request discussions.
>
> * The "bundle exec omnibus build chefdk" command uses the "Packager::RPM" tool to build the RPM on CentOS 5. Unfortunately, that tool builds only RPM's, not SRPM's. I wish I understood the tool well enough to fix that, but I don't today. That gets into security and source code management for RPM's: a build environment you construct from git, or anything else on the fly, does not necessarily bear a strong resemblance to the code used to build an earlier or later RPM, especially if the .spec file or any patches are generated dynamically. And sure enough, in this case, the .spec file is generated dynamically. So to analyze any '%post' or RPM macro controlled settings, you have to disassemble the whole RPM::Packager toolkit rather than having an SRPM to read.
It’s all in omnibus, which saves us a ton of time by standardizing our build across platforms, as much as is possible anyway. The package scripts are here if you want to look at them: https://github.com/chef/omnibus-chef/tree/master/package-scripts/chefdk I realize this means that we’re breaking some of the platform conventions you’re used to, but without it we couldn’t support making packages for as many platforms as we do at all. You’re of course welcome to mirror all of it and build the packages yourself if you need to, and we’ll try to help if you run into any issues with that.
>
> Overall, this looks like it's actually in reach for CentOS or RHEL 5: the key is the lack of a recent enough ruby in CentOS 5 itself to build the packages.
>
> Nico Kadel-Garcia
> Lead DevOps Engineer
>
" target="_blank">
(mailto:
" target="_blank">
)
--
Daniel DeLeo