[chef] Re: Re: Re: Re: Re: An deep topic

[Lamont Granquist < >]

On 7/8/14, 11:16 AM, John E. Vincent (lusis) wrote:
> So that post from Lamont is awesome but it kinda points to the thing I 
> was bringing up. This section right here:
>
> "This is the correct way to start building Chef Cookbooks"
>
> And then
>
> "Not that we're not using Test Kitchen to actually test our cookbook."
>
> Wait what? Then why is it "Test" Kitchen?
>
> When I asked all the `why` questions, this is a good example. Why 
> shouldn't I use the documented `knife cookbook create`? Is it not 
> acceptable to the job of creating a cookbook? What makes this more 
> confusing is that this is coming from someone who works for Chef.

The "knife cookbook create" command was probably written 5 years ago.  
ChefDK started being written a few months ago and the documentation 
clearly hasn't caught up.  As you like to point out, ChefDK is currently 
only an 0.1.0 version.  If you'd like a clear statement I think its 
fairly obvious that "chef generate cookbook" will become the way to do 
this and will entirely replace "knife cookbook create", "berks 
cookbook", and the 
knife-plugin-that-does-the-same-thing-with-the-funny-name-that-i-cant-think-of-right-now. 
We will not go out of our way to break the other commands, but we're 
putting our effort into ChefDK and 'chef generate' and there's 3 or 4 
software devs working on it right now, where 'knife cookbook create' is 
going to have to largely user driven for fixes and 'berks cookbook' was 
trying to solve this problem while being bolted onto the side of a 
dependency solver.  We're set up much better to be successful at 
producing a good cookbook skeleton generator, so its very likely that 
'chef generate cookbook' becomes the dominant way to solve that problem 
in the future.

And the problem here is that we have to balance being able to throw our 
effort at solving a problem against being 'arrogant'.  We do control the 
learnchef documentation and are the first place people go to and we have 
a lot of software devs working for us and can throw human resources at 
coding problems, which should set us up to be successful at tackling 
problems that we focus on.  At the same time we don't want to state that 
'berks cookbook is shit, don't use it' or something which would really 
be horribly arrogant.  We also recognize that we don't always do the 
right thing.  If 'chef generate cookbook' turns out to be some overly 
engineered terribleness that is difficult to use then we don't want to 
preclude someone in the community with a better way of doing things to 
go do it and show us a bitter/simpler/easier/cleaner/whatever way.  
That's largely what happened with test-kitchen and berkshelf, which were 
projects spun up entirely in the community.

And right now we problem can't say to replace whatever cookbook 
generators you're using with 'chef generate cookbook'.  It still is only 
a 0.1.0 version so it almost certainly doesn't cover all the use cases, 
and may not be "production ready".  In fact, I documented in that write 
up a major bug making it not even useful in that trivial case without 
hand editing your .kitchen.yml to fix something.  So we can't state that 
everyone should use it, because clearly everyone should not use it right 
now.  But I wrote that HOWTO up because we're getting there to the point 
where its useful in the simplest case to bootstrap someone from zero 
into writing a simple cookbook -- which is the first baby step.

And the problem with 'test-kitchen' is just in naming the thing.  In 
order to solve the TDD problem, test-kitchen had to solve the problem of 
spinning up and destroying virts, installing chef on the virts, using 
berkshelf to sync the cookbook under test and all its dependencies, 
supporting vagrant and various other clouds -- in addition to the goal 
of running tests.  Well, turns out that you can just not run the tests 
in test-kitchen and what you get is a nice facade where a single command 
can take the cookbook that you are developing and fire up a virt, 
install omnibus-chef, sync all the cookbooks and converge the node.  
Also if you use that utility as part of your cookbook development cycle 
then it becomes a no-brainer to later add tests to your cookbooks.  It 
wouldn't make any sense at this point to write something else to do 
everything that TK does *except* running tests.  It also doesn't make a 
whole lot of sense to rename TK at this point just because there's this 
edge case of TK-without-tests doing lots of incredibly useful work.

And one more thing I'll say on this thread is that it sounds like you've 
got a whole lot of frustration with the TDD cookbook design paradigm.  
And I can related because my initial impression was bad. I've been a 
actual software developer for only about 3 years now where my background 
is in operations and perl hacking.  And I initially hated TDD cookbooks 
-- Berksfile, Rakefile, Thorfile, various different spec directories, 
and a half dozen .yml dotfile turds and what does it get me?  So, fuck 
all these hippie kids and their ruby TDD shit (tableflip).  I can easily 
view TDD cookbooks from the perspective of a non-ruby non-programming 
system administrator coming to Chef for the first time, and its a 
completely horrific vision, and the casual way that you are expected to 
be already familiar with bundler, rake, thor, rubocop, etc and pretty 
much can't use TDD cookbooks unless you really grok ruby already.  Since 
I work here, I had to knuckle down and start using it, and found that it 
was *extremely* powerful when done right.  It is the 21st century way of 
doing infrastructure engineering and configuration management.  But its 
still very convoluted and with sharp edges all over the place.  The 
whole concept of ChefDK is trying to address all of that frustration and 
minimize it.  And that HOWTO I wrote up is an example of that.  And 
consider what it buys you -- I probably ran into the idea of spinning up 
a virt and testing CFEngine code against it back in 2003 when VMWare was 
first becoming popular and scoffed at it.  I used to just blow code into 
a master branch (that was sync'd to everything in production) protected 
by some CFEngine classes to only apply it to some workstations and then 
do "Testing" by gradually pushing the changes out to more and more 
servers across the enterprise until I crashed something and had to back 
off and fix it (but with experience I could anticipate and minimize that 
from happening and grow my virtual neckbeard out longer than anyone 
else).  Now I can spin up a virt on my laptop and throw production 
configuration code against it and see what happens.  In retrospect that 
is seriously fucking amazing.