[chef] Re: An deep topic

["John E. Vincent (lusis)" < >]

On Tue, Jul 8, 2014 at 5:00 PM, Adam Jacob < " target="_blank"> > wrote:

I'm going to hop in on top, but my intent isn't to stifle or ignore the resulting conversation (which has been great).

First off, one of the things that has gotten off in the way we work as a community is understanding what space we're in. You can see it in this email, and in resulting threads - if you post to this list from a Chef Software email address, are you speaking for yourself, or on behalf of the company? 

Since I'm probably the only person who can clear this up short of Barry Crist (who is awesome, but I don't think is on this thread,) lets go ahead and create a new rule. If you want to know the Official Position (tm) of Chef Software, you can go ahead and ask directly - and I, or someone the company designates, will respond with an Official Clarification (tm). :) Otherwise, all communication on this list, and in IRC, is the opinion of it's author, and not a representation of the Company. This is, I believe, the only tenable position for this space to remain one that is actually community driven, and not simply a channel for communicating with/around/to Chef Software.  I am adding this to the upcoming RFC on how the project is maintained - keep your eyes peeled for that before our first community meeting on Friday.

Now, on to your email. 

On Mon, Jul 7, 2014 at 8:05 PM, John E. Vincent (lusis) < " target="_blank"> > wrote:

Okay so I'm afraid I'm going to regret doing this but Dan was right that this is the best place for it. I thought about doing it on the dev list but figured there would be more (appropriate?) feedback here.

Why are you afraid you will regret it? Either you want to find solutions to your problems, or you don't - and this is the place for us to find solutions. Don't regret it - participate in it.

 

I don't think this fits with a blog post because (Dan is right here as well) that's dictation not discussion. I realize I'm an overly vocal critic of this topic but I've heard in private from quite a few folks about these same frustrations. I'll leave it up to them if they want to speak up. Some folks aren't comfortable with that though and feel like it would be burning a bridge. Please keep that in mind. Also please be aware that I'm going to point to specific statements by people but I'm not trying to call those people out and this should not be construed as a personal attack. I understand I can't stop people from feeling that way but I wanted to be clear about this.

I understand not wanting to break confidences, or people being concerned about hurting the feelings of others. At the same time, I'm wary of arguments that involve the specter of broad-scale assent - at several points in this email, you make reference to "how other people feel" and "the chef middle class" - all of which creates a self-defined rhetorical division within the community that, if it does exist, certainly can't be quantified easily without more discussion. As a rhetorical device, it is super effective - easy to think of a silent middle being abused by elites, which, as one of what you probably define as the upper class, I resent on a personal level, and I think my actual participation as a human being refutes it consistently. The thread on this list that ensues proves both the need for the discussion and the lack of actual division caused by anything other than lack of participation.

That was definitely never my intent but you are right. It's hard for anyone to prove. I don't know how better to communicate but suffice to say because I've been a vocal critic (not in any negative or even positive sense of the word), I've had quite a bit of private feed back mostly via twitter DMs. I'm not really comfortable discussing that particular point anymore and I never intended it as some sort of rhetorical "I win" button. 

To clarify the "middle class" bit, though, it wasn't even a matter of "abuse by elites". I was simply talking about the group of people in the middle between "Chef New Users" and "Chef Expert Level Users". Somewhere in that "Expert" range there are also people who have a freedom to work on chef ecosystem as part of their actual jobs. I'm not bitter about that and I don't look down on them (or up really) - we're talking horizontal not vertical. If that came off wrong I'm definitely sorry.

 

So I'm going to frame these questions again (at least the ones that haven't been clarified for ME):

1) Is Berkshelf becoming a core chef dependency?

I heard no but let's be clear when ALL of the tooling coming out of Chef proper has a dependency on Berkshelf, that's no different. Also ChefDK doesn't solve that problem (especially since it's not actually released yet). Related is "The Berkshelf Way"/model the prescribed  workflow for chef repos (note that I'm not saying cookbooks here for a reason).

As the CTO of Chef Software, let the next few paragraphs serve as an Official Clarification (tm). (You request it in a piece I snipped) As we move forward, I would like to see these conversations not require official clarification from Chef Software, because improvements in the transparency of how decisions are made and the roadmap is communicated obviates the need. Since we don't live in that world yet, here we go.

There are currently no plans for the "chef-client" omnibus package and rubygem to include Berkshelf as a dependency. It makes no sense to do so. There *are* conversations about doing fully client-side dependency resolution, as it would be more efficient for all concerned. 

The Chef DK is our attempt to bring a single, vetted, consistent user experience to those developing infrastructure with Chef. That includes having (at least one) blessed workflow that allows users to go from very little tooling to a fully built, best practice way of managing your infrastructure at scale. Step one of that is simply getting all the tooling in one place, released often, and vetted. Step two is creating and fixing those workflows, so that we can then talk about them more broadly. There are many more steps. Chef DK includes Berkshelf as its dependency resolver, and the workflows it brings to bear (test driven infrastructure primarily) use Berkshelf.

Chef Software is committed to ensuring that every tool shipped in the Chef DK works well with the rest of the DK, is vetted and tested. In many cases, that means working directly with the communities and authors of those tools, which we haven't taken away from those authors (and couldn't if we wanted to - they weren't "ours" to begin with.) That includes Berkshelf, ChefSpec, Test Kitchen, and more - all of which we participate in and support, but do not 'own'. 

End of Official Clarification (tm), and back to just plain old Adam.

Fair enough. official clarification was probably the wrong phrasing to get my request across.

 

 

2) What problem is Berkshelf trying to solve?

This is part of my frustration and other folks as well. The running joke is that Berkshelf solved a problem I didn't have. And it's true. Again I can totally accept that some people had a problem and found that Berkshelf solved it but is that reason enough to bake it into EVERYTHING?

Julian answered this well.

 

FWIW I didn't have a problem with Berkshelf originally. One: I didn't use it and didn't HAVE to use it. Two: It didn't have this entirely new service it depended on.

Now I can't avoid it and it requires a separate internet service (though now apparently rolled into the supermarket) just to be used.

I understand your frustration here, but you want to have your cake and eat it to. You want to stay up to date with an evolving, naturally leading edge of development. On the one hand, you don't have a problem that required client side dependency solving, because you're happy with the single repository, vendor branch approach. For the record, so was I - I created it because it worked for me, and I didn't struggle with the issues other people had. I was also perfectly happy to have my workflow be logging in to a machine and running chef client to observe if my shit worked. I was very proficient at it. :)

I don't think that's entirely a fair statement. Honestly I'm NOT trying to stay on the leading edge of development. That's not really a part of my job. I think if anything I get caught behind on changes because I'm not staying up to date. From my perspective it's pretty difficult to stay up to date because things are changing so frequently. Like I said, I'm in a middle ground here as a chef user. I use chef as a means to and end. It works well but I have to weigh the needs of our usage with the (maybe perceived) volatility in the community. It very much feels to me. as I said, that when I come up for air to grab a new cookbook or update some knife plugin, I spend a day just dealing with "what's broken now". It feels very much like fighting Vagrant releases did.

It's probably a fair statement that I should be better about tracking these changes but again what ends up happening is we want to use a cookbook from opscode-cookbook's repo or get a bug fix and everything has changed yet again. Perception? Maybe but that doesn't make it any less valid.

On the other hand, you want to be able to use tools like Test Kitchen, or the new generators in the Chef DK. The creators of those tools, over the course of the last several years, all think that having dependency resolution at the level of individual cookbooks is essential to its good functioning. After using those workflows (which I didn't want to do, because I didn't have those problems) - I agree with them, on a personal level. I also think the results in practice speak for themselves - it really does make it faster, and safer, to develop infrastructure this way. 

 

You still don't have to use it. Not using it means opting out of the things that, as the authors of those tools, people decided was necessary. That's cool, because, as you say - you don't have those problems anyway, right? ;)

 

I think this is an area I take issue with. The implication here is that we don't test our cookbooks or that we want to use test kitchen or that our workflow is somehow suboptimal. Are there things I would change? Sure but the implication still feels like "you'll come around. you just aren't REALLY using things yet. Wait till you have to do X". 

 

But this ties into bigger workflow questions. From almost every corner of the Chef proper universe (rough paraphrases):

"Berkshelf won"

"Stop doing devodd. Stop uploading unreleased artifacts to your Chef Servers"

"the separate repos per cookbook voice + Berkshelf is the loudest and with good reason"

"basically (in response to 'so individual cookbooks can be treated like district pieces of software?')"

"You don't have to change your workflow.. <two minutes passes> You have to change your workflow a bit"

And yet the Chef documentation is all over the place:

"The chef-dk defines a common workflow for cookbook development, including unit and integration testing, identifying lint-like behavior, dedicated tooling, and more"

"The chef-repo is located on a workstation and should be synchronized with a version control system, such as git. All of the data in the chef-repo should be treated like source code."

There is conflicting information here in that the docs describe one thing and yet there is semi-official direction/prescription/statements that you should use a single-cookbook-artifact-berkshelf-some-tool-that-isn't-actually-knife workflow.

But since this toolchain using this new workflow is so convoluted right now, there's ChefDK but it's not released yet and even if it was it's running an ENTIRELY different version of Ruby than the Omnibus chef-client.

We haven't ported the universe to a new workflow precisely because, in many cases, it just isn't ready for it yet. The difficulties in tying those tools together are sometimes deep, and pushing it out in front of the world doesn't help. I agree we need to get to a place where the best practice workflow is both documented and simple, and goes from using nothing but chef-apply on a single recipe all the way up to managing complex infrastructure with integrated testing and servers. The experience should be delightful and natural at each part of the way.

It isn't right now, because we're in transition. 

 

And in the end there's, and I hesitated to mention this but it's a part of it, a profound level of arrogance about this workflow as a whole laced with "but you can keep doing it the OLD way" or "you'll eventually figure out the right workflow" or "Berkshelf fits every use case". But just a year ago there was ANOTHER way that was named the same thing but done differently and it's not the right way anymore. 

As someone who is largely a run of the mill chef end-user (you can disagree with that. I have a larger than normal "voice" for better or worse), you have to appreciate the confusion here at the mixed messaging and "new shiny" every time I look up. As I've said before, Chef is a means to an end for me and not the defining thing. I don't like to fight my tools and when they keep fighting back I have to weigh the balance of how much time I invest with the tool versus getting actual work done. At some point, you figure out that you're fighting the wrong things.

And I realize that Chef is going through some transformative things right now and trying to get back to certain roots.

The arrogance exists on all sides of what, until this moment, hasn't even really been a debate. 

Just so we're being clear, cloaking yourself in a populist mantle (just a regular chef user, the chef middle class) and calling those who think differently than you 'arrogant' is, itself, pretty damn arrogant. 

Lets all drop that from the dialogue, and just focus on getting the mixed messaging and confusion that's at the heart of your message cleaned up.

Just a small comment here (unrelated to the arrogance bit). Just because nobody has spoken up doesn't mean there hasn't been discussion. I don't know what internal discussions/debates were had. I'm only speaking to what people have confided in me and my personal experiences. This isn't the first time this has come up from me for sure. 

 

mSo what am I asking here? What's the point of this rambling? 

Where's the communication? What's the intention? The post on Omnibus was awesome because it was clear. A line was drawn that said "look, omnibus was built for this and we think it's awesome that people are using it for this other thing but there's a conflict that we can't maintain"

I'm totally cool with that. What's hard for me is that when it comes to workflow, there's a defacto standard (driven by a vocal subset of the community and unofficially by Chef employees - my interpretation) that is infinitely more complicated than the one officially documented. And it's not just a question of workflow, it changes the entire model of how you use Chef when writing cookbooks (again in conflict with the official documentation).

Here is a proposal. Lets have two officially supported workflows.

1) The 'classic' workflow. This has a single chef-repo directory, that includes every artifact you need in your infrastructure. It its managed as a single git repository, uses vendor branches for importing external cookbooks via 'knife cookbook site', and either uploads directly to a Chef server or is bundled together for use in solo or chef-zero. The Chef DK should have generators that support this model as a first class citizen.

2) The 'TDI' workflow. This workflow involves breaking your single chef-repo into a git repository per cookbook, with all other assets tracked in a single common repository. It uses Berkshelf and Test-Kitchen to support test-driven infrastructure, and either uploads directly to a chef server or is bundled together for use in solo or chef-zero. The Chef DK should have generators that support this model as a first class citizen.

In either case, the glide path to either should be easy and basically identical, until you decide to start writing tests. 

I don't want to bikeshed names here but again the implication is that someone using the single chef-repo isn't doing any testing. The only difference between these two boils down to monolithic repo vs. per-cookbook. 

 

 

I realize the desire to straddle both ways but at some point you can't when they diverge so dramatically. And really one of them is in largely direct opposition to the way of working that's even possible for a subset of the community. I feel like there's an entire chef middle class being cut out of this entire topic.

Divergence isn't a bad thing. If you don't want, or don't see, the benefits of the TDI workflow - you shouldn't use it. 

For many of us, that workflow is consistently producing better results than the 'classic' one did.  

 

So am I asking for an "official workflow"? Part of me says no but part of me says "yes already. And then update the docs so I have somewhere to point new users that isn't being contradicted everywhere else". And frankly for me I almost want it to be done so I can make the difficult decisions around "is it time to move to another tool or not" because in the end my goal is pretty simple. I need to manage my servers and the tool that I have to fight the least to do that is the one I'm going to use.

 

Maybe I'm wrong about this. I could be the only person who feels this way but I don't think so.

As of today, I don't think you can say the TDI workflow is ready to be the default workflow for new users. My opinion is we should continue to develop it, and even recommend it for those whose problems it solves today and can handle being early adopters. We shouldn't put it in front of new users (see the revamped LearnChef content as an example.)

On another note, I'm glad you got involved in the discussion not from a "official chef policy" point but as an end-user. You've always been good about that and this is definitely no exception. For me personally it means a boatload.

 

Adam