[chef] Re: Re: An deep topic

[Adam Jacob < >]

On Tue, Jul 8, 2014 at 10:40 PM, John E. Vincent (lusis) < " target="_blank"> > wrote:

That was definitely never my intent but you are right. It's hard for anyone to prove. I don't know how better to communicate but suffice to say because I've been a vocal critic (not in any negative or even positive sense of the word), I've had quite a bit of private feed back mostly via twitter DMs. I'm not really comfortable discussing that particular point anymore and I never intended it as some sort of rhetorical "I win" button. 

To clarify the "middle class" bit, though, it wasn't even a matter of "abuse by elites". I was simply talking about the group of people in the middle between "Chef New Users" and "Chef Expert Level Users". Somewhere in that "Expert" range there are also people who have a freedom to work on chef ecosystem as part of their actual jobs. I'm not bitter about that and I don't look down on them (or up really) - we're talking horizontal not vertical. If that came off wrong I'm definitely sorry.

It's fine to talk about the fact that you've heard similar feedback from others - I have too. I don't think you need to break confidences, or anything even remotely like that. I do think you need to be careful when you use language like you did, particularly in a community. Regardless of putting a disclaimer on the front about how non-offensive you want to be, you absolutely framed this as an us-vs-them way, which makes the starting point for discourse at a net negative. 

Less blame, more fixes.

 

Fair enough. official clarification was probably the wrong phrasing to get my request across.

I dunno, it felt super healthy to me.

 

I don't think that's entirely a fair statement. Honestly I'm NOT trying to stay on the leading edge of development. That's not really a part of my job. I think if anything I get caught behind on changes because I'm not staying up to date. From my perspective it's pretty difficult to stay up to date because things are changing so frequently. Like I said, I'm in a middle ground here as a chef user. I use chef as a means to and end. It works well but I have to weigh the needs of our usage with the (maybe perceived) volatility in the community. It very much feels to me. as I said, that when I come up for air to grab a new cookbook or update some knife plugin, I spend a day just dealing with "what's broken now". It feels very much like fighting Vagrant releases did.

It's probably a fair statement that I should be better about tracking these changes but again what ends up happening is we want to use a cookbook from opscode-cookbook's repo or get a bug fix and everything has changed yet again. Perception? Maybe but that doesn't make it any less valid.

It doesn't make it less valid, but it does reflect on some of the difficulties inherent in your situation. A great example is a cookbook from the opscode-cookbooks repo - they are maintained pieces of software, by folks that are employed to maintain them. They try and keep them up to date, to accept pull requests, and to make them high quality. That work involves tools that you don't like or want, so when you (admittedly) go to use them after a long absence, they cause you pain.

This isn't unique to Chef - this is the same conversation everyone has when they talk about pinning a dependency. It's great because you stay stable, but you are paying a tax the whole time - you either pay that tax a little bit all the time (to stay current) or a whole lot on the day your taxes are due (you decide you need an upgrade for whatever reason). That's the price you pay for re-use, in any situation. 

I'm not saying we shouldn't be using semver, or communicating more clearly - folks should be. At the same time, I think the lack of sympathy you pick up on is a little bit real: it sucks to have that problem, but them's the breaks, and you make your bed, to some extent. There is no free lunch.

At the last community summit, I talked about the fact that, from my point of view, the idea of having community standard abstractions at the level of something like, say, "apache" actually turned out to be a failed experiment. If you come in alone, and you have to configure apache, you probably need less than 10 configuration statements to be perfect. It might take you 20 minutes to figure it out, say. Now, lets say you decide to automate your apache configuration (yay!) and you pop open the Apache 2 cookbook. What you get hit with is all the complexity of apache (cause that cookbook does *everything*) plus all the complexity of Chef (which is not small). Maybe you find your way to the couple statements in chef you need to get the result, and move on, maybe you don't. God help you if you actually find a bug, or need to edit the source - you'll be hit in the face with what is actually a complex software project, managing software on 10+ different platforms, and you'll be brutalized.

The alternative would have been maintaining your own apache cookbook that just does what you need. It's probably 5 resources to begin with. It might always be. Which situation is better? I know what I would choose. The whole world doesn't agree with me, but I really think the sweet spot for re-use is at the level of resources, providers, and libraries rather than recipes for most cases.

 

I think this is an area I take issue with. The implication here is that we don't test our cookbooks or that we want to use test kitchen or that our workflow is somehow suboptimal. Are there things I would change? Sure but the implication still feels like "you'll come around. you just aren't REALLY using things yet. Wait till you have to do X". 

That's not what I'm saying. I'm saying if you want to test your cookbooks using those tools, their authors have some opinions based on their experience about how your workflow needs to change. If you hate that, you still have choices - it just doesn't involve reusing other peoples work, because you opted out of that. 

 

Just a small comment here (unrelated to the arrogance bit). Just because nobody has spoken up doesn't mean there hasn't been discussion. I don't know what internal discussions/debates were had. I'm only speaking to what people have confided in me and my personal experiences. This isn't the first time this has come up from me for sure. 

Of course. I would wager this is the first conversation that's happened as a community, in public, about what decisions we should make to better accommodate people in your situation. 

 

I don't want to bikeshed names here but again the implication is that someone using the single chef-repo isn't doing any testing. The only difference between these two boils down to monolithic repo vs. per-cookbook. 

Sure. You are otherwise in favor? 

The second thing we'll need here is someone who wants to take on the role of shepherding the experience for both workflows. Asking folks who have moved away from the monolothic repo approach (for what, to them, are valid reasons) to maintain the other standard at parity is probably a loosing proposition. For example, the majority of the testing workflows assume that you use berkshelf to inject dependencies into the vm before you converge. That doesn't mean you have to be outside a monolithic repo, but it does change your Berkshelf configuration (for example, you could just have a loop that injects every cookbook in your repo). You could also write another way of injecting your resources.

On another note, I'm glad you got involved in the discussion not from a "official chef policy" point but as an end-user. You've always been good about that and this is definitely no exception. For me personally it means a boatload.

It's how it should actually have always worked.

Adam