[chef] Re: Re: Re: Re: Re: An deep topic

["Julian C. Dunn" < >]

On Tue, Jul 8, 2014 at 4:38 PM, Dylan Northrup 
<
 >
 wrote:

> Just to be explicit, please preface all of this with "In my opinion" and "In
> my experience" as appropriate.

Of course, I think that goes without saying (even for everything I'm
saying in this thread).

> On Tue, Jul 8, 2014 at 1:53 PM, Julian C. Dunn 
> <
 >
>  wrote:

>> > Every time I've talked to an Opscode/Chef employee, there's been this
>> > tremendous reluctance to say "This is a way that should be successful to
>> > you".  Every time, and I do mean EVERY TIME EVEN WHEN PRESSED, there's
>> > never
>> > been someone who will say anything other than "There's lots of ways to
>> > do
>> > it. Whatever's good for your environment should work" or something to
>> > that
>> > effect.
>>
>> It depends in what context you are asking, though.
>>
>> * If you ask us, as part of a paying services engagement for *your*
>> company, the answer should be along the lines of the "better answer"
>> you outlined above.
>> * If you ask us generally in a public forum, where there are many
>> readers, the answer we'll give is probably along the lines of "there's
>> more than one way to do it".
>
> I'll say that multiple times, even in a paying service engagement, the
> better answer has not happened.  I'll also say that in a public forum, the

If that's the case then I'm sorry - it would be my expectation that it is.

> "TMTOWTDI" answer should be followed by "and here's some suggested ways for
> typical use cases".  And those answers, scrubbed of incriminating details,
> should be informed by the solutions provided to enterprise customers.  This
> helps guide everyone in a similar direction so that the community as a whole
> will tend to move in the same directions.

Interesting. I'm not saying you're wrong here, it's a matter of
opinion, but my experience is that enterprise customers prefer the
information flow to go in the other direction: that they expect
ChefCo's consultants to show up armed with a filtered list of what OSS
people are doing, and what tends to work & what tends not to.

My experience has also been that people who participate in open source
have a high tolerance for the messiness, wide variety of different
workflows/technologies, and the sometimes wrong turns that open source
software takes. That's what I find orthogonal to "enterprise
readiness" and what the value proposition (excuse me for using
business jargon) is in paying ChefCo consultants money.

We've therefore been reluctant to clamp down on the OSS creativity --
some may call it chaos -- because the chaos leads to, and has led to,
interesting ideas, tools, and approaches. Perhaps I'm incorrect and
this community's expectations are different? I just think ChefCo has
explicitly NOT done that in the OSS space and we're unlikely to.

I'll give you an example: Jeremy Bingham has written a fully-fledged
Chef server in Go. Should ChefCo nip that in the bud because our
recommended Chef server is erchef? No, because it's an interesting
project and who knows, someone may find it useful, and anything that
contributes to the Chef ecosystem ultimately makes Chef stronger.

> As an ops guy with a programming background, I want a map of the green field
> so I know where the sink holes are so I can avoid them and where the paths
> are that others have used successfully (but which I may not be able to
> easily see).
>
> And as a Chef user, I want ChefCo, who has the most visibility into
> implementations across a wide variety of locations, to be able to distill
> that down and provide advice to the larger user base to help guide us as a
> community so we can all move in roughly the same direction (at least with
> regards to the larger trends).

I guess what I'm arguing is that if you participate in the OSS
ecosystem, you're likely to get better information about the state of
that ecosystem by asking on this mailing list & other OSS Chef fora
than by asking ChefCo employees directly about that. Then make up your
own mind.

> That being said, not having an omnibus client installer was a gaping hole in
> the Opscode/ChefCo offering for quite a while.  Not having a
> straightforward, platform independent method for installing knife has been a
> huge hurdle for getting folks up to speed on chef.

It's a mea culpa on our part, especially putting platforms like
"Windows 2008R2" on the download page just so you could get Knife on
your Windows 7 workstation (wat?)

>> They *should* be exemplars of good code. But often they are not. Would
>> we like them to be? Of course, but you can't erase 5 years of tech
>> debt overnight, and many of them date back to the days of HJK
>> Solutions, when we managed customer infrastructure with them. (I'm
>> assuming you mean the cookbooks under "opscode-cookbooks" when you say
>> "Chef site")
>>
>> Also, many of those cookbooks were written before today's cookbook
>> authoring practices & TDD were extant. It's a little unfair to measure
>> old work products against today's yardstick.
>
> In deference to paying down technical debt immediately (which, I agree, is
> not a reasonable expectation), I'd suggest having specific exemplars out
> there for cookbooks, LWRPs, etc. that can be pointed to as "shining
> cookbooks on a hill" on the site.  These could be used by others as
> guideposts in their own endeavors while old, creaky code is hand-waved away
> saying "Ignore that, we did that before we knew better".  Not all code needs
> to be good, but there do need to be examples folks can use to point the way.

I think we are discovering a number of things as the ecosystem evolves:

* The distribution of LWRPs (custom types) as part of cookbooks is not
optimal and maybe we should consider ingesting more of them into core
or distributing them as some other artifact type (this also addresses
some of the 'Ansible is easier to get started with' criticism that we
get)
* There are ways to write Chef cookbooks without them devolving into
total spaghetti recipe code by putting more things in LWRPs, HWRPs and
libraries to abstract them from recipe context.

To give you a sense of what a cookbook like that looks like, check out
the 'jenkins' or 'mysql' cookbooks. But we've also traded off the
reality that the code in there is not easily digestible by beginners.

>> > Opscode/ChefCo is the Source of Authority for implementations and, while
>> > it's awesome to have y'all be so open to suggestions from the community,
>> > that does not absolve you of the inherent responsibility.
>>
>> I'd like to know more about what you feel ChefCo's "responsibility"
>> should be. As I mentioned above, I'd also strongly suggest that the
>> "duty of care" is different for a commercial relationship versus the
>> open source ecosystem.
>
> I'd think one feeds into the other.  The needs of commercial clients
> provides use cases and solution spaces to test ideas, cookbooks, workflows,
> etc.  Once those practices have been fired in the crucible of actual
> day-to-day use, the survivors are used as suggestions to the greater
> community (either directly, or via the "shining code on a hill" method).
>
> I am, in no way, intending to impart a legal obligation to ChefCo to make
> Joe McOpensource's website work if Joe didn't pay anything to ChefCo.  But,
> as members of the greater community, I'd expect ChefCo to provide guidance
> in much the same way Linus does with Linux, that Tom Christiansen and Randal
> Schwartz have done for Perl, that Maxim Dounin does for nginx.  Suggest good
> ways of doing things and warn away from bad ways.  Point out de facto
> standards, and even help craft them.  Point in the right direction instead
> of saying "Whatever works for you" or something to that effect.  That's what
> I'd say is the "responsibility".  Not a legal obligation, but a social one.

I've certainly been trying to do that with some of my blog posts on
the Chef blog in the last 6-9 months, but I think that our blog isn't
as visible as it should be. Nor do I really think that is the right
place for "right direction" articles. But I hear what you are saying:
that we should use our substantial platform (as the creators of Chef)
more broadly, and I totally agree with that.

- Julian

-- 
[ Julian C. Dunn 
<
 >
          * Sorry, I'm    ]
[ WWW: http://www.aquezada.com/staff/julian    * only Web 1.0  ;]
[ gopher://sdf.org/1/users/keymaker/           * compliant!    ;]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9       ]