[chef] Re: Re: Re: Re: Re: Re: An deep topic

[Joseph Bowman < >]

Hi guys,

This has been an interesting thread to read. I'd like to chime in on
the side of it's hard to have a canonical or recommended way of doing
things. It's actually the flexibility that chef provides in being able
to do things different ways that's allowed me to introduce it to my
team.

It probably makes sense to just explain the make up of my team, our
environment and how we're using chef.

I don't work for a tech company. I work for one of the larger
environmental non-profits. I'm part of an 8 person enterprise team of
admins and engineers that support a mix of Widows, Redhat Linux and a
couple HPUX unix servers. Currently chef is our configuration
management tool we're rolling out on Redhat Linux. In the future we
plan to use chef to manage a nagios agent on our Windows hosts since
the server is built with chef. All other configuration for Windows is
done via GPO.

No one on our team uses ruby. Me and one other guy are the primary
Linux scripters, I prefer python and my other guy uses perl. We have a
couple other engineers that can kick around with both perl and python.
99% of our chef recipes are currently bash. Why? Because that's the
one "language" we all feel comfortable backing each other up on.
Recently I've for the most part gotten away from templates, just using
files to push stuff out. It was easier for the rest of the team to
follow along with rather than trying to learn the template language.

My team is constantly slammed. We all are heavily involved in project
work while also being responsible for all KLO. To get things like chef
inside our environment I had to do it in the scope of a project where
I padded some time in order to get the chef implementation in. My team
doesn't have the time to learn ruby and whole new way of doing things.
We also have some very opinionated people about how exactly everything
should be done... oh yea, we're sysadmins.

I imagine a lot of other people have environments similar to ours, not
in the tech sense but in the "have to get things done and get it done
now" lifestyle of the job. Otherwise, you wouldn't need a
configuration management system in the first place, you'd have time to
manage it yourself. Now, I can't imagine there's ever going to be a
chef guide that says write all yours recipes with code blocks of bash.
However, I can tell you it's the best possible implementation for my
team.

Operations environments are different every where. Especially with
older organizations. It's one thing to build ground up with a tool
like chef, but introducing it to an existing ecosystem of scripts,
policies and procedures is something that there's not going to be a
catch all solution for. The answer "there's a lot of ways to do it" is
exactly the answer that makes Chef work for us.


On Tue, Jul 8, 2014 at 9:37 PM, Julian C. Dunn 
<
 >
 wrote:
> On Tue, Jul 8, 2014 at 4:38 PM, Dylan Northrup 
> <
 >
>  wrote:
>
>> Just to be explicit, please preface all of this with "In my opinion" and 
>> "In
>> my experience" as appropriate.
>
> Of course, I think that goes without saying (even for everything I'm
> saying in this thread).
>
>> On Tue, Jul 8, 2014 at 1:53 PM, Julian C. Dunn 
>> <
 >
>>  wrote:
>
>>> > Every time I've talked to an Opscode/Chef employee, there's been this
>>> > tremendous reluctance to say "This is a way that should be successful to
>>> > you".  Every time, and I do mean EVERY TIME EVEN WHEN PRESSED, there's
>>> > never
>>> > been someone who will say anything other than "There's lots of ways to
>>> > do
>>> > it. Whatever's good for your environment should work" or something to
>>> > that
>>> > effect.
>>>
>>> It depends in what context you are asking, though.
>>>
>>> * If you ask us, as part of a paying services engagement for *your*
>>> company, the answer should be along the lines of the "better answer"
>>> you outlined above.
>>> * If you ask us generally in a public forum, where there are many
>>> readers, the answer we'll give is probably along the lines of "there's
>>> more than one way to do it".
>>
>> I'll say that multiple times, even in a paying service engagement, the
>> better answer has not happened.  I'll also say that in a public forum, the
>
> If that's the case then I'm sorry - it would be my expectation that it is.
>
>> "TMTOWTDI" answer should be followed by "and here's some suggested ways for
>> typical use cases".  And those answers, scrubbed of incriminating details,
>> should be informed by the solutions provided to enterprise customers.  This
>> helps guide everyone in a similar direction so that the community as a 
>> whole
>> will tend to move in the same directions.
>
> Interesting. I'm not saying you're wrong here, it's a matter of
> opinion, but my experience is that enterprise customers prefer the
> information flow to go in the other direction: that they expect
> ChefCo's consultants to show up armed with a filtered list of what OSS
> people are doing, and what tends to work & what tends not to.
>
> My experience has also been that people who participate in open source
> have a high tolerance for the messiness, wide variety of different
> workflows/technologies, and the sometimes wrong turns that open source
> software takes. That's what I find orthogonal to "enterprise
> readiness" and what the value proposition (excuse me for using
> business jargon) is in paying ChefCo consultants money.
>
> We've therefore been reluctant to clamp down on the OSS creativity --
> some may call it chaos -- because the chaos leads to, and has led to,
> interesting ideas, tools, and approaches. Perhaps I'm incorrect and
> this community's expectations are different? I just think ChefCo has
> explicitly NOT done that in the OSS space and we're unlikely to.
>
> I'll give you an example: Jeremy Bingham has written a fully-fledged
> Chef server in Go. Should ChefCo nip that in the bud because our
> recommended Chef server is erchef? No, because it's an interesting
> project and who knows, someone may find it useful, and anything that
> contributes to the Chef ecosystem ultimately makes Chef stronger.
>
>> As an ops guy with a programming background, I want a map of the green 
>> field
>> so I know where the sink holes are so I can avoid them and where the paths
>> are that others have used successfully (but which I may not be able to
>> easily see).
>>
>> And as a Chef user, I want ChefCo, who has the most visibility into
>> implementations across a wide variety of locations, to be able to distill
>> that down and provide advice to the larger user base to help guide us as a
>> community so we can all move in roughly the same direction (at least with
>> regards to the larger trends).
>
> I guess what I'm arguing is that if you participate in the OSS
> ecosystem, you're likely to get better information about the state of
> that ecosystem by asking on this mailing list & other OSS Chef fora
> than by asking ChefCo employees directly about that. Then make up your
> own mind.
>
>> That being said, not having an omnibus client installer was a gaping hole 
>> in
>> the Opscode/ChefCo offering for quite a while.  Not having a
>> straightforward, platform independent method for installing knife has been 
>> a
>> huge hurdle for getting folks up to speed on chef.
>
> It's a mea culpa on our part, especially putting platforms like
> "Windows 2008R2" on the download page just so you could get Knife on
> your Windows 7 workstation (wat?)
>
>>> They *should* be exemplars of good code. But often they are not. Would
>>> we like them to be? Of course, but you can't erase 5 years of tech
>>> debt overnight, and many of them date back to the days of HJK
>>> Solutions, when we managed customer infrastructure with them. (I'm
>>> assuming you mean the cookbooks under "opscode-cookbooks" when you say
>>> "Chef site")
>>>
>>> Also, many of those cookbooks were written before today's cookbook
>>> authoring practices & TDD were extant. It's a little unfair to measure
>>> old work products against today's yardstick.
>>
>> In deference to paying down technical debt immediately (which, I agree, is
>> not a reasonable expectation), I'd suggest having specific exemplars out
>> there for cookbooks, LWRPs, etc. that can be pointed to as "shining
>> cookbooks on a hill" on the site.  These could be used by others as
>> guideposts in their own endeavors while old, creaky code is hand-waved away
>> saying "Ignore that, we did that before we knew better".  Not all code 
>> needs
>> to be good, but there do need to be examples folks can use to point the 
>> way.
>
> I think we are discovering a number of things as the ecosystem evolves:
>
> * The distribution of LWRPs (custom types) as part of cookbooks is not
> optimal and maybe we should consider ingesting more of them into core
> or distributing them as some other artifact type (this also addresses
> some of the 'Ansible is easier to get started with' criticism that we
> get)
> * There are ways to write Chef cookbooks without them devolving into
> total spaghetti recipe code by putting more things in LWRPs, HWRPs and
> libraries to abstract them from recipe context.
>
> To give you a sense of what a cookbook like that looks like, check out
> the 'jenkins' or 'mysql' cookbooks. But we've also traded off the
> reality that the code in there is not easily digestible by beginners.
>
>>> > Opscode/ChefCo is the Source of Authority for implementations and, while
>>> > it's awesome to have y'all be so open to suggestions from the community,
>>> > that does not absolve you of the inherent responsibility.
>>>
>>> I'd like to know more about what you feel ChefCo's "responsibility"
>>> should be. As I mentioned above, I'd also strongly suggest that the
>>> "duty of care" is different for a commercial relationship versus the
>>> open source ecosystem.
>>
>> I'd think one feeds into the other.  The needs of commercial clients
>> provides use cases and solution spaces to test ideas, cookbooks, workflows,
>> etc.  Once those practices have been fired in the crucible of actual
>> day-to-day use, the survivors are used as suggestions to the greater
>> community (either directly, or via the "shining code on a hill" method).
>>
>> I am, in no way, intending to impart a legal obligation to ChefCo to make
>> Joe McOpensource's website work if Joe didn't pay anything to ChefCo.  But,
>> as members of the greater community, I'd expect ChefCo to provide guidance
>> in much the same way Linus does with Linux, that Tom Christiansen and 
>> Randal
>> Schwartz have done for Perl, that Maxim Dounin does for nginx.  Suggest 
>> good
>> ways of doing things and warn away from bad ways.  Point out de facto
>> standards, and even help craft them.  Point in the right direction instead
>> of saying "Whatever works for you" or something to that effect.  That's 
>> what
>> I'd say is the "responsibility".  Not a legal obligation, but a social one.
>
> I've certainly been trying to do that with some of my blog posts on
> the Chef blog in the last 6-9 months, but I think that our blog isn't
> as visible as it should be. Nor do I really think that is the right
> place for "right direction" articles. But I hear what you are saying:
> that we should use our substantial platform (as the creators of Chef)
> more broadly, and I totally agree with that.
>
> - Julian
>
> --
> [ Julian C. Dunn 
> <
 >
>           * Sorry, I'm    ]
> [ WWW: http://www.aquezada.com/staff/julian    * only Web 1.0  ;]
> [ gopher://sdf.org/1/users/keymaker/           * compliant!    ;]
> [ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9       ]