[chef-dev] Re: Re: Dialect support and loading enhancements

[Lamont Granquist < >]

On 9/20/13 12:17 PM, Brad Knowles wrote:
> On Sep 19, 2013, at 9:39 PM, Adam Jacob 
> <
>  >
>  wrote:
>
> > It's interesting to hear you say you feel like you don't know how to program, 
> > when i've seen you be an effective member of our community for a long time 
> > now. Is it completely out of school to say that perhaps you're arguing for 
> > your limitations? You might not be a CS major - lord knows, I'm not. But you 
> > do know how to program, and you're doing it in ruby. :) The closest you've 
> > come to a real programming language is Ruby: you're programming in it, and 
> > have been the entire time you've rocked the Chef, an you've done so 
> > successfully.
> I have a BSCS from the College of Engineering at the University of Oklahoma, technically 
> dated 1990 although I completed my last bit of work in 1989.  While at University, in terms 
> of formal programming languages I learned COBOL, FORTRAN, IBM System 360 Assembly, Prolog, 
> and others.  I'm old enough that I actually pre-date ANSI-C, and I have both 1st and 2nd 
> editions of K&R.  The first program I ever wrote was an implementation of a complete 
> "choose your own adventure" book in Basic, on a Commodore Vic-20, when I was 14 
> -- it was kinda-semi-sorta Zork-like, although I didn't learn about Zork until many years 
> later.
Okay, you totally opened the door there to me one-upping your 
grumpy-old-man all over this thread...

I learned 6502 assembly language in Junior High school and learned K&R C 
on a Tandy 6000 running Xenix.  I got my first Junior SA job in college 
managing a heterogenous mess of Digital Unix, Solaris, SunOS 4.1.x, 
NetBSD, Irix, Mac and NT.  I've always been a programmer (writing 
scripts to make the machine do a thing) rather than a software developer 
(designing objects and abstractions to make an extendable platform).  I 
wrote thousands of lines of (retrospectively) awful perl and cfengine 
scripts to keep Amazon and Real Networks running, but nothing that would 
be considered "software".  At the tender age of 39 I got sick of 
operations, and couldn't see myself landing another job in ops, and I 
was impatient with the devops revolution and so I switched to being a 
full time SDE and learning ruby and new rules like "use composition 
instead of inheritance" and started noodling on rails and angular 
websites in my spare time.  The shift was midly terrifying at times, but 
not as radical as I expected.   I think most of the barrier between 
programmer and software dev is mostly in learning a different *human* 
language, and it takes immersion in the culture to do it. For me, I 
don't learn well through theory and reading dissertations on a subject 
but more from rolling up my sleeves and trying it until it breaks, which 
is why I had a really hard time becoming a software dev without actually 
being a software dev.  I have to sit down and struggle with writing 
1,000s of lines of ugly rspec code before I understand why too much 
collaboration in an object model is bad -- its because you wind up with 
the unit tests for the file provider. A lot of software development 
isn't very creative at all, its just having experience, so that you can 
see that a bit of code is bad because its bad the way that other pieces 
of code were bad before it.  And learning from more senior devs who have 
already written all the bad code and learned to write better code and 
then just applying those patterns to the code you write in the future, 
and a lot of incremental improvement rather than any kind of insightful 
genius. A year or two later you just notice that you can sit down at a 
computer and the code that you type up is naturally become much better.

Anyway...

Compare these two examples:

SALT:

{% if grains['os'] != 'FreeBSD' %}
tcsh:
    pkg:
        - installed
{% endif %}

motd:
  file.managed:
    {% if grains['os'] == 'FreeBSD' %}
    - name: /etc/motd
    {% elif grains['os'] == 'Debian' %}
    - name: /etc/motd.tail
    {% endif %}
    - source: salt://motd


CHEF:

if node['platform'] != "FreeBSD"
  package "tcsh"
end

motd_path = case node['platform']
  when "FreeBSD"
    "/etc/motd"
  when "Debian"
    "/etc/motd.tail"
  end

cookbook_file motd_path do
  source "motd"
end

From my perspective, the idea that the Salt example is somehow "easier" 
is just flatly insane (and I should mention that I'm not speaking for 
Opscode here, this is just my own opinion).  You've still got a 
programming language in the Salt case or else where did those "if"'s and 
"endif" tokens come from?  You're just bracketing those tokens in 
magical {% %} tokens that you now need to explain what those are to 
anyone who is learning the language.  In fact now you have to explain 
that you're writing your config language in YAML and then go on to 
explain that when its not powerful enough they need to use the jinja 
templating language to get their jobs done. Now they have two things to 
learn, conceptually, and they have to context switch between all the 
leading -'s and trailing :'s in YAML markup and putting all the nasty 
programming language in between whatever template expansion operators {% 
%} that you are using.

I do not understand how this is even solving the stated problem at all.  
It does solve it for the absolutely simple and trivial (and largely 
totally useless) use cases, where you can do things without any 
programming language at all.  People can post those use cases to Hacker 
News and its like "ah, I've installed and configured a simple apache 
server without a line of a programming language!  winning!" while to do 
anything for a real job its just not going to be that easy.  And once 
you reach outside of the trivial it becomes *obviously* worse to me.  
Both conceptually its worse in the overhead of context switching between 
two things, and there is substantially more of the magical mark-up in 
the YAML case compared to ruby.  You are also training people to write a 
lot of "controller" code in a templating language which is exactly what 
we try to teach people not to do and keep templates simpler and more 
readable and with less control in them.  I think if we just wait awhile 
people are going to start posting non-trival examples of Salt and 
Ansible that will make your eyes bleed.  Our community cookbooks are 
awful in a lot of places precisely because they try to do so much, and I 
think if you translated those to jinja-YAML that you're going to see the 
readability deterioriate substantially.  The problem there I think lies 
in this idea that the community cookbooks are necessarily the 
objectively right way of doing Chef.

So I don't get it.  Even assuming you're right and people have vastly 
different capabilities, I think the argument here is between learning 
programming in the context of jinja-marked-up-YAML being somehow being 
better than just learning a programming language.  Are there any books 
out there on a programming language that *start* with the templating 
language?  Has anyone ever written an introductory chapter to a ruby 
language book that started with ERB? Again the config management world 
seems to have gone insane from my perspective.  Really I think this is 
the last gasp of the people who will not learn a programming language as 
the tide is changing to absolutely needing to know how to at least 
"program" in order to do operations, and they're trying to find any way 
to keep from changing as long as possible when it *MUST* change.  They 
don't have to learn how to design languages and write the next 'go' or 
'rust' themselves, but just basic control structures in the language, 
and enough ruby to understand that case assignment that I did up there. 
I managed to get that much down in my first month or two at opscode.  
They don't need to learn about monkey patching or instance_eval or 
writing DSLs or the difference between a module and a class.  And they 
might not understand what #each is doing, but they copypasta it so that 
it works (and after a few hundred times of doing that they might get a 
glimmer of what functional programming is...  I was certainly 
uncomfortable with that kind of programming when I got here 2 years ago 
and I last weekend I just breathed a *huge* sigh of relief when I found 
underscore.js because all those #each, #map, #select, etc functions have 
become fundamental bits of my toolbox and writing c-style for loops in 
javascript was making me cry...)

Anyway, having changed from SE to SDE and coming from your same 
background, I'm skeptical of the premise of your argument.  And even 
given your premise I'm skeptical that the proposed solution actually 
solves it for anyone in other than the most trivial of cases.

As a historical aside how did we even wind up in this situation anyway?  
Back when I was growing up 'system adminstrators' like Wietse Venema 
went off and wrote tcp_wrappers and postfix, and him and Dan Famer wrote 
SATAN/SAINT.  Having SAs who were proud of not programming back then 
would have seemed bizzare to me.   Back before then, there wasn't any 
good package management, and there wasn't even GNU autoconf when I 
started learning C and Unix.  There was no "./configure; make; make 
install" we had to hack up Makefiles and at least choose our CFLAGs from 
all the different Unix variants like 3B2 that we'd never heard of 
before, and then cobble together our own CFLAGs based on the compiler 
errors we got because the version of Dynix that we had on the Sequent 
Symmetry wasn't in the Makefile yet.  What happen to us in the late 
90s?  It seems like in the dotcom gold rush we hired warm butts in seats 
that could barely type 'lsof' at all and we wound up here...

And we were all naked, in the snow, pushing the compiler uphill both 
ways just so we could get tcsh installed, because bash hadn't even been 
written yet...

> > My concern really boil down to this: I don't want to say that there will be a 
> > simpler path that turns out to just be a ghetto - an underserved, less loved 
> > corner of Chef.
> Then don't let it turn into a ghetto.  ;-)
The problem is then we have two things to support.  We're going to get 
tickets which look like "i can't do this in the YAML syntax and I've 
inherited 8,000 lines of code from the guy who quit 2 months ago, please 
help" and we either wind up needing to implement some kind of functional 
parity in that dialect, or wind up telling the person that they need to 
rewrite everything in ruby.  Cynically, maybe we can offer them a 
consulting gig so they can pay us to do that, but that spot doesn't seem 
like a very healthy space to wind up in.   And if we go off and extend 
the dialect to attempt to get parity, which of the outstanding tickets 
of yours and everyone else's do we bump in order to fix the dialect?  
TANSTAAFL...

And all this is just my opinion, and I may just be a grumpy old man 
about it and the kids these days may just go nuts with their 
jinja-YAML...  I still don't see it though.  Noah *almost* made it 
through to me that there is a barrier to entry between the python 
community and chef because of the need to use ruby.  I can see that.  I 
certainly see that barrier from the other side as well. But at the same 
time I've managed to pick up javascript and angular in my free time 
because I was interested in that.  I also don't see this as a solution 
because it would push chef into the situation where if a python dialect 
of chef was actually successful you'd now need to learn *2* programming 
languages to interact fully with the community.  I also read hacker news 
and have some anxiety about 'go' and 'rust' and other languages, but I 
don't see how to get past the problem of building a barrier of needing 
to know N different languages to learn Chef.