[chef-dev] Re: Re: Dialect support and loading enhancements

Chronological Thread 
  • From: Lamont Granquist < >
  • To:
  • Subject: [chef-dev] Re: Re: Dialect support and loading enhancements
  • Date: Fri, 20 Sep 2013 15:58:29 -0700

On 9/20/13 12:17 PM, Brad Knowles wrote:
On Sep 19, 2013, at 9:39 PM, Adam Jacob 
< >

It's interesting to hear you say you feel like you don't know how to program, 
when i've seen you be an effective member of our community for a long time 
now. Is it completely out of school to say that perhaps you're arguing for 
your limitations? You might not be a CS major - lord knows, I'm not. But you 
do know how to program, and you're doing it in ruby. :) The closest you've 
come to a real programming language is Ruby: you're programming in it, and 
have been the entire time you've rocked the Chef, an you've done so 
I have a BSCS from the College of Engineering at the University of Oklahoma, technically 
dated 1990 although I completed my last bit of work in 1989.  While at University, in terms 
of formal programming languages I learned COBOL, FORTRAN, IBM System 360 Assembly, Prolog, 
and others.  I'm old enough that I actually pre-date ANSI-C, and I have both 1st and 2nd 
editions of K&R.  The first program I ever wrote was an implementation of a complete 
"choose your own adventure" book in Basic, on a Commodore Vic-20, when I was 14 
-- it was kinda-semi-sorta Zork-like, although I didn't learn about Zork until many years 
Okay, you totally opened the door there to me one-upping your grumpy-old-man all over this thread...

I learned 6502 assembly language in Junior High school and learned K&R C on a Tandy 6000 running Xenix. I got my first Junior SA job in college managing a heterogenous mess of Digital Unix, Solaris, SunOS 4.1.x, NetBSD, Irix, Mac and NT. I've always been a programmer (writing scripts to make the machine do a thing) rather than a software developer (designing objects and abstractions to make an extendable platform). I wrote thousands of lines of (retrospectively) awful perl and cfengine scripts to keep Amazon and Real Networks running, but nothing that would be considered "software". At the tender age of 39 I got sick of operations, and couldn't see myself landing another job in ops, and I was impatient with the devops revolution and so I switched to being a full time SDE and learning ruby and new rules like "use composition instead of inheritance" and started noodling on rails and angular websites in my spare time. The shift was midly terrifying at times, but not as radical as I expected. I think most of the barrier between programmer and software dev is mostly in learning a different *human* language, and it takes immersion in the culture to do it. For me, I don't learn well through theory and reading dissertations on a subject but more from rolling up my sleeves and trying it until it breaks, which is why I had a really hard time becoming a software dev without actually being a software dev. I have to sit down and struggle with writing 1,000s of lines of ugly rspec code before I understand why too much collaboration in an object model is bad -- its because you wind up with the unit tests for the file provider. A lot of software development isn't very creative at all, its just having experience, so that you can see that a bit of code is bad because its bad the way that other pieces of code were bad before it. And learning from more senior devs who have already written all the bad code and learned to write better code and then just applying those patterns to the code you write in the future, and a lot of incremental improvement rather than any kind of insightful genius. A year or two later you just notice that you can sit down at a computer and the code that you type up is naturally become much better.


Compare these two examples:


{% if grains['os'] != 'FreeBSD' %}
        - installed
{% endif %}

    {% if grains['os'] == 'FreeBSD' %}
    - name: /etc/motd
    {% elif grains['os'] == 'Debian' %}
    - name: /etc/motd.tail
    {% endif %}
    - source: salt://motd


if node['platform'] != "FreeBSD"
  package "tcsh"

motd_path = case node['platform']
  when "FreeBSD"
  when "Debian"

cookbook_file motd_path do
  source "motd"

From my perspective, the idea that the Salt example is somehow "easier" is just flatly insane (and I should mention that I'm not speaking for Opscode here, this is just my own opinion). You've still got a programming language in the Salt case or else where did those "if"'s and "endif" tokens come from? You're just bracketing those tokens in magical {% %} tokens that you now need to explain what those are to anyone who is learning the language. In fact now you have to explain that you're writing your config language in YAML and then go on to explain that when its not powerful enough they need to use the jinja templating language to get their jobs done. Now they have two things to learn, conceptually, and they have to context switch between all the leading -'s and trailing :'s in YAML markup and putting all the nasty programming language in between whatever template expansion operators {% %} that you are using.

I do not understand how this is even solving the stated problem at all. It does solve it for the absolutely simple and trivial (and largely totally useless) use cases, where you can do things without any programming language at all. People can post those use cases to Hacker News and its like "ah, I've installed and configured a simple apache server without a line of a programming language! winning!" while to do anything for a real job its just not going to be that easy. And once you reach outside of the trivial it becomes *obviously* worse to me. Both conceptually its worse in the overhead of context switching between two things, and there is substantially more of the magical mark-up in the YAML case compared to ruby. You are also training people to write a lot of "controller" code in a templating language which is exactly what we try to teach people not to do and keep templates simpler and more readable and with less control in them. I think if we just wait awhile people are going to start posting non-trival examples of Salt and Ansible that will make your eyes bleed. Our community cookbooks are awful in a lot of places precisely because they try to do so much, and I think if you translated those to jinja-YAML that you're going to see the readability deterioriate substantially. The problem there I think lies in this idea that the community cookbooks are necessarily the objectively right way of doing Chef.

So I don't get it. Even assuming you're right and people have vastly different capabilities, I think the argument here is between learning programming in the context of jinja-marked-up-YAML being somehow being better than just learning a programming language. Are there any books out there on a programming language that *start* with the templating language? Has anyone ever written an introductory chapter to a ruby language book that started with ERB? Again the config management world seems to have gone insane from my perspective. Really I think this is the last gasp of the people who will not learn a programming language as the tide is changing to absolutely needing to know how to at least "program" in order to do operations, and they're trying to find any way to keep from changing as long as possible when it *MUST* change. They don't have to learn how to design languages and write the next 'go' or 'rust' themselves, but just basic control structures in the language, and enough ruby to understand that case assignment that I did up there. I managed to get that much down in my first month or two at opscode. They don't need to learn about monkey patching or instance_eval or writing DSLs or the difference between a module and a class. And they might not understand what #each is doing, but they copypasta it so that it works (and after a few hundred times of doing that they might get a glimmer of what functional programming is... I was certainly uncomfortable with that kind of programming when I got here 2 years ago and I last weekend I just breathed a *huge* sigh of relief when I found underscore.js because all those #each, #map, #select, etc functions have become fundamental bits of my toolbox and writing c-style for loops in javascript was making me cry...)

Anyway, having changed from SE to SDE and coming from your same background, I'm skeptical of the premise of your argument. And even given your premise I'm skeptical that the proposed solution actually solves it for anyone in other than the most trivial of cases.

As a historical aside how did we even wind up in this situation anyway? Back when I was growing up 'system adminstrators' like Wietse Venema went off and wrote tcp_wrappers and postfix, and him and Dan Famer wrote SATAN/SAINT. Having SAs who were proud of not programming back then would have seemed bizzare to me. Back before then, there wasn't any good package management, and there wasn't even GNU autoconf when I started learning C and Unix. There was no "./configure; make; make install" we had to hack up Makefiles and at least choose our CFLAGs from all the different Unix variants like 3B2 that we'd never heard of before, and then cobble together our own CFLAGs based on the compiler errors we got because the version of Dynix that we had on the Sequent Symmetry wasn't in the Makefile yet. What happen to us in the late 90s? It seems like in the dotcom gold rush we hired warm butts in seats that could barely type 'lsof' at all and we wound up here...

And we were all naked, in the snow, pushing the compiler uphill both ways just so we could get tcsh installed, because bash hadn't even been written yet...

My concern really boil down to this: I don't want to say that there will be a 
simpler path that turns out to just be a ghetto - an underserved, less loved 
corner of Chef.
Then don't let it turn into a ghetto.  ;-)
The problem is then we have two things to support. We're going to get tickets which look like "i can't do this in the YAML syntax and I've inherited 8,000 lines of code from the guy who quit 2 months ago, please help" and we either wind up needing to implement some kind of functional parity in that dialect, or wind up telling the person that they need to rewrite everything in ruby. Cynically, maybe we can offer them a consulting gig so they can pay us to do that, but that spot doesn't seem like a very healthy space to wind up in. And if we go off and extend the dialect to attempt to get parity, which of the outstanding tickets of yours and everyone else's do we bump in order to fix the dialect? TANSTAAFL...

And all this is just my opinion, and I may just be a grumpy old man about it and the kids these days may just go nuts with their jinja-YAML... I still don't see it though. Noah *almost* made it through to me that there is a barrier to entry between the python community and chef because of the need to use ruby. I can see that. I certainly see that barrier from the other side as well. But at the same time I've managed to pick up javascript and angular in my free time because I was interested in that. I also don't see this as a solution because it would push chef into the situation where if a python dialect of chef was actually successful you'd now need to learn *2* programming languages to interact fully with the community. I also read hacker news and have some anxiety about 'go' and 'rust' and other languages, but I don't see how to get past the problem of building a barrier of needing to know N different languages to learn Chef.

Archive powered by MHonArc 2.6.16.