[chef-dev] Re: Re: Re: Dialect support and loading enhancements

[Avishai Ish-Shalom < >]

To me Chef's biggest power was its flexibility; i can easily monkey patch/wrap whatever cookbook/internal chef behavior, and i'm not the only doing this. just look at the number of cookbook that use rubygems or include internal chef mixins.

As someone working with various CM tools (chef, puppet, salt) and multiple programming languages, i always found chef to be more coherent and easy *because* it's all ruby. most programmers will agree that the biggest cognitive overhead is not the syntax of a programming language but rather the behavior of its constructs which are 95% standard library and frameworks. salt will not behave like chef even if both accept the same yaml catalog, ditto for puppet. adding another syntax will slightly increase cognitive overhead but as "crippled chef" user base groes it will lead to different cookbook patterns and different behaviors - and that will be a big cognitive overhead.

On top of this, consider that templating is applying logic to data, aka programming. If i have to program, i want tools, tests and libraries - which are the hallmarks of an evolved eco-system; if a significant number of users will use "cripple chef" you will have less eco-system, not more.

In many ways this reminds me of the age old "lisp vs everyone else" debate: at some point, the data IS the program. 

my 2cents.

On Fri, Sep 20, 2013 at 6:10 AM, Noah Kantrowitz < " target="_blank"> > wrote:

On Sep 19, 2013, at 7:39 PM, Adam Jacob < "> > wrote:

> We have not even scratched the surface of people who are already programmers, from an opscode customer base point of view. Of course, Chef isn't about Opscode, not really - but god bless you for thinking about it. :)

My worry isn't about customers per se, honestly its pretty selfish. I like Chef, I want to write my fix-the-world stuffs in or with Chef, and as such I need to get more people (in my case a lot of Python people) to use Chef so that I don't have to go duplicate my work in Ansible or whatever is The Thing at that moment.

> My concern is you won't get 80% of what you want out of it - you're going to get 20% of it. What's capable with a dialect vs the native ruby is hugely different - and it's going to drift naturally. I think Noah is smart to say that the low risk choice here is something like YAML - it's essentially just serialization, so it might be able to stretch relatively naturally with a serialized resource catalog. But if you compare it to what you will experience with a tool for whom the template+YAML approach is truly first class, the delight won't be the same.
>

I just want to be clear, I haven't said that. I added JSON and Yaml for attributes files, _not_ recipes. I don't think Yaml for recipes belongs in core for exactly the reasons I mentioned re: difficulty translating knowledge. I could certainly imagine a mapping, but needing non-declarative stuff in recipes is pretty close to a given, whereas in attr files it only crops up in very advanced cookbooks, usually in service of cross-platform unification. I do think the JS and Python dialects will be able to maintain a good portion of the expressive power of the Ruby DSL, but I don't think those belong in core just due to complexity of maintenance and general long road to stability (you can currently segfault Ruby from your Python code if you try and recursively load libffi for example). If a fully declarative (JSON/Yaml) dialect is added for recipes I think the target wouldn't be humans, but auto generated recipe code. I don't see why you would do that though, so I left it out. Other than that it would only work well for suuuuuuper simple cases, which do happen, but I don't have a clear picture of how thats evolving. I could certainly see a future where more and more stuff moves in to LWRPs and app-ish cookbooks really could be fully declarative invocations of those LWRPs, but we are a long way from that.

> It's interesting to hear you say you feel like you don't know how to program, when i've seen you be an effective member of our community for a long time now. Is it completely out of school to say that perhaps you're arguing for your limitations? You might not be a CS major - lord knows, I'm not. But you do know how to program, and you're doing it in ruby. :) The closest you've come to a real programming language is Ruby: you're programming in it, and have been the entire time you've rocked the Chef, an you've done so successfully.
>
> My concern really boil down to this: I don't want to say that there will be a simpler path that turns out to just be a ghetto - an underserved, less loved corner of Chef. I think we've made similar mistakes in the past, and they've hurt us (for example, the unintentional impacts Opscode's focus has had on the chef solo experience being improved upon) as a community.

Agreed, it would mean a lot of documentation work to ensure mobility between different formats. In my head I'm imagining something like how multi-lang SDK docs work sometimes with a format radio button at the top of the page which switches things within the page so you can see how the syntaxes interact, but maybe that is overkill.

>
> Having someone like Noah who is committed to seeing it through goes a long way to solving that concern, in that I totally believe he would be an active maintainer of that functionality.
>
> Thinking out loud: what if we wrapped it in a feature flag and marked it experimental?

I think because its already opt-in and there is no change to the Ruby formats this might not accomplish much. The only way a new user will find out this exists is via docs and blog posts and the like, and I think the "here be dragons" would belong at that level, rather than someone trying to use the new feature and either getting a message of "please pass --dialects=yaml,py to continue" (which is one of those just-do-what-I-mean rage inducing situations) or silent failure.

--Noah