[chef] Re: Installing large numbers of packages

[Haselwanter Edmund < >]

On Sep 1, 2011, at 7:54 AM, Adam Jacob wrote:

> To answer your original question, you use a loop, not pass an array to the 
> resource.

and I think this is how it should be. this is dsl vs. language. using chef is 
programming ruby, using puppet is using the puppet dsl (ok, things are 
changing)

   package %w{foo bar baz} do
       action :install
   end

might be a good shortcut in the dsl, but this comes with syntax clutter

imagine you would like to pass in exact versions for each package, then 
package would have to
support arrays of arrays (in this implementation), and if it does not you 
have to patch the resource 

   package [["foo","0.3"], ["bar","0.4.5"], ["baz",nil]]} do
       action :install
   end

if you allow for ruby in the recipe you can leave that up to the user:

[["foo","0.3"], ["bar","0.4.5"], ["baz",nil]].each do |pkg,pkg_version|
   package pkg do
       action :install
       version pkg_version if pkg_version
   end
end

all ruby, no special chef syntax

this is one reason why I favor the chef way

> 
> The only reason that the namevar being an array is "the obvious thing" is 
> because you are a previous puppet user :)
> 
> I'm not going to get involved in what appears to be a kind of crazy CM 
> trench war for the rest of the thread. Suffice it to say, in my opinion and 
> experience, you are likely to benefit from approaching things in Chef 
> differently than you did in puppet. The big ones:
> 
> 1) Much more of what you do will be data driven, and use the way we compose 
> node attributes to your benefit. This will be less common in Solo, but even 
> then, it's very handy.
> 
> 2) Several of the big bottlenecks for scaling Puppet are solved with Chef 
> Server - namely, we don't execute any arbitrary code or deal with compiling 
> the catalog, rendering templates, or anything else on the server. We 
> basically pass data in and back out again. That said, rock on with whatever 
> way works best for you.
> 
> 3) Chef solo does support data bags as of 0.10.4.
> 
> Best,
> Adam
> -- 
> Opscode, Inc.
> Adam Jacob, Chief Product Officer
> T: (206) 619-7151 E: 
> 
 
> 
> 
> On Wednesday, August 31, 2011 at 5:47 PM, Matt Palmer wrote:
> 
>> On Thu, Sep 1, 2011 at 10:34 AM, Brad Knowles 
>> <
 
>>  
>> (mailto:
 )>
>>  wrote:
>>> On Aug 31, 2011, at 7:08 PM, Matt Palmer wrote:
>>>> There's also resilience issues, a strong aversion to centralisation,
>>>> and too many painful memories of Puppet scaling nightmares to get
>>>> over.
>>> 
>>> Well, all CM systems are about centralization, regularization, 
>>> categorization, and management of information, so I don't think you're 
>>> going to get away from that. In the case of Chef, what you're doing is 
>>> trying to get all this information about your internal systems & network 
>>> infrastructure recorded into a reliable and version-controlled CM system.
>> 
>> Configuration management doesn't imply reliability on a giant central
>> server infrastructure that's going to have to be scaled and managed
>> itself.
>> 
>>>> Even without that, though, I'm having trouble working out how it's
>>>> better to have a list of packages in one place, and a resource
>>>> specification that installs those packages somewhere else. I can
>>>> almost convince myself that putting attributes in an external JSON
>>>> file makes sense for roles (although I think it's codifying the same
>>>> mistake that practically everyone makes using Puppet, where you define
>>>> a pile of global variables and cross your fingers that everything
>>>> works, rather than having locally-passed parameters that define how
>>>> you want to use something Here and Now), but making a list somewhere
>>>> external just so I can avoid having to walk an array is insane. My
>>>> recipe says "this is how you configure a workstation", and the list of
>>>> packages you have to install in order to do that should be in that
>>>> recipe.
>>> 
>>> I think it comes down to a separation of "code" from "data". You should 
>>> put your "code" into a code repository, but when the data that the code 
>>> is going to be operating needs to be changed, you shouldn't necessarily 
>>> have to change the code just to accommodate the change in the data.
>> 
>> My system configuration is *all* data... "this is what I want to
>> happen". A list of packages is no more code or data than the fact
>> that I want those packages installed, and not removed. I want to
>> revision control it all.
>> 
>>> The installation script should be simple and easy to read, regardless of 
>>> how many packages are being installed -- that information should come 
>>> from the database. And when all that is changing is the data, it should 
>>> work as you want with the existing code that is already in place on all 
>>> your nodes.
>> 
>> The installation script *is* simple and easy to read, if the syntax is
>> appropriate.
>> 
>>> When you're talking about a small number of nodes to be managed, I'm not 
>>> sure that this makes such a difference. But as you try to scale up, it's 
>>> going to become more and more important that you keep this separation 
>>> between code & data.
>>> 
>>> So, do you want to learn the right way from Day One, or do you want to 
>>> learn a single-file method that you will have to unlearn as you try to 
>>> scale up?
>> 
>> And this is where I feel like I should stop listening to you, because
>> you assume I've never managed large scale systems. I've done 500+
>> nodes with Puppet, and 2,500+ systems under management in bodgy
>> semi-manual ways.
>> 
>> If there are better ways to do it with Chef, I'm open to them, but I
>> *do* have plenty of experience in this field, and so far my
>> experiences are telling me that the way Chef does it is a monumental
>> pain in the arse at scale. However, I'm willing to learn that I'm
>> wrong, so point me at the documentation that explains clearly and
>> simply why the Chef way works better.
>> 
>>> Maybe your problem with CM systems isn't with the systems themselves, but 
>>> with the way you're trying to use them -- or maybe misuse them?
>> 
>> Perhaps. Feel free to show me where I'm wrong, but with specifics,
>> not platitudes.
>> 
>> - Matt
> 
> 

--
DI Edmund Haselwanter, 

 ,
 http://edmund.haselwanter.com/
http://www.iteh.at ;| http://facebook.com/iTeh.solutions ;| 
http://at.linkedin.com/in/haselwanteredmund ;