[chef] Re: omnibus thoughts

["John E. Vincent (lusis)" < >]

Comments inline

On Wed, Sep 4, 2013 at 5:53 PM, Seth Falcon < " target="_blank"> > wrote:

Lusis,

A few comments below...

lusis.org+ "> writes:
> Omnibus is obviously very focused on the idea of building fat packages for
> Opscode's various products. Not a big deal and totally understandable. It
> feels like at this point though, omnibus has grown into a product in its
> own right and has a community entirely outside of Chef users (folks are
> packaging puppet this way, for instance). I feel like there's a bit of
> conflict in those two things.

I think we'd all benefit from Omnibus being generally useful. The
conflict I'm aware of here at Opscode is one of available time. And it
is true that, at present, building mind share around omnibus is not a
top priority.

Yeah I think conflict might have been to harsh of a word. Sorry about that.

 

> Right now, the omnibus-software repo is the master source/fallback for all
> things. I haven't looked to see if this is hardcoded into omnibus itself or
> if I'm missing something simple to override that. The software definitions
> in omnibus-software are always going to be designed to be for
> building/packaging all the bits of a chef-server/chef-client package.
> They're carefully curated to ensure they all play nice with each other.
>
> This is great for reuse/community as long as the omnibus-software repo
> versions of packages are what you need. But if they aren't, you have to
> start copying out individual software files into your own project. If you
> have multiple projects internally that you build with omnibus, you have to
> duplicate that effort across all the projects.
>
> So I have a proposal that I think would at least help part of that (and
> leads to a bigger question):
>
> - Extended the project DSL to support defining an alternate source at the
> project level as well as at the individual dependency level. Obviously the
> sane default here (and sane default for fallback) is the opscode
> omnibus-software repo. Something like this (in config/projects/foo.rb)

Extending things to allow specifying a list of omnibus software repos
such that package search does PATH style lookup in the repo list sounds
like a nice enhancement.

This would be the simplest MVP kind of thing to implement I think while the specific semantics and needs get fleshed out.

 

> name 'foo'
> master_source '<some git uri or even a remote tarball>, :branch => 'master'
>
> dependency 'erlang', :source => '<http url or an alternate repository>'
> dependency 'nginx' # checks local 'software' dir first -> checks
> 'master_source' -> checks opscode upstream

If you can specify the location on a per dependency basis, what happens
when two different things depend on 'nginx' from two different sources?
Is the per dependency thing something you are sure you need? Seems
easier to reason about a list of software repos defining a software
package search path.

I think I ended up mixing two use cases here and didn't define it clearly. It would be nice to be able to define a dependency as a gist raw url source for instance. Ostensibly it's no different than me curling that raw gist url but the overall idea was that you could, by marking a dependency as coming from a versioned source somehow, "freeze" things a bit. Right now I'm always pulling from the opscode "master".

Then again this smells like a road that every damn language, project, framework and tool on the planet has been down. Maybe simply opting in to a search path at the project level does make more sense. Definitely much easier to implement. I'm not too keen on implementing a proper dependency graph here ;)

Thinking out loud here... I could see a desire to be able to install
different versions of the same thing in a given package. It's unclear to
me if that would be best managed within omnibus or by the user
(e.g. create packages nginx-1.2, nginx-2.0).

Yeah this is a use case I've thought about but haven't really fleshed out. I like what you're thinking though.

> The questions are if opscode would merge something like this functionality
> being that omnibus is so sensitive to the needs of building chef packages
> first and foremost?

The short answer is: yes, this sounds like a good change and we'd love
to see omnibus be more generally useful.

If our omnibus projects contined to work with the patch, that would make
merging it much easier.

> "The first version of this project used Opscode's tool, but they don't seem
> to take pull requests, so I enhanced bernd's superb fpm-cookery to create
> Omnibus packages, and switched this project to use it." (from here:
> https://github.com/andytinycat/puppet-omnibus/blob/master/README.md)

The context of that was very much: we open sourced a project as soon as
we could, but were super busy with six other projects and just couldn't
prioritize looking at PRs at the time. That sucks, I know, but it was
absolutely not a "we don't take pull requests" and instead a "we opened
this project while we were very busy and didn't plan ahead on responding
to PRs for it" thing.

Apologies because I realize now that section sounded totally harsh on you all. You didn't have to open a lick of code and I think everyone would say they're grateful for it being opened up. 

 

Cheers,

+ seth


--
Seth Falcon | Development Lead | Opscode | @sfalcon