It’d be absolutely stellar if the Chef Server also provided a universe endpoint; that would obviate our need for a Berkshelf API server entirely, especially since we don’t have any particular need for our own Supermarket right now. The alternative of having Supermarket baked-in to the Chef Server would also be welcome.
On the larger topic of Chef providing us with The Way to author cookbooks, I agree with Dylan’s points that we don’t necessarily need a One True Way, but some more direction on a Few Good Ways would be very welcome. And I wholeheartedly agree that the website examples should be solid examples.
On July 8, 2014 at 1:20:57 PM, Lamont Granquist (
">
) wrote: On 7/7/14, 8:05 PM, John E. Vincent (lusis) wrote:
> FWIW I didn't have a problem with Berkshelf originally. One: I didn't
> use it and didn't HAVE to use it. Two: It didn't have this entirely
> new service it depended on.
>
> Now I can't avoid it and it requires a separate internet service
> (though now apparently rolled into the supermarket) just to be used.
Just addressing this one issue...
I hated the Berkshelf API with a passion initially. I still think its a
hack, but its a hack around problems that needed to be solved. The
first problem is just speed. Berkshelf needs to depsolve against all
the cookbooks on the community site, and doing that with a single
request for every cookbook was slow. So the /universe endpoint was
created to slurp it all down. That endpoint is now integrated with the
community site, and what is going on is essentially exactly what happens
when you use 'gem install' and it goes and hits rubygems to pull down
the list of current gems so that the rubygems depsolver can figure out
what gems to install. Its that problem, and its now solved in a very
similar fashion. There's another obvious source of cookbooks -- the
chef server -- that you should be able to depsolve against as well, and
we should add a universe endpoint (probably
/organizations/<org>/cookbook_universe or something) to the chef-server
to solve it.
The other problem is one dealing with metadata.rb and metadata.json that
has its roots in configuration-as-code. The problem is that if you ship
around metadata.rb and it has pure-ruby code in it that references
external data you will not necessarily get the same metadata.json result
when you evaluate it. There are two chef tickets on this:
https://tickets.opscode.com/browse/CHEF-4810
https://tickets.opscode.com/browse/CHEF-4811
There's also Jamie's blog post on it:
http://blog.vialstudios.com/the-importance-of-compiled-metadata/
And there's my issue that I opened against berkshelf where you can see
it tied all together:
https://github.com/berkshelf/berkshelf/issues/1126
The result of that leads to the lack of being able to use transitive
github dependencies and the need to serve your own API server. The use
most important case which is driving it is an important one which is
that people are using ruby code in metadata.rb to drive their version
numbers off of CI/CD (e.g. jenkins) and in that case the artifacts need
to have compiled metadata.json so that all the actors involved in the
system agree on the version number in the metadata. Since CI/CD is
important, we don't want to break that.
Eliminating the need for 'finalized artifacts' and API servers may not
be a use case that you care about, but we can't just break the users
that do care about it because you don't. My initial reaction to the
problem is just to do a (tableflip) on configuration-as-code and think
to myself that we should deprecate metadata.rb and mandate using
metadata.json (back-to-the-future time). There might be a better way to
address the problem, though, which looks at the CD/CI use case and fixes
their version numbering use case a better way. I'm somewhat optimistic
that in the future that the berks API server will have completely gone
away.
I'd also be interested in solutions like baking supermarket into every
chef server deploy. Then everyone would have a local repo to push
cookbooks to and a /universe endpoint. Then that just becomes your
local berks API server.
So, if I've got any point here its that I'm very grouchy about the berks
API server because I see it as something which is a barrier to entry of
berkshelf adoption and its a problem that need to get solved. But
being grouchy about it without understanding why its there (Jamie is
pretty smart, he didn't just write it because he felt like writing some
server code) is not being productive. We need to understand the reasons
why things are there, and the use cases that drive them and come up with
solutions to the problems, rather than just complain. Just complaining
about the berks API is bad signalling since it just causes flak to the
people working on berkshelf. What you want to signal is the chef
server team that they should integrate the /universe endpoint, or that
supermarket should be bundled with chef server, or other solutions like
that. What the chef server team hears right now is that Berkshelf sucks,
which sounds entirely like a Berkshelf problem. That's not useful in
helping them to determine what they could do better with the server code
or packaging to help solve that problem -- and I'm fairly convinced that
this is an ecosystem problem, and that Berkshelf is what it is because
its been developed in isolation from work going on in the client and
server, and that needs to change in order to make it better.
|