[chef] Re: Re: Re: Chef Server vs. Not Guidance

[Daniel DeLeo < >]

On Wednesday, August 5, 2015 at 4:54 AM, Kelly Brownsberger wrote:

> Thanks Torben and Daniel - this was very helpful. I appreciate you taking 
> the time to respond.
>  
> Your comment about generating the environment json is interest... I've been 
> considering doing that. I was surprised to find that all cookbooks and 
> their versions have to be specified at the enviornment json level in order 
> to have versioning. But after thinking more about it, I guess that makes 
> sense... however it presents a maintenance problem. We already have a 
> pretty tight CI pipeline with strict versioning methodologies in place. It 
> doesn't make sense for us to maintain this list in an environment json file 
> when our Jenkins server already know this information.

You wouldn’t really be maintaining it, so much as producing a build artifact 
based on the information in the Jenkins server. You’d just have a script that 
takes the info from Jenkins and converts it to the format that the Chef 
Server understands, and you’d have Jenkins run this script as part of your 
build. As long as you only ever move data from Jenkins to the Chef Server, 
you will just have one source of truth and a well defined process for syncing 
that data. It does make using environment attributes more difficult, but that 
can be worked around in a number of ways depending on your needs/priorities. 
Just think of it as being like the step in your current workflow where you 
run `berks package` — hypothetically, you could unpack the tarball, and make 
some edits, but you just don’t do that.

> I'm still a bit confused on Nodes though. Here is a use case we're faced 
> with now: We're trying to automate our CI infrastructure - i.e. the build 
> out of a Jenkins Master and various flavors of Jenkins Agent machines. Part 
> of the master setup is configuring it to manage one or more agent machines. 
> We would very much like to have a single source of truth for this 
> information and not maintain it in multiple files. At a conceptual level it 
> seems to need to have some way from the recipes responsible for building 
> out the master machine be able to query for all chef nodes that are of a 
> certain role (or some means for knowing which machine have or will be 
> provisioned as an agent). How would one approach this? In our current 
> chef-server-less world it's not clear to me how you would define these node 
> in json files and then make them available for querying at chef-client 
> runtime. Does embracing chef-server make this problem any easier?
This is what the search feature of the Chef Server was created to do (well, 
one use case, anyway).


If you want to stay serverless, then you would maintain this data in some 
other system that you can query programmatically, the simplest being a 
JSON/YAML file you maintain by hand, with more effort you could write your 
own web service to do it. Alternatively, you could look into using a service 
discovery tool (e.g., etcd) and then integrating it with chef-client.

I know of one implementation which is built on top of a web service (which 
provides a bunch of other features) where they create minimal node JSON for 
every system in the cluster so that search mostly works for basic use cases, 
though it doesn’t have the full range of attributes available to search so 
it’s not 100% the same as the chef search you’d get with a Chef Server. And 
again, they already have a node database to start from.

As a final note, your workflow sounds like it could be an excellent fit for 
Policyfiles. They’re kinda new, but getting to “reasonably complete” very 
quickly. This is a draft of a blog post that I’m planning to put on the Chef 
blog once ChefDK 0.7 comes out, which should give you a better idea of how 
they work: https://gist.github.com/danielsdeleo/9112b6c7932452bdb7e8 Relevant ;
to this conversation is that all the version locking type stuff you’d have to 
script yourself would be done automatically by the `chef` tool during the 
development process, so you’d just confirm that it works in Ci and then 
publish it.

--  
Daniel DeLeo