[chef-dev] Re: Re: Re: MySQL, chef-solo and node.save

[John Vincent < >]

On Wed, Oct 3, 2012 at 2:50 PM, Jay Feldblum 
<
 >
 wrote:
> John, Alex,
>
> Chef isn't really an interactive or back-and-forth conversation between
> recipes and the node data. The Chef model is much more linear than that.
>

I wouldn't disagree with that per se.

> As you know, the general Chef model is that you specify the whole abstract
> configuration (run-lists, node-data) up front, and then recipes just expand
> it out by declaring resources based on the abstract configuration, and then
> chef finishes the job by converging the server to match the declared
> resources.
>

Still with you.

> If you need to generate a password during one run and save it on the server
> for use during the next run, you can use a simple trick as follows:
>
> You can have a well-known password-storage file wherever you like (under
> /var/chef or wherever else you like). During the recipe run ("compilation
> phase") you can test for the existence of the file. If it does not exist,
> you generate a new password into a local variable and write it to the file
> (still during the "compilation phase"); that's what happens during the first
> run. If it does exist, you read the password from the file into a local
> variable; that's what happens during all subsequent runs. After that step,
> you can always copy the password into the node data, but of course now
> there's no longer any need to save the new node data, because it will always
> be recomputed exactly with the same deterministic input and with the same
> deterministic algorithm on every subsequent chef run.
>

Here's where I'd disagree with you. If a node existed in totality on
its own, that would make AWESOME sense. Let's take a scenario though.

I have a DB server. I have an app server. Under the current Chef model
(and the design of the official Opscode cookbooks) is that the node
NEEDS the resource, declares the resource. If I need a db created, a
migration run or a user created the consumer of that resource (in this
case the app server) does that work under its Chef cookbooks. Mind you
I'd argue that the creator of that resource should be the DB server.
However that's just not possible in the current Chef world. I could
use Noah to do the orchestration but even I'd admit it's kludgy. What
we essentially are faced with his how to declare a resource on another
node from the node that needs it.

So in our case, we have the declaration of databases and app users in
the recipe for each application. That makes sense in a logical
grouping. The DB Server role is responsible for standing up MySQL and
creating the base state. The application role is responsible for
creating the databases it needs. But to do that, it needs to know the
root password. The root password that lives on the db server - under
the node data for that node.

Sure we can refactor a bit and move the application specific DB
information into a role that runs on the db server but that doesn't
fit how some folks think - that the database doesn't belong to the db
server but to the application using it. I've done it both ways and I
honestly don't know which way is entirely correct but it's much easier
to minimize the need for chef runs to be coordinated between nodes.

Did that make any sense at all cause I'm having to reread what I wrote myself 
;)

> Cheers,
> Jay
>
> On Wed, Oct 3, 2012 at 4:19 PM, Alex Soto 
> <
 >
>  wrote:
>>
>> I'm surprised that issue is still around.  I hit that years ago and
>> just ended up not using solo since.
>>
>> However, I'd vote for writing to a 'new' json file and that the
>> original node.json and the new one are merged during the run  with the
>> new json overriding node.json.  That way the original is not messed
>> with and you have the final end result available as a json file for
>> future runs.
>>
>> Alex
>>
>>
>> On Wed, Oct 3, 2012 at 12:51 PM, John Vincent
>> <
 >
>>  wrote:
>> > Hey all,
>> >
>> > Fletcher Nichol, Timberlake (jk) and I were bantering back and forth
>> > on the twitters originally about the mysql cookbook blowing up on
>> > failed initial runs on chef-solo (all related to the preseeded
>> > passwords). Fletcher evidently has a PR coming about explicitly
>> > failing out on chef-solo unless you specify all the passwords in your
>> > node json. That's a bit of a different discussion. What I wanted to
>> > noodle out is the possibility of a chef-solo friendly node.save. There
>> > were few options that came up:
>> >
>> > - Write back to the node json used during the call
>> > - Write somewhere else to disk
>> >
>> > The first one is fraught with edge cases and violating expectations
>> > when mutating that file. What if it's under version control? What if
>> > it's immutable in some way? The list goes on. I think we can sort of
>> > agree it's a bad idea. So the second one came up. My thought is that
>> > node.save when called under chef-solo should dump the node attributes
>> > to the cache path as something like node-data.json (or possibly
>> > something a bit more identifiable if you want to utilize the node data
>> > between chef-solo runs on other nodes.
>> >
>> > I realize that chef-solo will never replace server. I don't think
>> > Opscode would have objections to community-originated PRs for adding
>> > some kind of functionality like this? It would, however, allow us to
>> > unify how we create recipes that are friendly to both solo and server.
>> > It also has, as Ranjib Dey pointed out on twitter, immense benefit to
>> > the various testing initiatives out there.
>> >
>> > I'd also like to bring up a use case for reference that we have at
>> > enStratus:
>> >
>> > Much like OPC (as I understand it), we use chef-solo as our
>> > "installer". enStratus is fairly modular and while the default mode of
>> > installation is to collocate everything on a single machine, obviously
>> > that doesn't work. We typically support a single, two and four node
>> > configuration of our stack for customer installs. Right now we use a
>> > bit of a wrapper script and some manual error-prone data editing to
>> > perform the two and four node installs. What would be nice is if we
>> > could choose to persist node data between runs - primarily because of
>> > the mysql cookbook thing - but also because we need to track some very
>> > simple things like the mysql server IP but some passwords and
>> > encryption keys. I having each node's data persisted to
>> > cache/node-state/node{1-4}.json and loaded would be a godsend for
>> > making these things more idempotent.
>> >
>> > Anyway, just looking for feedback/thoughts.
>> >
>> > Thanks
>> > John E. Vincent
>> > @lusis
>
>