[chef] Re: Re: Simultaneous node edits

[Rafał Trójniak < >]

On Tue, 7 Jul 2015 14:56:17 -0700
Noah Kantrowitz 
<
 >
 wrote:

> 
> On Jul 7, 2015, at 2:33 PM, Rafał Trójniak 
> <
 >
>  wrote:
> 
> > On Mon, 06 Jul 2015 10:20:15 -0700
> > Lamont Granquist 
> > <
 >
> >  wrote:
> > 
> >> 
> >> 
> >> On 07/01/2015 11:40 PM, Maxime Brugidou wrote:
> >>> I would like to stop using Chef nodes as file but use the new
> >>> chefDK provision command with a special driver that would "pick" a
> >>> node from the firstboot pool (so basically my "cloud" provider is
> >>> the pool of firstboot nodes in Chef). Without dealing with
> >>> concurrent access to Chef provision, this seem doable: to allocate
> >>> a node I can "tag" a firstboot node and delete it once the machine
> >>> is ready.
> >>> 
> >>> But how to do this with concurrent access? It seems almost
> >>> impossible. And the way things are going with Policy files will
> >>> tend towards a separate git repo and provision cookbook per
> >>> policy, all sharing the same pool of firstboot nodes (for now I
> >>> don't use Policy files).
> >>> 
> >>> I wish I could have a way to "lock" a node or something like that.
> >>> 
> >>> 
> >> The way to do this is to make sure only one agent on your network
> >> can move the node between states.  A simple design would be to
> >> have the node responsible for publishing that its done with
> >> firstboot by tagging itself and then the node.save at the end
> >> publishes the write.  Then write a simple web endpoint which is
> >> your API to 'allocate' a new firstboot'ed node.  By centralizing
> >> it you don't have to worry about race conditions between multiple
> >> clients all trying to get the same node at the same time.  You can
> >> then write command line tools that talk to the endpoint you wrote
> >> to get a node, rather than wanting a distributed lock that the CLI
> >> commands can grab on the node object itself.  If you've already
> >> got etcd or something similar that you're using internally you
> >> could probably use that instead.
> >> 
> > 
> > Hello
> > 
> > Had anyone analysed lock-free and optimistic approach by using
> > 'If-Match:' HTTP header on the write stage ?
> > 
> > The scenario would look like :
> > - Every object (node, role, environment) would have some token
> >  (could be timestamp, or any other value changed on each edit)
> > - When the user invokes 'knife node edit' the version is sent to
> > client (possibly in HTTP Header)
> > - When the user edits the object, the value is stored somewhere
> > - When the user sends write API call to the server, it sends
> > 'If-Match' header with value received in first call
> >  - If the token matches the old one - the object is updated
> >  - If the token does not match the old one - the update is rejected.
> > 
> > That won't solve all the problems, but it will fix many of them
> > with (i suppose) less work and changes. Such behaviour would also be
> > non-braking change.
> 
> This was discussed way back at the first community summit, but no one
> has written the code. I'm sure it would be accepted if someone sent
> in a patch though.
> 
> --Noah

Hello Noah,

Thanks for the information. Just spent some time on the analysis on how
to achieve that.
- On the knife side
  - Extracting 'Etag' information from the response on the Chef::REST
  - Storing it during the edit (probably in Chef::Knife::NodeEditor
  - Updating node from the NodeEditor class instead of Node class
- On the chef-server side
  - I have completely no Idea where to start analysis from :/ I hadn't
    touched Erlang in few months.
  - We would generally require :
    - Generating Etag (dynamically or storing it somewhere) for the
      object (like node, environment...)
    - Sending Etag in HTTP Response header when requested by client
    - On Modification request, detecting 'If-Match' header
      - Comparing received value with current Etag, accepting/rejecting
        update

I hope that pattern will work for all objects (roles, nodes,
environments..)
Also this should work not generate braking change with older
chef-server/knife implementation.

Is there any way someone could help me with chef-server side ?

Regards,
-- 
Rafał Trójniak
WEB : http://trojniak.net/

 
 : 

 
Jid : 

 
GPG  key-ID : 9A9A9E98
ABC8 83DF E717 6B76 CE49
BAFD 4F6F 854F 9A9A 9E98

Attachment: pgpJ94ENAZdtQ.pgp
Description: OpenPGP digital signature