Hello Maxime,
On Wed, 8 Jul 2015 22:46:56 +0200
Maxime Brugidou
< >
wrote:
I don't know about the server side but wouldn't it be better to haveIf we want to use 'If-Match' HTTP header and 'Etag' value, those has
this feature directly usable from the Chef gem? Having something like
node.safe_save in addition to node.save instead of only implementing
this for the knife node edit command. This could help other gems use
the feature.
to be generated/validated on Server Side. It doesn't sound like good
idea to do it on client-side. It would generate many concurrency and
implementation-dependent problems (may clients in different versions)
The things that need to be done on client-side is : Remembering received
Etag, and sending IF-Match header whatever needed.
And should we do a safe_save or a save at the end of a Chef run?I would appreciate community/developers suggestions here.
Because that's where we get most of the concurrent edits: a Chef run
happening while the node gets edited, and doing a safe_save from
knife won't solve this. But do we want to make the Chef run fail for
that? Or retry?
Usually, situations like that are resolved in 'first-wins' way.
- If the knife save will finish first - chef-client run can fail
- If the chef-client will finish first - knife save will fail
How to respond to those failures are second problem.
Knife save can be re-tried by the operator. When the operator knows
what has to be changed - it can be applied to new object again.
Chef-client run can be re-run. I don't like any 'merging' idea because
the result of the chef-client run still bases on the old data.
I suppose we can threat it as just 'failed' chef-client run - just like
exception in process - we do not push result data to chef server.
Next chef-client run will base on the new object, and will probably
succeed.
What are your opinions ? Would you be able to help me implement
Etag/If-Match logic on chef server side ?
Regards,
On Jul 8, 2015 10:18 PM, "Rafał Trójniak"
< >
wrote:
On Tue, 7 Jul 2015 14:56:17 -0700
Noah Kantrowitz
< >
wrote:
On Jul 7, 2015, at 2:33 PM, Rafał TrójniakHello Noah,
< >
wrote:
On Mon, 06 Jul 2015 10:20:15 -0700This was discussed way back at the first community summit, but no
Lamont Granquist
< >
wrote:
Hello
On 07/01/2015 11:40 PM, Maxime Brugidou wrote:
I would like to stop using Chef nodes as file but use the newThe way to do this is to make sure only one agent on your
chefDK provision command with a special driver that would
"pick" a node from the firstboot pool (so basically my
"cloud" provider is the pool of firstboot nodes in Chef).
Without dealing with concurrent access to Chef provision,
this seem doable: to allocate a node I can "tag" a firstboot
node and delete it once the machine is ready.
But how to do this with concurrent access? It seems almost
impossible. And the way things are going with Policy files
will tend towards a separate git repo and provision cookbook
per policy, all sharing the same pool of firstboot nodes (for
now I don't use Policy files).
I wish I could have a way to "lock" a node or something like
that.
network can move the node between states. A simple design
would be to have the node responsible for publishing that its
done with firstboot by tagging itself and then the node.save
at the end publishes the write. Then write a simple web
endpoint which is your API to 'allocate' a new firstboot'ed
node. By centralizing it you don't have to worry about race
conditions between multiple clients all trying to get the same
node at the same time. You can then write command line tools
that talk to the endpoint you wrote to get a node, rather than
wanting a distributed lock that the CLI commands can grab on
the node object itself. If you've already got etcd or
something similar that you're using internally you could
probably use that instead.
Had anyone analysed lock-free and optimistic approach by using
'If-Match:' HTTP header on the write stage ?
The scenario would look like :
- Every object (node, role, environment) would have some token
(could be timestamp, or any other value changed on each edit)
- When the user invokes 'knife node edit' the version is sent to
client (possibly in HTTP Header)
- When the user edits the object, the value is stored somewhere
- When the user sends write API call to the server, it sends
'If-Match' header with value received in first call
- If the token matches the old one - the object is updated
- If the token does not match the old one - the update is
rejected.
That won't solve all the problems, but it will fix many of them
with (i suppose) less work and changes. Such behaviour would
also be non-braking change.
one has written the code. I'm sure it would be accepted if
someone sent in a patch though.
--Noah
Thanks for the information. Just spent some time on the analysis on
how to achieve that.
- On the knife side
- Extracting 'Etag' information from the response on the
Chef::REST
- Storing it during the edit (probably in Chef::Knife::NodeEditor
- Updating node from the NodeEditor class instead of Node class
- On the chef-server side
- I have completely no Idea where to start analysis from :/ I
hadn't touched Erlang in few months.
- We would generally require :
- Generating Etag (dynamically or storing it somewhere) for the
object (like node, environment...)
- Sending Etag in HTTP Response header when requested by client
- On Modification request, detecting 'If-Match' header
- Comparing received value with current Etag,
accepting/rejecting update
I hope that pattern will work for all objects (roles, nodes,
environments..)
Also this should work not generate braking change with older
chef-server/knife implementation.
Is there any way someone could help me with chef-server side ?
Regards,
--
Rafał Trójniak
WEB : http://trojniak.net/
:
Jid :
GPG key-ID : 9A9A9E98
ABC8 83DF E717 6B76 CE49
BAFD 4F6F 854F 9A9A 9E98
Archive powered by MHonArc 2.6.16.