[chef] Re: Antwort: Re: Antwort: Re: Testing idempotence of chef cookbooks

[Ranjib Dey < >]

yes, you can. But as roland is pointing out, you can have non-idempotent resources, i.e. resource that are always updating (like execute, bash, ruby_block), this does necessarily not mean something is wrong, neither it means they cant be made idempotent (with proper resource guards).

even if there are some resources which are always being updated, you can white-list them , since they are known, and then check for any additional resources (as you dont want new ones).

note. we are equating idempotency as  `not being updated in every run`,  but its lot more complicated than that. Sean Omeara wrote some stuff on idempotency vs convergence.

for all practical purpose, you can just drop statsd or something similar metrics about updated resources, and alert on increased rates (i.e. first derivative).

 

http://blog.afistfulofservers.net/post/2011/03/16/a-brief-chef-tutorial-from-concentrate/  

On Thu, May 21, 2015 at 11:45 PM, < " target="_blank"> > wrote:

Ok, I see. Well maybe I understood something wrong, but from my understanding idempotence is the property when you apply a cookbook several times and the result is always the same. So I always thought it is enough to run converge multiple times and see if the serverspec tests are always the same. That is also something you can easily automate for a CI environment.

The property that only resources are updated that are really needed is more some kind of efficiency of cookbooks, isn't it?

And as you described it is not always possible to have "0 resources updated" so I wonder if such idempotence tests can be standardized or automated at all.





Von:        Roland Moriz < " target="_blank"> >
An:         " target="_blank">
Datum:        22.05.2015 04:34
Betreff:        [chef] Re: Antwort: Re: Testing idempotence of chef cookbooks

---

Am 21.05.2015 um 17:30 schrieb Ranjib Dey < " target="_blank"> >:

yeah, its should be 0 resources updated if everything is idempotent and already converged. those two numbers relfect number of updated resources / number of total resources. Total resources is sum of all resources defined by all recipes, lwrps as part of the expanded run list, while updated resources are a subset of them which was modified (like absent file being created, absent packaged being installed etc) during the chef run.


Some resources are very hard to make „chef-idempotent“, e.g. ruby blocks. Even if you make sure your ruby code is idempotent, the execution of the block already counts as „updated resource“. Even popular community cookbooks do have that "problem“.  

You probably want to add each converge result to a temp file and calculate/check the difference or some kind of percentage instead.

regards
Roland




On Thu, May 21, 2015 at 8:01 AM, < " target="_blank"> > wrote:
Isn't your way the same as running the converge command twice and looking at the last line of the chef-client output? If so, this way sounds a little bit easier for me.

       Chef Client finished, 45/61 resources updated in 16.28070151 seconds

If so, what is really the goal of testing idempotence? Should I really have a "0 resources updated"?
By the way, what do the two numbers mean actually?

regards,
Johannes




Von:        Roland Moriz < " target="_blank"> >
An:         " target="_blank">
Datum:        20.05.2015 13:10
Betreff:        [chef] Re: Testing idempotence of chef cookbooks

---

Hi,


Am 20.05.2015 um 10:21 schrieb " target="_blank"> :

Hi,

is there a general way or best practice for testing idempotence of chef cookbooks?


An approach might be to use test-kitchen + duplicate suite names + a custom chef-handler that writes the number of updated resources to disk + serverspec to verify it.

e.g. .kitchen.yaml

-8<---
suites:
 - name: default
   run_list:
     - recipe[chef-handler]
     - recipe[build-essential]
   attributes:

 - name: default
   run_list:
     - recipe[chef-handler]
     - recipe[build-essential]
   attributes:
-8<---

this will converge the node twice on „kitchen converge“.

An example for a simple chef handler: https://github.com/jtimberman/chef-handler-updated-resources/blob/master/lib/chef/handler/updated_resources.rb - use https://supermarket.chef.io/cookbooks/chef_handler to deploy the handler to your testing node.

Instead of logging to Chef::Log you could just write the number of updated resources to a log file e.g. /tmp/updated_resources and use http://serverspec.org/resource_types.html#file to verify.

hth

best regards
Roland