[chef-dev] Re: Re: Re: REVIEW: Ohai Cloud Plugin Improvements (OHAI-542)

[Cary Penniman < >]

The cloud specific Mash structure (i.e. node['ec2']) can drastically vary from cloud to cloud.  Both ec2 and gce plugins follow their metadata tree structure -- which, IMHO, is a good thing.  

The problem I see with cloning the structure from the cloud specific Mash to node['cloud'], is that the keys will differ depending on which cloud you are running.  For example, getting the ip address running on EC2 would be: 

  node[:cloud][:public_ipv4]

whereas on GCE it would be something like:

  node['cloud']['network']['networkInterface'][0]['accessConfiguration'][0]['externalIp']

In this case, you would still need to add conditional logic like "if node.has_key?('ec2')", but now the recipe is harder to understand unless you know about the magic cloning that the cloud plugin does. Even if we wanted all the cloud plugins to produce the exact same Mash structure (or some subset), it would be difficult to enforce across all plugins when reviewing a simple Pull Request because the code would reside in multiple, seemingly unrelated files.

The cloud plugin today already creates a common Mash structure (or some subset) that is reviewable in a single file, but since it is an unstructured Hash it is very easy to miss subtle differences. The pull-request I have submitted uses a simple ruby object to help enforce the structure of the cloud Mash.  It does so by taking responsibility for writing the Mash itself.  By insuring a consistent interface, we should avoid the need to use of conditional logic like "if node.has_key?('ec2')" and the like.

Hopefully some of that makes sense.

Cheers!

On Fri, Dec 27, 2013 at 10:51 AM, Scott M. Likens < " target="_blank"> > wrote:

Hi,

I know my 2cents are likely only worth 1 hay penny.  But here goes.

Overloading node['cloud'] with virtually all the keys possible from node['ec2'] or node['rackspace'] could go either way in several regards.

• Instead of having nested loops of if node.has_key?('ec2') # for example with following if's for rackspace .. ..
• Make 1 central key base for all 'cloud based' keys.  This is likely a pro, not a con but your perspective may change this opinion.

If we cloned the objects from node['ec2'] into node['cloud'] memory allocation should not be affected much; which means we would not do away with node['ec2'] it would just have it's attributes cloned into node['cloud'].

I have never used the instance size for anything meaningful in chef; instead I typically use node['cpu'] (like node.cpu.total).  Maybe I just haven't had the opportunity :)

I imagine this is something good to have on the Radar making chef DRY (as well as useable) is always a win.

Scott

On 12/27/13, 9:50 AM, Cary Penniman wrote:

Joseph,

Thank you for the reply.

The meanings behind instance-id, instance-size and metadata are fairly specific to a given IaaS orchestration layer (aka "cloud).  For example, how does an "1GB Standard" Rackspace instance compare to a "m1-small" EC2 instance?  Typically using these values in a meaningful way will require cloud specific logic -- at which point, one might as well use the cloud-specific node['rackspace'] or node['ec2'] plugin values directly.  

The idea behind the "cloud" plugin (ironically?) is to provide cloud-agnostic details about the VM (like ip address and hostname) in a constant location in order to avoid adding logic that switches on node['cloud']['provider'].  Of course if you are writing more advanced recipes, like creating cloud volumes, you will likely need cloud specific stuff in your recipes.  But for simple things like setting up a public LAMP server you shouldn't.

The proposal below is an attempt to define the node['cloud'] key "interface" in a more rigorous way, to avoid the cloud-specific "key-creep" that has occurred over time.  This interface can absolutely be extended to provide other keys that are generic but provided/managed by a cloud orchestration layer.

In fact, reporting a list of attached volumes (e.g. ['/dev/xvde', '/dev/xvdf']) is an interesting idea.  I think something similar for ephemeral devices might also make sense.   The other items you mentioned I suggest we add to the rackspace plugin.

Best regards,

Cary P

On Mon, Dec 23, 2013 at 5:52 PM, Joseph Holsten < " target="_blank"> > wrote:

We have a need for much better info from the rackspace plugin in the near future. Notable things we're wanting:
- instance id
- instance size/flavor
- base image
- metadata
- attached volumes

Is it worth shooting for this across IaaS providers, and ultimately getting into node['cloud']?

On 2013-12-23, at 23:51, Cary Penniman < " target="_blank"> > wrote:

> Ohai!
>
> Just submitted a PR to help make the ohai cloud plugin more consistent across clouds:
>
> https://tickets.opscode.com/browse/OHAI-542
>
> There are some breaking changes for GCE and Azure as documented.
>
> Thoughts? Questions? Concerns?
>
> Best regards,
> Cary P
>