[chef] Re: Re: Re: Re: Re: Re: Re: Re: Managing an ec2 instance and security group together with chef?

[Aaron Abramson < >]

I'm still not understanding the aversion to using ec2 api tools on the node 
to modify a security group.  Using the command ec2-authorize / ec2-revoke, 
you can add/remove additional ports/services to your instances.  And if each 
instance has a unique instance-specific security group, it makes it quite 
easy to make sure that each node/instance has the appropriate ports open and 
available for its services.


Whether you're using Amazon security groups, or IPtables firewall rules on 
the servers directly, creating a security cookbook with enable/disable LWRP's 
to be called from your other cookbooks will give you the best possible 
solution.  If you move away from amazon, and use a different firewalling 
solution, you simply need to update your LWRP's to handle the new firewalling 
mechanism.


Regarding concerns for security key storage, the official opscode aws 
cookbook (which is used to bind elastic IP's and create/mount EBS volumes) 
says to store your AWS security credentials inside of a databag, which 
resides on the chef-server.  The key is read and used when the command is 
executed, and not stored on the instance itself.


The idea of having templates and having a cron run on a "master" server to 
update firewall rules and/or security groups based on what node has what 
roles is inefficient, and can cause delays/problems.  Especially when you get 
into different instances running different versions of the same cookbook 
which require different firewall rules. 

I'm experiencing a similar conceptual issue in regards to my splunk 
deployment. Do I let chef use a LWRP to tell the splunk-forwarder running on 
each node which inputs to watch, called from each recipe, or do I execute a 
cron on the splunk deployment server where chef updates the deployment conf 
file with which nodes belong in which input group...




On Sep 20, 2011, at 10:03 AM, Peter Norton wrote:

> I'm going to try to get this thread back on-track.  Here are some
> notes (not in a forwarded bit so people using gmail will see it by
> default):
> 
> Here are some thoughts:
> 
> 1) We may want to move [to another service than AWS] at some point, but I
> think that it may be off-topic.  To me, managing aws security groups
> with chef is
> orthogonal to managing firewall rules on another platform since it
> doesn't make sense to e.g. default to using a security group for
> everything that allows connections from everywhere to all ports.
> Someone else may disagree, but that just adds complexity to this
> environment.
> 
> Someone else may want to do this differently, but either way, we want
> to centralize the configuration and apply it to groups of hosts.  I'm
> asking this question to be able to control policy, that's all.
> 
> 2) Regarding boto vs. knife + fog, I'm trying to be agnostic and do
> this within knife so we only need one tool.  However, I don't see an
> example of how this is supposed to work.  This doesn't seem to create
> a group, for instance (it's based on the pieces I think I need from
> ec2_create_server.rb):
> 
>     def run
>       $stdout.sync = true
> 
>       secgroup_def = { :name => config[:name], :description =>
> config[:description], :ip_permissions => nil, :owner_id => nil }
> 
> 
>       conn = Fog::Compute.new(
>           :provider => 'AWS',
>           :aws_access_key_id => Chef::Config[:knife][:aws_access_key_id],
>           :aws_secret_access_key =>
> Chef::Config[:knife][:aws_secret_access_key],
>           :region => locate_config_value(:region)
>         )
>       # conn = connection.create()
>       secgroup = conn.security_groups.new(secgroup_def)
> 
> Does anyone know of a place I can look for an example of how to do
> this?  I don't want to spend much more time on this now, and I'm
> pretty sure I can do this in python+boto in about an hour, but I'd
> prefer to not have to write yet another tool.
> 
> -Peter
> 
> 
>> On Fri, Sep 16, 2011 at 9:51 PM, John E. Vincent (lusis)
>> <
 >
>>  wrote:
>>> On Fri, Sep 16, 2011 at 8:20 PM, James 
>>> <
 >
>>>  wrote:
>>>> Just want to point out that instead of working around all of these 
>>>> issues,
>>>> you could simply use something other than security groups. This approach
>>>> would be vendor agnostic, and allow you to transition off of EC2 if you 
>>>> ever
>>>> need to.
>>>> 
>>>> James
>>>> 
>>> 
>>> I was considering doing an EC2 security-group LWRP till I heard
>>> rumblings of a firewall cookbook that managed not only iptables but
>>> security groups and other forms of "firewall"ing.
>>> 
>>> For what it's worth, all this talk of jclouds and boto and shelling
>>> out is confusing. Amazon has a perfectly good ruby AWS SDK. Fog
>>> exists. You'll get +9000x greater flexibility doing it in an LWRP or
>>> as a knife plugin than trying to add yet another tool to the mix.
>>> 
>>> Am I missing something here?
>>> 
>>