[chef] Re: Managing an ec2 instance and security group together with chef?

[Aaron Abramson < >]

I would do it this way:

1) Launch EC2 servers with 3 security groups, default, chef, and unique 
(unique being unique for that server)
- Remember, your server has to allow ssh access from where you're working, 
and I prefer to keep the 'default' SG as amazon internal only, thus the 
'chef' SG.

2) Make sure server has ec2 tools installed, and can update security group 
settings.

3) Update all cookbooks which install/configure services to open necessary 
ports in the 'unique' security group.

Now when you install your http server of choice, part of the server setup is 
to open up the ports used in the security group.   You can store the 'unique' 
security group name as a node attribute.

I would suggest creating an EC2-controls cookbook, and make "sg-add" and 
"sg-del" LWRP's for use by your other cookbooks.  That way you simply need to 
tell the apache cookbook:

sg-add "apache ports" do
        group node[:ec2][:unique_group]
        tcp_ports [ "80", "443" ]
        udp_ports nil
end

This keeps it idempotent, only the packages/services which are installed get 
added to your security group. And you don't have to dink around with SG 
templates or anything else.  Let the security group be built/configured 
'on-the-fly'.


On Sep 16, 2011, at 2:22 PM, 
<
 >
 wrote:

> Hello,
> 
> I'm new to chef, but I'm trying it out as a way to manage amazon ec2
> instances, and I've found some places that I think chef can help manage,
> but which it can't completely manage yet.
> 
> My first challenge comes from the fact that it seems like security groups
> are inflexible, and can be limiting.
> 
> Specifically a use case that would be nice is to be able to create a new
> system, and have it attached to a security group after it's been allocated 
> to
> a service.  Since services require ports to be opened, this would mean
> changing security groups. However, since ec2 doesn't allow for security
> groups to change after a reservation is created, one way to get this
> behavior would be to have a security group created per-host.
> 
> I think I see how to do this with knife, so I think I can add the necessary
> knife ec2 command to create a group, and maybe tie the same function
> into creating an instance to make my use case easier (1 command instead
> of 2).
> 
> However, the second part that makes this useful is to have the host's
> security group be patterned off of an existing group, or off a configuration
> that chef maintains.  Either way, the important point is that changes to
> that template reflected in all groups that are patterned off of it.
> 
> One issue with this is that I don't see clearly how chef could be used to
> manage a security group in the way I envision it should work.  Here's what I
> imagine:
> 
> 1) Knife creates a new security group using a pseudo-random name - let's say
> 'sg-0xDEADBEEF' for this example.
> 2) The instance's reservation is created with two security groups:
> default,sg-0xDEADBEEF
> 3) The instance is created, and chef is bootstrapped
> 4) The host is created with an role called "available".  The "available"
> role includes the default security group's settings, so these are cloned
> into sg-0xDEADBEEF's settings and now it looks the same as default.
> 5) The server is needed an hour later, so it's prepared for a deployment by
> being assigned the roles "http,https,tomcat-2121" which each include the
> security groups needed as well as the packages.  The installation of apache,
> along with laying out conf files for port 80, port 443, and port 2121, the
> tomcat instance, java, etc. get pumped out, but each role also depends on an
> associated security group, let's say sg-http, sg-https, and sg-tomcat-2121,
> which allow port 80, port 443, and port 2121 respectively.
> 
> If later on the tomcat-2121 also needs port 2122 for some reason then
> port 2122 would be added to the recipe, and sg-0xDEADBEEF would have
> that port added as part of the regular chef operation.
> 
> So... is it possible to do this with chef?  If someone can point me to any
> examples of similar situations (e.g. a configuration that is related to a 
> host,
> managed along with that host, but is not part of the host) that I
> could use as a
> starting point, I'd be quite willing to point my zero knowledge of ruby at 
> this
> problem, and use fog to make this happen.
> 
> Thanks,
> 
> -Peter