Here's the problem:
IAM can't resolve to the granularity of only granting access to a
> Use amazon's IAM to create a user that ONLY has access to what you need, and store THAT api key/secret id in the databag.
user/group to operate on a specifc security group. It appears that as
long as that's the case, the right thing to do is to create the
security group, but stash the configuration template for the group
into revision control and have chef take care of applying
configuration to all instances' security groups from a central
location when changes need to be made.
-Peter
On Fri, Sep 16, 2011 at 4:37 PM, Aaron Abramson < "> > wrote:
>> Thanks for the input, I think that clarifies that this is possible.
>> One thing concerns me, though, and that is that this style of using
>> EC2 does require ec2 api keys be installed on every host, and that
>> means that if any host is compromised then the entire cloud can be
>> removed. If this could be managed centrally then that reduces the
>> proliferation of keys. Is there a method to proxy this sort of
>> meta-server management on either a second host, or on the chef server?
>
> I forgot to address this part...
>
> Write your controller cookbook to run the ec2 commandline tools giving the api-credentials at the commandline, and not through an environment variable. Then store your api-keys in a data bag, and use them in the cookbook when running your script resource that executes the ec2 tools.
>
> Use amazon's IAM to create a user that ONLY has access to what you need, and store THAT api key/secret id in the databag.
>
>
>
>
> On Sep 16, 2011, at 3:22 PM, Peter Norton wrote:
>
>> Questions in-line
>>
>> On Fri, Sep 16, 2011 at 4:02 PM, Aaron Abramson < "> > wrote:
>>> I would do it this way:
>>>
>>> 1) Launch EC2 servers with 3 security groups, default, chef, and unique (unique being unique for that server)
>>> - Remember, your server has to allow ssh access from where you're working, and I prefer to keep the 'default' SG as amazon internal only, thus the 'chef' SG.
>>>
>>
>> Good point. I was using default as a shorthand for "the things
>> everything needs", but it should be "default,common,sg-0xDEADBEEF"
>> where "default" should be everything off, and "common" should include
>> anything that all servers need (ssh being the most likely, or ssh on
>> an alternate port for example).
>>
>>> 2) Make sure server has ec2 tools installed, and can update security group settings.
>>>
>>
>> Ec2 tools? Would knife (with modules to handle security group
>> manipulations) be sufficient? I'm pretty sure the AMIs we use include
>> the tools, but they're slooooow.
>>
>>> 3) Update all cookbooks which install/configure services to open necessary ports in the 'unique' security group.
>>>
>>
>> So once we decide on our group manipulation tool, use that tool in the
>> recipes? OK.
>>
>>> Now when you install your http server of choice, part of the server setup is to open up the ports used in the security group. You can store the 'unique' security group name as a node attribute.
>>>
>>> I would suggest creating an EC2-controls cookbook, and make "sg-add" and "sg-del" LWRP's for use by your other cookbooks. That way you simply need to tell the apache cookbook:
>>>
>>> sg-add "apache ports" do
>>> group node[:ec2][:unique_group]
>>> tcp_ports [ "80", "443" ]
>>> udp_ports nil
>>> end
>>>
>>> This keeps it idempotent, only the packages/services which are installed get added to your security group. And you don't have to dink around with SG templates or anything else. Let the security group be built/configured 'on-the-fly'.
>>>
>>
>> Thanks for the input, I think that clarifies that this is possible.
>> One thing concerns me, though, and that is that this style of using
>> EC2 does require ec2 api keys be installed on every host, and that
>> means that if any host is compromised then the entire cloud can be
>> removed. If this could be managed centrally then that reduces the
>> proliferation of keys. Is there a method to proxy this sort of
>> meta-server management on either a second host, or on the chef server?
>>
>> -Peter
>>
>>> On Sep 16, 2011, at 2:22 PM, < "> > wrote:
>>>
>>>> Hello,
>>>>
>>>> I'm new to chef, but I'm trying it out as a way to manage amazon ec2
>>>> instances, and I've found some places that I think chef can help manage,
>>>> but which it can't completely manage yet.
>>>>
>>>> My first challenge comes from the fact that it seems like security groups
>>>> are inflexible, and can be limiting.
>>>>
>>>> Specifically a use case that would be nice is to be able to create a new
>>>> system, and have it attached to a security group after it's been allocated to
>>>> a service. Since services require ports to be opened, this would mean
>>>> changing security groups. However, since ec2 doesn't allow for security
>>>> groups to change after a reservation is created, one way to get this
>>>> behavior would be to have a security group created per-host.
>>>>
>>>> I think I see how to do this with knife, so I think I can add the necessary
>>>> knife ec2 command to create a group, and maybe tie the same function
>>>> into creating an instance to make my use case easier (1 command instead
>>>> of 2).
>>>>
>>>> However, the second part that makes this useful is to have the host's
>>>> security group be patterned off of an existing group, or off a configuration
>>>> that chef maintains. Either way, the important point is that changes to
>>>> that template reflected in all groups that are patterned off of it.
>>>>
>>>> One issue with this is that I don't see clearly how chef could be used to
>>>> manage a security group in the way I envision it should work. Here's what I
>>>> imagine:
>>>>
>>>> 1) Knife creates a new security group using a pseudo-random name - let's say
>>>> 'sg-0xDEADBEEF' for this example.
>>>> 2) The instance's reservation is created with two security groups:
>>>> default,sg-0xDEADBEEF
>>>> 3) The instance is created, and chef is bootstrapped
>>>> 4) The host is created with an role called "available". The "available"
>>>> role includes the default security group's settings, so these are cloned
>>>> into sg-0xDEADBEEF's settings and now it looks the same as default.
>>>> 5) The server is needed an hour later, so it's prepared for a deployment by
>>>> being assigned the roles "http,https,tomcat-2121" which each include the
>>>> security groups needed as well as the packages. The installation of apache,
>>>> along with laying out conf files for port 80, port 443, and port 2121, the
>>>> tomcat instance, java, etc. get pumped out, but each role also depends on an
>>>> associated security group, let's say sg-http, sg-https, and sg-tomcat-2121,
>>>> which allow port 80, port 443, and port 2121 respectively.
>>>>
>>>> If later on the tomcat-2121 also needs port 2122 for some reason then
>>>> port 2122 would be added to the recipe, and sg-0xDEADBEEF would have
>>>> that port added as part of the regular chef operation.
>>>>
>>>> So... is it possible to do this with chef? If someone can point me to any
>>>> examples of similar situations (e.g. a configuration that is related to a host,
>>>> managed along with that host, but is not part of the host) that I
>>>> could use as a
>>>> starting point, I'd be quite willing to point my zero knowledge of ruby at this
>>>> problem, and use fog to make this happen.
>>>>
>>>> Thanks,
>>>>
>>>> -Peter
>>>
>>>
>
>
Archive powered by MHonArc 2.6.16.