[chef] Re: Re: Re: Re: Re: Automatically starting a new EC2 instance with Chef

["Jeffrey E. Sussna" < >]

Re ELB, I have other use cases in mind. Nice to be able to bring an instance 
up to date using Chef on an auto-scale event, instead of baking everything 
into the AMI. But relying on a grace period seems iffy.

Good to know about knife bootstrap.


On Jul 5, 2011, at 5:47 PM, Edward Sargisson wrote:

>> Very nice. I presume that putting everything in the user data script is 
>> what makes ELB wait for the chef run to finish before it considers the 
>> instance in service?
> 
> No - the ELB is set with a grace period before it starts testing for
> up-ness. Mine's set for 12 minutes or something because a standard
> chef run for my config is about 10 minutes.
> However, for your requirements you probably don't need the ELB unless
> you particularly need load balancing in test.
> 
>> The question is how to dynamically assign roles to nodes. If I understand 
>> your solution (and chef) correctly, whatever creates the user data script 
>> could dynamically set the value of role[] in the runlist. Then, when 
>> chef-client registers the instance with chef-server, it would say "hi, my 
>> name is regression-box-42, and my role is regression-suite-23", which 
>> would cause chef to configure it with the appropriate software for 
>> "regression-suite-23". Am I right?
> 
> Yes. If you look in the user data script you will see that it there is
> a section where the roles are. At the end there is a setting to set
> the environment on chef (-E qa).
> So, for your requirements you could write a script to write the user
> data script you want and then call the AWS command line client to
> start it.
> 
> However, the easiest way would probably be to use knife bootstrap. You
> can pass the run list into that - it will call AWS to launch the
> instance, ssh in to configure chef then fire the chef-client.
> Mind you, the user-data method has the advantage that you could write
> in some node variables. i.e. setting a test version for your chef
> config to load.
> 
> Feel free to ask more questions if I haven't been clear.
> 
> Cheers,
> Edward
> 
> On Tue, Jul 5, 2011 at 2:59 PM, Jeffrey E. Sussna 
> <
 >
>  wrote:
>> Very nice. I presume that putting everything in the user data script is 
>> what makes ELB wait for the chef run to finish before it considers the 
>> instance in service?
>> 
>> I think your post may also answer a question I was about to pose to the 
>> group. I want to use chef/EC2 to build an on-demand regression test 
>> environment. When an instance gets tipped up, it doesn't know what 
>> regression suite it will run, so it doesn't know what software needs to be 
>> installed (the suite itself, plus needed libs and exes). The question is 
>> how to dynamically assign roles to nodes. If I understand your solution 
>> (and chef) correctly, whatever creates the user data script could 
>> dynamically set the value of role[] in the runlist. Then, when chef-client 
>> registers the instance with chef-server, it would say "hi, my name is 
>> regression-box-42, and my role is regression-suite-23", which would cause 
>> chef to configure it with the appropriate software for 
>> "regression-suite-23". Am I right?
>> 
>> On May 28, 2011, at 12:30 AM, Edward Sargisson wrote:
>> 
>>> Hi all,
>>> I followed the advice (and made great use of Avishai's user data
>>> script) and now have my site running so that AWS Elastic Load
>>> Balancing will automatically detect an instance going down and bring
>>> another back up - configured from the standard OS image using Chef.
>>> 
>>> I hope I don't breach the standards of this list by including a link
>>> to the blog post I've written which details all the steps.
>>> http://www.trailhunger.com/blog/technical/2011/05/28/keeping-an-amazon-elastic-compute-cloud-ec2-instance-up-with-chef-and-auto-scaling/
>>> 
>>> Many thanks,
>>> Edward
>>> 
>>> On Sat, May 14, 2011 at 12:36 PM, Avishai Ish-Shalom
>>> <
 >
>>>  wrote:
>>>> If you want to achieve this goal with the minimum amount of work, create
>>>> a launch configuration with a user data script that installs ruby,
>>>> installs chef, write the validation certificate, chef config and
>>>> bootstrap json then launch chef. Ubuntu images are bundle with
>>>> cloud-init which is able to read shell scripts from user data. In other
>>>> words, edit the attached script and use that as the user data for new
>>>> instances or autoscaling groups.
>>>> 
>>>> BTW, a native chef plugin for cloud-init is waiting to be merged. If and
>>>> when it's included, launching ubuntu images with chef will become
>>>> ridiculously easy.
>>>> 
>>>> Regards,
>>>> Avishai
>>>> 
>>>> 
>>>> On 14/05/11 16:55, Edward Sargisson wrote:
>>>> 
>>>>> Hi all,
>>>>> What tools and services are available to automatically start a new EC2
>>>>> instance with Chef?
>>>>> 
>>>>> My site has just one server and yesterday it failed. Unfortunately, I
>>>>> was at my day job and couldn't do anything about it. It's configured
>>>>> with Chef so starting up a new server with knife then using Opscode
>>>>> Platform to put the roles on was reasonably easy (it could be easier
>>>>> but I have some issues to fix).
>>>>> 
>>>>> What I would like is for some service to realise that my server is
>>>>> down and then call Chef to setup a new one. Amazon Auto Scaling almost
>>>>> does this - the issue is that it merely starts an AMI and doesn't do
>>>>> anything cleverer than that. My Chef setup assumes a brand new Ubuntu
>>>>> image and goes from there. I suppose that, for future use, whatever
>>>>> solution I use needs to also listen to the CloudWatch metrics to scale
>>>>> up and down when required.
>>>>> 
>>>>> So:
>>>>> Do I write an image which bootstraps chef automatically?
>>>>> Or is there some other way to solve this problem?
>>>>> 
>>>>> Thanks,
>>>>> Edward
>>>> 
>> 
>>