[chef] Re: Re: Launching spot instances?

[Mike < >]

James,

All the points about spot instances you reference are correct.

What I think I was trying to convey is that the behavior currently of
knife-ec2 is to perform the requested steps (provision, ssh,
bootstrap) in sequence, and exit upon completion.

How would you propose to do that with spot instances that are
_not_granted immediately? The request at the price is still made, and
will open up when available, but may not launch right now. There also
may be more than 1, and knife

Would the operator expect the shell session to remain open until _one_
spot instance is launched, so it can then perform bootstrapping, role
assignment etc? That is not a very tempting scenario to me.

I think a better use case for the _utterly specific_ use case of spot
instances would be to leverage AWS's api and model, potentially
pre-bake your own AMI with a default chef client.rb and validation.pem
for your organization (and roles?) , so that when a spot request
launches, it can boot, register with chef, get the latest cookbook
code and run until it's terminated.

That's why I don't think this is a good fit for `knife ec2 server create`.

On Fri, Oct 26, 2012 at 3:09 PM, Jeffrey Hulten 
<
 >
 wrote:
> I could see a "knife ec2 stack create" that used a default cloud formation 
> stack json and passed in the parameters. This could be customized then to 
> run spot instances.
>
> The issue with knife and spot instances is that they can take up to 15 
> minutes to start up. Basically (as I understand it) there is a batch job 
> that takes the pool of bids and the available capacity and determines what 
> to start and what to kill on a scheduled basis. That means that your 
> request for spot instances cannot be instantaneous. This is not a model 
> that the knife ec2 plugin can support.
>
> --
> Jeffrey Hulten
> Principal Consultant at Automated Labs, LLC
> 
 
>   206-853-5216
> Skype: jeffhulten
>
> On Oct 25, 2012, at 6:56 AM, Mike wrote:
>
>> While the underlying library fog may provide this functionality, this
>> actually seems less useful to be part of the knife-ec2 command set.
>>
>> `knife ec2 server create` attempts to create an instance, wait for it
>> to come up, bootstrap it via ssh, and start up chef-client (usually).
>>
>> From what I garner, placing a spot request with a bid provide is not
>> ensured to be delivered at the time pf placing the request, and
>> therefore the flow of what "ec2 server create" does today would not be
>> the same, rather it seems like it would be: "If I can get a spot
>> request now, please give it to me, otherwise...?" Fail? The bid
>> request was probably placed, so this isn't a failure per se.
>>
>> The other part of why I think this might not be a great idea is:  Spot
>> instances come up and down whenever their bid price is met or outbid
>> respectively. This can cause easy buildup of cruft in your node/client
>> list if you are allowing these to auto-register/validate themselves,
>> and then have to clean up after them once they are gone.
>>
>> All in all, I think spot instances have a use case, but I don't know
>> if during ec2 server create (and subsequently bootstrap phase) makes
>> sense to support this.
>>
>>
>> On Wed, Oct 24, 2012 at 1:44 PM, Christopher J. Bottaro
>> <
 >
>>  wrote:
>>> How can I launch spot instances using "knife ec2 server create"?  I don't
>>> see any relevant command line switches.
>>>
>>> Thanks for the help.
>