[chef] Re: Re: Request for input on orchestration

[Hedge Hog < >]

On Sun, Jan 30, 2011 at 6:27 PM, Hedge Hog 
<
 >
 wrote:
> On Sat, Jan 29, 2011 at 8:19 AM, Dan Nemec 
> <
 >
>  wrote:
>> Mandatory disclosure. We've been using Control Tier for over a year.
>> I don't know that I am interested in Chef being the orchestration engine. I
>> would like to see Chef more friendly with other existing orchestration
>> tools. Control Tier is great if you want to code much of your customized
>> commands into the tool. The spinoff RunDeck is great if you have your own
>> scripts but just need them orchestrated.
>> We envision a scenario where Chef is mainly configuration and Control Tier
>> is orchestration. We would go so far as having Chef configure Control Tier
>> which could then call back to Chef to initiate Chef actions.
>> Orchestration is a difficult problem and Control Tier/RunDeck is already
>> fairly mature. Now that they support arbitrary workflows in Jobcenter
>> Control Tier is't missing any feature we require for orchestrating our very
>> complex deployments.
>> I don't know that I have much to add to your questions above. We're fairly
>> new to Chef and it being Friday afternoon is making my head hurt thinking 
>> of
>> how to incorporate orchestration into Chef. All I can imagine is how we're
>> doing it today with Control Tier.
>> 1) You hit the high points in your overview of scope.
>> 2/3) All I can say is to read Control Tier's documentation. In Control Tier
>> you define a "command" that performs a piece of work. You string a bunch of
>> commands together into a workflow. Then you can also string different
>> workflows into larger events. You add to that the error-handling and gating
>> so that the next step only executes after the first one completed
>> successfully. Threading concurrently across servers is mandatory to make it
>> go fast.
>> Happy Friday,
>> Dan
>>
>> On Fri, Jan 28, 2011 at 3:26 PM, Chris Walters 
>> <
 >
>>  wrote:
>>>
>>> Ohai Chefs!
>>> We're in the preliminary stages of designing possible solutions for
>>> orchestration and would like to understand the community's
>>> requirements.
>>> I'm going to write down my thoughts and questions. Nothing is gospel,
>>> so please feel free to comment on everything, including the framing.
>>> Background:
>>> Chef, as currently conceived, does a great job of exposing a model for
>>> how to get a system from either an embryonic state or a slightly
>>> misconfigured state to the desired state, mainly via the mechanism of
>>> resource idempotence.
>>> What I think is not yet well-modeled is how to go from one
>>> well-configured state to a completely different well-configued
>>> state. It also doesn't yet model synchronization of actions across
>>> multiple boxes in that there isn't a first-class way to gate actions
>>> that are dependent on the completion of steps on other servers. For
>>> example, a complex migration or deployment might require bringing
>>> boxes up or down, copying data, cleanly removing artifacts or services
>>> installed by previous chef runs, not restarting load balancers until
>>> some quorum of webservers have re-started, etc.
>>> We'd like to collect the use cases, requirements, and thoughts that
>>> best serve the community.
>>> 1) What do you think the scope of orchestration is and is not?
>>> 2) What are the use cases that you would like to see an orchestration
>>> system/DSL accommodate? The more specific and granular the steps of
>>> the orchestration, the better. (If you would not like your use case
>>> made public but would nonetheless like it considered during design,
>>> validation, and testing, please send it to me directly at
>>> 
 )
>>> 3) What generic primitives do you think would be useful in such a
>>> system?
>>> Thanks!
>>> Chris Walters
>>
>
> Orchestration is something I have yet to deal with.  It will loom and
> I had in mind to try and use Ruote[1].
> RunDeck sounds like it might be a specialized (aka limited?) workflow 
> engine.
> Given ruote-kit (RESTful)[2], ruote-jig[3] decision tables[4],
> persistence, ampq, etc. etc. I'd be surprised if solving Chef
> orchestration didn't involve adopting/employing a subset of ruotes
> existing functionality.
> As mentioned any chef specifics could be placed in a ruote-chef project?
> I think a route-chef project might give biggest bang-for-the-buck.
>
> [1] http://openwferu.rubyforge.org/
> [2] https://github.com/kennethkalmer/ruote-kit
> [3] https://github.com/jmettraux/rufus-jig
> [4] http://jmettraux.wordpress.com/?s=decision+tables

I know this is getting very long in the tooth...
In case anyone is still interested in this topic... RightScale have an
interesting blog on a Ruote+Amazon's Simple Workflow:

http://blog.rightscale.com/2012/02/22/rightscale-server-orchestration-and-amazon-swf-launch/

HTH


>
> HTH
>
> --
> πόλλ' οἶδ ἀλώπηξ, ἀλλ' ἐχῖνος ἓν μέγα
> [The fox knows many things, but the hedgehog knows one big thing.]
>   Archilochus, Greek poet (c. 680 BC – c. 645 BC)
> http://wiki.hedgehogshiatus.com



-- 
πόλλ' οἶδ ἀλώπηξ, ἀλλ' ἐχῖνος ἓν μέγα
[The fox knows many things, but the hedgehog knows one big thing.]
  Archilochus, Greek poet (c. 680 BC – c. 645 BC)
http://hedgehogshiatus.com