[chef] Re: Re: Request for input on orchestration

[Hedge Hog < >]

On Sat, Jan 29, 2011 at 8:19 AM, Dan Nemec 
<
 >
 wrote:
> Mandatory disclosure. We've been using Control Tier for over a year.
> I don't know that I am interested in Chef being the orchestration engine. I
> would like to see Chef more friendly with other existing orchestration
> tools. Control Tier is great if you want to code much of your customized
> commands into the tool. The spinoff RunDeck is great if you have your own
> scripts but just need them orchestrated.
> We envision a scenario where Chef is mainly configuration and Control Tier
> is orchestration. We would go so far as having Chef configure Control Tier
> which could then call back to Chef to initiate Chef actions.
> Orchestration is a difficult problem and Control Tier/RunDeck is already
> fairly mature. Now that they support arbitrary workflows in Jobcenter
> Control Tier is't missing any feature we require for orchestrating our very
> complex deployments.
> I don't know that I have much to add to your questions above. We're fairly
> new to Chef and it being Friday afternoon is making my head hurt thinking of
> how to incorporate orchestration into Chef. All I can imagine is how we're
> doing it today with Control Tier.
> 1) You hit the high points in your overview of scope.
> 2/3) All I can say is to read Control Tier's documentation. In Control Tier
> you define a "command" that performs a piece of work. You string a bunch of
> commands together into a workflow. Then you can also string different
> workflows into larger events. You add to that the error-handling and gating
> so that the next step only executes after the first one completed
> successfully. Threading concurrently across servers is mandatory to make it
> go fast.
> Happy Friday,
> Dan
>
> On Fri, Jan 28, 2011 at 3:26 PM, Chris Walters 
> <
 >
>  wrote:
>>
>> Ohai Chefs!
>> We're in the preliminary stages of designing possible solutions for
>> orchestration and would like to understand the community's
>> requirements.
>> I'm going to write down my thoughts and questions. Nothing is gospel,
>> so please feel free to comment on everything, including the framing.
>> Background:
>> Chef, as currently conceived, does a great job of exposing a model for
>> how to get a system from either an embryonic state or a slightly
>> misconfigured state to the desired state, mainly via the mechanism of
>> resource idempotence.
>> What I think is not yet well-modeled is how to go from one
>> well-configured state to a completely different well-configued
>> state. It also doesn't yet model synchronization of actions across
>> multiple boxes in that there isn't a first-class way to gate actions
>> that are dependent on the completion of steps on other servers. For
>> example, a complex migration or deployment might require bringing
>> boxes up or down, copying data, cleanly removing artifacts or services
>> installed by previous chef runs, not restarting load balancers until
>> some quorum of webservers have re-started, etc.
>> We'd like to collect the use cases, requirements, and thoughts that
>> best serve the community.
>> 1) What do you think the scope of orchestration is and is not?
>> 2) What are the use cases that you would like to see an orchestration
>> system/DSL accommodate? The more specific and granular the steps of
>> the orchestration, the better. (If you would not like your use case
>> made public but would nonetheless like it considered during design,
>> validation, and testing, please send it to me directly at
>> 
 )
>> 3) What generic primitives do you think would be useful in such a
>> system?
>> Thanks!
>> Chris Walters
>

Orchestration is something I have yet to deal with.  It will loom and
I had in mind to try and use Ruote[1].
RunDeck sounds like it might be a specialized (aka limited?) workflow engine.
Given ruote-kit (RESTful)[2], ruote-jig[3] decision tables[4],
persistence, ampq, etc. etc. I'd be surprised if solving Chef
orchestration didn't involve adopting/employing a subset of ruotes
existing functionality.
As mentioned any chef specifics could be placed in a ruote-chef project?
I think a route-chef project might give biggest bang-for-the-buck.

[1] http://openwferu.rubyforge.org/
[2] https://github.com/kennethkalmer/ruote-kit
[3] https://github.com/jmettraux/rufus-jig
[4] http://jmettraux.wordpress.com/?s=decision+tables

HTH

-- 
πόλλ' οἶδ ἀλώπηξ, ἀλλ' ἐχῖνος ἓν μέγα
[The fox knows many things, but the hedgehog knows one big thing.]
  Archilochus, Greek poet (c. 680 BC – c. 645 BC)
http://wiki.hedgehogshiatus.com