[chef] Re: Request for input on orchestration

[Dan Nemec < >]

Mandatory disclosure. We've been using Control Tier for over a year.

I don't know that I am interested in Chef being the orchestration engine. I would like to see Chef more friendly with other existing orchestration tools. Control Tier is great if you want to code much of your customized commands into the tool. The spinoff RunDeck is great if you have your own scripts but just need them orchestrated.

We envision a scenario where Chef is mainly configuration and Control Tier is orchestration. We would go so far as having Chef configure Control Tier which could then call back to Chef to initiate Chef actions.

Orchestration is a difficult problem and Control Tier/RunDeck is already fairly mature. Now that they support arbitrary workflows in Jobcenter Control Tier is't missing any feature we require for orchestrating our very complex deployments.

I don't know that I have much to add to your questions above. We're fairly new to Chef and it being Friday afternoon is making my head hurt thinking of how to incorporate orchestration into Chef. All I can imagine is how we're doing it today with Control Tier.

1) You hit the high points in your overview of scope.

2/3) All I can say is to read Control Tier's documentation. In Control Tier you define a "command" that performs a piece of work. You string a bunch of commands together into a workflow. Then you can also string different workflows into larger events. You add to that the error-handling and gating so that the next step only executes after the first one completed successfully. Threading concurrently across servers is mandatory to make it go fast.

Happy Friday,

Dan

On Fri, Jan 28, 2011 at 3:26 PM, Chris Walters < "> > wrote:

Ohai Chefs!

We're in the preliminary stages of designing possible solutions for

orchestration and would like to understand the community's

requirements.

I'm going to write down my thoughts and questions. Nothing is gospel,

so please feel free to comment on everything, including the framing.

Background:

Chef, as currently conceived, does a great job of exposing a model for

how to get a system from either an embryonic state or a slightly

misconfigured state to the desired state, mainly via the mechanism of

resource idempotence.

What I think is not yet well-modeled is how to go from one

well-configured state to a completely different well-configued

state. It also doesn't yet model synchronization of actions across

multiple boxes in that there isn't a first-class way to gate actions

that are dependent on the completion of steps on other servers. For

example, a complex migration or deployment might require bringing

boxes up or down, copying data, cleanly removing artifacts or services

installed by previous chef runs, not restarting load balancers until

some quorum of webservers have re-started, etc.

We'd like to collect the use cases, requirements, and thoughts that

best serve the community.

1) What do you think the scope of orchestration is and is not?

2) What are the use cases that you would like to see an orchestration

system/DSL accommodate? The more specific and granular the steps of

the orchestration, the better. (If you would not like your use case

made public but would nonetheless like it considered during design,

validation, and testing, please send it to me directly at

" target="_blank"> .)

3) What generic primitives do you think would be useful in such a

system?

Thanks!

Chris Walters