[chef] Re: Chef Deployment System for Swift - a proposed design - feedback?

[Sahil R Cooner < >]

sounds great look forward to checking out the code. however I think you might want to have attributes defined instead of trying to load configuration options through a yaml file as that might be more chef like, and then possibly having overrides when necessary for the appropriate attributes as you apply them to nodes.

-- 
Sahil R Cooner

On Wednesday, April 27, 2011 at 4:45 PM, Judd Maltin wrote:

Hi Folks,  (sorta cross posted with "> )

I've been hacking away at creating an automated deployment system for Swift using Chef.  I'd like to drop a design idea on you folks (most of which I've already implemented) and get feedback from this esteemed group.

My end goal is to have a "manifest" (apologies to Puppet) which will define an entire swift cluster, deploy it automatically, and allow edits to the ingredients to manage the cluster.  In this case, a "manifest" is a combination of a chef databag describing the swift settings, and a spiceweasel infrastructure.yaml file describing the OS configuration.

Ingredients:
- swift cookbook with base, proxy and server recipes.  proxy nodes also (provisionally) contain auth services. storage nodes handle object, container and account services.
-- Base recipe handles common package install, OS user creation.  Sets up keys.
-- Proxy recipe handles proxy nodes: network config, package install, memcache config, proxy and auth package config, user creation, ring management (including builder file backup), user management
-- Storage recipe handles storage nodes: network config, storage device config, package install, ring management.

- chef databag that describes a swift cluster (eg: mycluster_databag.json)
-- proxy config settings
-- memcached settings
-- settings for all rings and devices
-- basic user settings
-- account management

- chef "spiceweasel" file that auto-vivifies the infrastructure: (eg: mycluster_infra.yaml)
-- uploads cookbooks
-- uploads roles
-- uploads the cluster's databag
-- kicks off node provisioning by requesting from infrastructure API (ec2 or what have you) the following:
--- chef roles applied (role[swift:proxy] or role[swift:storage])
--- server flavor
--- storage device configs
--- hostname
--- proxy and storage network details

By calling this spiceweasel file, the infrastructure can leap into existence.

I'm more or less done with all this stuff - and I'd really appreciate conceptual feedback before I take out all the non-sense code I have in the files and publish.

Many thanks!  Happy spring, northern hemispherians!
-judd



--
Judd Maltin
T: 917-882-1270
F: 501-694-7809
A loving heart is never wrong.