[chef] Re: Re: Re: Scaling erchef horizontally

[Dennis Lovely < >]

We used this approach and happy with the results:

http://chef-docs.readthedocs.org/en/latest/private_chef_1x_install_tiered.html

It's not quite as desirable as the high availability approach, but it has made it easier to scale horizontally, and currently fits our needs.  Our frontend workers just run erchef while the backend runs the rest of the suite.

-Dennis

On Thu, Apr 24, 2014 at 1:14 PM, Daniel DeLeo < " target="_blank"> > wrote:

On Thursday, April 24, 2014 at 1:04 PM, Stephen Delano wrote:

> There should be some more crash logs from the console telling you what's going on with erchef, but you're also going to have some other issues with the setup you've described. If you're running enough erchef servers, you might want to check that you're not exceeding the available connections of the PostgreSQL server.
>
> Multiple Bookshelfs:
> Bookshelf was not designed to be run on multiple nodes. It has local disk-based storage for the contents of your cookbooks.
>
> Multiple Chef Expanders / RabbitMQ / Solr:
> You also don't want to run multiple search stacks. When indexable objects are stored on the chef server, their contents are shuffled off to a RabbitMQ queue for which there is a chef-expander listener that's ready to consume that data, "expand" it, and send it to Solr for indexing. First, if you have multiple expanders as consumers to the rabbit queue, you're introducing the chance that the data is indexed out-of-order. This problem is exacerbated when you start to add multiple RabbitMQs (which erchef talk to which queues) and multiple Solrs (which erchefs and expanders talk to which Solr).
>
>
>

> On Thu, Apr 24, 2014 at 9:42 AM, Darío Ezequiel Nievas < "> (mailto: "> )> wrote:
> > Hi Guys,
> > I'm having a bit of a problem trying to scale erchef between several nodes
> >
> > First, let me give you guys an overview of my environment
> > -2 (there will be more) servers behind a load balancer, running the following services:
> > -bookshelf
> > -chef-expander
> > -chef-server-webui
> > -erchef
> > -nginx
> >
> > -2 servers behind a load balancer, runing these services:
> > -chef-solr
> > -rabbitmq
> >
> > -a Postgresql cluster (using pgpool) for the chefdb

Take a look at the docs for scaling enterprise chef: http://docs.opscode.com/server_deploy_fe.html Enterprise Chef has some extra services that support the paid features, but aside from that, should give you a good idea of how to design your cluster.

HTH,

--
Daniel DeLeo