[chef] Re: Re: Re: Re: Re: Scaling erchef horizontally


Chronological Thread 
  • From: Cameron Cope < >
  • To:
  • Subject: [chef] Re: Re: Re: Re: Re: Scaling erchef horizontally
  • Date: Fri, 25 Apr 2014 02:59:18 -0400

I am currently running a Chef 10 cluster in EC2 with the bookshelf stored on a Gluster volume mounted on all the servers, but only one node at a time is receiving traffic for rabbitmq, solr, or couchdb. I've been holding off on upgrading to Chef 11 server because I haven't had time to figure out HA, so I would be very interested in seeing your cluster configuration for Chef 11.

From what I understand, it is possible to run rabbitmq in an HA mode where the queues are mirrored to slaves, but during failure there can be some consistency loss.
http://www.rabbitmq.com/ha.html

I think solr can be made HA as well, but it looks complicated: https://cwiki.apache.org/confluence/display/solr/SolrCloud

When I last checked into pgpool, I got scared off by the SQL restrictions, but if it works and is stable that would be awesome! http://www.pgpool.net/docs/latest/pgpool-en.html#restriction



On Thu, Apr 24, 2014 at 5:42 PM, James Scott < " target="_blank"> > wrote:
FWIW, the link that Dan added to this thread is the current documentation (for Enterprise Chef) and the link that Dennis added is the old documentation (for what was previously called Private Chef) ... both describe roughly the same approach, but it should be noted that one cannot actually follow the steps/requirements for Private Chef with Enterprise Chef.

Please use this link to find all the current docs: http://docs.opscode.com/enterprise/#the-server

James


On Thu, Apr 24, 2014 at 1:38 PM, Dennis Lovely < " target="_blank"> > wrote:
We used this approach and happy with the results:

http://chef-docs.readthedocs.org/en/latest/private_chef_1x_install_tiered.html

It's not quite as desirable as the high availability approach, but it has made it easier to scale horizontally, and currently fits our needs.  Our frontend workers just run erchef while the backend runs the rest of the suite.

-Dennis


On Thu, Apr 24, 2014 at 1:14 PM, Daniel DeLeo < " target="_blank"> > wrote:


On Thursday, April 24, 2014 at 1:04 PM, Stephen Delano wrote:

> There should be some more crash logs from the console telling you what's going on with erchef, but you're also going to have some other issues with the setup you've described. If you're running enough erchef servers, you might want to check that you're not exceeding the available connections of the PostgreSQL server.
>
> Multiple Bookshelfs:
> Bookshelf was not designed to be run on multiple nodes. It has local disk-based storage for the contents of your cookbooks.
>
> Multiple Chef Expanders / RabbitMQ / Solr:
> You also don't want to run multiple search stacks. When indexable objects are stored on the chef server, their contents are shuffled off to a RabbitMQ queue for which there is a chef-expander listener that's ready to consume that data, "expand" it, and send it to Solr for indexing. First, if you have multiple expanders as consumers to the rabbit queue, you're introducing the chance that the data is indexed out-of-order. This problem is exacerbated when you start to add multiple RabbitMQs (which erchef talk to which queues) and multiple Solrs (which erchefs and expanders talk to which Solr).
>
>
>
> On Thu, Apr 24, 2014 at 9:42 AM, Darío Ezequiel Nievas < " target="_blank"> (mailto: " target="_blank"> )> wrote:
> > Hi Guys,
> > I'm having a bit of a problem trying to scale erchef between several nodes
> >
> > First, let me give you guys an overview of my environment
> > -2 (there will be more) servers behind a load balancer, running the following services:
> > -bookshelf
> > -chef-expander
> > -chef-server-webui
> > -erchef
> > -nginx
> >
> > -2 servers behind a load balancer, runing these services:
> > -chef-solr
> > -rabbitmq
> >
> > -a Postgresql cluster (using pgpool) for the chefdb
Take a look at the docs for scaling enterprise chef: http://docs.opscode.com/server_deploy_fe.html Enterprise Chef has some extra services that support the paid features, but aside from that, should give you a good idea of how to design your cluster.

HTH,

--
Daniel DeLeo








--
Cameron Cope | Systems Engineer
Brightcove, Inc
290 Congress Street, 4th Floor, Boston, MA 02110
" target="_blank">



Archive powered by MHonArc 2.6.16.

§