[chef] Re: Re: Latency on search indicies

[Peter Donald < >]

Hi,

On Tue, Jun 25, 2013 at 10:46 AM, Brad Knowles 
<
 >
 wrote:
>> Just be warned that the lag can be much more significant. We run the
>> open source variant of chef for now and have noticed lags of anywhere
>> between <1s to ~120s between the data being inserted and it being
>> accessible via search.
>
> What version of chef-server are you running?  Is it Chef 11.x, 10.x, or 
> something older?

We actually use Chef 11 but to be honest most of the strategies we
used to address this were developed when we were on 10.x. There is
still a delay between when a data bag item is inserted/updates and
when it appears in the search results so I *assume* the lag is still
present for node data but have not tested.

> Windows nodes put a heavier load on the Chef server, but I would think that 
> chef-server 11.x should be a lot more capable of handling large numbers of 
> clients (even large numbers of Windows nodes) much better than the older 
> 10.x-based versions.
>
> After all, my understanding is that Hosted Chef is basically the world's 
> largest instance of Private Chef 11.x set up in a multi-tiered structure, 
> and I believe that Private Chef has been proven by partners like Facebook, 
> Etsy, Netflix, etc... to scale to at least tens of thousands of nodes on a 
> single Private Chef 11.x cluster.

To be fair the chef 11 server does not seem to be particularly loaded
even when there is lag.

Also most of those other sites can probably afford to rely on eventual
consistency. However it would be nice to have the parameter to the
rest endpoints to block until indexing of the changed data is
complete.

-- 
Cheers,

Peter Donald