[chef] Re: chef-solo: Is there a way to stream stderr/stdout output of executed processes without buffering?

[AJ Christensen < >]

Looks like mixlib-shellout is configured to stream the output under
certain conditions:

https://github.com/opscode/chef/blob/master/chef/lib/chef/mixin/shell_out.rb#L27-29

HTH

--AJ

On 6 June 2012 03:41, Joshua Blatt 
<
 >
 wrote:
> chef-solo seems to buffer all stderr/stdout output of any process created by
> the execute resource and then dumps it at the end of the run.   This makes
> debugging issues more difficult because you don't see errors when they
> actually occur.
>
> In one particular case, I had a long running execute'd script that was
> causing chef-solo to increase it's memory consumption rapidly.   Soon
> chef-solo was consuming 3.5GB heap at which point the machine was swapping
> heavily.   After waiting hours for the executed process to finally finish,
> chef-solo dumped thousands of lines that looked like this:
>
> Re-format filesystem in /var/hadoop/hdfs/data ? (Y or N) Re-format
> filesystem in /var/hadoop/hdfs/data ? (Y or N) Re-format filesystem in
> /var/hadoop/hdfs/data ? (Y or N) Re-format filesystem in
> /var/hadoop/hdfs/data ? (Y or N) ...
>
> If chef-solo had simply relayed that instead of buffering it without bound
> ...
>
> Thanks,
>
> Josh
>