[chef] chef-solo: Is there a way to stream stderr/stdout output of executed processes without buffering?


Chronological Thread 
  • From: Joshua Blatt < >
  • To:
  • Subject: [chef] chef-solo: Is there a way to stream stderr/stdout output of executed processes without buffering?
  • Date: Tue, 05 Jun 2012 08:41:52 -0700

chef-solo seems to buffer all stderr/stdout output of any process created by the execute resource and then dumps it at the end of the run. This makes debugging issues more difficult because you don't see errors when they actually occur.

In one particular case, I had a long running execute'd script that was causing chef-solo to increase it's memory consumption rapidly. Soon chef-solo was consuming 3.5GB heap at which point the machine was swapping heavily. After waiting hours for the executed process to finally finish, chef-solo dumped thousands of lines that looked like this:

Re-format filesystem in /var/hadoop/hdfs/data ? (Y or N) Re-format filesystem in /var/hadoop/hdfs/data ? (Y or N) Re-format filesystem in /var/hadoop/hdfs/data ? (Y or N) Re-format filesystem in /var/hadoop/hdfs/data ? (Y or N) ...

If chef-solo had simply relayed that instead of buffering it without bound ...

Thanks,

Josh




Archive powered by MHonArc 2.6.16.

§