[chef] Re: Re: Re: Parsing Chef's Multiline Output

[Daniel DeLeo < >]

The chef formatter output (well, the default one, anyway) is designed for 
human consumption, but the same API is available to you to do whatever you 
want.  

Here’s some examples:

* Enterprise Chef reporting: 
https://github.com/opscode/chef/blob/master/lib/chef/resource_reporter.rb
* Foreman reporting: https://github.com/theforeman/chef-handler-foreman
* Nyan Cat: https://github.com/andreacampi/nyan-cat-chef-formatter

The Enterprise Chef reporting code isn’t real-time (AFAIK this is true of the 
foreman plugin as well), but this is a design choice of the server 
implementation and not a limitation of the events API in chef-client.

I don’t know of anything that does exactly what you want out of the box, so 
you’ll have to do some assembling of the parts yourself.  

--  
Daniel DeLeo


On Sunday, March 2, 2014 at 7:01 PM, Greg Zapp wrote:

> Hi Larry and Noah,
>  
> Thanks for the feedback. Currently though I'm working on type "chef_log", 
> but I will most likely swing back around to "chef_reports" in the future:) 
> I plan to ship all the output and store a high level of verbosity for a 
> reduced duration from the rest of the stuff. I'm doing some long Windows 
> client runs with application/database deploys and it's been really nice to 
> be able to view all this stuff from one location.  
>  
> -Greg
>  
>  
> On Mon, Mar 3, 2014 at 3:47 PM, Larry Wright 
> <
 
>  
> (mailto:
 )>
>  wrote:
> > Do you want all of your Chef logs in Logstash, or do you just want 
> > exceptions and Chef run outcomes? If you don’t want all of the logs, the 
> > easiest way to handle that is to use a handler like this one: 
> > https://github.com/lusis/logstash_handler.
> >  
> > - Larry
> >  
> > On Mar 2, 2014, at 6:41 PM, Greg Zapp 
> > <
 
> >  
> > (mailto:
 )>
> >  wrote:
> > > HI Brian,
> > >  
> > > Yes, I'm wanting to get all the log data into logstash. I'm looking at 
> > > the ruby logger class and I don't see anything about single line mode? 
> > > It does take a formatter block so that's likely where I can set up my 
> > > ambush. Unfortunately it looks like I will need to customize my Chef 
> > > client installation to achieve what I want :| .  
> > >  
> > > -Greg
> > >  
> > >  
> > > On Mon, Mar 3, 2014 at 12:54 PM, Brian Hatfield 
> > > <
 
> > >  
> > > (mailto:
 )>
> > >  wrote:
> > > > What's your end goal? Just to get the data in Logstash, or do you 
> > > > have a specific view into the data you'd like to see?
> > > >  
> > > > If the goal is to get the data into Logstash, you might try forcing 
> > > > the logger back to single-line mode.
> > > >  
> > > > If the goal is as an intermediate hop to a specific view, might I 
> > > > recommend setting up a post-run handler to directly share the run 
> > > > information you need?
> > > >  
> > > >  
> > > > On Sun, Mar 2, 2014 at 6:46 PM, Greg Zapp 
> > > > <
 
> > > >  
> > > > (mailto:
 )>
> > > >  wrote:
> > > > > Hello,
> > > > >  
> > > > > I would like to know if anyone is dealing with Chef's multiline 
> > > > > output when shipping to something like logstash->elasticsearch. 
> > > > > Currently Chef does not delineate log entries when they span 
> > > > > multiple lines. I'm considering monkey patching the logger to add a 
> > > > > frame around the log entries but I'm not sure where the best place 
> > > > > to do this is? Ideally I would get it implemented before chef makes 
> > > > > a single log entry...  
> > > > >  
> > > > >  
> > > > > -Greg