[chef] Re: Re: Re: Re: Parsing Chef's Multiline Output

[Greg Zapp < >]

Hi Daniel,

Thanks for those links; that stuff definitely looks interesting.  I think for grabbing the logs out as they come I may just monkey patch my chef client to add a frame around what it would normally output to the log.  That should let me grab out multi-line log entries no worries.

-Greg

On Mon, Mar 3, 2014 at 4:12 PM, Daniel DeLeo < " target="_blank"> > wrote:

The chef formatter output (well, the default one, anyway) is designed for human consumption, but the same API is available to you to do whatever you want.

Here’s some examples:

* Enterprise Chef reporting: https://github.com/opscode/chef/blob/master/lib/chef/resource_reporter.rb
* Foreman reporting: https://github.com/theforeman/chef-handler-foreman
* Nyan Cat: https://github.com/andreacampi/nyan-cat-chef-formatter

The Enterprise Chef reporting code isn’t real-time (AFAIK this is true of the foreman plugin as well), but this is a design choice of the server implementation and not a limitation of the events API in chef-client.

I don’t know of anything that does exactly what you want out of the box, so you’ll have to do some assembling of the parts yourself.

--
Daniel DeLeo

On Sunday, March 2, 2014 at 7:01 PM, Greg Zapp wrote:

> Hi Larry and Noah,
>
> Thanks for the feedback. Currently though I'm working on type "chef_log", but I will most likely swing back around to "chef_reports" in the future:) I plan to ship all the output and store a high level of verbosity for a reduced duration from the rest of the stuff. I'm doing some long Windows client runs with application/database deploys and it's been really nice to be able to view all this stuff from one location.
>
> -Greg
>
>

> On Mon, Mar 3, 2014 at 3:47 PM, Larry Wright < "> (mailto: "> )> wrote:
> > Do you want all of your Chef logs in Logstash, or do you just want exceptions and Chef run outcomes? If you don’t want all of the logs, the easiest way to handle that is to use a handler like this one: https://github.com/lusis/logstash_handler.
> >
> > - Larry
> >

> > On Mar 2, 2014, at 6:41 PM, Greg Zapp < "> (mailto: "> )> wrote:
> > > HI Brian,
> > >
> > > Yes, I'm wanting to get all the log data into logstash. I'm looking at the ruby logger class and I don't see anything about single line mode? It does take a formatter block so that's likely where I can set up my ambush. Unfortunately it looks like I will need to customize my Chef client installation to achieve what I want :| .
> > >
> > > -Greg
> > >
> > >

> > > On Mon, Mar 3, 2014 at 12:54 PM, Brian Hatfield < "> (mailto: "> )> wrote:
> > > > What's your end goal? Just to get the data in Logstash, or do you have a specific view into the data you'd like to see?
> > > >
> > > > If the goal is to get the data into Logstash, you might try forcing the logger back to single-line mode.
> > > >
> > > > If the goal is as an intermediate hop to a specific view, might I recommend setting up a post-run handler to directly share the run information you need?
> > > >
> > > >

> > > > On Sun, Mar 2, 2014 at 6:46 PM, Greg Zapp < "> (mailto: "> )> wrote:
> > > > > Hello,
> > > > >
> > > > > I would like to know if anyone is dealing with Chef's multiline output when shipping to something like logstash->elasticsearch. Currently Chef does not delineate log entries when they span multiple lines. I'm considering monkey patching the logger to add a frame around the log entries but I'm not sure where the best place to do this is? Ideally I would get it implemented before chef makes a single log entry...
> > > > >
> > > > >
> > > > > -Greg