I would add a debug output to logstash, might help see what's going on.output {stdout => { codec => rubydebug }}On Sun, Apr 5, 2015 at 3:52 PM, Fabien Delpierre < " target="_blank"> > wrote:Hi folks,I'm trying to do a very basic implementation of an Elasticsearch/Logstash/Kibana stack, all running on the same box. I hoped someone had done the hard work of figuring out the mostly undocumented lusis/chef-logstash cookbooks and found the elkstack cookbook.Ultimately, I just need Logstash to read an HAProxy log at /var/log/haproxy.log.I'm using the elkstack::recipe but removed the rsyslog::client include at the end because I don't need it.I also don't actually have HAProxy running, just a log file with a few sample entries from an actual HAProxy instance.Here's what my recipe looks like. The kibana bits are disabled because I don't need them for troubleshooting and it just slows down the provisioning process.include_recipe 'elkstack::_server'include_recipe 'elkstack::elasticsearch'include_recipe 'elkstack::logstash'# node.run_state['elkstack_kibana_username'] = 'kibana'# node.run_state['elkstack_kibana_password'] = 'kibana'# include_recipe "elkstack::kibana"instance_name = node['elkstack']['config']['logstash']['instance_name']basedir = node['logstash']['instance'][instance_name]['basedir']# Create a dummy HAProxy file containing a few entriestemplate node['cloudmine']['haproxy_log_file'] dosource "haproxy.log.erb"owner "syslog"group "adm"mode 00644not_if { ::File.exist?(node['cloudmine']['haproxy_log_file']) }notifies :restart, "logstash_service[#{instance_name}]", :delayedendAt this point, there's nothing set up to tell Logstash to read my HAProxy log file. So I added this simple bit to the recipe:cookbook_file "input_haproxy" dopath "/opt/logstash/server/etc/conf.d/input_haproxy"owner "logstash"group "logstash"mode 00664endAnd this is the content of the input_haproxy file:input {file {path => "/var/log/haproxy.log"start_position => "beginning"type => "haproxy-http"}}If this recipe runs with the cookbook_file block and I restart the Logstash service... I get nothing in Elasticsearch, like my HAProxy log file is not being read at all, ever.However, if I run the recipe without the cookbook_file block, SSH into the instance, do a $ sudo su -l logstash, manually create the input_haproxy file in the correct location (I just copy-paste the contents from the file in my git repo) and assign it the right ownership/permissions, then restart the Logstash service, after about 30 seconds I can see that my HAProxy log was read and I can see it in Elasticsearch.It's incredibly frustrating and I don't understand what I'm doing wrong. It's like Chef does things differently than me when I do it manually, resulting in this silent "failure". I realize Chef does do this a little bit differently, since I believe it would create the file as root and then set the correct permissions, whereas I'm doing it directly as the logstash user... But still, I don't see why this would matter since, in the end, the file is owned by the logstash user, with the rw-rw-r-- permissions I want.Any ideas?
Archive powered by MHonArc 2.6.16.