[chef] recurring bad perms on /var/log/chef, chef-solr returns 404

hi. i thought i'd update subject in case it helps get more eyeballs
on this. the details are right here:

On Mon, 09 Apr 2012, 

 
 wrote:

> i suspect the following 2 dirs have the wrong permissions: /var/run/chef
> and /var/log/chef. wrong in that they are root:root owned. if you chown
> them to chef:chef, restart chef-solr, it should fix this.
> 
> i keep running into this problem where chef-solr dies for whatever reason,
> and for reasons i haven't figured out yet, these 2 dirs get root:root
> pwned.
> 
> anyone know why these bad perms keep getting laid down?
> 
> i'm about to cron a babysitter script to restart chef-solr and keep
> these dirs chowned properly :<
> 
> (yes, the chef-server recipe i run on my chef server managers proper
> perms on these dirs, but something else flips them back to root:root)
> 
> kallen
> 
> 
> On Mon, 09 Apr 2012, Paul Welch wrote:
> 
> > Output from my  /var/log/chef/solr.log:
> > 
> > [Mon, 09 Apr 2012 12:45:04 -0400] INFO: About to change privilege to 
> > chef:chef
> > [Mon, 09 Apr 2012 12:45:04 -0400] INFO: Daemonizing..
> > [Mon, 09 Apr 2012 12:45:04 -0400] INFO: Forked, in 895. Priveleges: 107 
> > 115
> > [Mon, 09 Apr 2012 12:45:04 -0400] INFO: Starting Solr with java
> > -Xmx256M -Xms256M -Dsolr.data.dir=/var/cache/chef/solr/data
> > -Dsolr.solr.home=/var/lib/chef/solr
> > -DSTART=/var/lib/chef/solr/solr-jetty/etc/start.config -jar
> > /var/lib/chef/solr/solr-jetty/start.jar
> > 1 [main] INFO org.mortbay.log - Logging to
> > org.slf4j.impl.SimpleLogger(org.mortbay.log) via
> > org.mortbay.log.Slf4jLog
> > 196 [main] INFO org.mortbay.log - jetty-6.1.22
> > SLF4J: Class path contains multiple SLF4J bindings.
> > SLF4J: Found binding in
> > [jar:file:/usr/share/java/slf4j-simple-1.5.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> > SLF4J: Found binding in
> > [jar:file:/var/lib/chef/solr/solr-jetty/webapps/solr/WEB-INF/lib/slf4j-jdk14.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an ;
> > explanation.
> > Apr 9, 2012 12:45:17 PM org.apache.solr.core.SolrResourceLoader 
> > locateSolrHome
> > INFO: No /solr/home in JNDI
> > Apr 9, 2012 12:45:17 PM org.apache.solr.core.SolrResourceLoader 
> > locateSolrHome
> > INFO: using system property solr.solr.home: /var/lib/chef/solr
> > Apr 9, 2012 12:45:17 PM org.apache.solr.core.SolrResourceLoader <init>
> > INFO: Solr home set to '/var/lib/chef/solr/'
> > Apr 9, 2012 12:45:17 PM org.apache.solr.servlet.SolrDispatchFilter init
> > INFO: SolrDispatchFilter.init()
> > Apr 9, 2012 12:45:17 PM org.apache.solr.core.SolrResourceLoader 
> > locateSolrHome
> > INFO: No /solr/home in JNDI
> > Apr 9, 2012 12:45:17 PM org.apache.solr.core.SolrResourceLoader 
> > locateSolrHome
> > INFO: using system property solr.solr.home: /var/lib/chef/solr
> > Apr 9, 2012 12:45:17 PM org.apache.solr.core.CoreContainer$Initializer
> > initialize
> > INFO: looking for solr.xml: /var/lib/chef/solr/solr.xml
> > Apr 9, 2012 12:45:17 PM org.apache.solr.core.SolrResourceLoader <init>
> > INFO: Solr home set to '/var/lib/chef/solr/'
> > Apr 9, 2012 12:45:17 PM org.apache.solr.core.SolrConfig <init>
> > INFO: Loaded SolrConfig: solrconfig.xml
> > Apr 9, 2012 12:45:17 PM org.apache.solr.core.SolrCore <init>
> > INFO: Opening new SolrCore at /var/lib/chef/solr/,
> > dataDir=/var/cache/chef/solr/data/
> > Apr 9, 2012 12:45:17 PM org.apache.solr.schema.IndexSchema readSchema
> > INFO: Reading Solr Schema
> > Apr 9, 2012 12:45:17 PM org.apache.solr.schema.IndexSchema readSchema
> > INFO: Schema name=chef
> > Apr 9, 2012 12:45:17 PM org.apache.solr.util.plugin.AbstractPluginLoader 
> > load
> > INFO: created string: org.apache.solr.schema.StrField
> > Apr 9, 2012 12:45:17 PM org.apache.solr.util.plugin.AbstractPluginLoader 
> > load
> > INFO: created boolean: org.apache.solr.schema.BoolField
> > Apr 9, 2012 12:45:17 PM org.apache.solr.util.plugin.AbstractPluginLoader 
> > load
> > INFO: created integer: org.apache.solr.schema.IntField
> > Apr 9, 2012 12:45:17 PM org.apache.solr.util.plugin.AbstractPluginLoader 
> > load
> > INFO: created long: org.apache.solr.schema.LongField
> > Apr 9, 2012 12:45:17 PM org.apache.solr.util.plugin.AbstractPluginLoader 
> > load
> > INFO: created float: org.apache.solr.schema.FloatField
> > Apr 9, 2012 12:45:17 PM org.apache.solr.util.plugin.AbstractPluginLoader 
> > load
> > INFO: created double: org.apache.solr.schema.DoubleField
> > Apr 9, 2012 12:45:18 PM org.apache.solr.util.plugin.AbstractPluginLoader 
> > load
> > INFO: created sint: org.apache.solr.schema.SortableIntField
> > Apr 9, 2012 12:45:18 PM org.apache.solr.util.plugin.AbstractPluginLoader 
> > load
> > INFO: created slong: org.apache.solr.schema.SortableLongField
> > Apr 9, 2012 12:45:18 PM org.apache.solr.util.plugin.AbstractPluginLoader 
> > load
> > INFO: created sfloat: org.apache.solr.schema.SortableFloatField
> > Apr 9, 2012 12:45:18 PM org.apache.solr.util.plugin.AbstractPluginLoader 
> > load
> > INFO: created sdouble: org.apache.solr.schema.SortableDoubleField
> > Apr 9, 2012 12:45:18 PM org.apache.solr.util.plugin.AbstractPluginLoader 
> > load
> > INFO: created stuff: org.apache.solr.schema.TextField
> > Apr 9, 2012 12:45:18 PM org.apache.solr.util.plugin.AbstractPluginLoader 
> > load
> > INFO: created date: org.apache.solr.schema.DateField
> > Apr 9, 2012 12:45:18 PM org.apache.solr.schema.IndexSchema readSchema
> > INFO: default search field is content
> > Apr 9, 2012 12:45:18 PM org.apache.solr.schema.IndexSchema readSchema
> > INFO: query parser default operator is OR
> > Apr 9, 2012 12:45:18 PM org.apache.solr.schema.IndexSchema readSchema
> > INFO: unique key field: X_CHEF_id_CHEF_X
> > Apr 9, 2012 12:45:18 PM org.apache.solr.core.JmxMonitoredMap <init>
> > INFO: No JMX servers found, not exposing Solr information with JMX.
> > Apr 9, 2012 12:45:18 PM org.apache.solr.core.RequestHandlers
> > initHandlersFromConfig
> > INFO: created standard: solr.SearchHandler
> > Apr 9, 2012 12:45:18 PM org.apache.solr.core.RequestHandlers
> > initHandlersFromConfig
> > INFO: created /update: solr.XmlUpdateRequestHandler
> > Apr 9, 2012 12:45:18 PM org.apache.solr.core.RequestHandlers
> > initHandlersFromConfig
> > INFO: created /update/javabin: solr.BinaryUpdateRequestHandler
> > Apr 9, 2012 12:45:18 PM org.apache.solr.core.RequestHandlers
> > initHandlersFromConfig
> > INFO: created /analysis/document: solr.DocumentAnalysisRequestHandler
> > Apr 9, 2012 12:45:18 PM org.apache.solr.core.RequestHandlers
> > initHandlersFromConfig
> > INFO: created /analysis/field: solr.FieldAnalysisRequestHandler
> > Apr 9, 2012 12:45:18 PM org.apache.solr.core.RequestHandlers
> > initHandlersFromConfig
> > INFO: adding lazy requestHandler: solr.CSVRequestHandler
> > Apr 9, 2012 12:45:18 PM org.apache.solr.core.RequestHandlers
> > initHandlersFromConfig
> > INFO: created /update/csv: solr.CSVRequestHandler
> > Apr 9, 2012 12:45:18 PM org.apache.solr.core.RequestHandlers
> > initHandlersFromConfig
> > INFO: created /admin/: org.apache.solr.handler.admin.AdminHandlers
> > Apr 9, 2012 12:45:18 PM org.apache.solr.core.RequestHandlers
> > initHandlersFromConfig
> > INFO: created /admin/ping: PingRequestHandler
> > Apr 9, 2012 12:45:18 PM org.apache.solr.core.RequestHandlers
> > initHandlersFromConfig
> > INFO: created /debug/dump: solr.DumpRequestHandler
> > Apr 9, 2012 12:45:18 PM org.apache.solr.search.SolrIndexSearcher <init>
> > INFO: Opening 
> > 
 
> >  main
> > Apr 9, 2012 12:45:18 PM
> > org.apache.solr.update.DirectUpdateHandler2$CommitTracker <init>
> > INFO: AutoCommit: if uncommited for 10000ms; if 100 uncommited docs
> > Apr 9, 2012 12:45:18 PM org.apache.solr.handler.component.SearchHandler 
> > inform
> > INFO: Adding  
> > component:
 
> > Apr 9, 2012 12:45:18 PM org.apache.solr.handler.component.SearchHandler 
> > inform
> > INFO: Adding  
> > component:
 
> > Apr 9, 2012 12:45:18 PM org.apache.solr.handler.component.SearchHandler 
> > inform
> > INFO: Adding  
> > component:
 
> > Apr 9, 2012 12:45:18 PM org.apache.solr.handler.component.SearchHandler 
> > inform
> > INFO: Adding  
> > component:
 
> > Apr 9, 2012 12:45:18 PM org.apache.solr.handler.component.SearchHandler 
> > inform
> > INFO: Adding  
> > component:
 
> > Apr 9, 2012 12:45:18 PM org.apache.solr.handler.component.SearchHandler 
> > inform
> > INFO: Adding  debug
> > component:
 
> > Apr 9, 2012 12:45:18 PM org.apache.solr.core.SolrCore registerSearcher
> > INFO: Registered new searcher 
> > 
 
> >  main
> > Apr 9, 2012 12:45:18 PM org.apache.solr.core.CoreContainer register
> > INFO: registering core:
> > Apr 9, 2012 12:45:18 PM org.apache.solr.servlet.SolrDispatchFilter init
> > INFO: user.dir=/var/lib/chef/solr/solr-jetty
> > Apr 9, 2012 12:45:18 PM org.apache.solr.servlet.SolrDispatchFilter init
> > INFO: SolrDispatchFilter.init() done
> > Apr 9, 2012 12:45:18 PM org.apache.solr.servlet.SolrServlet init
> > INFO: SolrServlet.init()
> > Apr 9, 2012 12:45:18 PM org.apache.solr.core.SolrResourceLoader 
> > locateSolrHome
> > INFO: No /solr/home in JNDI
> > Apr 9, 2012 12:45:18 PM org.apache.solr.core.SolrResourceLoader 
> > locateSolrHome
> > INFO: using system property solr.solr.home: /var/lib/chef/solr
> > Apr 9, 2012 12:45:18 PM org.apache.solr.servlet.SolrServlet init
> > INFO: SolrServlet.init() done
> > Apr 9, 2012 12:45:18 PM org.apache.solr.core.SolrResourceLoader 
> > locateSolrHome
> > INFO: No /solr/home in JNDI
> > Apr 9, 2012 12:45:18 PM org.apache.solr.core.SolrResourceLoader 
> > locateSolrHome
> > INFO: using system property solr.solr.home: /var/lib/chef/solr
> > Apr 9, 2012 12:45:18 PM org.apache.solr.servlet.SolrUpdateServlet init
> > INFO: SolrUpdateServlet.init() done
> > 13144 [main] WARN org.mortbay.log - failed
> > 
 :
> >  java.io.IOException: Cannot
> > write log directory /var/log/chef
> > 13144 [main] WARN org.mortbay.log - failed 
> > 
 :
> > java.io.IOException: Cannot write log directory /var/log/chef
> > 13144 [main] WARN org.mortbay.log - failed 
> > 
 :
> > java.io.IOException: Cannot write log directory /var/log/chef
> > 13144 [main] WARN org.mortbay.log - Error starting handlers
> > java.io.IOException: Cannot write log directory /var/log/chef
> >     at 
> > org.mortbay.util.RolloverFileOutputStream.setFile(RolloverFileOutputStream.java:217)
> >     at 
> > org.mortbay.util.RolloverFileOutputStream.<init>(RolloverFileOutputStream.java:164)
> >     at org.mortbay.jetty.NCSARequestLog.doStart(NCSARequestLog.java:428)
> >     at 
> > org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> >     at 
> > org.mortbay.jetty.handler.RequestLogHandler.doStart(RequestLogHandler.java:117)
> >     at 
> > org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> >     at 
> > org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152)
> >     at 
> > org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> >     at 
> > org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
> >     at org.mortbay.jetty.Server.doStart(Server.java:224)
> >     at 
> > org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> >     at org.mortbay.xml.XmlConfiguration.main(XmlConfiguration.java:985)
> >     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >     at 
> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >     at 
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >     at java.lang.reflect.Method.invoke(Method.java:616)
> >     at org.mortbay.start.Main.invokeMain(Main.java:194)
> >     at org.mortbay.start.Main.start(Main.java:534)
> >     at org.mortbay.start.Main.start(Main.java:441)
> >     at org.mortbay.start.Main.main(Main.java:119)
> > 13164 [main] INFO org.mortbay.log - Started 
> > 
 :8983
> > 
> > On Mon, Apr 9, 2012 at 2:04 PM, Bryan McLellan 
> > <
 >
> >  wrote:
> > > On Mon, Apr 9, 2012 at 12:36 PM, Paul Welch 
> > > <
 >
> > >  wrote:
> > >> If im not mistaken it looks like an issue talking to Solr. However, I
> > >> am not sure why it would work on initial install but stop working
> > >> later...
> > >
> > > Anything interesting in the solr log? Usually /var/log/chef/solr.log.
> > >
> > > Bryan