[chef] Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: segfault with CentOS 5.4/5.5 + ruby 1.8.7 + chef 0.10.0

[Sergio Rubio < >]

On Thu, Jun 9, 2011 at 10:56 AM, James < "> > wrote:

Sergio,

The problem is definitely the ruby 1.8.7 build for el5 on i386. If I remember correctly, we built this using mock, and had a lot of trouble with it. We have since switched to using native VMs for building packages, and will be releasing a new round very soon. This round will also resolve CHEF-2388.

Cool.

All rbel packages are built using mock. Can't really understand why mocking them make them segfault randomly, but let me know if non-mock builds fix the issue.

On a side note, ruby 1.8.7 p302 segfaults too from time to time.

I'm not opposed to switching to, or at least including 1.9. One challenge is that every other ruby package in the CentOS repos use the 1.8 ABI; to run 1.9 you either need to not install any of these packages, or have both 1.8 and 1.9 installed.

Right. Replacing 1.8 with 1.9 means breaking stuff in RHEL. Making 1.9 parallel installable is the right thing to do IMO. We need an alternatives mechanism like Debian's to switch between different ruby implementations seamlessly though. 

Rgds.

James

On Thu, Jun 9, 2011 at 1:25 AM, Sergio Rubio < " target="_blank"> > wrote:

On Thu, Jun 9, 2011 at 9:46 AM, KC Braunschweig < " target="_blank"> > wrote:

These issues seem to keep cropping up. I'm wondering if this really
means that its just time to press on to Ruby 1.9.2 and leave 1.8.x
behind. I also noticed when I got my Velocity registration today that
the recommended steps for installing Chef for use in the Chef training
session suggested using Ruby 1.9.2.

Fortunately for me, I don't run Ruby infrastructure for anything other
than Chef so I don't really care much. Just want it to work and
minimize change. Still looking forward to a fatty installer.

I'm dumb testing previous patch levels of ruby right now. I've uploaded i386 pacakges to http://rbel.frameos.org/misc if you are interested:

rpm -Uvh --force http://rbel.frameos.org/misc/ruby-1.8.7.302-1.el5.i386.rpm http://rbel.frameos.org/misc/ruby-libs-1.8.7.302-1.el5.i386.rpm

should do the trick downgrading your ruby version if it's higher. Patch level 302 was included by F14.

Besides that, RBEL ruby-1.9 is parallel installable with 1.8 if that helps.

Cheers!

KC

On Thu, Jun 9, 2011 at 12:30 AM, Sergio Rubio < " target="_blank"> > wrote:
> Hey guys,
> On Fri, Jun 3, 2011 at 11:05 AM, Raymond Tham < " target="_blank"> > wrote:
>>
>> That's right, we noticed the seg faults happening on our i386 systems; we
>> don't have chef-client set up on any x86_64 instances at the moment.
>
>
> I wonder if you made any progress with this stuff. Been swamped lately, but
> I closely follow they list and RPM related threads. I'll get some fresh air
> soon and I was wondering if testing i386 packages is the best way to help
> here right now.
> Thanks!
>
>>
>> Thanks for looking into this!
>>
>> On Fri, Jun 3, 2011 at 1:44 AM, James < " target="_blank"> > wrote:
>>>
>>> The problem is in our (aegisco) build of 1.8.7 for i386, which RBEL is
>>> using. We'll work on rebuilding that this weekend. All of your systems are
>>> i386, correct?
>>> Thanks a lot for the report and information!
>>> James
>>> On Thu, Jun 2, 2011 at 11:21 PM, Raymond Tham < " target="_blank"> >
>>> wrote:
>>>>
>>>> Hi James,
>>>> Sorry, I should have mentioned that the AMI (ami-31df8e74) is in US
>>>> West, not East. It should be an S3 backed RightScale AMI.
>>>> I checked /root/.gemrc, and it points
>>>> to http://ec2-us-west-mirror.rightscale.com/rubygems/archive/latest/.
>>>> However, the gems should have been installed through yum through the RBEL5
>>>> i386 repo.
>>>> If it helps, here's the output from "gem list" after installing through
>>>> yum.
>>>> ~]# gem list
>>>> *** LOCAL GEMS ***
>>>> abstract (1.0.0)
>>>> activesupport (3.0.3)
>>>> allison (2.0.3)
>>>> bunny (0.6.0)
>>>> chef (0.10.0)
>>>> diff-lcs (1.1.2)
>>>> echoe (4.5.6)
>>>> erubis (2.6.6)
>>>> highline (1.6.1)
>>>> json (1.4.6)
>>>> mime-types (1.16)
>>>> mixlib-authentication (1.1.4)
>>>> mixlib-cli (1.2.0)
>>>> mixlib-config (1.1.2)
>>>> mixlib-log (1.3.0)
>>>> moneta (0.6.0)
>>>> net-sftp (2.0.4)
>>>> net-ssh (2.1.4, 2.0.23)
>>>> net-ssh-gateway (1.0.1)
>>>> net-ssh-multi (1.0.1)
>>>> ohai (0.6.4)
>>>> polyglot (0.3.1)
>>>> rake (0.8.7)
>>>> rake-compiler (0.7.5)
>>>> rest-client (1.6.1)
>>>> rspec (2.5.0)
>>>> rspec-core (2.5.1)
>>>> rspec-expectations (2.5.0)
>>>> rspec-mocks (2.5.0)
>>>> systemu (1.2.0)
>>>> treetop (1.4.9)
>>>> uuidtools (2.1.1)
>>>> xml-simple (1.0.12)
>>>> yajl-ruby (0.8.2)
>>>>
>>>> On Thu, Jun 2, 2011 at 10:31 PM, James < " target="_blank"> > wrote:
>>>>>
>>>>> Ray,
>>>>> The ami you gave doesn't seem to be public, but I found one which the
>>>>> same CentOS and RightImage numbers: ami-2342a94a.
>>>>> Can you double-check that it's a public image, so I can make sure to
>>>>> mirror you as closely as possible?
>>>>> Did you also experience
>>>>> this: http://tickets.opscode.com/browse/CHEF-2402
>>>>> Thanks,
>>>>> James
>>>>> On Thu, Jun 2, 2011 at 8:19 PM, Raymond Tham < " target="_blank"> >
>>>>> wrote:
>>>>>>
>>>>>> Hi James,
>>>>>> I originally noticed this happening on our older physical servers
>>>>>> running i386 CentOS 5.4 and 5.5, but I just created an EC2 instance and was
>>>>>> able to reproduce it. Here's what I did:
>>>>>> 1. Created a small instance using RightScale's 5.4 i386 v.5.5.9 AMI
>>>>>> (ami-31df8e74)
>>>>>> 2. Added RBEL: rpm -Uvh http://rbel.frameos.org/rbel5
>>>>>> 3. Installed rubygem-chef (excluding EPEL so all the ruby packages are
>>>>>> from RBEL): yum install --disablerepo=epel rubygem-chef
>>>>>> 4. Manually created a /etc/chef/client.rb and copied over
>>>>>> validation.pem.
>>>>>> 5. Ran chef-client for the first time to register the node:
>>>>>> /usr/bin/chef-client
>>>>>> 6. Added recipe[logrotate] to the run list (this is the cookbook from
>>>>>> opscode)
>>>>>> 7. Re-ran chef-client which picked up the recipe and seg faults trying
>>>>>> to run it
>>>>>> With regards to the ruby 1.8.6 that ended up working:
>>>>>> 1. I took Fedora 13's ruby-1.8.6.399-1.fc13.src.rpm, dropped in
>>>>>> 1.8.6.420 and repackaged on CentOS 5.5.
>>>>>> 2. Took Fedora 13's rubygems-1.3.6-1.fc13.src.rpm and repackaged it on
>>>>>> CentOS 5.5 as is (but had to make a minor tweak to one of the patch files)
>>>>>> 3. Excluded ruby-*, rubygems-*, and rubygem-chef-0.10* in the RBEL yum
>>>>>> config.
>>>>>> 4. Ran yum install rubygem-chef with my new RPMs.
>>>>>> It pulled my repackaged RPMs (for ruby, ruby-lib, rubygems, and
>>>>>> rubygem-chef) and everything else from RBEL.
>>>>>> With regards to my Chef server (if that helps), it's running Chef 0.10
>>>>>> with all RBEL packages on a custom CentOS 5.6 x86_64 AMI.
>>>>>> Hope that helps!
>>>>>> Thanks,
>>>>>> Ray
>>>>>> On Thu, Jun 2, 2011 at 6:45 PM, James < " target="_blank"> > wrote:
>>>>>>>
>>>>>>> Ray,
>>>>>>> Can you provide the means to reproduce, specifically: steps to
>>>>>>> install from the RBEL repo, and the AMI / image info? Are these i386
>>>>>>> instances?
>>>>>>> We maintain packages at rpm.aegisco.com, but I'm happy to try to help
>>>>>>> debug the issues you're having. I'm surprised that Ruby 1.8.6 worked, I had
>>>>>>> trouble getting 0.10 to work on anything older than 1.8.7 and rubygems
>>>>>>> 1.6.2.
>>>>>>> James
>>>>>>>
>>>>>>> On Thu, Jun 2, 2011 at 6:29 PM, < " target="_blank"> > wrote:
>>>>>>>>
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>> I noticed a couple of email threads in the past about seg faults and
>>>>>>>> didn't see
>>>>>>>> any resolution so I wanted to share my experience with the same
>>>>>>>> thing happening
>>>>>>>> to us on a few CentOS 5.4 and 5.5 servers (we haven't tested the
>>>>>>>> client on 5.6
>>>>>>>> yet). I'm consistently able to get our chef-client to segfault
>>>>>>>> whenever it
>>>>>>>> handles this package resource (haven't had time to try other
>>>>>>>> packages yet):
>>>>>>>>
>>>>>>>> [Thu, 02 Jun 2011 17:38:17 -0700] INFO: Processing
>>>>>>>> package[logrotate] action
>>>>>>>> upgrade (logrotate::default line 20)
>>>>>>>>
>>>>>>>> /usr/lib/ruby/gems/1.8/gems/chef-0.10.0/bin/../lib/chef/provider/package/yum.rb:75:
>>>>>>>> [BUG] Segmentation fault
>>>>>>>>
>>>>>>>> Thu, 02 Jun 2011 17:38:27 -0700] INFO: Processing package[logrotate]
>>>>>>>> action
>>>>>>>> upgrade (logrotate::default line 20)
>>>>>>>>
>>>>>>>> /usr/lib/ruby/gems/1.8/gems/chef-0.10.0/bin/../lib/chef/provider/package/yum.rb:64:
>>>>>>>> [BUG] Segmentation fault
>>>>>>>>
>>>>>>>> [Thu, 02 Jun 2011 17:38:53 -0700] INFO: Processing
>>>>>>>> package[logrotate] action
>>>>>>>> upgrade (logrotate::default line 20)
>>>>>>>>
>>>>>>>> /usr/lib/ruby/gems/1.8/gems/chef-0.10.0/bin/../lib/chef/provider/package/yum.rb:66:
>>>>>>>> [BUG] Segmentation fault
>>>>>>>>
>>>>>>>> We were trying this with ruby 1.8.7 p334 from the FrameOS rbel5 i386
>>>>>>>> repo.
>>>>>>>> Since it sounded like an issue with ruby, reinstalling with ruby
>>>>>>>> 1.8.6 p420
>>>>>>>> (and rebuilding the rbel5 rubygem-chef rpm to relax the ruby version
>>>>>>>> requirement) seemed to solve the problem.
>>>>>>>>
>>>>>>>> Is anyone else noticing seg fauls with a combination of CentOS 5.x,
>>>>>>>> ruby 1.8.7
>>>>>>>> and Chef?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Ray
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
>