- From: Kenneth Stailey <
>
- To:
- Subject: [chef] Re: Re: Sudden Increase in Merb RAM Use
- Date: Wed, 3 Feb 2010 13:58:44 -0800 (PST)
- Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=fVuJcY7JEgw0sPuiq7aWe6hKnWQ42BCMF2v82uYx11wzg5cZuN4//x0dq2rbpGKPSEtEE5UTqYQQQa6hw4yvvvPv5Us2x0Kgkx50RCW0Jh4mg1OrqsE4bQvi1oZ63pmO6FPj9tS2rKC6XiON/RECHVTDXURtO/WIZseCs8JX2Bg=;
Particulars:
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 9.10
Release: 9.10
Codename: karmic
$ uname -m
x86_64
$ COLUMNS=40 dpkg -l | grep chef
ii chef 0.7.16-1 configuration management system written in R
ii chef-indexer 0.7.16-1 Creates search indexes of Chef node attribut
ii chef-server 0.7.16-1 Merb application providing centralized manag
ii chef-server-sl 0.7.16-1 Merb app slice providing centralized managem
ii libchef-ruby 0.7.16-1 Ruby libraries for Chef configuration manage
ii libchef-ruby1. 0.7.16-1 Ruby 1.8 libraries for Chef configuration ma
Looking closer the increase in RAM usage is not sudden. I was looking at a
chart of "free memory" *blush* so the "spikes" were really reboots resulting
in sudden increase of RAM.
However, the merb worker was using an rss of ~2.3 GiB
$ ps vax --sort=-rss |head
PID TTY STAT TIME MAJFL TRS DRS RSS %MEM COMMAND
29513 ? R 2192:16 1141997 2 2549253 2352616 79.9 merb : worker
(port 4000)
9882 ? S 39:31 31673 2 172461 111188 3.7 ruby
/usr/bin/chef-indexer -d -c /etc/chef/indexer.rb
29514 ? S 20:39 55304 2 132805 58284 1.9 merb : worker (port
4001)
29512 ? S 811:15 11118 2 104457 29028 0.9 merb : spawner
(ports 4000)
Since the reboot the rss of the restarted merb worker has climbed steadily
from about 185 MiB to 320 MiB
$ ps vax --sort=-rss |head -n 5
PID TTY STAT TIME MAJFL TRS DRS RSS %MEM COMMAND
1728 ? S 105:02 1 2 400393 322248 6.6 merb : worker
(port 4000)
1613 ? S 0:12 0 2 128161 78968 1.6 /usr/bin/ruby1.8
/usr/bin/stompserver -C /etc/stompserver.conf
1729 ? S 1:08 0 2 109313 36864 0.7 merb : worker (port
4001)
1727 ? S 41:55 3 2 105145 32532 0.6 merb : spawner
(ports 4000)
Does it have a memory leak or is it merely a resource hog?
Thanks,
Ken
--- On Wed, 2/3/10, Adam Jacob
<
>
wrote:
>
From: Adam Jacob
>
<
>
>
Subject: [chef] Re: Sudden Increase in Merb RAM Use
>
To:
>
"
"
>
>
<
>
>
Cc:
>
"
"
>
>
<
>
>
Date: Wednesday, February 3, 2010, 11:16 AM
>
What's the frequency of the
>
spike? Does it correlate to a particular set of nodes
>
checking in?
>
Just to be sure, this is on 0.7.x?
>
>
Sent from my iPhone
>
On Feb 3, 2010, at 8:10 AM, Kenneth Stailey
>
<
>
>
wrote:
>
>
Hi,
>
>
merb RAM usage (rss) appears to stay flat for a long time
>
and then suddenly shoots up.
>
>
After the RAM usage has spiked it looks like this:
>
>
$ ps vax --sort=-rss |head
>
PID TTY
>
STAT TIME MAJFL
>
TRS DRS RSS %MEM COMMAND
>
29513 ?
>
R 2192:16 1141997 2 2549253
>
2352616 79.9 merb : worker (port
>
>
4000)
>
>
9882 ?
>
S 39:31
>
31673 2 172461 111188 3.7 ruby
>
/usr/bin/chef-indexer -d -c /etc/chef/indexer.rb
>
29514 ?
>
S 20:39
>
55304 2 132805 58284 1.9 merb
>
: worker (port
>
4001)
>
>
29512 ?
>
S 811:15
>
11118 2 104457 29028 0.9 merb
>
: spawner (ports
>
>
4000)
>
>
10213 ?
>
S 3:37
>
119973 2 301561 18080 0.6
>
/usr/bin/ruby1.8 /usr/bin/stompserver -C
>
/etc/stompserver.conf
>
15666 ?
>
Sl 0:15 1473 1617
>
111930 10768 0.3
>
/usr/lib/erlang/erts-5.7.2/bin/beam.smp -Bd -K true -- -root
>
/usr/lib/erlang -progname erl -- -home /var/lib/couchdb
>
-noshell -noinput -smp auto -sasl errlog_type error -pa
>
/usr/lib/couchdb/erlang/lib/couch-0.10.0/ebin
>
/usr/lib/couchdb/erlang/lib/mochiweb-r97/ebin
>
/usr/lib/couchdb/erlang/lib/ibrowse-1.5.2/ebin
>
/usr/lib/couchdb/erlang/lib/erlang-oauth/ebin -eval
>
application:load(ibrowse) -eval application:load(oauth)
>
-eval application:load(crypto) -eval application:load(couch)
>
-eval crypto:start() -eval ssl:start() -eval ibrowse:start()
>
-eval couch_server:start([
>
"/etc/couchdb/default.ini",
>
"/etc/couchdb/local.ini",
>
"/etc/couchdb/default.ini",
>
"/etc/couchdb/local.ini"]), receive done ->
>
done end. -pidfile /var/run/couchdb/couchdb.pid -heart
>
>
Bumped up VM that is running Chef server from 3 GB RAM to 5
>
GB RAM.
>
>
After the reboot merb is only using about 185 MiB rss vs.
>
2.3 GiB
>
>
$ ps vax --sort=-rss |head
>
PID TTY
>
STAT TIME MAJFL
>
TRS DRS RSS %MEM COMMAND
>
29513 ?
>
R 2192:16
>
1141997 2 2549253 2352616 79.9 merb : worker
>
(port
>
4000)
>
>
9882 ?
>
S 39:31
>
31673 2 172461 111188 3.7 ruby
>
/usr/bin/chef-indexer -d -c /etc/chef/indexer.rb
>
29514 ?
>
S 20:39
>
55304 2 132805 58284 1.9 merb
>
: worker (port
>
>
4001)
>
>
29512 ?
>
S 811:15
>
11118 2 104457 29028 0.9 merb
>
: spawner (ports
>
4000)
>
>
10213
>
?
>
S 3:37
>
119973 2 301561 18080 0.6
>
/usr/bin/ruby1.8 /usr/bin/stompserver -C
>
/etc/stompserver.conf
>
15666 ?
>
Sl 0:15 1473 1617
>
111930 10768 0.3
>
/usr/lib/erlang/erts-5.7.2/bin/beam.smp -Bd -K true -- -root
>
/usr/lib/erlang -progname erl -- -home /var/lib/couchdb
>
-noshell -noinput -smp auto -sasl errlog_type error -pa
>
/usr/lib/couchdb/erlang/lib/couch-0.10.0/ebin
>
/usr/lib/couchdb/erlang/lib/mochiweb-r97/ebin
>
/usr/lib/couchdb/erlang/lib/ibrowse-1.5.2/ebin
>
/usr/lib/couchdb/erlang/lib/erlang-oauth/ebin -eval
>
application:load(ibrowse) -eval application:load(oauth)
>
-eval application:load(crypto) -eval application:load(couch)
>
-eval crypto:start() -eval ssl:start() -eval ibrowse:start()
>
-eval couch_server:start([
>
"/etc/couchdb/default.ini",
>
"/etc/couchdb/local.ini",
>
"/etc/couchdb/default.ini",
>
"/etc/couchdb/local.ini"]), receive done ->
>
done end. -pidfile /var/run/couchdb/couchdb.pid -heart
>
>
We are keeping charts of RAM use over time via Nagios/RRD
>
and they show the increase in RAM usage happens suddenly not
>
gradually. It also repeats over time.
>
>
Do other people see this issue?
>
>
Thanks,
>
Ken
>
>
Archive powered by MHonArc 2.6.16.