[chef] Re: Re: Re: disk running out of space

[Spike Grobstein < >]

Hi Mark,

I did that and ran my compaction script again...

in the couchdb web UI, the size of the DB went from 41.9GB to 41.3GB and 
doesn't seem to be getting smaller. Additionally I did:

$ curl localhost:5984/chef
{"db_name":"chef","doc_count":11178,"doc_del_count":204,"update_seq":497447,"purge_seq":0,"compact_running":false,"disk_size":44302975077,"instance_start_time":"1353113098236259","disk_format_version":5,"committed_update_seq":497447}

and it says it's does not have a compact_running.

Is there anything else I can do?

Is there any chance that this will reduce the amount of space used on-disk? I 
know some database systems will keep the space on-disk requisitioned, but I'm 
not super experienced with couchdb.

Thanks.



...spike

On Nov 16, 2012, at 7:36 PM, Mark Anderson wrote:

> Check that you've set the _revs_limit for the db, otherwise compaction
> will retain more versions than you need.
> 
> Something like:
> % curl -X PUT -d '1' 'localhost:5984/chef/_revs_limit'
> 
> Sets it to 1, which is fine for chef.
> 
> 
> From:  Spike Grobstein 
> <
 >
> Reply-To:  
> "
 "
>  
> <
 >
> Date:  Friday, November 16, 2012 4:26 PM
> To:  
> "
 "
>  
> <
 >
> Subject:  [chef] Re: disk running out of space
> 
> 
> Hi Ranjib,
> 
> I may have had 20 version bumps ever. I only just recently started
> versioning as I needed to keep separate versions for prod and staging
> environments.
> 
> I can't imagine version bumps causing 40GB+ of disk usage, though. I may
> have uploaded my cookbooks a few thousand times, but I've only made 2 or 3
> changes in the last month.
> 
> could there be something else?
> 
> 
> ...spike
> 
> 
> 
> On Nov 16, 2012, at 7:23 PM, Ranjib Dey wrote:
> 
> 
> Do you bump up the cookbook versions too often? Check if you have older
> versions of cookbooks thst you dont use,
> 
> On Nov 16, 2012 3:40 PM, "Spike Grobstein" 
> <
 >
> wrote:
> 
> btw, this is my chef compaction script:
> 
> https://gist.github.com/4091940
> 
> forgot to include that before.
> 
> 
> 
> ...spike
> 
> On Nov 16, 2012, at 6:38 PM, Spike Grobstein wrote:
> 
> 
> Hi
> 
> I've been running chef-server since about March of this year on our
> infrastructure for 81 nodes. After running out of space twice due to
> couchDB compaction being incorrectly configured, I finally got it working
> right, but I'm still beginning to run out of space
> at a rate of around 1.5GB per week.
> 
> I began poking around in the couchDB web UI and we've got over 11,000
> documents stored in the 'chef' database taking up 42GB. Looking at the
> docs, they're mostly 'sandbox' objects from what I can tell, with the
> occasional 'data_bag' object. Also, the vast majority
> of the sandbox objects I've looked at have a 'create_time' of march or
> april of this year.
> 
> So my questions are:
> 
> 1. is there any way to clean up these old documents
> 2. is there any way to prevent couchdb from getting larger? I have a 60GB
> disk allocated exclusively for couchdb data and that's a lot larger than
> I'd really like
> 3. could it be something else that's misconfigured?
> 
> Some details about the installation:
> 
> The server is Ubuntu and chef-server was installed via the OpsCode PPA.
> We're running chef 0.10.8.
> 
> I've got a couple of recipes that set node attributes when they run;
> primarily datestamps for when some recipes are run so they don't get run
> on a regular basis and things like that, but I'm not aware of anything
> else I could be doing that would cause such
> growth in the database.
> 
> Hopefully you guys can shed some light on this.
> 
> Thanks!
> 
> 
> 
> ...spike
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
>