[Lustre-discuss] Out of Memory on MDS

[Seger, Mark]

If you're looking to collect slab info, collectl exceeds at this.  Just download it from http://collectl.sourceforge.net/, install and "/etc/init.d/collectl start" and will collectl MDS stats every 10 seconds and slab stats every minute (easily changeable) along with a ton of other stats.  You can then play back the recorded data showing slabs that changed during each interval or even report the 10-n slabs (default=10) sorted by a variety of fields (collectl -showtopopts for help on top options).
-mark

From: lustre-discuss-bounces at lists.lustre.org [mailto:lustre-discuss-bounces at lists.lustre.org] On Behalf Of Roger Spellman
Sent: Tuesday, June 23, 2009 12:56 PM
To: lustre-discuss at lists.lustre.org
Subject: [Lustre-discuss] Out of Memory on MDS

I have an MDS that is crashing with out-of-memory.

Prior to the crash, I started collecting /proc/slabinfo.  I see that ldlm_locks is up to 4,500,000, and each one is 512 bytes, for a total of 2.2GB, which is more than half my RAM.

Is there a way to limit this?

Other heavy memory users are ldisk_inode_cache (421 MB) and ldlm_resources (137 MB).  Is there a way to limit these too?

Thanks.

Roger Spellman
Staff Engineer
Terascala, Inc.
508-588-1501
www.terascala.com http://www.terascala.com/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20090623/c8bb8134/attachment.htm>