[Lustre-discuss] Out of Memory on MDS

[Andreas Dilger]

On Jun 23, 2009  16:50 -0400, Roger Spellman wrote:
> The servers are 1.6.7.1.  The clients are a mix of 1.6.7.1 and 1.6.6.
> 
> Is lru_size an upper limit on the number of entries?
> 
> Also, lru_max_age does not seem to be working.  I set it to 10 seconds,
> and it did not clean anything up.

This is worth filing a bug on, if it isn't working.  The lock LRU size
should be limited by the size of the RAM.  The upper limit on the number
of locks being granted can be gotten via:

	lctl get_param ldlm.namespaces.*.pool.limit

The default limit should be 50 locks per 1MB of RAM.  In your case,
4GB is 4096MB, so the LRU limit should be 50 * 4096 = 204800 locks.

> > Roger Spellman wrote:
> > > I have an MDS that is crashing with out-of-memory.
> > >
> > > Prior to the crash, I started collecting /proc/slabinfo.  I see that
> > > ldlm_locks is up to 4,500,000, and each one is 512 bytes, for a
> > > total of 2.2GB, which is more than half my RAM.
> > >
> > > Is there a way to limit this?
> > 
> > You don't mention the version of Lustre - lru_size might have an
> > impact, I am not certain. I believe it is the only lock tuneable
> > of note. (and is auto-sized in recent Lustre)

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.