[lustre-discuss] Memory Management in Lustre

[Jeff Johnson]

Ellis,

I haven't messed with it much personally but if you look at some of
the Lustre module parameters, like in the case of module obdclass, you
will see some options that could be of interest like lu_cache_percent.

I'm sure a Whamcloud person might chime in with more detail.

# modinfo obdclass
filename:       /lib/modules/3.10.0-957.27.2.el7.DPC.x86_64/extra/obdclass.ko.xz
license:        GPL
version:        2.12.2
description:    Lustre Class Driver
author:         OpenSFS, Inc. <http://www.lustre.org/>
alias:          fs-lustre
retpoline:      Y
rhelversion:    7.6
srcversion:     3D7126D7BB611F089C67867
depends:        libcfs,lnet,crc-t10dif
vermagic:       3.10.0-957.27.2.el7.DPC.x86_64 SMP mod_unload modversions
parm:           lu_cache_percent:Percentage of memory to be used as
lu_object cache (int)
parm:           lu_cache_nr:Maximum number of objects in lu_object cache (long)
parm:           lprocfs_no_percpu_stats:Do not alloc percpu data for
lprocfs stats (int)

--Jeff


On Wed, Jan 19, 2022 at 6:35 PM Ellis Wilson via lustre-discuss
<lustre-discuss at lists.lustre.org> wrote:
>
> Hi folks,
>
> Broader (but related) question than my current malaise with OOM issues on 2.14/2.15:  Is there any documentation or can somebody point me at some code that explains memory management within Lustre?  I've hunted through Lustre manuals, the Lustre internals doc, and a bunch of code, but can find nothing that documents the memory architecture in place.  I'm specifically looking at PTLRPC and OBD code right now, and I can't seem to find anywhere that explicitly limits the amount of allocations Lustre will perform.  On other filesystems I've worked on there are memory pools that you can explicitly size with maxes, and while these may be discrete between areas or reference counters used to leverage a system-shared pool, I expected to see /something/ that might bake in limits of some kind.  I'm sure I'm just not finding it.  Any help is greatly appreciated.
>
> Best,
>
> ellis
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org



-- 
------------------------------
Jeff Johnson
Co-Founder
Aeon Computing

jeff.johnson at aeoncomputing.com
www.aeoncomputing.com
t: 858-412-3810 x1001   f: 858-412-3845
m: 619-204-9061

4170 Morena Boulevard, Suite C - San Diego, CA 92117

High-Performance Computing / Lustre Filesystems / Scale-out Storage