[Lustre-discuss] Lustre client memory usage very high

Guillaume Demillecamps guillaume at multipurpose.be
Thu Jul 30 23:56:22 PDT 2009


Hello again,


Not sure if it is interesting to be noted, but if I use the following  
command, my memory is freed:
sync; echo 3 > /proc/sys/vm/drop_caches
What is surprising, though, is that the cache never expires (at least  
it remains in memory for several days at the least).


Regards,


Guillaume Demillecamps





----- Message de adilger at sun.com ---------
     Date : Thu, 30 Jul 2009 16:45:47 -0600
      De : Andreas Dilger <adilger at sun.com>
  Objet : Re: [Lustre-discuss] Lustre client memory usage very high
       À : Guillaume Demillecamps <guillaume at multipurpose.be>
      Cc : lustre-discuss at lists.lustre.org


> On Jul 30, 2009  09:52 +0200, Guillaume Demillecamps wrote:
>>> On Jul 22, 2009  11:45 +0200, Guillaume Demillecamps wrote:
>>>> Lustre 1.8.0 on all servers / clients involved in this. OS is SLES 10
>>>> SP2 with un-patched kernel on the clients. I however has put the same
>>>> kernel revision downloaded from suse.com on the clients as the version
>>>> used in the Lustre-patched MGS/MDS/OSS servers. File system is only
>>>> several GBs, with ~500000 files. All inter-connections are through TCP.
>>>>
>>>> We have some “manual” replication of an active lustre file system to a
>>>> passive lustre file system. We have “sync” clients that just basically
>>>> mount both file systems and run large sync jobs from the active Lustre
>>>> to the passive Lustre. So far, so good (apart that it is quite a slow
>>>> process). However my issue is that Lustre is rising memory so high
>>>> that rsync cannot get enough RAM to finish its job before kswap kicks
>>>> in and slows things down drastically.
>
>> # name           <active>  <total> <size> <obj/slab>: slabdata  
>> <active> <num>
>> lustre_inode_cache  385652  385652    960    4      : slabdata  96413  96413
>> lov_oinfo          2929548 2929548    320   12      : slabdata 244129 244129
>> ldlm_locks          136262  254424    512    8      : slabdata  31803  31803
>> ldlm_resources      136183  256120    384   10      : slabdata  25612  25612
>
> This shows that we have 385k Lustre inodes, yet there are 2.9M "lov_oinfo"
> structs (there should only be a single one per inode).  I'm not sure
> why that is happening, but that is consuming about 1GB of RAM.  The 385k
> inode count is reasonable, given you have 500k files, per above.  There
> are 136k locks, which is also fine (probably so much lower than the inode
> count because of your short lock expiry time).
>
> So, it seems like a problem of some kind, and is probably deserving of
> filing a bug.
>
>
> Cheers, Andreas
> --
> Andreas Dilger
> Sr. Staff Engineer, Lustre Group
> Sun Microsystems of Canada, Inc.
>
>


----- Fin du message de adilger at sun.com -----





More information about the lustre-discuss mailing list