[lustre-discuss] Missing Files in /proc/fs/lustre after Upgrading to Lustre 2.15.X

[Jane Liu]

Hi Andreas,

Thanks. This helps.

Jane


On 2023-05-04 20:28, Andreas Dilger wrote:
>> On May 4, 2023, at 16:43, Jane Liu via lustre-discuss
>> <lustre-discuss at lists.lustre.org> wrote:
>> 
>> Hi,
>> 
>> We previously had a monitoring tool in Lustre 2.12.X that relied on
>> files located under /proc/fs/lustre for gathering metrics. However,
>> after upgrading our system to version 2.15.2, we noticed that at
>> least five files previously found under /proc/fs/lustre are no
>> longer present. Here is a list of these files as an example:
>> 
>> /proc/fs/lustre/osd-ldiskfs/fsname-OST0078/brw_stats
>> /proc/fs/lustre/osd-ldiskfs/fsname-OST0078/kbytestotal
>> /proc/fs/lustre/osd-ldiskfs/fsname-OST0078/kbytesfree
>> /proc/fs/lustre/osd-ldiskfs/fsname-OST0078/filestotal
>> /proc/fs/lustre/osd-ldiskfs/fsname-OST0078/filesfree
>> 
>> We have been unable to locate these files in the new version. We can
>> still obtain size information using the following commands:
>> 
>> lctl get_param obdfilter.*.kbytestotal
>> lctl get_param obdfilter.*.kbytesfree
>> lctl get_param obdfilter.*.filestotal
>> lctl get_param obdfilter.*.filesfree
>> 
>> However, we are unsure how to access the information previously
>> available in the brw_stats file. Any guidance or suggestions would
>> be greatly appreciated.
> 
>  You've already partially answered your own question - the parameters
> for "lctl get_param" are under
> "osd-ldiskfs.*.{brw_stats,kbytes*,files*}" and not "obdfilter.*.*",
> but they have (mostly) moved from /proc/fs/lustre/osd-ldiskfs to
> /sys/fs/lustre/osd-ldiskfs.  In the case of brw_stats they are under
> /sys/kernel/debug/lustre/osd-ldiskfs.
> 
> These stats actually moved from obdfilter to osd-ldiskfs back in
> Lustre 2.4 when the ZFS backend was added, and a symlink has been kept
> until now for compatibility.  That means your monitoring tool should
> still work with any modern Lustre version if you change the path. The
> move of brw_stats to /sys/kernel/debug/lustre was mandated by the
> upstream kernel and only happened in 2.15.0.
> 
> Cheers, Andreas
> --
> Andreas Dilger
> Lustre Principal Architect
> Whamcloud