[lustre-discuss] Missing Files in /proc/fs/lustre after Upgrading to Lustre 2.15.X

[Andreas Dilger]

On May 4, 2023, at 16:43, Jane Liu via lustre-discuss <lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>> wrote:

Hi,

We previously had a monitoring tool in Lustre 2.12.X that relied on files located under /proc/fs/lustre for gathering metrics. However, after upgrading our system to version 2.15.2, we noticed that at least five files previously found under /proc/fs/lustre are no longer present. Here is a list of these files as an example:

/proc/fs/lustre/osd-ldiskfs/fsname-OST0078/brw_stats
/proc/fs/lustre/osd-ldiskfs/fsname-OST0078/kbytestotal
/proc/fs/lustre/osd-ldiskfs/fsname-OST0078/kbytesfree
/proc/fs/lustre/osd-ldiskfs/fsname-OST0078/filestotal
/proc/fs/lustre/osd-ldiskfs/fsname-OST0078/filesfree

We have been unable to locate these files in the new version. We can still obtain size information using the following commands:

lctl get_param obdfilter.*.kbytestotal
lctl get_param obdfilter.*.kbytesfree
lctl get_param obdfilter.*.filestotal
lctl get_param obdfilter.*.filesfree

However, we are unsure how to access the information previously available in the brw_stats file. Any guidance or suggestions would be greatly appreciated.

You've already partially answered your own question - the parameters for "lctl get_param" are under "osd-ldiskfs.*.{brw_stats,kbytes*,files*}" and not "obdfilter.*.*", but they have (mostly) moved from /proc/fs/lustre/osd-ldiskfs to /sys/fs/lustre/osd-ldiskfs.  In the case of brw_stats they are under /sys/kernel/debug/lustre/osd-ldiskfs.

These stats actually moved from obdfilter to osd-ldiskfs back in Lustre 2.4 when the ZFS backend was added, and a symlink has been kept until now for compatibility.  That means your monitoring tool should still work with any modern Lustre version if you change the path. The move of brw_stats to /sys/kernel/debug/lustre was mandated by the upstream kernel and only happened in 2.15.0.

Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Whamcloud







-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20230505/3145b17e/attachment.htm>