[lustre-discuss] osc and llite stats files are not exported to clients

Fri Jul 26 01:56:34 PDT 2019

:-)  Found them.

Many thanks to Chris Horn.

---------------------------------------------------------------------------------------
Utane Sawangwit, PhD.
Research Astronomer
National Astronomical Research Institute of Thailand (NARIT)
Sirindhon AstroPark, 260, Mue 4, Donkaew
Mae-Rim, Chiang Mai, 50180, Thailand
----------------------------------------------------------------------------------------
ดร. อุเทน แสวงวิทย์
นักวิจัยชำนาญการ
สถาบันวิจัยดาราศาสตร์แห่งชาติ(องค์การมหาชน)
อุทยานดาราศาสตร์สิรินธร 260 หมู่ 4 ต.ดอนแก้ว
อ.แม่ริม จ.เชียงใหม่ 50180
---------------------------------------------------------------------------------------

On Fri, 26 Jul 2019 at 10:09, Chris Horn <hornc at cray.com> wrote:

> The stats files are in /sys/kernel/debug. Elevated privileges are required
> to read them. See https://jira.whamcloud.com/browse/LU-11850
>
> # find /sys/kernel/debug/lustre -name stats
>
> /sys/kernel/debug/lustre/ldlm/services/ldlm_canceld/stats
>
> /sys/kernel/debug/lustre/ldlm/services/ldlm_cbd/stats
>
>
> /sys/kernel/debug/lustre/ldlm/namespaces/lustre-OST0001-osc-MDT0000/pool/stats
>
>
> /sys/kernel/debug/lustre/ldlm/namespaces/lustre-OST0000-osc-MDT0000/pool/stats
>
>
> /sys/kernel/debug/lustre/ldlm/namespaces/lustre-OST0001-osc-ffff92312e7e2800/pool/stats
>
>
> /sys/kernel/debug/lustre/ldlm/namespaces/lustre-OST0000-osc-ffff92312e7e2800/pool/stats
>
>
> /sys/kernel/debug/lustre/ldlm/namespaces/lustre-MDT0000-mdc-ffff92312e7e2800/pool/stats
>
>
> /sys/kernel/debug/lustre/ldlm/namespaces/lustre-MDT0000-lwp-OST0001/pool/stats
>
>
> /sys/kernel/debug/lustre/ldlm/namespaces/filter-lustre-OST0001_UUID/pool/stats
>
>
> /sys/kernel/debug/lustre/ldlm/namespaces/lustre-MDT0000-lwp-OST0000/pool/stats
>
>
> /sys/kernel/debug/lustre/ldlm/namespaces/filter-lustre-OST0000_UUID/pool/stats
>
>
> /sys/kernel/debug/lustre/ldlm/namespaces/lustre-MDT0000-lwp-MDT0000/pool/stats
>
> /sys/kernel/debug/lustre/ldlm/namespaces/mdt-lustre-MDT0000_UUID/pool/stats
>
> /sys/kernel/debug/lustre/ldlm/namespaces/MGC192.168.2.30 at tcp/pool/stats
>
> /sys/kernel/debug/lustre/ldlm/namespaces/MGS/pool/stats
>
> /sys/kernel/debug/lustre/llite/lustre-ffff92312e7e2800/stats
>
> /sys/kernel/debug/lustre/mdc/lustre-MDT0000-mdc-ffff92312e7e2800/stats
>
> /sys/kernel/debug/lustre/ost/OSS/ost_out/stats
>
> /sys/kernel/debug/lustre/ost/OSS/ost_seq/stats
>
> /sys/kernel/debug/lustre/ost/OSS/ost_io/stats
>
> /sys/kernel/debug/lustre/ost/OSS/ost_create/stats
>
> /sys/kernel/debug/lustre/ost/OSS/ost/stats
>
> /sys/kernel/debug/lustre/osc/lustre-OST0001-osc-ffff92312e7e2800/stats
>
> /sys/kernel/debug/lustre/osc/lustre-OST0000-osc-ffff92312e7e2800/stats
>
> /sys/kernel/debug/lustre/lwp/lustre-MDT0000-lwp-OST0001/stats
>
> /sys/kernel/debug/lustre/lwp/lustre-MDT0000-lwp-OST0000/stats
>
> /sys/kernel/debug/lustre/lwp/lustre-MDT0000-lwp-MDT0000/stats
>
> /sys/kernel/debug/lustre/osp/lustre-OST0001-osc-MDT0000/stats
>
> /sys/kernel/debug/lustre/osp/lustre-OST0000-osc-MDT0000/stats
>
> /sys/kernel/debug/lustre/mds/MDS/mdt_io/stats
>
> /sys/kernel/debug/lustre/mds/MDS/mdt_fld/stats
>
> /sys/kernel/debug/lustre/mds/MDS/mdt_seqm/stats
>
> /sys/kernel/debug/lustre/mds/MDS/mdt_seqs/stats
>
> /sys/kernel/debug/lustre/mds/MDS/mdt_out/stats
>
> /sys/kernel/debug/lustre/mds/MDS/mdt_setattr/stats
>
> /sys/kernel/debug/lustre/mds/MDS/mdt_readpage/stats
>
> /sys/kernel/debug/lustre/mds/MDS/mdt/stats
>
> /sys/kernel/debug/lustre/mgs/MGS/mgs/stats
>
>
>
> Chris Horn
>
>
>
> *From: *lustre-discuss <lustre-discuss-bounces at lists.lustre.org> on
> behalf of U Sawangwit <u.sawangwit at gmail.com>
> *Date: *Thursday, July 25, 2019 at 9:58 PM
> *To: *"Degremont, Aurelien" <degremoa at amazon.com>
> *Cc: *"lustre-discuss at lists.lustre.org" <lustre-discuss at lists.lustre.org>
> *Subject: *Re: [lustre-discuss] osc and llite stats files are not
> exported to clients
>
>
>
> Hi
>
>
>
>     Thank you very much for your reply.
>
>
>
> I looked in the /sys but also did not find the osc/*/stats or
> llite/*/stats files.
>
>
>
> I understand what you are saying about the lctl becoming the new standard
> way.
>
> But I just want to make sure that this is not due to sth. wrong with my
> system or setup.
>
>
>
> FYI, I am using CentOS 7.6 kernel 3.10.0-957.el7.x86_64 and *_lustre* for
> my clients and
>
> lustre servers with Lustre version 2.12.0 using IB FDR as the LNET.
>
>
>
> Would the missing stats files in /sys/fs/lustre or /proc/fs/lustre
> considered normal?
>
>
>
>  Any monitoring tool you would suggest to use instead of lustre
> gmond_python_modules in order
>
> to avoid reading stats files? My initial review of LMT3 also indicate that
> it too reading data from
>
> stats file rather than using lctl.
>
>
>
> Cheers,
>
> Utane
>
>
>
>
> ---------------------------------------------------------------------------------------
> Utane Sawangwit, PhD.
> Research Astronomer
> National Astronomical Research Institute of Thailand (NARIT)
> Sirindhon AstroPark, 260, Mue 4, Donkaew
>
> Mae-Rim, Chiang Mai, 50180, Thailand
>
>
> ----------------------------------------------------------------------------------------
> ดร. อุเทน แสวงวิทย์
> นักวิจัยชำนาญการ
> สถาบันวิจัยดาราศาสตร์แห่งชาติ(องค์การมหาชน)
> อุทยานดาราศาสตร์สิรินธร 260 หมู่ 4 ต.ดอนแก้ว
>
> อ.แม่ริม จ.เชียงใหม่ 50180
>
>
> ---------------------------------------------------------------------------------------
>
>
>
>
>
>
>
> On Thu, 25 Jul 2019 at 22:05, Degremont, Aurelien <degremoa at amazon.com>
> wrote:
>
> Lustre is transitionning /proc files to /sys per kernel developer requests.
>
>
>
> Instead of chasing where these files are located now depending on your
> Lustre version, 'lctl get_param' is the official way to read them.
>
> This command will figure out where to properly read the files.
>
>
>
> *De : *lustre-discuss <lustre-discuss-bounces at lists.lustre.org> au nom de
> U Sawangwit <u.sawangwit at gmail.com>
> *Date : *jeudi 25 juillet 2019 à 13:15
> *À : *"lustre-discuss at lists.lustre.org" <lustre-discuss at lists.lustre.org>
> *Objet : *[lustre-discuss] osc and llite stats files are not exported to
> clients
>
>
>
> Dear all,
>
>
>
>    I am having trouble with lustre monitoring using Ganglia
> gmond_python_modules
>
> (https://github.com/ganglia/gmond_python_modules/tree/master/lustre).
>
>
>
>     Upon closer inspections, I found that the stats files in
> /proc/fs/lustre/osc/*/
>
> and /proc/fs/lustre/llite are missing (among other similar files in these
> directories).
>
>
>
>      However, lctl get_param osc.*.stats and lctl get_param llite.*.stats
> return
>
> parameters and values as you would expect. The stats files are what the
>
> gmond_python_modules wants hence the problem monitoring with Ganglia.
>
>
>
>     After many searching around, could not figure out what I could have
> done to
>
> have stats export normally to clients and remounting did not help. On MDS
> and OSS,
>
> I can see the obdfiler/*/export/*/stats or mdt/*/export/*/stats
>
>
>
>     The lustre manual also said that the exporting of these info to client
> should be by
>
> default.
>
>
>
>   Thank you in advance.
>
>
>
> Cheers,
>
> Utane
>
>
>
>
> ---------------------------------------------------------------------------------------
> Utane Sawangwit, PhD.
> Research Astronomer
> National Astronomical Research Institute of Thailand (NARIT)
> Sirindhon AstroPark, 260, Mue 4, Donkaew
>
> Mae-Rim, Chiang Mai, 50180, Thailand
>
>
> ----------------------------------------------------------------------------------------
> ดร. อุเทน แสวงวิทย์
> นักวิจัยชำนาญการ
> สถาบันวิจัยดาราศาสตร์แห่งชาติ(องค์การมหาชน)
> อุทยานดาราศาสตร์สิรินธร 260 หมู่ 4 ต.ดอนแก้ว
>
> อ.แม่ริม จ.เชียงใหม่ 50180
>
>
> ---------------------------------------------------------------------------------------
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20190726/54590a7f/attachment-0001.html>