[lustre-discuss] User find out OST configuration

[Andreas Dilger]

On Jan 23, 2023, at 10:01, Anna Fuchs <anna.fuchs at uni-hamburg.de<mailto:anna.fuchs at uni-hamburg.de>> wrote:

Thanks!

Is it planned to introduce some metric propagation to the user?
For advanced users who are benchmarking stuff on remote systems it remains unclear which performance to expect if they can not access underlaying hardware metrics.  Sure, they can ask the admin to share the config, but it might be more convenient to be able to look it up, maybe.

Yes,  there is a longstanding open ticket to export some server statistics to the clients - https://jira.whamcloud.com/browse/LU-7880 "add performance statistics to obd_statfs".  As the summary describes, this would export basic performance stats for each OST/MDT device to the client for current/peak x read/write x IOPS/bandwidth.  There are a number of potential uses for this, such as client/MDS selection of OSTs based on bandwidth/IOPS (beyond just "rotational" or "non-rotational"), userspace applications/libraries using it to determine if the OSTs are less busy (e.g. when to checkpoint), etc.

There aren't any plans to be able to export the storage "config" (e.g. RAID geometry) via Lustre since this is often opaque even on the server, and doesn't have any use on the client.  There was a discussion in the context of IO500 to write a script for collecting storage system configuration metadata for Lustre and other filesystems (e.g. list of OST/MDT devices, list of SCSI devices, PCI devices, CPU, RAM, network interfaces, etc.)

Additionally: if I try to find out the stripe location (lfs getstripe) and map this information to OST-specs (lctl get_param osc.*.*ost_conn_uuid), to find out how many different servers and networks are involved, the obdidx seems to be in dec-format, but the OST index in connections list is hex, which is not always obvious.  Is there a way to display it both in dec or both in hex?

There isn't currently an option for "lfs getstripe" to print in hex, but it would be possible to add a "--hex" option to print the fields in hex.  I've filed https://jira.whamcloud.com/browse/LU-16503 for tracking this issue.  I don't think it would be terribly complex to implement, but I also don't know if anyone is available to do this work right now.

Are there generally any tools for doing similar things?
We plan a student project for building kind of GUI for visualizing stripings and mappings, so I would try to avoid reinventing the wheel.

Depending on how complex your tool is, it may be better to use llapi_layout_from_file() or llapi_layout_from_xattr() to parse the binary layout directly from the file, rather than printing it out and then parsing the text again in userspace.

Cheers, Andreas

Am 21.01.2023 um 17:08 schrieb Andreas Dilger:
Hi Anna,
Beyond the number and size of OSTs and MDTs there isn't much information about the underlying storage available on the client.

The "lfs df -v" command will print a "f" at the end for flash (non-rotational) devices, if the storage is properly configured.  The "osc*.imports " parameter file will contain some information about the grant_block_size that can be used to distinguish ldiskfs (4096) vs. zfs backends (131072 or 1048576).

The size of the disks can often be inferred from 1/8 of the total OST size for standard 8+2 RAID configs, but this may vary and no actual device-level metrics are available on the client.

Even on the server, Lustre itself doesn't know or care much about the underlying storage devices beyond (non-)rotational state, so we don't track any of that.

Cheers, Andreas

On Jan 21, 2023, at 01:16, Anna Fuchs via lustre-discuss <lustre-discuss at lists.lustre.org><mailto:lustre-discuss at lists.lustre.org> wrote:

 Hi,

is it possible for a user (no root, so ssh to server) to find out the configuration of an OST?
How many devices are there in one OST 'pool' (for both ldiskfs and ZFS) and even which type of devices they are (nvme, ssd, hdd)? Maybe even speeds and raid-levels?

Additionally, how can a user find out the mapping of all available OSTs to OSSs easily?

Thanks
Anna

--
Anna Fuchs
Universität Hamburg
https://wr.informatik.uni-hamburg.de<https://wr.informatik.uni-hamburg.de/>

anna.fuchs at informatik.uni-hamburg.de<mailto:anna.fuchs at informatik.uni-hamburg.de>
https://wr.informatik.uni-hamburg.de/people/anna_fuchs

_______________________________________________
lustre-discuss mailing list
lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org


--
Anna Fuchs
Universität Hamburg
https://wr.informatik.uni-hamburg.de<https://wr.informatik.uni-hamburg.de/>

anna.fuchs at informatik.uni-hamburg.de<mailto:anna.fuchs at informatik.uni-hamburg.de>
https://wr.informatik.uni-hamburg.de/people/anna_fuchs

Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Whamcloud







-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20230124/8a994327/attachment-0001.htm>