[lustre-discuss] User find out OST configuration

[Anna Fuchs]

Thank you!

> Depending on how complex your tool is, it may be better to use 
> llapi_layout_from_file() or llapi_layout_from_xattr() to parse the 
> binary layout directly from the file, rather than printing it out and 
> then parsing the text again in userspace.
at first the tool should be runable by the user in userspace and help 
the user to understand the system/his data distribution without being 
dependent on admins, installed tools or versions.
The advanced one might include further features requiring root.

Best regards
Anna

On 1/24/23 03:13, Andreas Dilger wrote:
> On Jan 23, 2023, at 10:01, Anna Fuchs <anna.fuchs at uni-hamburg.de> wrote:
>>
>> Thanks!
>>
>> Is it planned to introduce some metric propagation to the user?
>> For advanced users who are benchmarking stuff on remote systems it 
>> remains unclear which performance to expect if they can not access 
>> underlaying hardware metrics.  Sure, they can ask the admin to share 
>> the config, but it might be more convenient to be able to look it up, 
>> maybe.
>
> Yes,  there is a longstanding open ticket to export some server 
> statistics to the clients - https://jira.whamcloud.com/browse/LU-7880 
> "add performance statistics to obd_statfs".  As the summary describes, 
> this would export basic performance stats for each OST/MDT device to 
> the client for current/peak x read/write x IOPS/bandwidth.  There are 
> a number of potential uses for this, such as client/MDS selection of 
> OSTs based on bandwidth/IOPS (beyond just "rotational" or 
> "non-rotational"), userspace applications/libraries using it to 
> determine if the OSTs are less busy (e.g. when to checkpoint), etc.
>
> There aren't any plans to be able to export the storage "config" (e.g. 
> RAID geometry) via Lustre since this is often opaque even on the 
> server, and doesn't have any use on the client.  There was a 
> discussion in the context of IO500 to write a script for collecting 
> storage system configuration metadata for Lustre and other filesystems 
> (e.g. list of OST/MDT devices, list of SCSI devices, PCI devices, CPU, 
> RAM, network interfaces, etc.)
>
>> Additionally: if I try to find out the stripe location (lfs 
>> getstripe) and map this information to OST-specs (lctl get_param 
>> osc.*.*ost_conn_uuid), to find out how many different servers and 
>> networks are involved, the obdidx seems to be in dec-format, but the 
>> OST index in connections list is hex, which is not always obvious. 
>>  Is there a way to display it both in dec or both in hex?
>
> There isn't currently an option for "lfs getstripe" to print in hex, 
> but it would be possible to add a "--hex" option to print the fields 
> in hex.  I've filed https://jira.whamcloud.com/browse/LU-16503 for 
> tracking this issue.  I don't think it would be terribly complex to 
> implement, but I also don't know if anyone is available to do this 
> work right now.
>
>> Are there generally any tools for doing similar things?
>> We plan a student project for building kind of GUI for visualizing 
>> stripings and mappings, so I would try to avoid reinventing the wheel.
>
>
> Cheers, Andreas
>
>> Am 21.01.2023 um 17:08 schrieb Andreas Dilger:
>>> Hi Anna,
>>> Beyond the number and size of OSTs and MDTs there isn't much 
>>> information about the underlying storage available on the client.
>>>
>>> The "lfs df -v" command will print a "f" at the end for flash 
>>> (non-rotational) devices, if the storage is properly configured. 
>>>  The "osc*.imports " parameter file will contain some information 
>>> about the grant_block_size that can be used to distinguish ldiskfs 
>>> (4096) vs. zfs backends (131072 or 1048576).
>>>
>>> The size of the disks can often be inferred from 1/8 of the total 
>>> OST size for standard 8+2 RAID configs, but this may vary and no 
>>> actual device-level metrics are available on the client.
>>>
>>> Even on the server, Lustre itself doesn't know or care much about 
>>> the underlying storage devices beyond (non-)rotational state, so we 
>>> don't track any of that.
>>>
>>> Cheers, Andreas
>>>
>>>> On Jan 21, 2023, at 01:16, Anna Fuchs via lustre-discuss 
>>>> <lustre-discuss at lists.lustre.org> wrote:
>>>>
>>>>  Hi,
>>>>
>>>> is it possible for a user (no root, so ssh to server) to find out 
>>>> the configuration of an OST?
>>>> How many devices are there in one OST 'pool' (for both ldiskfs and 
>>>> ZFS) and even which type of devices they are (nvme, ssd, hdd)? 
>>>> Maybe even speeds and raid-levels?
>>>>
>>>> Additionally, how can a user find out the mapping of all available 
>>>> OSTs to OSSs easily?
>>>>
>>>> Thanks
>>>> Anna
>>>> -- 
>>>> Anna Fuchs
>>>> Universität Hamburg
>>>> https://wr.informatik.uni-hamburg.de
>>>>
>>>> anna.fuchs at informatik.uni-hamburg.de
>>>> https://wr.informatik.uni-hamburg.de/people/anna_fuchs
>>>> _______________________________________________
>>>> lustre-discuss mailing list
>>>> lustre-discuss at lists.lustre.org
>>>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>>
>> -- 
>> Anna Fuchs
>> Universität Hamburg
>> https://wr.informatik.uni-hamburg.de
>>
>> anna.fuchs at informatik.uni-hamburg.de
>> https://wr.informatik.uni-hamburg.de/people/anna_fuchs
>
> Cheers, Andreas
> --
> Andreas Dilger
> Lustre Principal Architect
> Whamcloud
>
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20230209/78438e28/attachment.htm>