<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
On Jan 23, 2023, at 10:01, Anna Fuchs <<a href="mailto:anna.fuchs@uni-hamburg.de" class="">anna.fuchs@uni-hamburg.de</a>> wrote:<br class="">
<div>
<blockquote type="cite" class=""><br class="Apple-interchange-newline">
<div class="">
<div class="">Thanks!<br class="">
<br class="">
Is it planned to introduce some metric propagation to the user?<br class="">
For advanced users who are benchmarking stuff on remote systems it remains unclear which performance to expect if they can not access underlaying hardware metrics. Sure, they can ask the admin to share the config, but it might be more convenient to be able
to look it up, maybe.<br class="">
</div>
</div>
</blockquote>
<div><br class="">
</div>
<div>Yes, there is a longstanding open ticket to export some server statistics to the clients - <a href="https://jira.whamcloud.com/browse/LU-7880" class="">https://jira.whamcloud.com/browse/LU-7880</a> "add performance statistics to obd_statfs". As the summary
describes, this would export basic performance stats for each OST/MDT device to the client for current/peak x read/write x IOPS/bandwidth. There are a number of potential uses for this, such as client/MDS selection of OSTs based on bandwidth/IOPS (beyond
just "rotational" or "non-rotational"), userspace applications/libraries using it to determine if the OSTs are less busy (e.g. when to checkpoint), etc.</div>
<div><br class="">
</div>
<div>There aren't any plans to be able to export the storage "config" (e.g. RAID geometry) via Lustre since this is often opaque even on the server, and doesn't have any use on the client. There was a discussion in the context of IO500 to write a script for
collecting storage system configuration metadata for Lustre and other filesystems (e.g. list of OST/MDT devices, list of SCSI devices, PCI devices, CPU, RAM, network interfaces, etc.)</div>
<br class="">
<blockquote type="cite" class="">
<div class="">
<div class="">Additionally: if I try to find out the stripe location (lfs getstripe) and map this information to OST-specs (lctl get_param osc.*.*ost_conn_uuid), to find out how many different servers and networks are involved, the obdidx seems to be in dec-format,
but the OST index in connections list is hex, which is not always obvious. Is there a way to display it both in dec or both in hex?<br class="">
</div>
</div>
</blockquote>
<div><br class="">
</div>
There isn't currently an option for "lfs getstripe" to print in hex, but it would be possible to add a "--hex" option to print the fields in hex. I've filed
<a href="https://jira.whamcloud.com/browse/LU-16503" class="">https://jira.whamcloud.com/browse/LU-16503</a> for tracking this issue. I don't think it would be terribly complex to implement, but I also don't know if anyone is available to do this work right
now.</div>
<div><br class="">
<blockquote type="cite" class="">
<div class="">
<div class="">Are there generally any tools for doing similar things?<br class="">
We plan a student project for building kind of GUI for visualizing stripings and mappings, so I would try to avoid reinventing the wheel.<br class="">
</div>
</div>
</blockquote>
<div><br class="">
</div>
Depending on how complex your tool is, it may be better to use llapi_layout_from_file() or llapi_layout_from_xattr() to parse the binary layout directly from the file, rather than printing it out and then parsing the text again in userspace.</div>
<div><br class="">
</div>
<div>Cheers, Andreas</div>
<div><br class="">
<blockquote type="cite" class="">
<div class="">
<div class="">
<div class="moz-cite-prefix">Am 21.01.2023 um 17:08 schrieb Andreas Dilger:<br class="">
</div>
<blockquote type="cite" cite="mid:2B2973D8-CDF1-4643-9147-012C93A18EF4@whamcloud.com" class="">
Hi Anna,
<div class="">Beyond the number and size of OSTs and MDTs there isn't much information about the underlying storage available on the client. </div>
<div class=""><br class="">
</div>
<div class="">The "lfs df -v" command will print a "f" at the end for flash (non-rotational) devices, if the storage is properly configured. The "osc*.imports " parameter file will contain some information about the grant_block_size that can be used to distinguish
ldiskfs (4096) vs. zfs backends (131072 or 1048576). </div>
<div class=""><br class="">
</div>
<div class="">The size of the disks can often be inferred from 1/8 of the total OST size for standard 8+2 RAID configs, but this may vary and no actual device-level metrics are available on the client. </div>
<div class=""><br class="">
</div>
<div class="">Even on the server, Lustre itself doesn't know or care much about the underlying storage devices beyond (non-)rotational state, so we don't track any of that. <br class="">
<br class="">
<div dir="ltr" class="">Cheers, Andreas</div>
<div dir="ltr" class=""><br class="">
<blockquote type="cite" class="">On Jan 21, 2023, at 01:16, Anna Fuchs via lustre-discuss
<a class="moz-txt-link-rfc2396E" href="mailto:lustre-discuss@lists.lustre.org"><lustre-discuss@lists.lustre.org></a> wrote:<br class="">
<br class="">
</blockquote>
</div>
<blockquote type="cite" class="">
<div dir="ltr" class="">\ufeff Hi, <br class="">
<br class="">
is it possible for a user (no root, so ssh to server) to find out the configuration of an OST?<br class="">
How many devices are there in one OST 'pool' (for both ldiskfs and ZFS) and even which type of devices they are (nvme, ssd, hdd)? Maybe even speeds and raid-levels?<br class="">
<br class="">
Additionally, how can a user find out the mapping of all available OSTs to OSSs easily?<br class="">
<br class="">
Thanks<font size="4" class=""><font face="monospace" class=""><br class="">
</font></font>Anna<font size="4" class=""><font face="monospace" class=""><br class="">
</font></font>
<pre class="moz-signature" cols="72">--
Anna Fuchs
Universität Hamburg
<a class="moz-txt-link-freetext" href="https://wr.informatik.uni-hamburg.de/" moz-do-not-send="true">https://wr.informatik.uni-hamburg.de</a>
<a class="moz-txt-link-abbreviated moz-txt-link-freetext" href="mailto:anna.fuchs@informatik.uni-hamburg.de" moz-do-not-send="true">anna.fuchs@informatik.uni-hamburg.de</a>
<a class="moz-txt-link-freetext" href="https://wr.informatik.uni-hamburg.de/people/anna_fuchs" moz-do-not-send="true">https://wr.informatik.uni-hamburg.de/people/anna_fuchs</a></pre>
<span class="">_______________________________________________</span><br class="">
<span class="">lustre-discuss mailing list</span><br class="">
<span class=""><a class="moz-txt-link-abbreviated" href="mailto:lustre-discuss@lists.lustre.org">lustre-discuss@lists.lustre.org</a></span><br class="">
<span class=""><a class="moz-txt-link-freetext" href="http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org">http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org</a></span><br class="">
</div>
</blockquote>
</div>
</blockquote>
<br class="">
<pre class="moz-signature" cols="72">--
Anna Fuchs
Universität Hamburg
<a class="moz-txt-link-freetext" href="https://wr.informatik.uni-hamburg.de/">https://wr.informatik.uni-hamburg.de</a>
<a class="moz-txt-link-abbreviated" href="mailto:anna.fuchs@informatik.uni-hamburg.de">anna.fuchs@informatik.uni-hamburg.de</a>
<a class="moz-txt-link-freetext" href="https://wr.informatik.uni-hamburg.de/people/anna_fuchs">https://wr.informatik.uni-hamburg.de/people/anna_fuchs</a></pre>
</div>
</div>
</blockquote>
</div>
<br class="">
<div class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div>Cheers, Andreas</div>
<div>--</div>
<div>Andreas Dilger</div>
<div>Lustre Principal Architect</div>
<div>Whamcloud</div>
<div><br class="">
</div>
<div><br class="">
</div>
<div><br class="">
</div>
</div>
</div>
</div>
</div>
</div>
<br class="Apple-interchange-newline">
</div>
<br class="Apple-interchange-newline">
<br class="Apple-interchange-newline">
</div>
<br class="">
</body>
</html>