<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
On Jun 12, 2024, at 16:54, Patrick Farrell <<a href="mailto:patrick.farrell@oracle.com" class="">patrick.farrell@oracle.com</a>> wrote:<br class="">
<div>
<blockquote type="cite" class="">
<div class="">
<div style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;" class="">
<br class="">
</div>
<div class="elementToProof" style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">
A few clarifying questions, partly for me and hopefully for Saisha as well as a non-Lustre expert. The ldiskfs file system supported for Lustre disk targets is a modified ext4, so it inherits ext4 features for the local volume, but not all of them are supported
for ldiskfs formatted volumes. So does Lustre support use of metadata_csum (that's the relevant option, isn't it?) when formatting ldiskfs? Is it on by default?</div>
</div>
</blockquote>
<div><br class="">
</div>
I _thought_ that xattrs have had built-in checksums since they were first added to ext3, long before metadata_csum existed. However, looking at the metadata_csum patch handling xattrs, I see that it is adding the 'h_checksum' field itself. I guess I was conflating
the xattr *checksum* with the xattr *hash* which is used for xattr block sharing, but is never actually verified by the kernel. </div>
<div><br class="">
<blockquote type="cite" class="">
<div class="">
<div class="elementToProof" style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">
Inferring from your comments, I think you're saying metadata_csums are (or can be) used on an ldiskfs volume. But since they are just a local disk feature, they contribute only minimally to resiliency in Lustre, and Lustre makes no use of them. They can help
protect against some damage to the local volume, but Lustre is unaware of them.<br class="">
</div>
</div>
</blockquote>
<div><br class="">
</div>
Unfortunately, the Lustre "dirdata" feature did not previously interoperate well with metadata_csum (which was developed many years later), so currently mkfs.lustre explicitly disables the metadata_csum feature when the filesystem is formatted. There have
been incremental fixes related to dirdata and metadata_csum over the years, so it is possible that these would work together today, but it has not been on anyone's radar to test and verify that these features are OK to use together. <a href="https://jira.whamcloud.com/browse/LU-13650" class="">https://jira.whamcloud.com/browse/LU-13650</a> is
open to track this, maybe someone would be interested to work on that...</div>
<div><br class="">
<blockquote type="cite" class="">
<div class="">
<div class="elementToProof" style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">
Do the metadata checksums extend to all ext4/ldiskfs extended attributes? Are they done separately for individual xattrs or just for the whole inode?</div>
</div>
</blockquote>
<div><br class="">
</div>
<div>There is an h_checksum field for the xattr header that covers all of the xattrs, but only the e_hash field for the individual entry:</div>
<div><br class="">
</div>
<div>
<div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Monaco; color: rgba(242, 242, 242, 0.901961); background-color: rgba(0, 0, 0, 0.862745);" class="">
<span style="font-variant-ligatures: no-common-ligatures; color: #2fb41d" class="">struct</span><span style="font-variant-ligatures: no-common-ligatures" class=""> ext4_xattr_header {</span></div>
<div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Monaco; color: rgb(46, 174, 187); background-color: rgba(0, 0, 0, 0.862745);" class="">
<span style="font-variant-ligatures: no-common-ligatures; color: rgba(242, 242, 242, 0.9)" class=""> __le32 h_magic;
</span><span style="font-variant-ligatures: no-common-ligatures" class="">/* magic number for identification */</span></div>
<div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Monaco; color: rgba(242, 242, 242, 0.901961); background-color: rgba(0, 0, 0, 0.862745);" class="">
<span style="font-variant-ligatures: no-common-ligatures" class=""> __le32 h_refcount;
</span><span style="font-variant-ligatures: no-common-ligatures; color: #2eaebb" class="">/* reference count */</span></div>
<div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Monaco; color: rgba(242, 242, 242, 0.901961); background-color: rgba(0, 0, 0, 0.862745);" class="">
<span style="font-variant-ligatures: no-common-ligatures" class=""> __le32 h_blocks;
</span><span style="font-variant-ligatures: no-common-ligatures; color: #2eaebb" class="">/* number of disk blocks used */</span></div>
<div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Monaco; color: rgb(46, 174, 187); background-color: rgba(0, 0, 0, 0.862745);" class="">
<span style="font-variant-ligatures: no-common-ligatures; color: rgba(242, 242, 242, 0.9)" class=""> __le32 h_hash;
</span><span style="font-variant-ligatures: no-common-ligatures" class="">/* hash value of all attributes */</span></div>
<div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Monaco; color: rgba(242, 242, 242, 0.901961); background-color: rgba(0, 0, 0, 0.862745);" class="">
<span style="font-variant-ligatures: no-common-ligatures" class=""> __le32 h_checksum;
</span><span style="font-variant-ligatures: no-common-ligatures; color: #2eaebb" class="">/* crc32c(uuid+id+xattrblock) */</span></div>
<div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Monaco; color: rgb(46, 174, 187); background-color: rgba(0, 0, 0, 0.862745);" class="">
<span style="font-variant-ligatures: no-common-ligatures; color: rgba(242, 242, 242, 0.9)" class="">
</span><span style="font-variant-ligatures: no-common-ligatures" class="">/* id = inum if refcount=1, blknum otherwise */</span></div>
<div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Monaco; color: rgba(242, 242, 242, 0.901961); background-color: rgba(0, 0, 0, 0.862745);" class="">
<span style="font-variant-ligatures: no-common-ligatures" class=""> __u32 h_reserved[</span><span style="font-variant-ligatures: no-common-ligatures; color: #b42419" class="">3</span><span style="font-variant-ligatures: no-common-ligatures" class="">];
</span><span style="font-variant-ligatures: no-common-ligatures; color: #2eaebb" class="">/* zero right now */</span></div>
<div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Monaco; color: rgba(242, 242, 242, 0.901961); background-color: rgba(0, 0, 0, 0.862745);" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">};</span></div>
<div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Monaco; color: rgba(242, 242, 242, 0.901961); background-color: rgba(0, 0, 0, 0.862745);" class="">
<span style="font-variant-ligatures: no-common-ligatures" class=""><br class="">
</span></div>
<div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Monaco; color: rgba(242, 242, 242, 0.901961); background-color: rgba(0, 0, 0, 0.862745);" class="">
<span style="font-variant-ligatures: no-common-ligatures" class="">
<div style="margin: 0px; font-stretch: normal; line-height: normal;" class=""><span style="font-variant-ligatures: no-common-ligatures; color: #2fb41d" class="">struct</span><span style="font-variant-ligatures: no-common-ligatures" class=""> ext4_xattr_entry
{</span></div>
<div style="margin: 0px; font-stretch: normal; line-height: normal;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> __u8 e_name_len;
</span><span style="font-variant-ligatures: no-common-ligatures; color: #2eaebb" class="">/* length of name */</span></div>
<div style="margin: 0px; font-stretch: normal; line-height: normal;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> __u8 e_name_index;
</span><span style="font-variant-ligatures: no-common-ligatures; color: #2eaebb" class="">/* attribute name index */</span></div>
<div style="margin: 0px; font-stretch: normal; line-height: normal; color: rgb(46, 174, 187);" class="">
<span style="font-variant-ligatures: no-common-ligatures; color: rgba(242, 242, 242, 0.9)" class=""> __le16 e_value_offs;
</span><span style="font-variant-ligatures: no-common-ligatures" class="">/* offset in disk block of value */</span></div>
<div style="margin: 0px; font-stretch: normal; line-height: normal; color: rgb(46, 174, 187);" class="">
<span style="font-variant-ligatures: no-common-ligatures; color: rgba(242, 242, 242, 0.9)" class=""> __le32 e_value_block;
</span><span style="font-variant-ligatures: no-common-ligatures" class="">/* disk block attribute is stored on (n/i) */</span></div>
<div style="margin: 0px; font-stretch: normal; line-height: normal;" class=""><span style="font-variant-ligatures: no-common-ligatures" class=""> __le32 e_value_size;
</span><span style="font-variant-ligatures: no-common-ligatures; color: #2eaebb" class="">/* size of attribute value */</span></div>
<div style="margin: 0px; font-stretch: normal; line-height: normal; color: rgb(46, 174, 187);" class="">
<span style="font-variant-ligatures: no-common-ligatures; color: rgba(242, 242, 242, 0.9)" class=""> __le32 e_hash;
</span><span style="font-variant-ligatures: no-common-ligatures" class="">/* hash value of name and value */</span></div>
<div style="margin: 0px; font-stretch: normal; line-height: normal;" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">
</span><span style="font-variant-ligatures: no-common-ligatures; color: #2fb41d" class="">char</span><span style="font-variant-ligatures: no-common-ligatures" class=""> e_name[</span><span style="font-variant-ligatures: no-common-ligatures; color: #b42419" class="">0</span><span style="font-variant-ligatures: no-common-ligatures" class="">];
</span><span style="font-variant-ligatures: no-common-ligatures; color: #2eaebb" class="">/* attribute name */</span></div>
<div style="margin: 0px; font-stretch: normal; line-height: normal;" class=""><span style="font-variant-ligatures: no-common-ligatures" class="">};</span></div>
</span></div>
</div>
<br class="">
<blockquote type="cite" class="">
<div class="">
<div class="elementToProof" style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">
The kernel wiki describes a possible implementation that credits someone named 'Andreas Dilger' ( :) ) with suggesting checksumming them under the inode checksum rather than separately, but I don't know what was actually implemented.</div>
</div>
</blockquote>
<div><br class="">
</div>
I think that was for xattrs stored within the inode itself, since in-inode xattrs do not have the "ext4_xattr_header" that is used in external xattr blocks, but apparently you can't trust everything that guy writes... :-) Thanks for keeping me honest.</div>
<div><br class="">
</div>
<div>Cheers, Andreas</div>
<div><br class="">
<blockquote type="cite" class="">
<div class="">
<div class="elementToProof" style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">
<br class="">
Regards,</div>
<div class="elementToProof" style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt;">
Patrick</div>
<div id="appendonsend" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class="">
</div>
<hr tabindex="-1" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; display: inline-block; width: 1288.6875px;" class="">
<span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class=""></span>
<div id="divRplyFwdMsg" dir="ltr" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class="">
<font face="Calibri, sans-serif" style="font-size: 11pt;" class=""><b class="">From:</b><span class="Apple-converted-space"> </span>lustre-devel <<a href="mailto:lustre-devel-bounces@lists.lustre.org" class="">lustre-devel-bounces@lists.lustre.org</a>> on behalf
of Andreas Dilger via lustre-devel <<a href="mailto:lustre-devel@lists.lustre.org" class="">lustre-devel@lists.lustre.org</a>><br class="">
<b class="">Sent:</b><span class="Apple-converted-space"> </span>Wednesday, June 12, 2024 5:27 PM<br class="">
<b class="">To:</b><span class="Apple-converted-space"> </span>Saisha Kamat <<a href="mailto:skamat1@charlotte.edu" class="">skamat1@charlotte.edu</a>><br class="">
<b class="">Cc:</b><span class="Apple-converted-space"> </span>lustre-devel <<a href="mailto:lustre-devel@lists.lustre.org" class="">lustre-devel@lists.lustre.org</a>><br class="">
<b class="">Subject:</b><span class="Apple-converted-space"> </span>[External] : Re: [lustre-devel] Inquiry Regarding Checksum Support for Lustre Extended Attributes</font>
<div class=""> </div>
</div>
<div class="" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; line-break: after-white-space;">
On Jun 12, 2024, at 12:35, Saisha Kamat via lustre-devel <<a href="mailto:lustre-devel@lists.lustre.org" class="">lustre-devel@lists.lustre.org</a>> wrote:<br class="">
<div class="">
<blockquote type="cite" class=""><br class="x_Apple-interchange-newline">
<div class="">
<div dir="ltr" class="">Hello,<br class="">
<br class="">
I hope this email finds you well.<br class="">
<br class="">
My name is Saisha, and I am currently pursuing my Ph.D. at UNC-Charlotte, with a focus on research related to the Lustre File System. As part of my project, I am exploring the possibility of utilizing checksums to verify Lustre extended attributes.<br class="">
<br class="">
My understanding is that the ext4 file system supports checksums for extended attributes (xattrs). However, I am interested in whether this functionality extends to Lustre as well.<br class="">
</div>
</div>
</blockquote>
<div class=""><br class="">
</div>
Yes, the ext4 (and ZFS) xattr values have checksums. No, the xattr checksums are neither managed or verified by Lustre, and only come into effect when they are passed on to the backing filesystem. Conceivably, it would be possible to have a checksum (e.g.
crc32c) for the xattr values in the MDS_GETXATTR and MDS_SETXATTR RPCs, if this is something you are interested to contribute.</div>
<div class=""><br class="">
</div>
<div class="">This could probably be done by overloading one of the 32-bit fields in the mdt_body for getxattr, and one in mdt_rec_reint for setxattr, but there is also "opportunistic" xattr prefetching done in the lookup RPC, so that would need to be covered
as well.</div>
<div class=""><br class="">
</div>
<div class="">Also, the checksum would also need to be kept with the xattrs in cache and verified on access, otherwise they could become corrupted in memory after the RPC processing had completed.</div>
<div class=""><br class="">
</div>
<div class="">Finally, there is no interface to specify or verify the xattr checksum in the syscall interface, so there can be no guarantee that the data supplied in the setxattr is correct, or remains correct after supplied to getxattr, but the window there
is very small.</div>
<div class=""><br class="">
</div>
<div class="">Cheers, Andreas</div>
<div class=""><br class="">
<blockquote type="cite" class="">
<div class="">
<div dir="ltr" class="">I would greatly appreciate it if you could provide some insights or direct me to relevant documentation on this matter. Any information or guidance you can offer would be invaluable to my research.<br class="">
<br class="">
Thank you very much for your time and assistance.
<div class=""><br class="">
</div>
<div class="">Thanks and Regards,</div>
<div class="">Saisha</div>
</div>
_______________________________________________<br class="">
lustre-devel mailing list<br class="">
<a href="mailto:lustre-devel@lists.lustre.org" class="">lustre-devel@lists.lustre.org</a><br class="">
<a href="https://urldefense.com/v3/__http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org__;!!ACWV5N9M2RV99hQ!NxZsGb9nGU3ZLJa1pn9PclSTh2QUuTOnSdBLFvT4cYZxHl-jq2z3TrARWnu9vlkaS-diipwr7e73a1J0pgTw4Htt05quzd0a$" class="">http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org</a><br class="">
</div>
</blockquote>
</div>
<br class="">
<div class="">
<div dir="auto" class="" style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none; word-wrap: break-word; line-break: after-white-space;">
<div dir="auto" class="" style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none; word-wrap: break-word; line-break: after-white-space;">
<div dir="auto" class="" style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none; word-wrap: break-word; line-break: after-white-space;">
<div dir="auto" class="" style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none; word-wrap: break-word; line-break: after-white-space;">
<div dir="auto" class="" style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none; word-wrap: break-word; line-break: after-white-space;">
<div dir="auto" class="" style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none; word-wrap: break-word; line-break: after-white-space;">
<div class="">Cheers, Andreas</div>
<div class="">--</div>
<div class="">Andreas Dilger</div>
<div class="">Lustre Principal Architect</div>
<div class="">Whamcloud</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br class="">
<div class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div>Cheers, Andreas</div>
<div>--</div>
<div>Andreas Dilger</div>
<div>Lustre Principal Architect</div>
<div>Whamcloud</div>
<div><br class="">
</div>
<div><br class="">
</div>
<div><br class="">
</div>
</div>
</div>
</div>
</div>
</div>
<br class="Apple-interchange-newline">
</div>
<br class="Apple-interchange-newline">
<br class="Apple-interchange-newline">
</div>
<br class="">
</body>
</html>