<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);" class="elementToProof">
I ended up installing lustre 2.15.1 on the new servers. Now compression works as expected when PFL is enabled.</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
What I have tested so far is:</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
AlmaLinux 9.4 - Lustre 2.16.1 - ZFS 2.1.16 - No compression with PFL</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);" class="elementToProof">
AlmaLinux 8.10 - Lustre 2.15.7 - ZFS 2.2.8 - No compression with PFL</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);" class="elementToProof">
AlmaLinux 8.10 - Lustre 2.15.5 - ZFS 2.1.15 - No compression with PFL</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
AlmaLinux 8.6 - Lustre 2.15.1 - ZFS 2.1.5 - PFL + compression works as expected</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);" class="elementToProof">
I tried installing 2.15.2, 2.15.3 and 2.15.4 but that failed with ksym errors.</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
It seems that somewhere between lustre 2.15.1 and 2.15.5 the PFL + compression breaks.</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
gr,</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
Martin Balvers</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
AlmaLinux 8.6 - Lustre 2.15.1 - ZFS 2.1.5</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
[root@microbio pfl]# ls -lh *</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
-rw-r--r-- 1 root root 6.3G Jul 1 13:28 test1.txt</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
-rw-r--r-- 1 root root 6.3G Jul 23 09:34 test2.txt</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
-rw-r--r-- 1 root root 6.3G Jul 23 09:35 test3.txt</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
LUSTRE:SVNAME COMPRESS NAME RATIO REFRATIO USED LUSED</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lustre-MDT0000 zstd-fast mdt/mdt 1.51x 1.51x 8.23M 6.80M</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lustre-OST0000 zstd lustre/ost 4.60x 4.60x 1.37G 6.29G</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lustre-OST0001 zstd lustre/ost 4.60x 4.60x 1.37G 6.29G</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lustre-OST0002 zstd lustre/ost 4.60x 4.60x 1.37G 6.29G</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
[root@microbio pfl]# lfs getstripe test1.txt</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
test1.txt</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lcm_layout_gen: 40</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lcm_mirror_count: 1</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lcm_entry_count: 5</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lcme_id: 1</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lcme_mirror_id: 0</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lcme_flags: init</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lcme_extent.e_start: 0</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lcme_extent.e_end: 1048576</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lmm_stripe_count: 0</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lmm_stripe_size: 1048576</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lmm_pattern: mdt</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lmm_layout_gen: 0</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lmm_stripe_offset: 0</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lcme_id: 2</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lcme_mirror_id: 0</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lcme_flags: init</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lcme_extent.e_start: 1048576</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lcme_extent.e_end: 134217728</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lmm_stripe_count: 1</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lmm_stripe_size: 4194304</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lmm_pattern: raid0</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lmm_layout_gen: 0</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lmm_stripe_offset: 0</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lmm_objects:</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
- 0: { l_ost_idx: 0, l_fid: [0x100000000:0x6e:0x0] }</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lcme_id: 3</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lcme_mirror_id: 0</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lcme_flags: init</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lcme_extent.e_start: 134217728</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lcme_extent.e_end: 2147483648</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lmm_stripe_count: 2</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lmm_stripe_size: 4194304</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lmm_pattern: raid0</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lmm_layout_gen: 0</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lmm_stripe_offset: 2</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lmm_objects:</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
- 0: { l_ost_idx: 2, l_fid: [0x100020000:0x6e:0x0] }</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
- 1: { l_ost_idx: 1, l_fid: [0x100010000:0x6f:0x0] }</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lcme_id: 5</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lcme_mirror_id: 0</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lcme_flags: init</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lcme_extent.e_start: 2147483648</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lcme_extent.e_end: 6979321856</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lmm_stripe_count: 3</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lmm_stripe_size: 4194304</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lmm_pattern: raid0</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lmm_layout_gen: 0</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lmm_stripe_offset: 2</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lmm_objects:</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
- 0: { l_ost_idx: 2, l_fid: [0x100020000:0x6f:0x0] }</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
- 1: { l_ost_idx: 1, l_fid: [0x100010000:0x70:0x0] }</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
- 2: { l_ost_idx: 0, l_fid: [0x100000000:0x6f:0x0] }</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lcme_id: 6</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lcme_mirror_id: 0</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lcme_flags: extension</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lcme_extent.e_start: 6979321856</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lcme_extent.e_end: EOF</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lmm_stripe_count: 0</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lmm_extension_size: 268435456</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lmm_pattern: raid0</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lmm_layout_gen: 0</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
lmm_stripe_offset: -1</div>
<div id="appendonsend"></div>
<div style="direction: ltr;"><a name="x__MailOriginal"></a></div>
<div style="direction: ltr;"><br>
</div>
<div style="direction: ltr; font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<hr style="direction: ltr; display: inline-block; width: 98%;">
<div style="direction: ltr; font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<b>From:</b> lustre-discuss <lustre-discuss-bounces@lists.lustre.org> on behalf of BALVERS Martin via lustre-discuss <lustre-discuss@lists.lustre.org><br>
<b>Sent:</b> Tuesday, July 29, 2025 09:19<br>
<b>To:</b> Vicker, Darby J. (JSC-EG111)[Jacobs Technology, Inc.] <darby.vicker-1@nasa.gov>; lustre-discuss <lustre-discuss@lists.lustre.org><br>
<b>Subject:</b> Re: [lustre-discuss] [EXTERNAL] [BULK] No ZFS compression on OST's when using PFL</div>
<div style="direction: ltr; font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<table style="direction: ltr; border-width: 3px; border-style: solid; border-color: rgb(255, 0, 51); background-color: rgb(255, 251, 224); width: 100%; box-sizing: border-box; border-collapse: collapse; border-spacing: 0px;" cellpadding="0" cellspacing="0">
<tbody>
<tr>
<td style="direction: ltr; line-height: 1; padding: 4px 8px; color: black;" align="center">
<div style="direction: ltr; line-height: 1; font-size: 11px;"><b>WARNING - EXTERNAL SENDER - BE CYBERSAFE</b></div>
</td>
</tr>
</tbody>
</table>
<div style="direction: ltr;"><br>
</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: black;">
Hi,</div>
<div id="x_appendonsend"></div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: black;">
<br>
</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: black;">
My initial copy from old lustre to new lustre was with rsync, using this syntax:</div>
<div style="direction: ltr; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 11pt; color: black;">
SOURCE="/lustre/workgroups" DEST="${SOURCE#/lustre/}" && find "$SOURCE" -maxdepth 1 -mindepth 1 -type d | xargs -t -n1 -P6 -I% rsync -aPA --delete-during % "/mnt/lustre_new/$DEST"</div>
<div style="direction: ltr; font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: black;">
<br>
</div>
<div style="direction: ltr; font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: black;">
I do indeed need to copy the linux ACL's</div>
<div style="direction: ltr; font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: black;">
<br>
</div>
<div style="direction: ltr; font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: black;">
But when debugging the issue, I used files without any extended attributes, from other filesystems than lustre.</div>
<div style="direction: ltr; font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: black;">
Just enabling DoM with "lfs setstripe -E 1M -L mdt -E -1 -c 1 /mnt/lustre_new/dom" will disable compression on the OST's. (dom_stripesize is set to 1048576)</div>
<div style="direction: ltr; font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: black;">
<br>
</div>
<div style="direction: ltr; font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: black;">
I'm thinking about trying 2.15.1 on the new servers...</div>
<div style="direction: ltr; font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: black;">
<br>
</div>
<div style="direction: ltr; font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: black;">
Regards,</div>
<div style="direction: ltr; font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: black;">
Martin Balvers</div>
<div style="direction: ltr; font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: black;">
<br>
</div>
<div style="direction: ltr; font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: black;">
<br>
</div>
<hr style="direction: ltr; display: inline-block; width: 98%;">
<div style="direction: ltr; font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: black;">
<b>From:</b> Vicker, Darby J. (JSC-EG111)[Jacobs Technology, Inc.] <darby.vicker-1@nasa.gov><br>
<b>Sent:</b> Monday, July 28, 2025 16:57<br>
<b>To:</b> BALVERS Martin <Martin.BALVERS@danone.com>; lustre-discuss <lustre-discuss@lists.lustre.org><br>
<b>Subject:</b> Re: [EXTERNAL] [BULK] [lustre-discuss] No ZFS compression on OST's when using PFL</div>
<div style="direction: ltr; font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: black;">
<br>
</div>
<table style="direction: ltr; border-width: 3px; border-style: solid; border-color: rgb(255, 0, 51); background-color: rgb(255, 251, 224); width: 100%; box-sizing: border-box; border-collapse: collapse; border-spacing: 0px;" cellpadding="0" cellspacing="0">
<tbody>
<tr>
<td style="direction: ltr; line-height: 1; padding: 4px 8px; color: black;" align="center">
<div style="direction: ltr; line-height: 1; font-size: 11px;"><b>WARNING - EXTERNAL SENDER - BE CYBERSAFE</b></div>
</td>
</tr>
</tbody>
</table>
<div style="direction: ltr;"><br>
</div>
<p style="direction: ltr; margin: 0px; font-family: Aptos, sans-serif; font-size: 10pt;">
<span style="font-size: 11pt;">I’m not sure if this is related to your problem but you need to be careful when copying data between two lustre filesystems. If you use something that copies the extended attributes (e.g. rsync -X), you can copy the lustre xattrs,
which is probably not what you want. This will override the PFL you have on the new lustre FS. You can either not copy the extended attributes (which isn’t a great option if you are using ACL’s or other xattrs) or use a lustre-aware tool like mpifileutils. </span></p>
<p style="direction: ltr; margin: 0px; font-family: Aptos, sans-serif; font-size: 10pt;">
<span style="font-size: 11pt;"> </span></p>
<div id="x_x_mail-editor-reference-message-container">
<div style="padding-top: 3pt; border-top: 1pt solid rgb(181, 196, 223);">
<p style="direction: ltr; margin: 0px 0px 12pt; font-family: Aptos, sans-serif; font-size: 10pt;">
<span style="font-size: 12pt; color: black;"><b>From: </b>lustre-discuss <lustre-discuss-bounces@lists.lustre.org> on behalf of BALVERS Martin via lustre-discuss <lustre-discuss@lists.lustre.org><br>
<b>Date: </b>Friday, July 25, 2025 at 7:07</span><span style="font-family: Arial, sans-serif; font-size: 12pt; color: black;"> </span><span style="font-size: 12pt; color: black;">AM<br>
<b>To: </b>lustre-discuss <lustre-discuss@lists.lustre.org><br>
<b>Subject: </b>[EXTERNAL] [BULK] [lustre-discuss] No ZFS compression on OST's when using PFL</span></p>
</div>
<table style="direction: ltr; border-width: 1.5pt; border-style: solid; border-color: black; box-sizing: border-box; border-collapse: collapse; border-spacing: 0px;" cellpadding="0" cellspacing="0" align="left">
<tbody>
<tr>
<td style="direction: ltr; background-color: rgb(255, 235, 156); padding: 3.75pt; width: 100%;">
<p style="direction: ltr; margin: 0px; font-family: Aptos, sans-serif; font-size: 10pt;">
<span style="color: black;"><b>CAUTION:</b></span><span style="font-size: 12pt; color: black;">
</span><span style="color: black;">This email originated from outside of NASA. Please take care when clicking links or opening attachments. Use the "Report Message" button to report suspicious messages to the NASA SOC.</span></p>
</td>
</tr>
</tbody>
</table>
<p style="direction: ltr; margin: 0px 0px 12pt; font-family: Aptos, sans-serif; font-size: 10pt;">
<span style="font-size: 12pt;"><br>
<br>
</span></p>
<p style="direction: ltr; margin: 0px; font-family: Aptos, sans-serif; font-size: 10pt;">
<span style="font-size: 11pt; color: black;">Hi,</span></p>
<p style="direction: ltr; margin: 0px; font-family: Aptos, sans-serif; font-size: 10pt;">
<span style="font-size: 11pt; color: black;"> </span></p>
<p style="direction: ltr; margin: 0px; font-family: Aptos, sans-serif; font-size: 10pt;">
<span style="font-size: 11pt; color: black;">I am using a lustre 2.15.1 cluster with PFL. I'm using the following layout:</span></p>
<p style="direction: ltr; margin: 0px; font-family: Aptos, sans-serif; font-size: 10pt;">
<span style="font-size: 11pt; color: black;">lfs setstripe -E 1M -L mdt -E 128M -c 1 -S 4M -E 2G -c 2 -z 64M -E -1 -c -1 -z 256M /lustre</span></p>
<p style="direction: ltr; margin: 0px; font-family: Aptos, sans-serif; font-size: 10pt;">
<span style="font-size: 11pt; color: black;"> </span></p>
<p style="direction: ltr; margin: 0px; font-family: Aptos, sans-serif; font-size: 10pt;">
<span style="font-size: 11pt; color: black;">OST's have zpools with zstd compression enabled.</span></p>
<p style="direction: ltr; margin: 0px; font-family: Aptos, sans-serif; font-size: 10pt;">
<span style="font-size: 11pt; color: black;">This has worked fine for me for many years.</span></p>
<p style="direction: ltr; margin: 0px; font-family: Aptos, sans-serif; font-size: 10pt;">
<span style="font-size: 11pt; color: black;"> </span></p>
<p style="direction: ltr; margin: 0px; font-family: Aptos, sans-serif; font-size: 10pt;">
<span style="font-size: 11pt; color: black;">I now have new servers and installed 2.16.1 on them. I used the same PFL layout, and after some benchmarking I proceeded to copy data from the old lustre to the new.</span></p>
<p style="direction: ltr; margin: 0px; font-family: Aptos, sans-serif; font-size: 10pt;">
<span style="font-size: 11pt; color: black;">After a while I noticed that it was filling up faster that expected, and say that the compression ratio on the zpools was 1.</span></p>
<p style="direction: ltr; margin: 0px; font-family: Aptos, sans-serif; font-size: 10pt;">
<span style="font-size: 11pt; color: black;"> </span></p>
<p style="direction: ltr; margin: 0px; font-family: Aptos, sans-serif; font-size: 10pt;">
<span style="font-size: 11pt; color: black;">Compression on the zpools is enabled, it should have worked as far as I can tell.</span></p>
<p style="direction: ltr; margin: 0px; font-family: Aptos, sans-serif; font-size: 10pt;">
<span style="font-size: 11pt; color: black;">When I disable PFL, compression works as expected.</span></p>
<p style="direction: ltr; margin: 0px; font-family: Aptos, sans-serif; font-size: 10pt;">
<span style="font-size: 11pt; color: black;"> </span></p>
<p style="direction: ltr; margin: 0px; font-family: Aptos, sans-serif; font-size: 10pt;">
<span style="font-size: 11pt; color: black;">I have attached the some results in a txt file.</span></p>
<p style="direction: ltr; margin: 0px; font-family: Aptos, sans-serif; font-size: 10pt;">
<span style="font-size: 11pt; color: black;"> </span></p>
<p style="direction: ltr; margin: 0px; font-family: Aptos, sans-serif; font-size: 10pt;">
<span style="font-size: 11pt; color: black;">I have also tried 2.15.7 with the same results. Enabling PFL, or just DoM disables compression on the OST's.</span></p>
<p style="direction: ltr; margin: 0px; font-family: Aptos, sans-serif; font-size: 10pt;">
<span style="font-size: 11pt; color: black;"> </span></p>
<p style="direction: ltr; margin: 0px; font-family: Aptos, sans-serif; font-size: 10pt;">
<span style="font-size: 11pt; color: black;">Am I missing something obvious here? </span></p>
<p style="direction: ltr; margin: 0px; font-family: Aptos, sans-serif; font-size: 10pt;">
<span style="font-size: 11pt; color: black;"> </span></p>
<p style="direction: ltr; margin: 0px; font-family: Aptos, sans-serif; font-size: 10pt;">
<span style="font-size: 11pt; color: black;">Regards,</span></p>
<p style="direction: ltr; margin: 0px; font-family: Aptos, sans-serif; font-size: 10pt;">
<span style="font-size: 11pt; color: black;">Martin Balvers</span></p>
<p style="direction: ltr; margin: 0px; font-family: Aptos, sans-serif; font-size: 10pt;">
<span style="font-size: 12pt;">Ce message électronique et tous les fichiers attachés qu'il contient sont confidentiels et destinés exclusivement à l'usage de la personne à laquelle ils sont adressés. Si vous avez reçu ce message par erreur, merci de le retourner
à son émetteur. Les idées et opinions présentées dans ce message sont celles de son auteur, et ne représentent pas nécessairement celles de DANONE ou d'une quelconque de ses filiales. La publication, l'usage, la distribution, l'impression ou la copie non autorisée
de ce message et des attachements qu'il contient sont strictement interdits.<br>
<br>
This e-mail and any files transmitted with it are confidential and intended solely for the use of the individual to whom it is addressed. If you have received this email in error please send it back to the person that sent it to you. Any views or opinions presented
are solely those of its author and do not necessarily represent those of DANONE or any of its subsidiary companies. Unauthorized publication, use, dissemination, forwarding, printing or copying of this email and its associated attachments is strictly prohibited.</span></p>
</div>
</body>
</html>