[lustre-discuss] Increased file block space consumption when the stripe size is set 8MB

[Homer Li]

Hi All,
    Server ver: lustre 2.15.4 + zfs 2.1.15-1, raidz3
    Client 2.15.4(3x file size = file block size) , 2.12 is OK
    It appears that the -S 8M and -c 8 stripe parameter causes Lustre
file block space to be consumed exponentially as the file size
increases.
     Is it a bug ?

client# ls -lhs
total 277T
 512 -rw------- 1 175536 2772   0 Feb 17 09:22 nohup.out
143T -rw-r--r-- 1 175536 2772 50T Feb 17 09:16 N0007899.tar.gz
135T -rw-r--r-- 1 175536 2772 52T Feb 17 09:17 Z33578900.tar.gz

man ls
       -s, --size
              print the allocated size of each file, in blocks

# stat N0007899.tar.gz
  File: N0007899.tar.gz
  Size: 54371049154560 Blocks: 306805052685 IO Block: 4194304 regular file
Device: 4fdb1a5ch/1339759196d Inode: 144115507497271299  Links: 1
Access: (0644/-rw-r--r--)  Uid: (175536/ UNKNOWN)   Gid: ( 2772/ UNKNOWN)
Access: 2025-02-17 09:16:31.000000000 +0800
Modify: 2025-02-17 09:16:28.000000000 +0800
Change: 2025-02-17 09:16:28.000000000 +0800
 Birth: 2025-01-20 14:36:30.000000000 +0800

# stat Z33578900.tar.gz
  File: Z33578900.tar.gz
  Size: 56675110215680 Blocks: 287895780252 IO Block: 4194304 regular file
Device: 4fdb1a5ch/1339759196d Inode: 144115507514048514  Links: 1
Access: (0644/-rw-r--r--)  Uid: (175536/ UNKNOWN)   Gid: ( 2772/ UNKNOWN)
Access: 2025-02-17 09:54:16.000000000 +0800
Modify: 2025-02-17 09:17:14.000000000 +0800
Change: 2025-02-17 09:17:14.000000000 +0800
 Birth: 2025-01-20 14:33:59.000000000 +0800

 # lfs getstripe ST_NOTEBOOK_TRANSFER
ST_NOTEBOOK_TRANSFER
stripe_count:  8 stripe_size:   8388608 pattern:       raid0 stripe_offset: -1

ST_NOTEBOOK_TRANSFER/nohup.out
lmm_stripe_count:  1
lmm_stripe_size:   8388608
lmm_pattern:       raid0
lmm_layout_gen:    0
lmm_stripe_offset: 1
obdidx objid objid group
     1        4305626      0x41b2da              0

ST_NOTEBOOK_TRANSFER/Z33578900.tar.gz
lmm_stripe_count:  8
lmm_stripe_size:   8388608
lmm_pattern:       raid0
lmm_layout_gen:    0
lmm_stripe_offset: 2
obdidx objid objid group
     2        4550986      0x45714a              0
     7        4355242      0x4274aa              0
     4        4485210      0x44705a              0
     1        4304762      0x41af7a              0
     8        4509498      0x44cf3a              0
     5        4455274      0x43fb6a              0
     6        4539370      0x4543ea              0
     0        4495466      0x44986a              0

ST_NOTEBOOK_TRANSFER/N0007899.tar.gz
lmm_stripe_count:  8
lmm_stripe_size:   8388608
lmm_pattern:       raid0
lmm_layout_gen:    0
lmm_stripe_offset: 3
obdidx objid objid group
     3        4304634      0x41aefa              0
     2        4550987      0x45714b              0
     7        4355243      0x4274ab              0
     4        4485211      0x44705b              0
     1        4304763      0x41af7b              0
     8        4509499      0x44cf3b              0
     5        4455275      0x43fb6b              0
     6        4539371      0x4543eb              0

-----------------------------------> reproduce the issue

# mkdir stripe stripe8
# lfs setstripe -c 8 -S 1M stripe
# lfs setstripe -c 8 -S 8M stripe8
client 2.15.4 # tar cf stripe/N0007899.tar-2.15.4 N0007899
client 2.15.4 # tar cf stripe8/N0007899.tar-2.15.4 N0007899
client 2.12.8 # tar cf stripe/N0007899.tar-2.12.8 N0007899
client 2.12.8 # tar cf stripe8/N0007899.tar-2.12.8 N0007899

# ls -lhs stripe stripe8
stripe:
total 8.1T
4.0T -rw-r--r-- 1 99 99 4.8T Feb 27 08:47 N0007899.tar-2.12.8
4.1T -rw-r--r-- 1 99 99 5.0T Feb 27 08:47 N0007899.tar-2.15.4

stripe8/:
total 17T
 12T -rw-r--r-- 1 99 99 5.0T Feb 27 08:47 N0007899.tar-2.15.4
<-------File system block space has been allocated at twice the file
size
5.4T -rw-r--r-- 1 99 99 4.8T Feb 27 08:47 N0007899.tar.gz-2.12.8


# stat stripe/N0007899.tar-2.12.8
stripe/N0007899.tar-2.15.4 stripe8/N0007899.tar-2.15.4
stripe8/N0007899.tar.gz-2.12.8

  File: stripe/N0007899.tar-2.12.8
  Size: 5173207121920 Blocks: 8574556072 IO Block: 4194304 regular file

  File: stripe/N0007899.tar-2.15.4
  Size: 5406169067520 Blocks: 8797185712 IO Block: 4194304 regular file

  File: stripe8/N0007899.tar-2.15.4
  Size: 5406169067520 Blocks: 24171346606 IO Block: 4194304 regular file

  File: stripe8/N0007899.tar.gz-2.12.8
  Size: 5173207121920 Blocks: 11436837355 IO Block: 4194304 regular file

-- 
May the wind guide your path; May the stars light your way