[lustre-discuss] [EXTERNAL] Unable to mount the OST

Thu Oct 23 16:33:11 PDT 2025

The OST objects do not *require* extended attributes, only the MDT inodes (to hold file layouts, FIDs, etc).

On Oct 23, 2025, at 02:37, Ihsan Ur Rahman <ihsanurr67 at gmail.com> wrote:

Hell,

Unfortunately the extended attributes were not copied during the backup. so we have lost the data.

Regards,

Ihsan

On Wed, Oct 22, 2025 at 8:36 PM Ihsan Ur Rahman <ihsanurr67 at gmail.com<mailto:ihsanurr67 at gmail.com>> wrote:
Dear Rick,
Thank you for your reply. Yes i have unmounted before the mounting with "lustre"

regards,
Ihsan

On Wed, Oct 22, 2025 at 5:19 PM Mohr, Rick <mohrrf at ornl.gov<mailto:mohrrf at ornl.gov>> wrote:
Just curious, after you mounted /dev/sdd as ldiskfs, did you unmount it before trying to mount it as type "lustre"?  I'm wondering if the "File exists" error is a result of trying to double mount it.

--Rick

On Tue, Oct 21, 2025 at 12:06 AM Ihsan Ur Rahman <ihsanurr67 at gmail.com<mailto:ihsanurr67 at gmail.com> <mailto:ihsanurr67 at gmail.com<mailto:ihsanurr67 at gmail.com>> <mailto:ihsanurr67 at gmail.com<mailto:ihsanurr67 at gmail.com> <mailto:ihsanurr67 at gmail.com<mailto:ihsanurr67 at gmail.com>>>> wrote:

Hello Lustre community,

One of the OSTs failed. We have copied the data using rsync and reformatted the OST with the following command:

mkfs.lustre --ost --index=29 --replace --reformat --fsname=lustre --backfstype=ldiskfs --mgsnode=X.X.X.X at o2ib --mkfsoptions="-i 7000" /dev/sdd

The OST has been deactivated on both the client side and the MDS side. We mounted it with the following command and copied the data back using rsync:

mount -t ldiskfs /dev/sdd /mnt/ost29

Once the copying was completed, I tried to mount it as a Lustre OST, but I encountered the following errors:

# mount -t lustre /dev/sdd /mnt/29ost
mount.lustre: mount /dev/sdd at /mnt/29ost failed: File exists

When I tried a second time, I got this error:

# mount -t lustre /dev/sdd /mnt/29ost
mount.lustre: mount /dev/sdd at /mnt/29ost failed: Operation now in progress

I also tried:

# mount -t lustre -o abort_recov /dev/sdd /mnt/ost29
mount.lustre: mount /dev/sdd at /mnt/ost29 failed: File exists

On the OSS, when I run:

# lctl dl | grep OST001d
12 UP osd-ldiskfs lustre-OST001d-osd lustre-OST001d-osd_UUID 3

The OST still appears. I tried to remove it using:

lctl --device lustre-OST001d-osd cleanup
lctl --device lustre-OST001d-osd detach

However, when I run mount -t lustre /dev/sdd /mnt/ost29 again, OST001d appears once more.

I have also found Lustre-related errors in the OSS kernel logs.

[74585.030434] LustreError: 15c-8: MGC10.1x.x.x.132 at o2ib: The configuration from log 'lustre-OST001d' failed (-115). This may be the result of communication errors between this node and the MGS, a bad configuration, or other errors. See the syslog for more information.
[74585.034979] LustreError: 8737:0:(obd_mount_server.c:1397:server_start_targets()) failed to start server lustre-OST001d: -115
[74585.036522] LustreError: 8737:0:(obd_mount_server.c:1992:server_fill_super()) Unable to start targets: -115
[74585.037997] LustreError: 8737:0:(obd_config.c:610:class_cleanup()) Device 13 not setup
[74585.039532] Lustre: server umount lustre-OST001d complete
[74585.039537] LustreError: 8737:0:(obd_mount.c:1608:lustre_fill_super()) Unable to mount /dev/sdd (-115)
[74744.284257] LDISKFS-fs (sdd): file extents enabled, maximum tree depth=5
[74745.013194] LDISKFS-fs (sdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc
[74745.477281] LustreError: 8763:0:(obd_mount_server.c:110:server_register_mount()) Already registered lustre-OST001d
[74745.478835] LustreError: 8763:0:(obd_mount_server.c:1992:server_fill_super()) Unable to start targets: -17
[74745.480369] LustreError: 8763:0:(obd_mount_server.c:1600:server_put_super()) no obd lustre-OST001d
[74746.236387] Lustre: server umount lustre-OST001d complete
[74746.236393] LustreError: 8763:0:(obd_mount.c:1608:lustre_fill_super()) Unable to mount (-17)
[74751.705099] LDISKFS-fs (sdd): file extents enabled, maximum tree depth=5
[74752.436188] LDISKFS-fs (sdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc
[74752.902198] LustreError: 8776:0:(llog_osd.c:987:llog_osd_next_block()) lustre-OST001d-osd: invalid llog tail at log id [0xb46f60d6:0xa1cf2803:0x0]:0 offset 20672 last_rec idx 0 tail idx 2250830848lrt len 2048 read_size 4288
[74752.905177] LustreError: 8776:0:(llog.c:1419:llog_backup()) MGC10.19.4.132 at o2ib: failed to backup log lustre-OST001d: rc = -22
[74752.908120] LustreError: 8776:0:(mgc_request.c:1879:mgc_llog_local_copy()) MGC10.x.x.x.132 at o2ib: failed to copy remote log lustre-OST001d: rc = -22
[74752.916309] LustreError: 8788:0:(nodemap_storage.c:206:nodemap_cache_find_create()) cannot destroy nodemap index: rc = -115
[74753.304620] LustreError: 8788:0:(obd_config.c:559:class_setup()) setup lustre-OST001d failed (-115)
[74753.306167] LustreError: 8788:0:(obd_config.c:1835:class_config_llog_handler()) MGC10.19.4.132 at o2ib: cfg command failed: rc = -115
[74753.309113] Lustre: cmd=cf003 0:lustre-OST001d 1:dev 2:0 3:f

[74753.309227] LustreError: 15c-8: MGC10.19.4.132 at o2ib: The configuration from log 'lustre-OST001d' failed (-115). This may be the result of communication errors between this node and the MGS, a bad configuration, or other errors. See the syslog for more information.
[74753.313693] LustreError: 8776:0:(obd_mount_server.c:1397:server_start_targets()) failed to start server lustre-OST001d: -115
[74753.315270] LustreError: 8776:0:(obd_config.c:610:class_cleanup()) Device 13 not setup
[75312.334648] LDISKFS-fs (sdd): file extents enabled, maximum tree depth=5
[75313.081093] LDISKFS-fs (sdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc
[75313.081112] LustreError: 8822:0:(osd_handler.c:7815:osd_mount()) lustre-OST001d-osd: failed to set lma on /dev/sdd root inode
[75313.082671] LustreError: 8822:0:(obd_config.c:559:class_setup()) setup lustre-OST001d-osd failed (-30)
[75313.084163] LustreError: 8822:0:(obd_mount.c:202:lustre_start_simple()) lustre-OST001d-osd setup error -30
[75313.085645] LustreError: 8822:0:(obd_mount_server.c:1958:server_fill_super()) Unable to start osd on /dev/sdd: -30
[75313.087122] LustreError: 8822:0:(obd_mount.c:1608:lustre_fill_super()) Unable to mount (-30)
[75313.088583] LustreError: 8822:0:(obd_mount.c:1608:lustre_fill_super()) Skipped 1 previous similar message
[75330.678571] LDISKFS-fs (sdd): file extents enabled, maximum tree depth=5
[75331.407996] LDISKFS-fs (sdd): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc
[75331.408015] LustreError: 8837:0:(osd_handler.c:7815:osd_mount()) lustre-OST001d-osd: failed to set lma on /dev/sdd root inode
[75331.409564] LustreError: 8837:0:(obd_config.c:559:class_setup()) setup lustre-OST001d-osd failed (-30)
[75331.411040] LustreError: 8837:0:(obd_mount.c:202:lustre_start_simple()) lustre-OST001d-osd setup error -30
[75331.412528] LustreError: 8837:0:(obd_mount_server.c:1958:server_fill_super()) Unable to start osd on /dev/sdd: -30

You guys' support will be highly appreciated as we are stuck very badly. lustre version is 2.12.6 .

Regards,

Ihsan Ur Rahman

Cheers, Andreas
—
Andreas Dilger
Lustre Principal Architect
Whamcloud/DDN

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20251023/2f3d7195/attachment-0001.htm>