[lustre-discuss] backup zfs MDT or migrate from ZFS back to ldiskfs

[Stu Midgley]

Afternoon

I have an MDS running on spinning media and wish to migrate it to SSD's.

    Lustre 2.9.52
    ZFS 0.7.0-rc3

How do I do it?


This is a loaded question :)

The MDT is using ~2TB of space.  I used the zfs send | zfs receive method
to no avail.  It was just too slow (I killed it after a week), gave no
indication of how far though it was and the destination FS was using ~4TB
of space (more than the 2TB of the source FS).

I have tried creating a snapshot, mounting it and rsyncing the contents to
the new FS.  That fails with


mount.lustre: mount localData/t1-MDT0001 at /localData/t1-MDT0001 failed:
Invalid argument

This may have multiple causes.

Are the mount options correct?

Check the syslog for more info.

​
and from dmesg


​[6982824.344872] Lustre: 401:0:(client.c:2113:ptlrpc_expire_one_request())
@@@ Request sent has timed out for slow reply: [sent 1500606249/real
1500606249]  req at ffff880d70b56300 x1566221596712816/t0(0)
o38->t1-MDT0000-lwp-OST0000 at 0@lo:12/10 lens 520/544 e 0 to 1 dl 1500606260
ref 1 fl Rpc:XN/0/ffffffff rc 0/-1

[6982829.865323] LustreError: 18881:0:(obd_config.c:574:class_setup())
setup t1-MDT0000 failed (-22)

[6982829.868476] LustreError:
18881:0:(obd_config.c:1710:class_config_llog_handler()) MGC172.16.250.40 at tcp:
cfg command failed: rc = -22

[6982829.874691] Lustre:    cmd=cf003 0:t1-MDT0000  1:t1-MDT0000_UUID  2:0
3:t1-MDT0000-mdtlov  4:f

[6982829.881030] LustreError: 15b-f: MGC172.16.250.40 at tcp: The
configuration from log 't1-MDT0000'failed from the MGS (-22).  Make sure
this client and the MGS are running compatible versions of Lustre.

[6982829.887555] LustreError:
18811:0:(obd_mount_server.c:1352:server_start_targets()) failed to start
server t1-MDT0000: -22

[6982829.890670] LustreError:
18811:0:(obd_mount_server.c:1844:server_fill_super()) Unable to start
targets: -22

[6982829.893651] LustreError: 18811:0:(obd_config.c:621:class_cleanup())
Device 443 not setup

[6982830.300528] Lustre: server umount t1-MDT0000 complete

[6982830.303158] LustreError:
18811:0:(obd_mount.c:1496:lustre_fill_super()) Unable to mount  (-22)


​While doing the rsync -aXv I get weird error messages


​file has vanished: "/mnt/mdt.s1/oi.7/0x200000007:0x1:0x0/mpartition"


​which is on the ro snapshot of the original FS.  It gets even weirder...


170721112705 pud40:mnt# ls -l
/mnt/mdt.s1/oi.7/0x200000007:0x1:0x0/mpartition

ls: cannot read symbolic link
/mnt/mdt.s1/oi.7/0x200000007:0x1:0x0/mpartition: No such file or directory

lrwxrwxrwx 1 root root 6 Mar  9 07:50
/mnt/mdt.s1/oi.7/0x200000007:0x1:0x0/mpartition

# strace -f -e readlink -- ls -l
/mnt/mdt.s1/oi.7/0x200000007:0x1:0x0/mpartition

readlink("/mnt/mdt.s1/oi.7/0x200000007:0x1:0x0/mpartition", 0xc8c320, 7) =
-1 ENOENT (No such file or directory)

ls: cannot read symbolic link
/mnt/mdt.s1/oi.7/0x200000007:0x1:0x0/mpartition: No such file or directory

lrwxrwxrwx 1 root root 6 Mar  9 07:50
/mnt/mdt.s1/oi.7/0x200000007:0x1:0x0/mpartition

+++ exited with 2 +++



# stat /mnt/mdt.s1/oi.7/0x200000007:0x1:0x0/mpartition

  File: ‘/mnt/mdt.s1/oi.7/0x200000007:0x1:0x0/mpartition’stat: cannot read
symbolic link ‘/mnt/mdt.s1/oi.7/0x200000007:0x1:0x0/mpartition’: No such
file or directory


  Size: 6               Blocks: 2          IO Block: 512    symbolic link

Device: 41h/65d Inode: 738         Links: 1

Access: (0777/lrwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/    root)

Access: 2017-07-21 10:44:27.000000000 +0800

Modify: 2017-03-09 07:50:43.000000000 +0800

Change: 2017-07-21 10:44:27.000000000 +0800

 Birth: -


so the file is in the directory listing as a link but not on the file
system??​

So, this leads me to think that lustre is using features of ZFS that linux
can't access?  or something??

It also raised the issue... how would I migrate off ZFS if I wanted to go
back to ldiskfs for the MDT?

Any ideas appreciated

Thanks.


-- 
Dr Stuart Midgley
sdm900 at sdm900.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20170721/3954a2f8/attachment.htm>