[lustre-discuss] lustre-discuss Digest, Vol 158, Issue 10

Kurt Strosahl strosahl at jlab.org
Thu May 9 13:38:42 PDT 2019

Presently I'm experimenting with the following:

alias ost01d1shasl00  /dev/disk/by-id/dm-uuid-mpath-35000cca26b825d6c
alias ost01d2shasl01  /dev/disk/by-id/dm-uuid-mpath-35000cca26b860178
alias ost01d3shasl02  /dev/disk/by-id/dm-uuid-mpath-35000cca26c1e2cb4
alias ost01d4shasl03  /dev/disk/by-id/dm-uuid-mpath-35000cca2680a8280

and using that to make sure that the disks have persistent names across reboots.  I still need to test pulling out one of the SAS cables to make sure that the multipathing works, and test replacing a disk.

I read on http://wiki.lustre.org/ZFS_OSD_Hardware_Considerations that

"The configuration of the device multi-mapper service is quite complex and can affect the performance characteristics of the solution. In some cases, JBODs can exhibit bad behavior from using load-balanced IO balancing, when in fact all the requests for a disk are expected to arrive from a single interface. For this reason, when working with JBODS it is recommended to use the path_grouping_policy that enables failover-only capability."

so I presently set my systems to failover mode.


From: lustre-discuss <lustre-discuss-bounces at lists.lustre.org> on behalf of lustre-discuss-request at lists.lustre.org <lustre-discuss-request at lists.lustre.org>
Sent: Thursday, May 9, 2019 4:21 PM
To: lustre-discuss at lists.lustre.org
Subject: lustre-discuss Digest, Vol 158, Issue 10

Send lustre-discuss mailing list submissions to
        lustre-discuss at lists.lustre.org

To subscribe or unsubscribe via the World Wide Web, visit
or, via email, send a message with subject or body 'help' to
        lustre-discuss-request at lists.lustre.org

You can reach the person managing the list at
        lustre-discuss-owner at lists.lustre.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of lustre-discuss digest..."

Today's Topics:

   1. Enable multipath for existing Lustre OST with ZFS backend
      (Tung-Han Hsieh)


Message: 1
Date: Fri, 10 May 2019 03:25:52 +0800
From: Tung-Han Hsieh <thhsieh at twcp1.phys.ntu.edu.tw>
To: lustre-discuss at lists.lustre.org
Subject: [lustre-discuss] Enable multipath for existing Lustre OST
        with ZFS backend
Message-ID: <20190509192551.GA5793 at twcp1.phys.ntu.edu.tw>
Content-Type: text/plain; charset=big5


Recently we have a new storage device. It has dual RAID controllers
with two fibre connections to the file server which map the LUM of
the storage to the server:

# lsscsi -g
[5:0:0:0]    disk    IFT      DS 1000 Series   661J  /dev/sdb   /dev/sg4
[6:0:0:0]    disk    IFT      DS 1000 Series   661J  /dev/sdc   /dev/sg6

# /lib/udev/scsi_id -g -u /dev/sdb

# /lib/udev/scsi_id -g -u /dev/sdc

So /dev/sdb and /dev/sdc are actually the same LUM of the storage.

We have created the Lustre OST with ZFS backend on /dev/sdb:

# mkfs.lustre --ost --fsname chome --mgsnode=<host> --index=0 \
              --backfstype=zfs chome_ost/ost /dev/sdb

It works fine. But soon after that, I was told that I should setup
multipath to take the advantage of dual fibre channel for load
balance and HA. I am wondering whether it is too late or not because
we already have data of Lustre file system running on it.

I read the documents of multipath. It seems that after setting
multipath, both /dev/sdb and /dev/sdc are re-mapped to, say,
/dev/mapper/mpath0. The existing data is probably not affacted.
What we need to do is just to replace the device name /dev/sdb
by /dev/mapper/mpath0 (please correct me if I am wrong). So the
problem seems leading to ZFS. Now my OST pool "chome_ost/ost" was
created on /dev/sdb. Could we replace the pool device name to
/dev/mapper/mpath0 ?

Thanks very much for you suggestions in advance :)

Best Regards,



Subject: Digest Footer

lustre-discuss mailing list
lustre-discuss at lists.lustre.org


End of lustre-discuss Digest, Vol 158, Issue 10
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20190509/0bcd787a/attachment.html>

More information about the lustre-discuss mailing list