<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
<title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
Hi Matthias,<br>
<br>
As far as I know, RDAC also supports multipathing. So, you should have
only one of RDAC or dm-multipath enabled on the MDS server. Moreover,
your HBA driver could be doing multipathing and failover as well. <br>
<br>
I usually disable failover in HBA driver and use RDAC only for reliable
multupathing. For example, if using qlogic FC HBAs, set "options
qla2xxx ql2xfailover=0" in /etc/modprobe.conf and see if all shared
luns are visible through both controllers and RDAC (ls -lR /proc/mpp).<br>
<br>
Cheers,<br>
-Atul<br>
<br>
<br>
On 06/16/2010 07:23 PM, lustre wrote:
<blockquote cite="mid:4C18981E.2010502@navum.de" type="cite">
<meta http-equiv="Content-Type"
content="text/html; charset=ISO-8859-1">
<meta name="Generator" content="Microsoft Word 11 (filtered medium)">
<style>
<!--
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman";}
a:link, span.MsoHyperlink
{color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:Arial;
color:windowtext;}
@page Section1
{size:595.3pt 841.9pt;
margin:70.85pt 70.85pt 2.0cm 70.85pt;}
div.Section1
{page:Section1;}
-->
</style><font
face="Arial">Hello Folks,</font><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB"><br>
<br>
</span></font><font face="Arial">we have one LUN on our MGS|MGT
Server.<br>
The LUN is availible over two pathes.<br>
( Multipathing with OS embeded rdac driver, SLES11)<br>
<br>
</font>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB">snowball-mds2:/proc/fs
#
multipath -ll <o:p></o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB">3600a0b80005a7215000002034b952b00
dm-10
SUN,LCSM100_S<o:p></o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB">[size=419G][features=1
queue_if_no_path][hwhandler=1
rdac][rw]<o:p></o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB">\_
round-robin 0 [prio=6][active]<o:p></o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB"> \_ 6:0:1:1
sdd 8:48 [active][ready]<o:p></o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB">\_
round-robin 0 [prio=1][enabled]<o:p></o:p></span></font></p>
<font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB"> \_ 6:0:0:1
sdb 8:16 [active][ghost]</span></font><font face="Arial"><br>
<br>
We create an LVM device on this LUN.<br>
<br>
</font>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB">snowball-mds2:~
#
lvscan<o:p></o:p></span></font></p>
<font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB"><o:p></o:p></span></font><font
face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB">
ACTIVE '/dev/mds2/mgs2' [418.68 GB]
inherit</span></font><font face="Arial"><br>
<br>
</font>
<meta http-equiv="Content-Type"
content="text/html; charset=ISO-8859-1">
<meta name="Generator" content="Microsoft Word 11 (filtered medium)">
<style>
<!--
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman";}
a:link, span.MsoHyperlink
{color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:Arial;
color:windowtext;}
@page Section1
{size:595.3pt 841.9pt;
margin:70.85pt 70.85pt 2.0cm 70.85pt;}
div.Section1
{page:Section1;}
-->
</style>
<div class="Section1">
<pre><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB"><o:p></o:p></span></font><big><font
face="Arial" size="2"><big><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB"><o:p>everything works fine.</o:p></span></big></font></big></pre>
<pre><big><font face="Arial" size="2"><big><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB"><o:p></o:p></span></big></font></big></pre>
<pre><big><font face="Arial" size="2"><big><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB"><o:p>Now we switched the controller on the storage to simulate a path failover:</o:p></span></big></font></big></pre>
<pre><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB"><o:p></o:p></span></font></pre>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB"># multipath
-ll<o:p></o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB">3600a0b80005a7215000002034b952b00
dm-10
SUN,LCSM100_S<o:p></o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB">[size=419G][features=1
queue_if_no_path][hwhandler=1
rdac][rw]<o:p></o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB">\_
round-robin 0 [prio=1][enabled]<o:p></o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB"> \_ 6:0:1:1
sdd 8:48 [active][ghost]<o:p></o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB">\_
round-robin 0 [prio=6][enabled]<o:p></o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB"> \_ 6:0:0:1
sdb 8:16 [active][ready]<o:p></o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB"><o:p> <br>
after that the MDT Device is unhealthy:<br>
<br>
</o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB">snowball-mds2:/proc/fs
#
cat
/proc/fs/lustre/health_check<o:p></o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB">device
tools-MDT0000 reported unhealthy<o:p></o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB">NOT HEALTHY<o:p></o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB"><o:p> </o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB"><o:p>and we
can not remount the filesystem -> the filesystem is not writeable</o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB"><o:p>We can
see this in /var/log/messages, as there is a warning about this
filesystem beeing in read-only mode.<br>
</o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB"><o:p><br>
</o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB">snowball-mds2:/proc/fs
#
tunefs.lustre --dryrun
/dev/mds2/mgs2<o:p></o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB">checking for
existing Lustre data: found
CONFIGS/mountdata<o:p></o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB">Reading
CONFIGS/mountdata<o:p></o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB"><o:p> </o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB"> Read
previous values:<o:p></o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB">Target:<o:p></o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB">Index:
unassigned<o:p></o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB">Lustre FS:
lustre<o:p></o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB">Mount type:
ldiskfs<o:p></o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB">Flags:
0x70<o:p></o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB">
(needs_index
first_time update )<o:p></o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB">Persistent
mount opts:<o:p></o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB">Parameters:<o:p></o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB"><o:p> <br>
</o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB">tunefs.lustre
FATAL:
must set target type:
MDT,OST,MGS<o:p></o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB">tunefs.lustre:
exiting
with 22 (Invalid argument)<o:p></o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB"><o:p> <br>
<br>
</o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB"><o:p>after a
reboot everything works fine again.<br>
Is there a problem this the lvm configuration?<br>
We found an document to enable multipathing on lvm2, but it doesent
work.</o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB"><o:p>Is
lustre 1.8.2 supported on lvm and multipathing?<br>
</o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB"><o:p><br>
We are concerning about the availability and consistency about the
lustre filesystem e.g. metadata. Because<br>
the metadata isn't correctly acailable after a path failover of the
metadata-(MDT)-device. The path-failover should<br>
be absolutely transparent to the LVM LUN used for the MDT and the
Lustre-FS on it. Is this correct? <br>
We tested the path-failover functionality with a simple ext3-fs on the
device and we could not see any problem.<br>
Also I think it is not recommended to configure the lustre filesystem
to remain in write-mode when an "error" accours, isnt't it?<br>
</o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB"><o:p><br>
Does anyone have experiance with the above mentioned configuration? Are
there any known bugs?<br>
</o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB"><o:p><br>
</o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB"><o:p>Thanks
and regards<br>
</o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB"><o:p>Matthias<br>
<br>
<br>
</o:p></span></font></p>
<font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB"><o:p></o:p></span></font>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB"><o:p>Additional
Information:<br>
</o:p></span></font></p>
<p class="MsoNormal"><font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB">snowball-mds2:/proc/fs
#
uname -a<o:p></o:p></span></font></p>
<font face="Arial" size="2"><span
style="font-size: 10pt; font-family: Arial;" lang="EN-GB">Linux
snowball-mds2
2.6.27.39-0.3_lustre.1.8.2-default #1 SMP 2009-11-23 12:57:38 +0100
x86_64
x86_64 x86_64 GNU/Linux</span></font><font face="Arial"><br>
</font></div>
<pre wrap="">
<fieldset class="mimeAttachmentHeader"></fieldset>
_______________________________________________
Lustre-discuss mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Lustre-discuss@lists.lustre.org">Lustre-discuss@lists.lustre.org</a>
<a class="moz-txt-link-freetext" href="http://lists.lustre.org/mailman/listinfo/lustre-discuss">http://lists.lustre.org/mailman/listinfo/lustre-discuss</a>
</pre>
</blockquote>
<br>
</body>
</html>