<HTML><BODY style="word-wrap: break-word; -khtml-nbsp-mode: space; -khtml-line-break: after-white-space; ">Check <DIV><A href="http://manual.lustre.org">http://manual.lustre.org</A></DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>When you mount the mds/mgs with -t lustre it activates the server. </DIV><DIV>You will also need 1 or more OSS (mkfs.lustre --ost )</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>You will then need to also mount it </DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>Then on the client you mount like nfs</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>mount -t lustre mgshost:/lustre /test_share/</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>All the lustre information is built on top of ext3+patches, but you can not access it directly (unless you mount without -t lustre which mounts it ext3)</DIV><DIV><BR class="khtml-block-placeholder"></DIV><DIV>This is all spelled out in the manual.<BR><DIV> <SPAN class="Apple-style-span" style="border-collapse: separate; border-spacing: 0px 0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: auto; -khtml-text-decorations-in-effect: none; text-indent: 0px; -apple-text-size-adjust: auto; text-transform: none; orphans: 2; white-space: normal; widows: 2; word-spacing: 0px; "><SPAN class="Apple-style-span" style="border-collapse: separate; border-spacing: 0px 0px; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: auto; -khtml-text-decorations-in-effect: none; text-indent: 0px; -apple-text-size-adjust: auto; text-transform: none; orphans: 2; white-space: normal; widows: 2; word-spacing: 0px; "><DIV><BR class="khtml-block-placeholder"></DIV><DIV>Brock Palen</DIV><DIV>Center for Advanced Computing</DIV><DIV><A href="mailto:brockp@umich.edu">brockp@umich.edu</A></DIV><DIV>(734)936-1985</DIV><BR class="Apple-interchange-newline"></SPAN></SPAN> </DIV><BR><DIV><DIV>On Oct 25, 2007, at 9:50 AM, Iain Grant wrote:</DIV><BR class="Apple-interchange-newline"><BLOCKQUOTE type="cite"> <DIV class="Section1"><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial">Hi there,<O:P></O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial"><O:P> </O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial">I have just started a new position with a company that has a nice 20 node cluster.<O:P></O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial"><O:P> </O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial">This cluster is a nice setup, except they have not got any shared storage setup, they did have NFS configured but the read times they were getting for processing data was slowing the cluster down considerably.<O:P></O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial">After looking around I have found lustre and thought I would configure it on a spare node and see what kind of results I can get from it.<O:P></O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial"><O:P> </O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial">So I have a system running Redhat Enterprise V 4 and have downloaded the the pre built RPM’s from the following area on clusterfs.com<O:P></O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial"><O:P> </O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial"><A href="http://www.clusterfs.com/downloads/public/Lustre/v1.6/Production/latest/rhel-2.6-i686/">http://www.clusterfs.com/downloads/public/Lustre/v1.6/Production/latest/rhel-2.6-i686/</A><O:P></O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial"><O:P> </O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial">I’ve installed the new kernel and modified the grub.conf file so it is being booted up.<O:P></O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial">Also I have installed<O:P></O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial"><O:P> </O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial">Kernel-lustre-source<O:P></O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial">Lustre-1.6.3-…<O:P></O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial">Lustre-debuginfo<O:P></O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial">Lustre-ldiskfs<O:P></O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial">Lustre-modules<O:P></O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial">Lustre source<O:P></O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial"><O:P> </O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial">Now at the moment I have a second 70Gb sata drive on the system I would like to use as the lustre drive, so following the instructions I have used the following command<O:P></O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial"><O:P> </O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial">Mkfs.lustre –mdt –mgs /dev/sdb<O:P></O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial"><O:P> </O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial">Which shows up all the right messages and is successful, <O:P></O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial">I then mount the drive <O:P></O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial"><O:P> </O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial">Mount –t lustre /dev/sdb /test_share<O:P></O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial"><O:P> </O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial">which successfully mounts and I can look at “cat /proc/fs/lustre/devices” and can see all the right information.<O:P></O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial"><O:P> </O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial">Now the thing is if I try and write to this filesystem I get an error<O:P></O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial"><O:P> </O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial">Not a directory ????<O:P></O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial"><O:P> </O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial">So I’m kinda lost. What am I doing wrong ???????<O:P></O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial"><O:P> </O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial">If I mount the filesystem with a standard command<O:P></O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial"><O:P> </O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial">Mount /dev/sdb /test_share<O:P></O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial"><O:P> </O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial">I can see the following files and directories listed<O:P></O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial"><O:P> </O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial">[root@tester /]# ls /test_share<O:P></O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial">CONFIGS last_rcvd lost+found OBJECTS ROOT<O:P></O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial">health_check LOGS lov_objid PENDING<O:P></O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial"><O:P> </O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial">Help.<O:P></O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial"><O:P> </O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial"><O:P> </O:P></SPAN></FONT></P><P class="MsoNormal"><FONT size="2" face="Arial"><SPAN lang="EN-GB" style="font-size: 10.0pt;font-family:Arial"><O:P> </O:P></SPAN></FONT></P> </DIV> <BR><BR><TABLE bgcolor="white" style="color:black"><TBODY><TR><TD><BR>_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _<BR> <BR> SCRI, Invergowrie, Dundee, DD2 5DA. <BR> The Scottish Crop Research Institute is a charitable company limited by guarantee. <BR> Registered in Scotland No: SC 29367.<BR> Recognised by the Inland Revenue as a Scottish Charity No: SC 006662.<BR> <BR> <BR> DISCLAIMER:<BR> <BR> This email is from the Scottish Crop Research Institute, but the views <BR> expressed by the sender are not necessarily the views of SCRI and its <BR> subsidiaries. This email and any files transmitted with it are confidential <BR> to the intended recipient at the e-mail address to which it has been <BR> addressed. It may not be disclosed or used by any other than that addressee.<BR> If you are not the intended recipient you are requested to preserve this <BR> confidentiality and you must not use, disclose, copy, print or rely on this <BR> e-mail in any way. Please notify <A href="mailto:postmaster@scri.ac.uk">postmaster@scri.ac.uk</A> quoting the <BR> name of the sender and delete the email from your system.<BR> <BR> Although SCRI has taken reasonable precautions to ensure no viruses are <BR> present in this email, neither the Institute nor the sender accepts any <BR> responsibility for any viruses, and it is your responsibility to scan the email <BR> and the attachments (if any).<BR> </TD></TR></TBODY></TABLE><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">_______________________________________________</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Lustre-discuss mailing list</DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><A href="mailto:Lustre-discuss@clusterfs.com">Lustre-discuss@clusterfs.com</A></DIV><DIV style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><A href="https://mail.clusterfs.com/mailman/listinfo/lustre-discuss">https://mail.clusterfs.com/mailman/listinfo/lustre-discuss</A></DIV> </BLOCKQUOTE></DIV><BR></DIV></BODY></HTML>