<br><tt><font size=2><br>
Hi Daire,</font></tt>
<br>
<br><tt><font size=2>In your thread discussing "e2scan MDT backup",
I was very interested in finding out more about how you initially setup
your OST's and MDT's with LVM. We're implementing a new set of production
Lustre servers and had Sun come on-site last fall to help get us started.
Our Sun Lustre consultant was very knowledgeable and helpful and
he setup our environment as follows:</font></tt>
<br>
<br><tt><font size=2>MGS/MDT Servers:</font></tt>
<br>
<br><tt><font size=2>MGS# df -h</font></tt>
<br><tt><font size=2>Filesystem Size
Used Avail Use% Mounted on</font></tt>
<br><tt><font size=2>/dev/mapper/vgroot-lvroot</font></tt>
<br><tt><font size=2>
9.7G 5.3G 3.9G 58% /</font></tt>
<br><tt><font size=2>/dev/mapper/vgroot-lvtmp</font></tt>
<br><tt><font size=2>
992M 34M 908M 4% /tmp</font></tt>
<br><tt><font size=2>/dev/mapper/vgroot-lvlawsonusr</font></tt>
<br><tt><font size=2>
2.0G 68M 1.8G 4% /lawson/usr</font></tt>
<br><tt><font size=2>/dev/xvda1 122M
85M 31M 74% /boot</font></tt>
<br><tt><font size=2>tmpfs
8.0G 0 8.0G 0% /dev/shm</font></tt>
<br><tt><font size=2><b>/dev/xvdb 1008M
34M 923M 4% /mnt/mgs</b></font></tt>
<br>
<br><tt><font size=2>MDT# df -h</font></tt>
<br><tt><font size=2>Filesystem Size
Used Avail Use% Mounted on</font></tt>
<br><tt><font size=2>/dev/mapper/vgroot-lvroot</font></tt>
<br><tt><font size=2>
9.7G 6.5G 2.8G 71% /</font></tt>
<br><tt><font size=2>/dev/mapper/vgroot-lvtmp</font></tt>
<br><tt><font size=2>
992M 34M 908M 4% /tmp</font></tt>
<br><tt><font size=2>/dev/mapper/vgroot-lvlawsonusr</font></tt>
<br><tt><font size=2>
2.0G 1.3G 609M 68% /lawson/usr</font></tt>
<br><tt><font size=2>/dev/xvda1 122M
85M 31M 74% /boot</font></tt>
<br><tt><font size=2>tmpfs
8.0G 0 8.0G 0% /dev/shm</font></tt>
<br><tt><font size=2><b>/dev/xvdj
3.5G 173M 3.2G 6% /mnt/lusfs01/mdt</b></font></tt>
<br><tt><font size=2><b>/dev/xvdk
3.5G 170M 3.2G 6% /mnt/lusfs02/mdt</b></font></tt>
<br><tt><font size=2><b>/dev/xvdl
3.5G 169M 3.2G 5% /mnt/lusfs03/mdt</b></font></tt>
<br><tt><font size=2><b>/dev/xvdm
3.5G 173M 3.2G 6% /mnt/lusfs04/mdt</b></font></tt>
<br><tt><font size=2><b>/dev/xvdn
3.5G 170M 3.2G 6% /mnt/lusfs05/mdt</b></font></tt>
<br>
<br><tt><font size=2>OSS Servers:</font></tt>
<br>
<br><tt><font size=2>OSS1# df -h</font></tt>
<br><tt><font size=2>Filesystem Size
Used Avail Use% Mounted on</font></tt>
<br><tt><font size=2>/dev/mapper/vgroot-lvroot</font></tt>
<br><tt><font size=2>
9.7G 7.7G 1.6G 84% /</font></tt>
<br><tt><font size=2>/dev/mapper/vgroot-lvtmp</font></tt>
<br><tt><font size=2>
992M 34M 908M 4% /tmp</font></tt>
<br><tt><font size=2>/dev/mapper/vgroot-lvlawsonusr</font></tt>
<br><tt><font size=2>
2.0G 68M 1.8G 4% /lawson/usr</font></tt>
<br><tt><font size=2>/dev/xvda1 122M
85M 31M 74% /boot</font></tt>
<br><tt><font size=2>tmpfs
8.0G 0 8.0G 0% /dev/shm</font></tt>
<br><tt><font size=2><b>/dev/xvde1 174G
461M 165G 1% /mnt/lusfs01/ost01</b></font></tt>
<br><tt><font size=2><b>/dev/xvdg1 174G
461M 165G 1% /mnt/lusfs01/ost03</b></font></tt>
<br><tt><font size=2><b>/dev/xvde2 106G
461M 100G 1% /mnt/lusfs02/ost01</b></font></tt>
<br><tt><font size=2><b>/dev/xvdg2 106G
461M 100G 1% /mnt/lusfs02/ost03</b></font></tt>
<br><tt><font size=2><b>/dev/xvde3
16G 439M 14G 3% /mnt/lusfs03/ost01</b></font></tt>
<br><tt><font size=2><b>/dev/xvdg3
16G 439M 14G 3% /mnt/lusfs03/ost03</b></font></tt>
<br><tt><font size=2><b>/dev/xvde5 7.6G
331M 6.9G 5% /mnt/lusfs04/ost01</b></font></tt>
<br><tt><font size=2><b>/dev/xvdg5 7.6G
331M 6.9G 5% /mnt/lusfs04/ost03</b></font></tt>
<br><tt><font size=2><b>/dev/xvde6
16G 2.2G 13G 15% /mnt/lusfs05/ost01</b></font></tt>
<br><tt><font size=2><b>/dev/xvdg6
16G 2.9G 12G 21% /mnt/lusfs05/ost03</b></font></tt>
<br>
<br><tt><font size=2>OSS2# df -h</font></tt>
<br><tt><font size=2>Filesystem Size
Used Avail Use% Mounted on</font></tt>
<br><tt><font size=2>/dev/mapper/vgroot-lvroot</font></tt>
<br><tt><font size=2>
9.7G 7.2G 2.1G 78% /</font></tt>
<br><tt><font size=2>/dev/mapper/vgroot-lvtmp</font></tt>
<br><tt><font size=2>
992M 34M 908M 4% /tmp</font></tt>
<br><tt><font size=2>/dev/mapper/vgroot-lvlawsonusr</font></tt>
<br><tt><font size=2>
2.0G 68M 1.8G 4% /lawson/usr</font></tt>
<br><tt><font size=2>/dev/xvda1 122M
85M 31M 74% /boot</font></tt>
<br><tt><font size=2>tmpfs
8.0G 0 8.0G 0% /dev/shm</font></tt>
<br><tt><font size=2><b>/dev/xvdd1 174G
461M 165G 1% /mnt/lusfs01/ost00</b></font></tt>
<br><tt><font size=2><b>/dev/xvdd2 106G
461M 100G 1% /mnt/lusfs02/ost00</b></font></tt>
<br><tt><font size=2><b>/dev/xvdf2 106G
461M 100G 1% /mnt/lusfs02/ost02</b></font></tt>
<br><tt><font size=2><b>/dev/xvdd3
16G 439M 14G 3% /mnt/lusfs03/ost00</b></font></tt>
<br><tt><font size=2><b>/dev/xvdf3
16G 439M 14G 3% /mnt/lusfs03/ost02</b></font></tt>
<br><tt><font size=2><b>/dev/xvdd5 7.6G
331M 6.9G 5% /mnt/lusfs04/ost00</b></font></tt>
<br><tt><font size=2><b>/dev/xvdf5 7.6G
331M 6.9G 5% /mnt/lusfs04/ost02</b></font></tt>
<br><tt><font size=2><b>/dev/xvdd6
16G 2.5G 12G 18% /mnt/lusfs05/ost00</b></font></tt>
<br><tt><font size=2><b>/dev/xvdf6
16G 1.1G 14G 8% /mnt/lusfs05/ost02</b></font></tt>
<br><tt><font size=2><b>/dev/xvdf1 174G
461M 165G 1% /mnt/lusfs01/ost02</b></font></tt>
<br>
<br><tt><font size=2>Lustre client server:</font></tt>
<br>
<br><tt><font size=2>CLIENT# df -h</font></tt>
<br><tt><font size=2>Filesystem Size
Used Avail Use% Mounted on</font></tt>
<br><tt><font size=2>/dev/mapper/vgroot-lvroot</font></tt>
<br><tt><font size=2>
9.7G 3.6G 5.7G 39% /</font></tt>
<br><tt><font size=2>/dev/mapper/vgroot-lvlawsonusr</font></tt>
<br><tt><font size=2>
2.0G 68M 1.8G 4% /lawson/usr</font></tt>
<br><tt><font size=2>/dev/mapper/vgroot-lvwebsphere</font></tt>
<br><tt><font size=2>
9.7G 1.7G 7.6G 18% /websphere</font></tt>
<br><tt><font size=2>/dev/mapper/vgroot-lvtest</font></tt>
<br><tt><font size=2>
2.0G 68M 1.8G 4% /test</font></tt>
<br><tt><font size=2>/dev/mapper/vgroot-lvtmp</font></tt>
<br><tt><font size=2>
992M 34M 908M 4% /tmp</font></tt>
<br><tt><font size=2>/dev/xvda1 122M
77M 39M 67% /boot</font></tt>
<br><tt><font size=2>tmpfs
8.0G 0 8.0G 0% /dev/shm</font></tt>
<br><tt><font size=2>/dev/mapper/vgroot-lvlogs</font></tt>
<br><tt><font size=2>
5.0G 139M 4.6G 3% /logs</font></tt>
<br><tt><font size=2><b>10.203.4.100@tcp,10.203.4.101@tcp:/lusfs01</b></font></tt>
<br><tt><font size=2><b>
694G 1.9G 657G 1% /content</b></font></tt>
<br><tt><font size=2><b>10.203.4.100@tcp,10.203.4.101@tcp:/lusfs02</b></font></tt>
<br><tt><font size=2><b>
423G 1.8G 400G 1% /newproducts</b></font></tt>
<br><tt><font size=2><b>10.203.4.100@tcp,10.203.4.101@tcp:/lusfs04</b></font></tt>
<br><tt><font size=2><b>
31G 1.3G 28G 5% /gid</b></font></tt>
<br><tt><font size=2><b>10.203.4.100@tcp,10.203.4.101@tcp:/lusfs03</b></font></tt>
<br><tt><font size=2><b>
61G 1.8G 56G 3% /products</b></font></tt>
<br><tt><font size=2><b>10.203.4.100@tcp,10.203.4.101@tcp:/lusfs05</b></font></tt>
<br><tt><font size=2><b>
61G 8.6G 49G 15% /scs/content</b></font></tt>
<br>
<br><tt><font size=2>As you can see, our Sun consultant set our production
environment up to use standard disk partitions that are striped across
two sets of servers for our five Lustre filesystems (which is working fine
and we can grow it no problems). We're running RHEL 5.2 XEN MGS/MDT/OSS
servers using Linux Heartbeat for STONITH and failover. The XEN domU's
are LVM-backed on EMC SAN. Lustre was compiled into the XEN kernel
and we've compiled and are using the patchless Lustre client (Lustre 1.6.6).</font></tt>
<br>
<br><tt><font size=2>What I'm really curious about is how to go about setting
up this environment under LVM2. Is it as simple as taking the "xvd?"
disk on the OSS and MDS servers, doing a pvcreate, vgcreate, lvcreate,
mkfs? Do you have to be careful about anything (striping as seen
above)? May I ask if you might be able to give an example? I'd
like to test LVM in our Lustre certification environment, but would like
to make sure I have an idea of what I'm doing before I mess up my working
cert environment. Are there any pros / cons for LVM2 versus disk
partitions? Backup issues, speed, performance, etc?</font></tt>
<br>
<br><tt><font size=2>> Once you move to LVM then you can snapshot the
MDT and mount that without<br>
> unmounting the production MDT. You should probably destroy the snapshot<br>
> afterwards as it affects performance somewhat.<br>
><br>
> Regards,<br>
><br>
> Daire<br>
</font></tt>
<br><font size=2 face="sans-serif">Cheers and many thanks for your consideration
and time,<br>
<br>
Ms. Andrea D. Rucks<br>
Sr. Unix Systems Administrator,<br>
Lawson ITS Unix Server Team</font>