[Lustre-discuss] SAN, shared storage, iscsi using lustre?

[Brian J. Murrell]

On Tue, 2008-08-12 at 23:00 +0300, Alex wrote:
> 
> I mean that i'm exporting via iscsi a block device

Ahhh.  Now that's nomenclature I'm grasping.  :-)

> In this case is and entire hard disk, 120GB large, 
> named for simplicity volX (vol1, vol2... vol8) because i have 8 computers 
> doing this.

Right.  So you have 8 iscsi disks.

> For example could be a GFS shared volume over volX. Here on lustre i don't 
> know ... You tell me ...

These 3 servers are still unclear to me.  What do you see their function
as being?  Would they be the Lustre filesystem servers to which Lustre
clients go to get access to the shared filesystem composed of the 8
iscsi disks?

If so, then it sounds like your 3 servers are looking to create a Lustre
filesystem with the 8 iscsi disks.  This is doable but not a typical
scenario.  You would probably dedicate one of the 8 targets to the MDT
and the other 7 as OSTs.

> I don't know lustre. I asked about. I just want to know if is possibile ... If 
> the answer is yes, my question is: who will be MDS and WHO will be OSSes. How 
> MANY MDS and HOW MANY OSSes I NEED in order to obtain what i want!

Well, from your explanation above I would imagine you would use your
three servers to create 1 active MDS and 2 active OSSes with each OSS
hosting 4 and 3 OSTs respectively.  You could pair the machines so that
one would pick up the slack of a failed machine in a failover event.
Something like:

Server 1         Server 2          Server 3
Primary MDS      Primary OSS1      Primary OSS2
Backup OSS1      Backup OSS2       Backup MDS

> Can you run Lustre on LVM volumes, software RAID, etc?
> 
> Yes. You can use any Linux block device as storage for a backend Lustre server 
> file system, including LVM or software RAID devices. 
> [end snip]

Oh, hrm.  You want no SPOF.  Given that you have 8 targets all in 8
different machines, I'm not quite sure how you are going to achieve
that.  I suppose you could mirror two of the 8 iscsi devices for the MDT
and RAID5/6 the remaining 6 iscsi devices into a single volume.  You
could then have 1 of your 3 machines be Primary MDS, the other Primary
OSS and the third backup for both of the first.  It does seem a bit
wasteful to have a single machine doing nothing but waiting for failure
but this is frequently the case when you are working on the low end with
so little.


> So, for me, reading this, is very clear without being an expert that lustre 
> support BLOCK DEVICES in any RAID/LVM configuration...

Indeed.

> Also lustre, can work 
> with my iscsi block devices /dev/sd[a-h] ... 

Sure.  But you will have to build your redundancy on those devices
before giving them to Lustre.  Lustre provides no redundancy itself and
relies on the underlying block device to be redundant.

> This is not clear at all... Generally speaking ext3 is a local file system 
> (used on one computer). Reading FAQ, didn't find an answer, so i asked 
> here...

Right.  It is in fact too much information for the Lustre beginner.  You
should just be told that Lustre operates on and manages the block
device.  That it does so through ext3 only serves to confuse the Lustre
beginner.  Later when you have a better grasp on the architecture it
might be worthwhile understanding that each Lustre server does it's
management of the block device via ext3.  So please, don't worry about
the traditional uses of ext3 and confuse it's limitations with Lustre.
Lustre simply didn't want to invent a new on-disk management library and
used ext3 for it.

> No i can't ignore ... I want to be sure that ext3 used by lustre is a 
> clustered file system.

Well, you are just going to have to take my word for it or start digging
deeper into the architecture of Lustre to be sure of that.  Beyond what
I've already explained I'm not going to get any deeper into the
internals of Lustre.

> Redhat NEVER indicated their ext3 as file system for 
> clusters.

First of all, ext3 is not RedHat's.  Second, ext3, in and of itself is
not for clusters, this is true.

> So, if lustre's ext3 file system 
> is clustered, why nobody add a note to the FAQ about that: "we are using a 
> patched ext3 version, which differ by redhat ext3 because it support cluster 
> file systems like GFS"...

Because it's not like that.  As I have said, we simply use ext3 as a
storage mechanism.

b.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20080812/37e588da/attachment.pgp>