[Lustre-discuss] How smart is Lustre?

[Philippe Weill]

Le 19/12/2012 18:36, Jason Brooks a écrit :
> Hello,
>
> I am building a 2.3.x filesystem right now, and I am looking at setting up some active-active
> failover abilities to my oss's. I have been looking at Dell's md3xxx arrays, as they have redundant
> controllers, and allow up to four hosts to connect to each controller.
>
> I can see how linux multi-path can be used with redundant disk controllers. I can even (slightly)
> understand how lustre fails over when an oss goes down.
>
>  1. Is lustre smart enough to use redundant paths, or failover oss's if an oss is congested? (it
>     would be cool, no?)
>  2. Does the linux multi-path module slow performance?
>  3. How much does a raid array such as the one listed above act as a bottleneck, say if I have as
>     many volumes available on the raid controllers as there are oss hosts?
>  4. Are there arrays similar to Dell's model that would work?

I'm using 1 dell MD3660F  on lustre (60x3to) disk
and also nec sgi ibm netapp-lsi this the same hardware (netapp 2660 ;-) on 1.8.8wc1 and IB
2 diskrack are in production for more than one year now without any problem
we ave now 5 rack like this + 2 jbod extension with 60 disk connected - each jbod connected to one 
primary rack

with Hyper perf license controllers do peak performance 2Go/s write on 6 raid6 8+2
to achieve this we used 3 server connected directly to diskrack with two FC ports over IB
each server with 2 OST

on the rack write caching is disable ( cache mirror divide perf by two )

we work with 4 or 6 OST by OSS on production ( sufficient for our need )
failover per oss pair   ( not on congestion only when an oss goes down )

I didn't notice many penalty with multipath ( active/passive mode for this material )

-- 
  Weill Philippe -  Administrateur Systeme et Reseaux
  CNRS/UPMC/IPSL   LATMOS (UMR 8190)
  Email:philippe.weill at latmos.ipsl.fr