[Lustre-discuss] is lustre for us?

[Mag Gam]

thankyou all

answered my question


On Mon, Jul 7, 2008 at 12:45 PM, Peter Grandi <pg_lus at lus.for.sabi.co.uk> wrote:
>
>>> At my university we have a big storage problem. Currently we
>>> have 25 Linux servers each equiped with 3 TB of data. Totaling
>>> to 75TB of storage. We use each server as a NFS server and
>>> have clients mount up the filesystem. They are all connected
>>> by a GigE network.
>
>>> The problem we are facing is performance and scalibility.
>>> When a users access data, many times they access the same
>>> server. This is becoming a huge performance problem. The
>>> files sizes range from 8GB to 70GB.
>
> The impression I get from the above is that it is not clear what
> kind of performance and scalability problems you have. It is far
> from clear what is the status of any of the following "details":
>
> * Storage system structure and speed.
> * Network performance tuning.
> * How many users tend to access the same fileserver at once.
> * Speed of the clients.
> * Performance of the network switches/routers.
> * ....
>
> You have large scale storage issues, and perhaps the way they
> are expressed above is less clear than it should be.
>
> For example, in the best of cases you can get around 100MB/s
> reads out of a single NFS server to a single NFS (or Lustre)
> client, and 3GB will take 30s and 70GB over 10 minutes just to
> transfer at this speed, irrespective of filesystem performance
> and scalability. In a similar environment I have been pushing
> 10G switches/routers and 10G cards for servers and even for
> clients (and too bad that 10GBASE-T products are still a dream).
>
>>> Would Lustre solve our performance and management
>>> problem?
>
> Which management problem? Above you mention only performance.
> You also don't mention any reliability/availalability goals,
> or backup.
>
> Also, the performance is not at all clear. Single client speed?
> Aggregate speed? Server load? ...
>
>>> How would lustre scale?
>
> To what? A petabyte? A thousand clients? A 10G network? ...
>
>>> I have been reading the quickstart guide, is it easy to
>>> implement?
>
> Implementing a high performance, scalable storage pool for
> 3-70GB files starting at 75TB is not a trivial task and requires
> a lot of clarity and experience irrespective of the network
> filesystem used...
>
> BTW, I have written some of my ineffable wisdom :-) on the
> subject here:
>
>  http://WWW.sabi.co.UK/blog/0804apr.html#080417b
>
> However as AndreasD says, Lustre overall seems pretty good at
> medium-large scientific data storage systems, but the vague
> nature of your concerns makes a bit difficult to evaluate them.
>
> Perhaps those that have only a vague idea of their goals should
> read a few articles detailing users experiences with Lustre,
> there are quite a few pointers on the Lustre Wiki, and in
> various years of the HEPiX workshop.
>
>> hello andreas, i have totally different enviroment than mag's
>> we have many small file which less than 1Mb and i didn't
>> (actually i can't) know exactly how many it is.  it seems more
>> than 10 millions of files..  in our situation, is lustre
>> suitable??
>
> 10^6 each less than 10^6 large. Looks like 1TB, for which an
> external FW800 or eSATA drive with a small-file friendly
> filesystem seems a good match. Perhaps JFS or ReiserFS; but
> 'fsck' times are going however to be potentially long (days to
> weeks if the filesystems gets damaged). Lustre has the great
> advantage that it can 'fsck' its OSTs in parallel, so perhaps
> dividing that 1TB into 50-100G OSTs may be of benefit as each
> filesystem would end up having a lot less files.
>
> Note however that Lustre in effect is based on the idea of
> clients mounting individual files (or file slices) instead of
> filetrees (with the MDS in the role of the automount directory
> server), so each file open involves at least two network round
> trips, which means that latency on metadata-intensive loads will
> be less awesome.
>
> >From the very vague goals you give above it is not at all clear
> to me whether any of this matters or not.
>
> As to the several and often underrated aspects on which a file
> system design can be evaluated, here is more of my incredible
> insight :-):
>
>  http://WWW.sabi.co.UK/blog/0804apr.html#080415
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>