[Lustre-discuss] Performance Expectations of Lustre

[Brian J. Murrell]

On Mon, 2009-01-26 at 16:51 +0100, Nick Jennings wrote:
> Hello (and a special hello to all my ex-co-workers from the CFS days :)

And MVD days too.  ;-)

>   The company where I work now has grown fast in the past year and we 
> suddenly find ourselves in need of a lot of storage. For 5 years the 
> company ran on a 60gig server, last year we got a 1TB RAID that is now 
> almost full. In 1-2 years we could easily be using 10-15TB of storage.

Good on y'all for keeping the storage industry busy.  :-)

>   Instead of just adding another 1TB server, I need to plan for a more 
> scalable solution. Immediately Lustre came to mind, but I'm wondering 
> about the performance. Basically our company does niche web-hosting for 
> "Creative Professionals" so we need fast access to the data in order to 
> have snappy web services for our clients. Typically these are smaller 
> files (2MB pictures, 50MB videos, .swf files, etc.).

Well, I'm not sure those files would fall within our general
classification of "small files" (wherein we know we don't perform very
well).  Our small-file issues are usually characterized by "kernel
builds" and ~ use, where files are usually much smaller than 1MB.

>   Also I'm wondering about the best way set this up in terms of speed 
> and ease of growth. I want the web-servers and the storage pool to be 
> independent of each other. So I can add web-servers as the web traffic 
> increases, and add more storage ass our storage needs grow.

Well, your web-servers would be Lustre clients.  There is no
relationship, or rather requirements in terms of the number of clients
and servers being used.  You use as many servers as your client load
demands.  So you could imagine both ends of the spectrum where only a
relatively few clients could be used to tax quite a few servers or the
opposite where a lot of clients with modest demand requires only a few
servers.

>   I was thinking initially we could start with 2 servers, both attached 
> to the storage array. setup as OSS' and functioning as (load balanced) 
> web-servers as well.

Sounds like you are describing 2 storage servers, which would require at
least 3 servers total.  Don't forget about the MDS.  Also don't forget
about HA if that's a concern for you.  You could make the 2 OSSes
failover partners for each other if you are willing to accept a possibly
lower performance impact when one of the OSSes failing.

If HA is important to you however, you need to address an MDS failover
with a second server to pick up the MDT should the active MDS fail.

As for OSSes being web-servers, that would require the OSS/Webservers
also be clients and that is an unsupported configuration due to the risk
of deadlock due to memory pressure.  The recommended architecture would
be to make the webservers Lustre clients.

>   Now, it's been years since I've played with Lustre, I'm sure some 
> stuff will come back to me as I start using it again, other things I'll 
> probably have to re-learn. I wanted to get some input from the Lustre 
> community on whether or not this seems like a reasonable use for Lustre? 

It's most certainly reasonable, if you make modifications to your
architecture as above.

> performance can I expect, am I out of touch to expect something similar 
> to a directly attached RAID array?

I think our generally talked about numbers are something on the order of
achieving 80% of the raw storage bandwidth (assuming a capable network
and so on).  Maybe somebody who is closer to the benchmarking that we
are constantly doing can comment further on how close-to-raw-disk we are
achieving lately.

b.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20090126/5576099e/attachment.pgp>