[Lustre-discuss] Performance Expectations of Lustre

[Arden Wiebe]

Nick:

In case I had a capitalization in the links I sent you mixed up.  http://www.ioio.ca/Lustre-tcp-bonding/Lustre-notes/images.html and http://www.ioio.ca/Lustre-tcp-bonding/images.html should work.  Go easy on my old girl she only has one processor and is a complete hack to recover data after root stroke and jail riot last year on main drive that I couldn't salvage.  Pity it had the only copy of the code I needed yesterday.

Aside from the webserver it originates from it should put a pretty clear visual into how far you can take it for roughly how much TCO.  Again with valid points for tuning to small file size as best as possible.  If you would like to see a specific small file benchmark from some view I would do my best to produce if you tell me what to write.

Ardenly;

Arden Wiebe  

 
--- On Mon, 1/26/09, Nick Jennings <nick at creativemotiondesign.com> wrote:

From: Nick Jennings <nick at creativemotiondesign.com>
Subject: [Lustre-discuss] Performance Expectations of Lustre
To: lustre-discuss at lists.lustre.org
Date: Monday, January 26, 2009, 7:51 AM

Hello (and a special hello to all my ex-co-workers from the CFS days :)

  The company where I work now has grown fast in the past year and we 
suddenly find ourselves in need of a lot of storage. For 5 years the 
company ran on a 60gig server, last year we got a 1TB RAID that is now 
almost full. In 1-2 years we could easily be using 10-15TB of storage.

  Instead of just adding another 1TB server, I need to plan for a more 
scalable solution. Immediately Lustre came to mind, but I'm wondering 
about the performance. Basically our company does niche web-hosting for 
"Creative Professionals" so we need fast access to the data in order to 
have snappy web services for our clients. Typically these are smaller 
files (2MB pictures, 50MB videos, .swf files, etc.).

  Also I'm wondering about the best way set this up in terms of speed 
and ease of growth. I want the web-servers and the storage pool to be 
independent of each other. So I can add web-servers as the web traffic 
increases, and add more storage ass our storage needs grow. We have the 
option of an MD3000 or MD3000i for back-end storage.

  I was thinking initially we could start with 2 servers, both attached 
to the storage array. setup as OSS' and functioning as (load balanced) 
web-servers as well. In the future I could separate this out so that we 
have the web-servers on the "front line" mounting the data from the OSS' 
which will be on a private (gigE) network.

  Now, it's been years since I've played with Lustre, I'm sure some 
stuff will come back to me as I start using it again, other things I'll 
probably have to re-learn. I wanted to get some input from the Lustre 
community on whether or not this seems like a reasonable use for Lustre? 
Are there alternatives out there which might fit my needs more? 
(specifically speed and a shared storage pool). Also, what kind of 
performance can I expect, am I out of touch to expect something similar 
to a directly attached RAID array?

  I appreciate any and all feedback, suggestions, comments etc.

Thanks,
- Nick

--
Nick Jennings
Senior Programmer & Systems Administrator
Creative Motion Design
nick at creativemotiondesign.com
_______________________________________________
Lustre-discuss mailing list
Lustre-discuss at lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss