[Lustre-discuss] Performance Expectations of Lustre

[Arden Wiebe]

Nick:

On another note I just had to do a mysqlcheck -p --auto-repair on a 23266 table database tonight so probably not a good idea doing direct copies of /var/lib/mysql to the lustre filesystem.  Correlated or not would be better to mysqldump there instead.

Ardently;

Arden Wiebe

--- On Mon, 1/26/09, Nick Jennings <nick at creativemotiondesign.com> wrote:

From: Nick Jennings <nick at creativemotiondesign.com>
Subject: [Lustre-discuss] Performance Expectations of Lustre
To: lustre-discuss at lists.lustre.org
Date: Monday, January 26, 2009, 7:51 AM

Hello (and a special hello to all my ex-co-workers from the CFS days :)

  The company where I work now has grown fast in the past year and we 
suddenly find ourselves in need of a lot of storage. For 5 years the 
company ran on a 60gig server, last year we got a 1TB RAID that is now 
almost full. In 1-2 years we could easily be using 10-15TB of storage.

  Instead of just adding another 1TB server, I need to plan for a more 
scalable solution. Immediately Lustre came to mind, but I'm wondering 
about the performance. Basically our company does niche web-hosting for 
"Creative Professionals" so we need fast access to the data in order to 
have snappy web services for our clients. Typically these are smaller 
files (2MB pictures, 50MB videos, .swf files, etc.).

  Also I'm wondering about the best way set this up in terms of speed 
and ease of growth. I want the web-servers and the storage pool to be 
independent of each other. So I can add web-servers as the web traffic 
increases, and add more storage ass our storage needs grow. We have the 
option of an MD3000 or MD3000i for back-end storage.

  I was thinking initially we could start with 2 servers, both attached 
to the storage array. setup as OSS' and functioning as (load balanced) 
web-servers as well. In the future I could separate this out so that we 
have the web-servers on the "front line" mounting the data from the OSS' 
which will be on a private (gigE) network.

  Now, it's been years since I've played with Lustre, I'm sure some 
stuff will come back to me as I start using it again, other things I'll 
probably have to re-learn. I wanted to get some input from the Lustre 
community on whether or not this seems like a reasonable use for Lustre? 
Are there alternatives out there which might fit my needs more? 
(specifically speed and a shared storage pool). Also, what kind of 
performance can I expect, am I out of touch to expect something similar 
to a directly attached RAID array?

  I appreciate any and all feedback, suggestions, comments etc.

Thanks,
- Nick

--
Nick Jennings
Senior Programmer & Systems Administrator
Creative Motion Design
nick at creativemotiondesign.com
_______________________________________________
Lustre-discuss mailing list
Lustre-discuss at lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss