[Lustre-discuss] Performance Expectations of Lustre
albert682 at yahoo.com
Wed Jan 28 00:42:14 PST 2009
On another note I just had to do a mysqlcheck -p --auto-repair on a 23266 table database tonight so probably not a good idea doing direct copies of /var/lib/mysql to the lustre filesystem. Correlated or not would be better to mysqldump there instead.
--- On Mon, 1/26/09, Nick Jennings <nick at creativemotiondesign.com> wrote:
From: Nick Jennings <nick at creativemotiondesign.com>
Subject: [Lustre-discuss] Performance Expectations of Lustre
To: lustre-discuss at lists.lustre.org
Date: Monday, January 26, 2009, 7:51 AM
Hello (and a special hello to all my ex-co-workers from the CFS days :)
The company where I work now has grown fast in the past year and we
suddenly find ourselves in need of a lot of storage. For 5 years the
company ran on a 60gig server, last year we got a 1TB RAID that is now
almost full. In 1-2 years we could easily be using 10-15TB of storage.
Instead of just adding another 1TB server, I need to plan for a more
scalable solution. Immediately Lustre came to mind, but I'm wondering
about the performance. Basically our company does niche web-hosting for
"Creative Professionals" so we need fast access to the data in order to
have snappy web services for our clients. Typically these are smaller
files (2MB pictures, 50MB videos, .swf files, etc.).
Also I'm wondering about the best way set this up in terms of speed
and ease of growth. I want the web-servers and the storage pool to be
independent of each other. So I can add web-servers as the web traffic
increases, and add more storage ass our storage needs grow. We have the
option of an MD3000 or MD3000i for back-end storage.
I was thinking initially we could start with 2 servers, both attached
to the storage array. setup as OSS' and functioning as (load balanced)
web-servers as well. In the future I could separate this out so that we
have the web-servers on the "front line" mounting the data from the OSS'
which will be on a private (gigE) network.
Now, it's been years since I've played with Lustre, I'm sure some
stuff will come back to me as I start using it again, other things I'll
probably have to re-learn. I wanted to get some input from the Lustre
community on whether or not this seems like a reasonable use for Lustre?
Are there alternatives out there which might fit my needs more?
(specifically speed and a shared storage pool). Also, what kind of
performance can I expect, am I out of touch to expect something similar
to a directly attached RAID array?
I appreciate any and all feedback, suggestions, comments etc.
Senior Programmer & Systems Administrator
Creative Motion Design
nick at creativemotiondesign.com
Lustre-discuss mailing list
Lustre-discuss at lists.lustre.org
More information about the lustre-discuss