[Lustre-devel] LustreFS performance

[Andreas Dilger]

On Mar 08, 2009  22:50 -0400, Oleg Drokin wrote:
> On Mar 5, 2009, at 4:27 PM, Andreas Dilger wrote:
>> Even better, if you have some development skills, would be to  
>> implement (or possibly resurrect) an fsfilt-tmpfs layer.  Since
>> tmpfs isn't going to be recoverable anyways (I assume you just
>> reformat from scratch when there is a crash), then you can make
>> all of the transaction handling as no-ops, and just implement
>> the minimal interfaces needed to work.
>>
>> That would allow unlinked files to release space from tmpfs, and also
>> avoid the fixed allocation overhead and journaling of ldiskfs, probably
>> saving you 5% of RAM (more on the MDS) and a LOT of memcpy() overhead.
>
> This is exactly what I was trying to avoid.
> I tried to measure things as if I had an infinitely fast disk only, and I
> still needed all the journal/blockdevice and other such things to take
> the CPU they would normally take.
> After all we cannot expect people to actually run real MDSes on tmpfs
> unless they have some means to replicate that MDS somewhere else.

In fact, it wasn't my proposal for the metadata performance testing
itself, but rather SiCortex are apparently running with RAM-backed
filesystems for some kind of fast cache filesystem (e.g. distributed
shared memory or similar).  I was only proposing their implementation
could be more efficient.

I could imagine that for flash-cache type applications that storing
checkpoints for a short time in a RAM-backed OST pool before migrating
it to persistent storage.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.