[Lustre-discuss] Anybody actually using Flash (Fusion IO specifically) for meta data?

[Carlson, Timothy S]

> On May 19, 2011, at 10:28, Kevin Van Maren wrote:
> > Dardo D Kleiner - CONTRACTOR wrote:
> > As for putting the entire filesystem on flash, sure that would be
> pretty
> > nifty, but expensive.  Not being able to do failover, with storage on
> > internal PCIe cards, is a downside.
>
>  [Andreas added this comment]
> I doubt this will be possible for a long time to come, due to cost,
> even if
> the PCI cards have external interfaces (as I've heard some high-end
> ones do).

I hate to snip out most of a thread, but I want to focus on the issues of cost and failover.

As for cost, I really don't think this is an issue. If I am investing in a file system that is either approaching a Petabyte or is larger than a Petabyte then I don't see that purchasing a 5K-10K flash device is really a cost factor. It is not quite in the noise, but it is going to be less than 5% of the total purchase price of a the file system. 

Failover is an issue. I've been keeping some loose statistics on my current Lustre configurations (a Petabyte or so in total) and looking at what components fail and where redundancy/failover could be improved. So far, metadata server failure hasn't entered the picture.  The problem with Lustre is it is now just too damn robust to random reboots :). 

Thanks

Tim