[lustre-discuss] Designing a new Lustre system

Carlson, Timothy S Timothy.Carlson at pnnl.gov
Thu Dec 21 16:53:46 PST 2017

Isilon is truly an enterprise solution. We have one (about a dozen bricks worth) and use it for home directories on our super computers and it allows easy access via CIFS to users on Windows/Mac.

It is highly configurable with “smart pools” and policies to move data around based on age/size/access time/etc.    How you “stripe” the data depends on the policy you set.

It is a BSD with Isilon magic sauce. It uses its own Infiniband networking on the backend to move data around and pipe it out various front ends that you have configured. But as has been pointed out, it doesn’t really do parallel I/O.

For our users with millions (or hundreds of millions of files), it works as a solution.


From: lustre-discuss [mailto:lustre-discuss-bounces at lists.lustre.org] On Behalf Of John Bent
Sent: Thursday, December 21, 2017 4:44 PM
To: Glenn Lockwood <glock at lbl.gov>
Cc: lustre-discuss at lists.lustre.org
Subject: Re: [lustre-discuss] Designing a new Lustre system

Last I looked at Isilon, it serializes parallel writes to a single file.  Ultimately, the data is striped across multiple data servers but it all channels through a single data server.  If you only have file-per-process workloads, and you have a lot of money, then Isilon is considered a solid enterprise solution.

On Thu, Dec 21, 2017 at 7:15 PM, Glenn Lockwood <glock at lbl.gov<mailto:glock at lbl.gov>> wrote:

On Wed, Dec 20, 2017 at 8:21 AM, E.S. Rosenberg <esr+lustre at mail.hebrew.edu<mailto:esr+lustre at mail.hebrew.edu>> wrote:

4. One of my colleagues likes Isilon very much, I have not been able to find any literature on if/how Lustre compares any pointers/knowledge on the subject is very welcome.

I haven't looked at Isilon in a while, but my recollection was that

1. It's phenomenally expensive, especially at smaller scales.  This is the most obvious detractor vs. Lustre, especially at low node counts.

2. It's completely proprietary and architecturally complex, so management and support model is difficult to shape into existing operations.  There are also cost implications here.

3. It uses NFS for transport, so it doesn't offer POSIX consistency between clients.  This makes shared-file parallel I/O extremely hazardous.

In my experience, Isilon is popular in markets flush with money but scarce in institutional storage expertise.  In such cases, #1 and #2 are non-issues, and #3 often doesn't apply because such industries' workloads are throughput-oriented and rarely use MPI.


lustre-discuss mailing list
lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20171222/87e7c036/attachment.html>

More information about the lustre-discuss mailing list