[Lustre-discuss] Lustre on WAN

[Peter Grandi]

[ ... ]

>>> Lustre is originally designed to target at HPC clusters,
>>> i.e., systems on a single LAN environment.

It is not so much single LAN, but streaming and low latency.

>>> On the other hand, the cloud we are building is physically
>>> distributed at different cities in the province of Alberta.
>>> [ ... ] performance is impressively good, partly due to the
>>> fast network we are running in the province.

The question here is whether the *clients* and/or the *servers*
are physically distributed.

If the servers are physically distributed then what are the
resilience requirements, and this largely relates to what is the
redundancy strategy for the underlying storage, and whether
files are striped across OSSes at different sites.

In the example below the servers are centralized but maybe this
is not what you mean by a "cloud".

>> This is very similar to the environment that is being used at
>> Indiana University.  They have the Lustre servers at a
>> central site, but several labs/campuses in other cities are
>> mounting the filesystem and they can saturate 10GigE links
>> between the sites.

That is quite plausible, but relevant performance depends on
whether access patterns are streaming or not, and doing some
decent TCP setup to maximize link utilization.

> [ ... ] one application we've tried over the WAN didn't fare
> very well out of the box - lots of small random reads (read
> ~4k, seek a bunch, read ~4k, etc ad nauseum).

That looks like some people have unrealistic expectations as to
latency and synchronous IO more than something to which Lustre
is relevant. Even if Lustre is mostly targeted at streaming
workloads on low latency networks, and even if it is not too bad
in different circumstances.