[Lustre-discuss] performance tuning

[Brian J. Murrell]

On Thu, 2009-07-02 at 13:34 -0600, Martin Pokorny wrote:
> 
> While my brief testing with IOR showed acceptable write throughput to 
> the Lustre filesystem,

Was the IOR test file-per-client or single-file, segment-per-client?  If
the latter, how big was the segment and was the file being written to
striped?  If yes, what were it's striping parameters?

> I have been unable to achieve anywhere near that 
> figure with our application doing the writes --- I'm concerned that the 
> write pattern being used is a severely limiting factor.

If you need more bandwidth to the file than a single client/network
connection/OSS/disk can provide, then you need to do some
parallellization, as you have surmised.  But the parallellization has to
be effective.

The most effective parallization you can get is by mapping a single
client to a single OST on an unshared OSS across an unshared network
connection.  Frequently one or more of those components will be
underutilized though so some economy can be introduced in sharing an OSS
or even OST, or network connection, etc. among several clients to the
limit of the performance of the shared resource.

> In this 
> situation, does anyone have any advice about what I ought to be looking 
> at to improve performance on Lustre?

Well, you need to figure out why your application is not able to take
advantage of the parallellization as IOR has demonstrated can be
achieved.

Perhaps you need to (more effectively) stripe your file across some OSTs
so that clients are not competing for the same OST in their write
operations.

b.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20090702/7d2d49b4/attachment.pgp>