[Lustre-discuss] Poor Direct-IO Performance with Lustre-2.1.5

[Drokin, Oleg]

Hello!

On Jun 22, 2013, at 7:50 PM, Chan Ching Yu, Patrick wrote:
>         Record Size 1024 KB

This is your problem.
DirectIO is synchronous in nature so only as much data as you fit into a single syscall will be in flight at any given moment.
In this case you only have 1 RPC in flight.
The key to adequately performing DirectIO is to have read/write syscalls with a lot of data. 8*num_stripes is the very minimum, I would think.
Even then you likely won't get cached read/write performance unless you run a very underpowered client that gets choked on memory copies.

Bye,
    Oleg