[lustre-devel] lustre and loopback device

[NeilBrown]

On Fri, Mar 23 2018, James Simmons wrote:

> Hi Neil
>
>       So once long ago lustre had its own loopback device due to the 
> upstream loopback device not supporting Direct I/O. Once it did we
> dropped support for our custom driver. Recently their has been interest
> in using the loopback driver and Jinshan discussed with me about reviving
> our custom driver which I'm not thrilled about. He was seeing problems
> with Direct I/O above 64K. Do you know the details why that limitation
> exist. Perhaps it can be resolved or maybe we are missing something?
> Thanks for your help.

Hi James, and Jinshan,
 What sort of problems do you see with 64K DIO requests?
 Is it a throughput problem or are you seeing IO errors?
 Would it be easy to demonstrate the problem in a cluster
 comprising a few VMs, or is real hardware needed?  If VMs are OK,
 can you tell me exactly how to duplicate the problem?

 If loop gets a multi-bio request, it will allocate a bvec array
 to hold all the bio_vecs.  If there are more than 256 pages (1Meg)
 in a request, this could easily fail. 5 consecutive 64K requests on a
 machine without much free memory could hit problems here.
 If that is the problem, it should be easy to fix (request the number
 given to blk_queue_max_hw_sectors).

Thanks,
NeilBrown
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20180326/63e2b2a2/attachment.sig>