[lustre-devel] lustre and loopback device

[James Simmons]

> On Fri, Mar 23 2018, James Simmons wrote:
> 
> > Hi Neil
> >
> >       So once long ago lustre had its own loopback device due to the 
> > upstream loopback device not supporting Direct I/O. Once it did we
> > dropped support for our custom driver. Recently their has been interest
> > in using the loopback driver and Jinshan discussed with me about reviving
> > our custom driver which I'm not thrilled about. He was seeing problems
> > with Direct I/O above 64K. Do you know the details why that limitation
> > exist. Perhaps it can be resolved or maybe we are missing something?
> > Thanks for your help.
> 
> Hi James, and Jinshan,
>  What sort of problems do you see with 64K DIO requests?
>  Is it a throughput problem or are you seeing IO errors?
>  Would it be easy to demonstrate the problem in a cluster
>  comprising a few VMs, or is real hardware needed?  If VMs are OK,
>  can you tell me exactly how to duplicate the problem?
> 
>  If loop gets a multi-bio request, it will allocate a bvec array
>  to hold all the bio_vecs.  If there are more than 256 pages (1Meg)
>  in a request, this could easily fail. 5 consecutive 64K requests on a
>  machine without much free memory could hit problems here.
>  If that is the problem, it should be easy to fix (request the number
>  given to blk_queue_max_hw_sectors).

Jinshan can you post a reproducer so we can see the problem.