[lustre-discuss] LU-8964/pio feature usage

Patrick Farrell paf at cray.com
Sun Dec 23 12:59:55 PST 2018


Good afternoon,

There was a recent discussion on the lustre-devel mailing list in which I floated removing the 'pio' feature from Lustre.  This is a client side i/o parallelization feature (splitting i/o in kernel space & using multiple worker threads) which is off by default and must be enabled manually with a setting in proc.

It is not on by default because while it helps performance for some workloads, it harms performance for many other common workloads.  I studied making it fit to be on by default some time ago (item #1 would be making sure it did not slow down any common workloads), and found a fair bit of work to do to make that possible.

In the time since then, the work to extend the utility of this feature to reads was unsuccessful, and a credible alternate proposal for writes ("write containers", basically simplifying the write path dramatically for sequential i/o, which should be able to offer similar or larger benefits at a lower cost in CPU usage) has come about.

I could go on - But I am really wondering this:
Are there any active users of this feature?  If so, what's your use case and what's the benefit you see?  Keep in mind it must be turned on explicitly, so if you are not familiar with it, it's pretty unlikely you're using it.

For those who'd like to learn more, I'd refer you to the original bug:
https://jira.whamcloud.com/browse/LU-8964

My LAD talk from a few years ago:
https://www.eofs.eu/_media/events/devsummit17/patrick_farrell_laddevsummit_pio.pdf

And the recent mailing list discussion:
http://lists.lustre.org/pipermail/lustre-devel-lustre.org/2018-November/thread.html
(Look for [PATCH 10/12] lustre: clio: Introduce parallel tasks framework <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/2018-November/008210.html> , and for my comments specifically)

  *   Patrick
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20181223/ab913db9/attachment.html>


More information about the lustre-discuss mailing list