From ckkccf at gmail.com  Sat Nov  3 17:41:44 2012
From: ckkccf at gmail.com (Kalyana C)
Date: Sat, 3 Nov 2012 12:41:44 -0500
Subject: [Lustre-devel] Control file striping
Message-ID: <CANuO2dvubpiw79KC_svR7HoGZ2XRrWJ=g4gGgeHjniuyh0YHhg@mail.gmail.com>

Hello,

I am trying to figure out a way to:

set striping across a set of OSTs of my choice.

lfs setstripe command and API call does not allow for this currently. The
user can only set the starting OST through the index parameter. I
appreciate why there is no such feature currently; you do not want to let
the users have control on how OSTs are used since it can result in OST
usage imbalance.

However, I do have a valid usage case for such a feature and I am trying to
figure out the best way to do it.

One way to achieve the same result is through the use of OST pools.
However, the userland tools do not work from a client node, it has to be
run on the MGS. There are also other restrictions as described here:
http://wiki.lustre.org/index.php/Creating_and_Managing_OST_Pools

I am also digging through the Lustre source and header files to see if
there are other (undocumented) ways to do this.

So, my questions are as follows:

1) Is there currently a way for a user to control the striping of a file /
directory on a selected set of OSTs? If yes, is it possible to do it
through an API call or a command line utility?

2) If answer to (1) is No, Is there a way to do this with admin rights from
the MDS or client nodes?

3) If pools are the only way to achieve this, is there an API call to
create, manage and destroy pools from a user application? Is there any
impact on performance of MDS components if pools are dynamically created
and deleted on demand? Can the API be called from a client node (root or
user) or does it have to be on the MGS?

Thank you.

Regards,
Kalyana Chadalavada
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20121103/1884bff0/attachment.html>

From atchleyes at ornl.gov  Mon Nov  5 17:47:05 2012
From: atchleyes at ornl.gov (Atchley, Scott)
Date: Mon, 5 Nov 2012 12:47:05 -0500
Subject: [Lustre-devel] Control file striping
In-Reply-To: <CANuO2dvubpiw79KC_svR7HoGZ2XRrWJ=g4gGgeHjniuyh0YHhg@mail.gmail.com>
References: <CANuO2dvubpiw79KC_svR7HoGZ2XRrWJ=g4gGgeHjniuyh0YHhg@mail.gmail.com>
Message-ID: <DE176288-9AC9-454F-89D2-8F1098400418@ornl.gov>

On Nov 3, 2012, at 1:41 PM, Kalyana C <ckkccf at gmail.com> wrote:

> Hello,
> 
> I am trying to figure out a way to:
> 
> set striping across a set of OSTs of my choice. 
> 
> lfs setstripe command and API call does not allow for this currently. The user can only set the starting OST through the index parameter. I appreciate why there is no such feature currently; you do not want to let the users have control on how OSTs are used since it can result in OST usage imbalance. 
> 
> However, I do have a valid usage case for such a feature and I am trying to figure out the best way to do it. 

I have a valid use for it as well. It would be nice to have this feature back.

Scott

> 
> One way to achieve the same result is through the use of OST pools. However, the userland tools do not work from a client node, it has to be run on the MGS. There are also other restrictions as described here: http://wiki.lustre.org/index.php/Creating_and_Managing_OST_Pools
> 
> I am also digging through the Lustre source and header files to see if there are other (undocumented) ways to do this. 
> 
> So, my questions are as follows:
> 
> 1) Is there currently a way for a user to control the striping of a file / directory on a selected set of OSTs? If yes, is it possible to do it through an API call or a command line utility?
> 
> 2) If answer to (1) is No, Is there a way to do this with admin rights from the MDS or client nodes? 
> 
> 3) If pools are the only way to achieve this, is there an API call to create, manage and destroy pools from a user application? Is there any impact on performance of MDS components if pools are dynamically created and deleted on demand? Can the API be called from a client node (root or user) or does it have to be on the MGS?
> 
> Thank you.
> 
> Regards,
> Kalyana Chadalavada 
> 
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-devel

From ckkccf at gmail.com  Mon Nov  5 18:01:10 2012
From: ckkccf at gmail.com (Kalyana C)
Date: Mon, 5 Nov 2012 12:01:10 -0600
Subject: [Lustre-devel] Control file striping
In-Reply-To: <2C7DE72B9BD00F44BAECA5B0CBB87395022AD490@hermes.terascala.com>
References: <CANuO2dvubpiw79KC_svR7HoGZ2XRrWJ=g4gGgeHjniuyh0YHhg@mail.gmail.com>
	<DE176288-9AC9-454F-89D2-8F1098400418@ornl.gov>
	<2C7DE72B9BD00F44BAECA5B0CBB87395022AD490@hermes.terascala.com>
Message-ID: <CANuO2dvGUgK=PB50P3ZG-rEMFv2WkHYQwaT6uD=0rf2MVgbAcA@mail.gmail.com>

> Have you considered using a combination of OST Pools and striping
fit your needs?

This is the only way possible that I currently know of.

But is it possible to manage OST pools via an API call? Can it be done from
a client under a normal user application? Or does it have the same
restrictions as described at
http://wiki.lustre.org/index.php/Creating_and_Managing_OST_Pools

Regards,
Kalyan

On Mon, Nov 5, 2012 at 11:57 AM, Ben Evans <Ben.Evans at terascala.com> wrote:

> Have you considered using a combination of OST Pools and striping fit
> your needs?
>
> -----Original Message-----
> From: lustre-devel-bounces at lists.lustre.org
> [mailto:lustre-devel-bounces at lists.lustre.org] On Behalf Of Atchley,
> Scott
> Sent: Monday, November 05, 2012 12:47 PM
> To: Kalyana C
> Cc: lustre-devel at lists.lustre.org
> Subject: Re: [Lustre-devel] Control file striping
>
> On Nov 3, 2012, at 1:41 PM, Kalyana C <ckkccf at gmail.com> wrote:
>
> > Hello,
> >
> > I am trying to figure out a way to:
> >
> > set striping across a set of OSTs of my choice.
> >
> > lfs setstripe command and API call does not allow for this currently.
> The user can only set the starting OST through the index parameter. I
> appreciate why there is no such feature currently; you do not want to
> let the users have control on how OSTs are used since it can result in
> OST usage imbalance.
> >
> > However, I do have a valid usage case for such a feature and I am
> trying to figure out the best way to do it.
>
> I have a valid use for it as well. It would be nice to have this feature
> back.
>
> Scott
>
> >
> > One way to achieve the same result is through the use of OST pools.
> However, the userland tools do not work from a client node, it has to be
> run on the MGS. There are also other restrictions as described here:
> http://wiki.lustre.org/index.php/Creating_and_Managing_OST_Pools
> >
> > I am also digging through the Lustre source and header files to see if
> there are other (undocumented) ways to do this.
> >
> > So, my questions are as follows:
> >
> > 1) Is there currently a way for a user to control the striping of a
> file / directory on a selected set of OSTs? If yes, is it possible to do
> it through an API call or a command line utility?
> >
> > 2) If answer to (1) is No, Is there a way to do this with admin rights
> from the MDS or client nodes?
> >
> > 3) If pools are the only way to achieve this, is there an API call to
> create, manage and destroy pools from a user application? Is there any
> impact on performance of MDS components if pools are dynamically created
> and deleted on demand? Can the API be called from a client node (root or
> user) or does it have to be on the MGS?
> >
> > Thank you.
> >
> > Regards,
> > Kalyana Chadalavada
> >
> > _______________________________________________
> > Lustre-devel mailing list
> > Lustre-devel at lists.lustre.org
> > http://lists.lustre.org/mailman/listinfo/lustre-devel
>
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20121105/3f279f1b/attachment.html>

From Ben.Evans at terascala.com  Mon Nov  5 17:57:20 2012
From: Ben.Evans at terascala.com (Ben Evans)
Date: Mon, 5 Nov 2012 12:57:20 -0500
Subject: [Lustre-devel] Control file striping
In-Reply-To: <DE176288-9AC9-454F-89D2-8F1098400418@ornl.gov>
References: <CANuO2dvubpiw79KC_svR7HoGZ2XRrWJ=g4gGgeHjniuyh0YHhg@mail.gmail.com>
	<DE176288-9AC9-454F-89D2-8F1098400418@ornl.gov>
Message-ID: <2C7DE72B9BD00F44BAECA5B0CBB87395022AD490@hermes.terascala.com>

Have you considered using a combination of OST Pools and striping fit
your needs?

-----Original Message-----
From: lustre-devel-bounces at lists.lustre.org
[mailto:lustre-devel-bounces at lists.lustre.org] On Behalf Of Atchley,
Scott
Sent: Monday, November 05, 2012 12:47 PM
To: Kalyana C
Cc: lustre-devel at lists.lustre.org
Subject: Re: [Lustre-devel] Control file striping

On Nov 3, 2012, at 1:41 PM, Kalyana C <ckkccf at gmail.com> wrote:

> Hello,
> 
> I am trying to figure out a way to:
> 
> set striping across a set of OSTs of my choice. 
> 
> lfs setstripe command and API call does not allow for this currently.
The user can only set the starting OST through the index parameter. I
appreciate why there is no such feature currently; you do not want to
let the users have control on how OSTs are used since it can result in
OST usage imbalance. 
> 
> However, I do have a valid usage case for such a feature and I am
trying to figure out the best way to do it. 

I have a valid use for it as well. It would be nice to have this feature
back.

Scott

> 
> One way to achieve the same result is through the use of OST pools.
However, the userland tools do not work from a client node, it has to be
run on the MGS. There are also other restrictions as described here:
http://wiki.lustre.org/index.php/Creating_and_Managing_OST_Pools
> 
> I am also digging through the Lustre source and header files to see if
there are other (undocumented) ways to do this. 
> 
> So, my questions are as follows:
> 
> 1) Is there currently a way for a user to control the striping of a
file / directory on a selected set of OSTs? If yes, is it possible to do
it through an API call or a command line utility?
> 
> 2) If answer to (1) is No, Is there a way to do this with admin rights
from the MDS or client nodes? 
> 
> 3) If pools are the only way to achieve this, is there an API call to
create, manage and destroy pools from a user application? Is there any
impact on performance of MDS components if pools are dynamically created
and deleted on demand? Can the API be called from a client node (root or
user) or does it have to be on the MGS?
> 
> Thank you.
> 
> Regards,
> Kalyana Chadalavada 
> 
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-devel

_______________________________________________
Lustre-devel mailing list
Lustre-devel at lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-devel

From cliff.white at intel.com  Mon Nov  5 18:16:09 2012
From: cliff.white at intel.com (White, Cliff)
Date: Mon, 5 Nov 2012 18:16:09 +0000
Subject: [Lustre-devel] Control file striping
In-Reply-To: <CANuO2dvGUgK=PB50P3ZG-rEMFv2WkHYQwaT6uD=0rf2MVgbAcA@mail.gmail.com>
References: <CANuO2dvubpiw79KC_svR7HoGZ2XRrWJ=g4gGgeHjniuyh0YHhg@mail.gmail.com>
	<DE176288-9AC9-454F-89D2-8F1098400418@ornl.gov>
	<2C7DE72B9BD00F44BAECA5B0CBB87395022AD490@hermes.terascala.com>
	<CANuO2dvGUgK=PB50P3ZG-rEMFv2WkHYQwaT6uD=0rf2MVgbAcA@mail.gmail.com>
Message-ID: <F46703DD-5873-4FB4-AFCE-059E868B4D6B@intel.com>

Creating/destroying pools must be done as per the manual, from MDS.
Do you need to frequently create/destroy pools, or would you have a fixed setup?

The normal method is to map various pools (and other striping policies) to directories, then user space applications
can access specific pools by pointing at the requisite directory, thus requiring no special API.
Once pools are created, the directory creation/striping is all done on a client via lfs setstripe, the liblustreapi can
also be used by clients to set striping via C calls.

Best
cliffw


On Nov 5, 2012, at 10:01 AM, Kalyana C <ckkccf at gmail.com<mailto:ckkccf at gmail.com>>
 wrote:

> Have you considered using a combination of OST Pools and striping fit your needs?

This is the only way possible that I currently know of.

But is it possible to manage OST pools via an API call? Can it be done from a client under a normal user application? Or does it have the same restrictions as described at http://wiki.lustre.org/index.php/Creating_and_Managing_OST_Pools

Regards,
Kalyan

On Mon, Nov 5, 2012 at 11:57 AM, Ben Evans <Ben.Evans at terascala.com<mailto:Ben.Evans at terascala.com>> wrote:
Have you considered using a combination of OST Pools and striping fit
your needs?

-----Original Message-----
From: lustre-devel-bounces at lists.lustre.org<mailto:lustre-devel-bounces at lists.lustre.org>
[mailto:lustre-devel-bounces at lists.lustre.org<mailto:lustre-devel-bounces at lists.lustre.org>] On Behalf Of Atchley,
Scott
Sent: Monday, November 05, 2012 12:47 PM
To: Kalyana C
Cc: lustre-devel at lists.lustre.org<mailto:lustre-devel at lists.lustre.org>
Subject: Re: [Lustre-devel] Control file striping

On Nov 3, 2012, at 1:41 PM, Kalyana C <ckkccf at gmail.com<mailto:ckkccf at gmail.com>> wrote:

> Hello,
>
> I am trying to figure out a way to:
>
> set striping across a set of OSTs of my choice.
>
> lfs setstripe command and API call does not allow for this currently.
The user can only set the starting OST through the index parameter. I
appreciate why there is no such feature currently; you do not want to
let the users have control on how OSTs are used since it can result in
OST usage imbalance.
>
> However, I do have a valid usage case for such a feature and I am
trying to figure out the best way to do it.

I have a valid use for it as well. It would be nice to have this feature
back.

Scott

>
> One way to achieve the same result is through the use of OST pools.
However, the userland tools do not work from a client node, it has to be
run on the MGS. There are also other restrictions as described here:
http://wiki.lustre.org/index.php/Creating_and_Managing_OST_Pools
>
> I am also digging through the Lustre source and header files to see if
there are other (undocumented) ways to do this.
>
> So, my questions are as follows:
>
> 1) Is there currently a way for a user to control the striping of a
file / directory on a selected set of OSTs? If yes, is it possible to do
it through an API call or a command line utility?
>
> 2) If answer to (1) is No, Is there a way to do this with admin rights
from the MDS or client nodes?
>
> 3) If pools are the only way to achieve this, is there an API call to
create, manage and destroy pools from a user application? Is there any
impact on performance of MDS components if pools are dynamically created
and deleted on demand? Can the API be called from a client node (root or
user) or does it have to be on the MGS?
>
> Thank you.
>
> Regards,
> Kalyana Chadalavada
>
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org<mailto:Lustre-devel at lists.lustre.org>
> http://lists.lustre.org/mailman/listinfo/lustre-devel

_______________________________________________
Lustre-devel mailing list
Lustre-devel at lists.lustre.org<mailto:Lustre-devel at lists.lustre.org>
http://lists.lustre.org/mailman/listinfo/lustre-devel

_______________________________________________
Lustre-devel mailing list
Lustre-devel at lists.lustre.org<mailto:Lustre-devel at lists.lustre.org>
http://lists.lustre.org/mailman/listinfo/lustre-devel

From atchleyes at ornl.gov  Mon Nov  5 18:28:06 2012
From: atchleyes at ornl.gov (Atchley, Scott)
Date: Mon, 5 Nov 2012 13:28:06 -0500
Subject: [Lustre-devel] Control file striping
In-Reply-To: <F46703DD-5873-4FB4-AFCE-059E868B4D6B@intel.com>
References: <CANuO2dvubpiw79KC_svR7HoGZ2XRrWJ=g4gGgeHjniuyh0YHhg@mail.gmail.com>
	<DE176288-9AC9-454F-89D2-8F1098400418@ornl.gov>
	<2C7DE72B9BD00F44BAECA5B0CBB87395022AD490@hermes.terascala.com>
	<CANuO2dvGUgK=PB50P3ZG-rEMFv2WkHYQwaT6uD=0rf2MVgbAcA@mail.gmail.com>
	<F46703DD-5873-4FB4-AFCE-059E868B4D6B@intel.com>
Message-ID: <FCE48434-4A4E-481C-BB3C-FB31F1527B6B@ornl.gov>

On Nov 5, 2012, at 1:16 PM, "White, Cliff" <cliff.white at intel.com> wrote:

> Creating/destroying pools must be done as per the manual, from MDS.
> Do you need to frequently create/destroy pools, or would you have a fixed setup?

In our case, we need to frequently change the OSTs. Possibly per file.

Scott

> 
> The normal method is to map various pools (and other striping policies) to directories, then user space applications
> can access specific pools by pointing at the requisite directory, thus requiring no special API.
> Once pools are created, the directory creation/striping is all done on a client via lfs setstripe, the liblustreapi can
> also be used by clients to set striping via C calls.
> 
> Best
> cliffw
> 
> 
> On Nov 5, 2012, at 10:01 AM, Kalyana C <ckkccf at gmail.com<mailto:ckkccf at gmail.com>>
> wrote:
> 
>> Have you considered using a combination of OST Pools and striping fit your needs?
> 
> This is the only way possible that I currently know of.
> 
> But is it possible to manage OST pools via an API call? Can it be done from a client under a normal user application? Or does it have the same restrictions as described at http://wiki.lustre.org/index.php/Creating_and_Managing_OST_Pools
> 
> Regards,
> Kalyan
> 
> On Mon, Nov 5, 2012 at 11:57 AM, Ben Evans <Ben.Evans at terascala.com<mailto:Ben.Evans at terascala.com>> wrote:
> Have you considered using a combination of OST Pools and striping fit
> your needs?
> 
> -----Original Message-----
> From: lustre-devel-bounces at lists.lustre.org<mailto:lustre-devel-bounces at lists.lustre.org>
> [mailto:lustre-devel-bounces at lists.lustre.org<mailto:lustre-devel-bounces at lists.lustre.org>] On Behalf Of Atchley,
> Scott
> Sent: Monday, November 05, 2012 12:47 PM
> To: Kalyana C
> Cc: lustre-devel at lists.lustre.org<mailto:lustre-devel at lists.lustre.org>
> Subject: Re: [Lustre-devel] Control file striping
> 
> On Nov 3, 2012, at 1:41 PM, Kalyana C <ckkccf at gmail.com<mailto:ckkccf at gmail.com>> wrote:
> 
>> Hello,
>> 
>> I am trying to figure out a way to:
>> 
>> set striping across a set of OSTs of my choice.
>> 
>> lfs setstripe command and API call does not allow for this currently.
> The user can only set the starting OST through the index parameter. I
> appreciate why there is no such feature currently; you do not want to
> let the users have control on how OSTs are used since it can result in
> OST usage imbalance.
>> 
>> However, I do have a valid usage case for such a feature and I am
> trying to figure out the best way to do it.
> 
> I have a valid use for it as well. It would be nice to have this feature
> back.
> 
> Scott
> 
>> 
>> One way to achieve the same result is through the use of OST pools.
> However, the userland tools do not work from a client node, it has to be
> run on the MGS. There are also other restrictions as described here:
> http://wiki.lustre.org/index.php/Creating_and_Managing_OST_Pools
>> 
>> I am also digging through the Lustre source and header files to see if
> there are other (undocumented) ways to do this.
>> 
>> So, my questions are as follows:
>> 
>> 1) Is there currently a way for a user to control the striping of a
> file / directory on a selected set of OSTs? If yes, is it possible to do
> it through an API call or a command line utility?
>> 
>> 2) If answer to (1) is No, Is there a way to do this with admin rights
> from the MDS or client nodes?
>> 
>> 3) If pools are the only way to achieve this, is there an API call to
> create, manage and destroy pools from a user application? Is there any
> impact on performance of MDS components if pools are dynamically created
> and deleted on demand? Can the API be called from a client node (root or
> user) or does it have to be on the MGS?
>> 
>> Thank you.
>> 
>> Regards,
>> Kalyana Chadalavada
>> 
>> _______________________________________________
>> Lustre-devel mailing list
>> Lustre-devel at lists.lustre.org<mailto:Lustre-devel at lists.lustre.org>
>> http://lists.lustre.org/mailman/listinfo/lustre-devel
> 
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org<mailto:Lustre-devel at lists.lustre.org>
> http://lists.lustre.org/mailman/listinfo/lustre-devel
> 
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org<mailto:Lustre-devel at lists.lustre.org>
> http://lists.lustre.org/mailman/listinfo/lustre-devel
> 
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-devel

From ckkccf at gmail.com  Mon Nov  5 18:28:38 2012
From: ckkccf at gmail.com (Kalyana C)
Date: Mon, 5 Nov 2012 12:28:38 -0600
Subject: [Lustre-devel] Control file striping
In-Reply-To: <F46703DD-5873-4FB4-AFCE-059E868B4D6B@intel.com>
References: <CANuO2dvubpiw79KC_svR7HoGZ2XRrWJ=g4gGgeHjniuyh0YHhg@mail.gmail.com>
	<DE176288-9AC9-454F-89D2-8F1098400418@ornl.gov>
	<2C7DE72B9BD00F44BAECA5B0CBB87395022AD490@hermes.terascala.com>
	<CANuO2dvGUgK=PB50P3ZG-rEMFv2WkHYQwaT6uD=0rf2MVgbAcA@mail.gmail.com>
	<F46703DD-5873-4FB4-AFCE-059E868B4D6B@intel.com>
Message-ID: <CANuO2ds0HF3kf_qJ_yoMDQ=F5k_4SqmRZTAm8v5x8ahiDCC85A@mail.gmail.com>

I will need to setup striping on specific OSTs on demand. Its not a fixed
setup.

Here is what I am thinking, with my current understanding:

*) user app sends the details (OST ids, name of the pool) to a listener on
MGS
*) a process on MGS executes the command and sends back an ack
*) user app sets up striping on the newly created pool
*) pool is deleted once striping is set

I was hoping to find out if there was a better (and easier) way to do this.

A simple usage case is as follows:
There are a mix of disks with varying performance in the file system.
Depending on the application, I would like to change the disks used for
that instance. Say, 10% spindle, 90% SSD, 50-50 for another etc.

The same can be applied to a mix of storage devices in the same file system
- disks with varying levels of redundancy or even varying devices.

Regards,
Kalyan

On Mon, Nov 5, 2012 at 12:16 PM, White, Cliff <cliff.white at intel.com> wrote:

>  Creating/destroying pools must be done as per the manual, from MDS.
> Do you need to frequently create/destroy pools, or would you have a fixed
> setup?
>
>  The normal method is to map various pools (and other striping policies)
> to directories, then user space applications
> can access specific pools by pointing at the requisite directory, thus
> requiring no special API.
> Once pools are created, the directory creation/striping is all done on a
> client via lfs setstripe, the liblustreapi can
> also be used by clients to set striping via C calls.
>
>  Best
> cliffw
>
>
>  On Nov 5, 2012, at 10:01 AM, Kalyana C <ckkccf at gmail.com>
>  wrote:
>
>  > Have you considered using a combination of OST Pools and striping
> fit your needs?
>
>  This is the only way possible that I currently know of.
>
>  But is it possible to manage OST pools via an API call? Can it be done
> from a client under a normal user application? Or does it have the same
> restrictions as described at
> http://wiki.lustre.org/index.php/Creating_and_Managing_OST_Pools
>
>  Regards,
> Kalyan
>
> On Mon, Nov 5, 2012 at 11:57 AM, Ben Evans <Ben.Evans at terascala.com>wrote:
>
>> Have you considered using a combination of OST Pools and striping fit
>> your needs?
>>
>> -----Original Message-----
>> From: lustre-devel-bounces at lists.lustre.org
>> [mailto:lustre-devel-bounces at lists.lustre.org] On Behalf Of Atchley,
>> Scott
>> Sent: Monday, November 05, 2012 12:47 PM
>> To: Kalyana C
>> Cc: lustre-devel at lists.lustre.org
>> Subject: Re: [Lustre-devel] Control file striping
>>
>> On Nov 3, 2012, at 1:41 PM, Kalyana C <ckkccf at gmail.com> wrote:
>>
>> > Hello,
>> >
>> > I am trying to figure out a way to:
>> >
>> > set striping across a set of OSTs of my choice.
>> >
>> > lfs setstripe command and API call does not allow for this currently.
>> The user can only set the starting OST through the index parameter. I
>> appreciate why there is no such feature currently; you do not want to
>> let the users have control on how OSTs are used since it can result in
>> OST usage imbalance.
>> >
>> > However, I do have a valid usage case for such a feature and I am
>> trying to figure out the best way to do it.
>>
>> I have a valid use for it as well. It would be nice to have this feature
>> back.
>>
>> Scott
>>
>> >
>> > One way to achieve the same result is through the use of OST pools.
>> However, the userland tools do not work from a client node, it has to be
>> run on the MGS. There are also other restrictions as described here:
>> http://wiki.lustre.org/index.php/Creating_and_Managing_OST_Pools
>> >
>> > I am also digging through the Lustre source and header files to see if
>> there are other (undocumented) ways to do this.
>> >
>> > So, my questions are as follows:
>> >
>> > 1) Is there currently a way for a user to control the striping of a
>> file / directory on a selected set of OSTs? If yes, is it possible to do
>> it through an API call or a command line utility?
>> >
>> > 2) If answer to (1) is No, Is there a way to do this with admin rights
>> from the MDS or client nodes?
>> >
>> > 3) If pools are the only way to achieve this, is there an API call to
>> create, manage and destroy pools from a user application? Is there any
>> impact on performance of MDS components if pools are dynamically created
>> and deleted on demand? Can the API be called from a client node (root or
>> user) or does it have to be on the MGS?
>> >
>> > Thank you.
>> >
>> > Regards,
>> > Kalyana Chadalavada
>> >
>> > _______________________________________________
>> > Lustre-devel mailing list
>> > Lustre-devel at lists.lustre.org
>> > http://lists.lustre.org/mailman/listinfo/lustre-devel
>>
>> _______________________________________________
>> Lustre-devel mailing list
>> Lustre-devel at lists.lustre.org
>> http://lists.lustre.org/mailman/listinfo/lustre-devel
>>
>
>  _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-devel
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20121105/86822b0b/attachment.html>

From morrone2 at llnl.gov  Mon Nov  5 18:42:29 2012
From: morrone2 at llnl.gov (Christopher J. Morrone)
Date: Mon, 05 Nov 2012 10:42:29 -0800
Subject: [Lustre-devel] Control file striping
In-Reply-To: <2C7DE72B9BD00F44BAECA5B0CBB87395022AD490@hermes.terascala.com>
References: <CANuO2dvubpiw79KC_svR7HoGZ2XRrWJ=g4gGgeHjniuyh0YHhg@mail.gmail.com>
	<DE176288-9AC9-454F-89D2-8F1098400418@ornl.gov>
	<2C7DE72B9BD00F44BAECA5B0CBB87395022AD490@hermes.terascala.com>
Message-ID: <50980895.6010809@llnl.gov>

Pools only address one kind of problem:  constraining object allocation 
to a set of OSTs.

But what if the user wants to guarantee that their files' objects are 
evenly distributed across the OSTs, whether in a pool or not?  Even a 
slight unevenness in object allocation (one OST getting more objects 
than the others) can drastically change the apparent aggregate 
throughput that the user sees.

Most of the applications that I am aware of essentially do:

   compute
   barrier
   write files
   barrier
   repeat

So any unevenness of object allocation means that the entire application 
sits and waits for that one OST that is more heavily loaded than the rest.

Which generally means for us that Lustre's object allocation algorithms 
that attempt to provide space-balancing have noticeably negative impacts 
on application performance.

So I agree that I would like to see the set-stripe API and wire 
protocols expanded to allow specifying all of the desired object 
indices, not just the first one.

It would probably be good to make the use of the ability to manually set 
object locations permission-based, and allow the sysadmin to decide if 
users can use that functionality.  Best would be if they could set it on 
a per-user basis, but I'd settle for a single large on/off switch.

Chris

On 11/05/2012 09:57 AM, Ben Evans wrote:
> Have you considered using a combination of OST Pools and striping fit
> your needs?
>
> -----Original Message-----
> From: lustre-devel-bounces at lists.lustre.org
> [mailto:lustre-devel-bounces at lists.lustre.org] On Behalf Of Atchley,
> Scott
> Sent: Monday, November 05, 2012 12:47 PM
> To: Kalyana C
> Cc: lustre-devel at lists.lustre.org
> Subject: Re: [Lustre-devel] Control file striping
>
> On Nov 3, 2012, at 1:41 PM, Kalyana C <ckkccf at gmail.com> wrote:
>
>> Hello,
>>
>> I am trying to figure out a way to:
>>
>> set striping across a set of OSTs of my choice.
>>
>> lfs setstripe command and API call does not allow for this currently.
> The user can only set the starting OST through the index parameter. I
> appreciate why there is no such feature currently; you do not want to
> let the users have control on how OSTs are used since it can result in
> OST usage imbalance.
>>
>> However, I do have a valid usage case for such a feature and I am
> trying to figure out the best way to do it.
>
> I have a valid use for it as well. It would be nice to have this feature
> back.
>
> Scott
>
>>
>> One way to achieve the same result is through the use of OST pools.
> However, the userland tools do not work from a client node, it has to be
> run on the MGS. There are also other restrictions as described here:
> http://wiki.lustre.org/index.php/Creating_and_Managing_OST_Pools
>>
>> I am also digging through the Lustre source and header files to see if
> there are other (undocumented) ways to do this.
>>
>> So, my questions are as follows:
>>
>> 1) Is there currently a way for a user to control the striping of a
> file / directory on a selected set of OSTs? If yes, is it possible to do
> it through an API call or a command line utility?
>>
>> 2) If answer to (1) is No, Is there a way to do this with admin rights
> from the MDS or client nodes?
>>
>> 3) If pools are the only way to achieve this, is there an API call to
> create, manage and destroy pools from a user application? Is there any
> impact on performance of MDS components if pools are dynamically created
> and deleted on demand? Can the API be called from a client node (root or
> user) or does it have to be on the MGS?
>>
>> Thank you.
>>
>> Regards,
>> Kalyana Chadalavada
>>
>> _______________________________________________
>> Lustre-devel mailing list
>> Lustre-devel at lists.lustre.org
>> http://lists.lustre.org/mailman/listinfo/lustre-devel
>
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-devel
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-devel
> .
>

From atchleyes at ornl.gov  Mon Nov  5 19:00:47 2012
From: atchleyes at ornl.gov (Atchley, Scott)
Date: Mon, 5 Nov 2012 14:00:47 -0500
Subject: [Lustre-devel] Control file striping
In-Reply-To: <50980895.6010809@llnl.gov>
References: <CANuO2dvubpiw79KC_svR7HoGZ2XRrWJ=g4gGgeHjniuyh0YHhg@mail.gmail.com>
	<DE176288-9AC9-454F-89D2-8F1098400418@ornl.gov>
	<2C7DE72B9BD00F44BAECA5B0CBB87395022AD490@hermes.terascala.com>
	<50980895.6010809@llnl.gov>
Message-ID: <B5E48401-5F24-4434-9A0E-089FFC54D4DF@ornl.gov>

On Nov 5, 2012, at 1:42 PM, Christopher J. Morrone <morrone2 at llnl.gov> wrote:

> Pools only address one kind of problem:  constraining object allocation 
> to a set of OSTs.
> 
> But what if the user wants to guarantee that their files' objects are 
> evenly distributed across the OSTs, whether in a pool or not?  Even a 
> slight unevenness in object allocation (one OST getting more objects 
> than the others) can drastically change the apparent aggregate 
> throughput that the user sees.
> 
> Most of the applications that I am aware of essentially do:
> 
>   compute
>   barrier
>   write files
>   barrier
>   repeat
> 
> So any unevenness of object allocation means that the entire application 
> sits and waits for that one OST that is more heavily loaded than the rest.
> 
> Which generally means for us that Lustre's object allocation algorithms 
> that attempt to provide space-balancing have noticeably negative impacts 
> on application performance.
> 
> So I agree that I would like to see the set-stripe API and wire 
> protocols expanded to allow specifying all of the desired object 
> indices, not just the first one.
> 
> It would probably be good to make the use of the ability to manually set 
> object locations permission-based, and allow the sysadmin to decide if 
> users can use that functionality.  Best would be if they could set it on 
> a per-user basis, but I'd settle for a single large on/off switch.
> 
> Chris

Ditto.

Scott

> 
> On 11/05/2012 09:57 AM, Ben Evans wrote:
>> Have you considered using a combination of OST Pools and striping fit
>> your needs?
>> 
>> -----Original Message-----
>> From: lustre-devel-bounces at lists.lustre.org
>> [mailto:lustre-devel-bounces at lists.lustre.org] On Behalf Of Atchley,
>> Scott
>> Sent: Monday, November 05, 2012 12:47 PM
>> To: Kalyana C
>> Cc: lustre-devel at lists.lustre.org
>> Subject: Re: [Lustre-devel] Control file striping
>> 
>> On Nov 3, 2012, at 1:41 PM, Kalyana C <ckkccf at gmail.com> wrote:
>> 
>>> Hello,
>>> 
>>> I am trying to figure out a way to:
>>> 
>>> set striping across a set of OSTs of my choice.
>>> 
>>> lfs setstripe command and API call does not allow for this currently.
>> The user can only set the starting OST through the index parameter. I
>> appreciate why there is no such feature currently; you do not want to
>> let the users have control on how OSTs are used since it can result in
>> OST usage imbalance.
>>> 
>>> However, I do have a valid usage case for such a feature and I am
>> trying to figure out the best way to do it.
>> 
>> I have a valid use for it as well. It would be nice to have this feature
>> back.
>> 
>> Scott
>> 
>>> 
>>> One way to achieve the same result is through the use of OST pools.
>> However, the userland tools do not work from a client node, it has to be
>> run on the MGS. There are also other restrictions as described here:
>> http://wiki.lustre.org/index.php/Creating_and_Managing_OST_Pools
>>> 
>>> I am also digging through the Lustre source and header files to see if
>> there are other (undocumented) ways to do this.
>>> 
>>> So, my questions are as follows:
>>> 
>>> 1) Is there currently a way for a user to control the striping of a
>> file / directory on a selected set of OSTs? If yes, is it possible to do
>> it through an API call or a command line utility?
>>> 
>>> 2) If answer to (1) is No, Is there a way to do this with admin rights
>> from the MDS or client nodes?
>>> 
>>> 3) If pools are the only way to achieve this, is there an API call to
>> create, manage and destroy pools from a user application? Is there any
>> impact on performance of MDS components if pools are dynamically created
>> and deleted on demand? Can the API be called from a client node (root or
>> user) or does it have to be on the MGS?
>>> 
>>> Thank you.
>>> 
>>> Regards,
>>> Kalyana Chadalavada
>>> 
>>> _______________________________________________
>>> Lustre-devel mailing list
>>> Lustre-devel at lists.lustre.org
>>> http://lists.lustre.org/mailman/listinfo/lustre-devel
>> 
>> _______________________________________________
>> Lustre-devel mailing list
>> Lustre-devel at lists.lustre.org
>> http://lists.lustre.org/mailman/listinfo/lustre-devel
>> _______________________________________________
>> Lustre-devel mailing list
>> Lustre-devel at lists.lustre.org
>> http://lists.lustre.org/mailman/listinfo/lustre-devel
>> .
>> 
> 
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-devel

From ckkccf at gmail.com  Mon Nov  5 19:41:29 2012
From: ckkccf at gmail.com (Kalyana C)
Date: Mon, 5 Nov 2012 13:41:29 -0600
Subject: [Lustre-devel] Control file striping
Message-ID: <CANuO2duTofrad9yBN42sOs6sGeq8asoAatTKaF6awKnbHDjmYQ@mail.gmail.com>

> So I agree that I would like to see the set-stripe API and wire
> protocols expanded to allow specifying all of the desired object
> indices, not just the first one.
> It would probably be good to make the use of the ability to manually set
> object locations permission-based, and allow the sysadmin to decide if
> users can use that functionality.  Best would be if they could set it on
> a per-user basis, but I'd settle for a single large on/off switch.


This would be nice to have.


Regards,
Kalyan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20121105/58f8aed9/attachment.html>

From andreas.dilger at intel.com  Sat Nov 10 00:09:09 2012
From: andreas.dilger at intel.com (Dilger, Andreas)
Date: Sat, 10 Nov 2012 00:09:09 +0000
Subject: [Lustre-devel] Seeking contributors for Lustre User Manual
In-Reply-To: <F4644F70-E8BA-437F-87B1-9A57FADF8254@intel.com>
Message-ID: <CCC2E817.24EA%andreas.dilger@intel.com>

In hopes of improving the quality, coverage, and currency of the Lustre
User Manual, I'm putting out a call to the Lustre community for
contributors to this important resource.

The user manual is especially important for new users to Lustre, but has
fallen into some disrepair now that there is no longer a dedicated
documentation writer for it.  As well, there are many sections of the
manual that have become outdated over the years (such as example output,
command descriptions, etc) that need to be refreshed.

The Lustre User Manual is also a component of Lustre that has a much
larger pool of potential contributors than the code itself.  If you want
to contribute to Lustre, but are not able to contribute with patches to
the code, then this is a great opportunity to help out.  If you benefit
from the open source development of Lustre, then contributing to the
manual is a chance to return something back to the community.  The Lustre
Manual is released under a Creative Commons license, so it is open to all
of us to improve.


While there is not currently a "todo" list for the areas of the manual
that need updating, looking through open LUDOC tickets is one option:

    
http://bugs.whamcloud.com/secure/QuickSearch.jspa?searchString=LUDOC%20open

There are a number of existing documentation tickets for features that are
under development for the Lustre 2.4 release (for which we expect to be
completed internally), but there are also some tickets from users pointing
out errors in the document that need to be fixed.

Another way to improve the manual is to simply fetch the manual and read
some section at random that you are either interested in, or have some
knowledge about, and see if any of the text is confusing, outdated, or
incorrect and needs to be updated.


The PDF and HTML versions of the current manual are available at:

    http://wiki.whamcloud.com/display/PUB/Documentation

The manual source is hosted in a Git/Gerrit repository in Docbook XML
format and can be downloaded at:

    git clone http://git.whamcloud.com/doc/manual lustre-manual

An account in Gerrit is required to download the manual and submit patches.

There are some wiki pages that describe the process and packages needed to
modify, build, and submit patches to the manual:

    
http://wiki.whamcloud.com/display/PUB/Making+changes+to+the+Lustre+Manual

Cheers, Andreas
--
Andreas Dilger
Lustre Software Architect
Intel Corporation

From nathan_rutman at xyratex.com  Sat Nov 10 03:12:38 2012
From: nathan_rutman at xyratex.com (Nathan Rutman)
Date: Fri, 9 Nov 2012 19:12:38 -0800
Subject: [Lustre-devel] New test framework
Message-ID: <3418A331-F07E-4CC0-A2C1-4C8DB7592C4F@xyratex.com>

For awhile now I've believed that the current built-in Lustre tests have a number of problems and limitations that are limiting their usefulness in finding Lustre bugs or providing a good proxy for Lustre quality.  We started this initially with a discussion on TWG, further refined some ideas for our LAD presentation, and have now finally articulated them in a semi-coherent manner here:
http://goo.gl/mCtwU

Chris Gearing at Intel is also heavily interested in this, and has started turning some of these thoughts into an actual spec, but before I integrate those I'd like to get a little feedback from the community on the high-level concepts.  I'd also like to hear back from others that are interested in the topic (and hopefully willing to work on it) so we can put together a special-interest list for discussion.

I've given everyone comment rights and Chris and Roman edit rights for this doc - if anyone else wants to be an editor, let me know.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20121109/aa4018da/attachment.html>

From uja at ornl.gov  Tue Nov 13 18:30:39 2012
From: uja at ornl.gov (James A Simmons)
Date: Tue, 13 Nov 2012 13:30:39 -0500
Subject: [Lustre-devel] newer e2fsprogs
Message-ID: <1352831439.25490.24.camel@bohr.ornl.gov>

Hello

	LU-1911 noted some problems with certain versions of e2fsprogs released
by intel. The ticket spoke of e2fsprogs-1.42.5.wc3 rpms but I can't find
them anywhere. Anyone have a copy?

From bass6 at llnl.gov  Tue Nov 13 18:33:02 2012
From: bass6 at llnl.gov (Ned Bass)
Date: Tue, 13 Nov 2012 10:33:02 -0800
Subject: [Lustre-devel] [wc-discuss] Seeking contributors for Lustre
	User Manual
In-Reply-To: <CCC2E817.24EA%andreas.dilger@intel.com>
References: <F4644F70-E8BA-437F-87B1-9A57FADF8254@intel.com>
	<CCC2E817.24EA%andreas.dilger@intel.com>
Message-ID: <20121113183302.GM31700@llnl.gov>

On Sat, Nov 10, 2012 at 12:09:09AM +0000, Dilger, Andreas wrote:
> The manual source is hosted in a Git/Gerrit repository in Docbook XML
> format and can be downloaded at:
> 
>     git clone http://git.whamcloud.com/doc/manual lustre-manual

That doesn't work for me:

  % git clone http://review.whamcloud.com/doc/manual  lustre-manual
  Cloning into lustre-manual...
  fatal: http://review.whamcloud.com/doc/manual/info/refs not found: did
    you run git update-server-info on the server?

I've only been able to check out the manual source directly from gerrit:

  git clone http://review.whamcloud.com/p/doc/manual lustre-manual

or,

  git clone ssh://nedbass at review.whamcloud.com:29418/doc/manual lustre-manual


Ned

From nathan_rutman at xyratex.com  Tue Nov 13 19:48:35 2012
From: nathan_rutman at xyratex.com (Nathan Rutman)
Date: Tue, 13 Nov 2012 11:48:35 -0800
Subject: [Lustre-devel] [wc-discuss] Seeking contributors for Lustre
	User Manual
In-Reply-To: <CCC2E817.24EA%andreas.dilger@intel.com>
References: <CCC2E817.24EA%andreas.dilger@intel.com>
Message-ID: <39D7B2E1-4DEE-4E13-9F35-A4E0B2C149C6@xyratex.com>

Would it be easier to move the manual back to a Wiki?  The low hassle factor of wikis has always been a draw for contribution.  The openSFS site is up and running with MediaWiki now (wiki.opensfs.org).


On Nov 9, 2012, at 4:09 PM, "Dilger, Andreas" <andreas.dilger at intel.com> wrote:

> In hopes of improving the quality, coverage, and currency of the Lustre
> User Manual, I'm putting out a call to the Lustre community for
> contributors to this important resource.
> 
> The user manual is especially important for new users to Lustre, but has
> fallen into some disrepair now that there is no longer a dedicated
> documentation writer for it.  As well, there are many sections of the
> manual that have become outdated over the years (such as example output,
> command descriptions, etc) that need to be refreshed.
> 
> The Lustre User Manual is also a component of Lustre that has a much
> larger pool of potential contributors than the code itself.  If you want
> to contribute to Lustre, but are not able to contribute with patches to
> the code, then this is a great opportunity to help out.  If you benefit
> from the open source development of Lustre, then contributing to the
> manual is a chance to return something back to the community.  The Lustre
> Manual is released under a Creative Commons license, so it is open to all
> of us to improve.
> 
> 
> While there is not currently a "todo" list for the areas of the manual
> that need updating, looking through open LUDOC tickets is one option:
> 
> 
> http://bugs.whamcloud.com/secure/QuickSearch.jspa?searchString=LUDOC%20open
> 
> There are a number of existing documentation tickets for features that are
> under development for the Lustre 2.4 release (for which we expect to be
> completed internally), but there are also some tickets from users pointing
> out errors in the document that need to be fixed.
> 
> Another way to improve the manual is to simply fetch the manual and read
> some section at random that you are either interested in, or have some
> knowledge about, and see if any of the text is confusing, outdated, or
> incorrect and needs to be updated.
> 
> 
> The PDF and HTML versions of the current manual are available at:
> 
>    http://wiki.whamcloud.com/display/PUB/Documentation
> 
> The manual source is hosted in a Git/Gerrit repository in Docbook XML
> format and can be downloaded at:
> 
>    git clone http://git.whamcloud.com/doc/manual lustre-manual
> 
> An account in Gerrit is required to download the manual and submit patches.
> 
> There are some wiki pages that describe the process and packages needed to
> modify, build, and submit patches to the manual:
> 
> 
> http://wiki.whamcloud.com/display/PUB/Making+changes+to+the+Lustre+Manual
> 
> Cheers, Andreas
> --
> Andreas Dilger
> Lustre Software Architect
> Intel Corporation
> 

From bass6 at llnl.gov  Tue Nov 13 21:26:42 2012
From: bass6 at llnl.gov (Ned Bass)
Date: Tue, 13 Nov 2012 13:26:42 -0800
Subject: [Lustre-devel] [wc-discuss] Seeking contributors for Lustre
	User Manual
In-Reply-To: <39D7B2E1-4DEE-4E13-9F35-A4E0B2C149C6@xyratex.com>
References: <CCC2E817.24EA%andreas.dilger@intel.com>
	<39D7B2E1-4DEE-4E13-9F35-A4E0B2C149C6@xyratex.com>
Message-ID: <20121113212642.GN31700@llnl.gov>

On Tue, Nov 13, 2012 at 11:48:35AM -0800, Nathan Rutman wrote:
> Would it be easier to move the manual back to a Wiki?  The low hassle
> factor of wikis has always been a draw for contribution.  The openSFS
> site is up and running with MediaWiki now (wiki.opensfs.org).

Easier? Yes, probably. Better? I personally don't think so.  Wikis are
great collaboration tools for informally sharing information, but I
don't think the paradigm scales well for documents of this size and
complexity. And a wiki isn't the right tool for producing a formal
professional-quality document, which is what I think the Lustre manual
should strive to be.

True, we would lower the bar for contributions, but for that we would
sacrifice the following features that I consider essential.

- Ability to export to multiple formats (pdf, html, epub) from one source
- Consistency of formatting and navigation elements
- A review process for proposed changes that assures a high standard of quality

However, there are some short articles that probably do belong in the
wiki that could be poached from the manual, i.e. installation and
configuration procedures, etc.

Ned

From steven.jenkins at gmail.com  Tue Nov 13 21:43:50 2012
From: steven.jenkins at gmail.com (Steven Jenkins)
Date: Tue, 13 Nov 2012 16:43:50 -0500
Subject: [Lustre-devel] [wc-discuss] Seeking contributors for Lustre
	User Manual
In-Reply-To: <20121113212642.GN31700@llnl.gov>
References: <CCC2E817.24EA%andreas.dilger@intel.com>
	<39D7B2E1-4DEE-4E13-9F35-A4E0B2C149C6@xyratex.com>
	<20121113212642.GN31700@llnl.gov>
Message-ID: <CAG7Q=16WOrrLeW6ChuZ7syjKQUTdRwBiR9p6sSCA87_ef1aYMg@mail.gmail.com>

+1

Steven

On Tue, Nov 13, 2012 at 4:26 PM, Ned Bass <bass6 at llnl.gov> wrote:
> On Tue, Nov 13, 2012 at 11:48:35AM -0800, Nathan Rutman wrote:
>> Would it be easier to move the manual back to a Wiki?  The low hassle
>> factor of wikis has always been a draw for contribution.  The openSFS
>> site is up and running with MediaWiki now (wiki.opensfs.org).
>
> Easier? Yes, probably. Better? I personally don't think so.  Wikis are
> great collaboration tools for informally sharing information, but I
> don't think the paradigm scales well for documents of this size and
> complexity. And a wiki isn't the right tool for producing a formal
> professional-quality document, which is what I think the Lustre manual
> should strive to be.
>
> True, we would lower the bar for contributions, but for that we would
> sacrifice the following features that I consider essential.
>
> - Ability to export to multiple formats (pdf, html, epub) from one source
> - Consistency of formatting and navigation elements
> - A review process for proposed changes that assures a high standard of quality
>
> However, there are some short articles that probably do belong in the
> wiki that could be poached from the manual, i.e. installation and
> configuration procedures, etc.
>
> Ned

From laisiyao at whamcloud.com  Wed Nov 14 02:43:34 2012
From: laisiyao at whamcloud.com (Lai Siyao)
Date: Wed, 14 Nov 2012 10:43:34 +0800
Subject: [Lustre-devel] newer e2fsprogs
In-Reply-To: <1352831439.25490.24.camel@bohr.ornl.gov>
References: <1352831439.25490.24.camel@bohr.ornl.gov>
Message-ID: <CALxHnhKg_7o6gCqpfhBVdcunW8Z9eDZJtGSLpwCgQfug=AE=kw@mail.gmail.com>

You can always find the latest build for e2fsprogs on build.whamcloud.com,
after choosing correct arch and distro.

On Wed, Nov 14, 2012 at 2:30 AM, James A Simmons <uja at ornl.gov> wrote:

> Hello
>
>         LU-1911 noted some problems with certain versions of e2fsprogs
> released
> by intel. The ticket spoke of e2fsprogs-1.42.5.wc3 rpms but I can't find
> them anywhere. Anyone have a copy?
>
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20121114/d3f7d612/attachment.html>

From andreas.dilger at intel.com  Wed Nov 14 03:21:01 2012
From: andreas.dilger at intel.com (Dilger, Andreas)
Date: Wed, 14 Nov 2012 03:21:01 +0000
Subject: [Lustre-devel] [wc-discuss] Seeking contributors for Lustre
	User Manual
In-Reply-To: <20121113212642.GN31700@llnl.gov>
References: <CCC2E817.24EA%andreas.dilger@intel.com>
	<39D7B2E1-4DEE-4E13-9F35-A4E0B2C149C6@xyratex.com>
	<20121113212642.GN31700@llnl.gov>
Message-ID: <C0AA90B2-FE89-4290-8640-D55EF49AF618@intel.com>

On 2012-11-13, at 14:26, Ned Bass <bass6 at llnl.gov> wrote:

> On Tue, Nov 13, 2012 at 11:48:35AM -0800, Nathan Rutman wrote:
>> Would it be easier to move the manual back to a Wiki?  The low hassle
>> factor of wikis has always been a draw for contribution.  The openSFS
>> site is up and running with MediaWiki now (wiki.opensfs.org).
> 
> Easier? Yes, probably. Better? I personally don't think so.  Wikis are
> great collaboration tools for informally sharing information, but I
> don't think the paradigm scales well for documents of this size and
> complexity. And a wiki isn't the right tool for producing a formal
> professional-quality document, which is what I think the Lustre manual
> should strive to be.
> 
> True, we would lower the bar for contributions, but for that we would
> sacrifice the following features that I consider essential.
> 
> - Ability to export to multiple formats (pdf, html, epub) from one source
> - Consistency of formatting and navigation elements
> - A review process for proposed changes that assures a high standard of quality

Ned, these are the reasons we never moved the manual to a wiki in the first place.  The original manual at Oracle was in FrameMaker format and only two people could edit it. When we regenerated the manual again, it was converted into Docbook XML.

Since Docbook is a relatively lightweight plain text markup language, and we have good tools for validating the syntax along with WYSIWYG editing, so it is hopefully a good tradeoff between ease of editing and the feature set needed to produce the manual. 

The other major drawback of a wiki is that it only works online, so it can't (easily?) be saved to read on a plane or in a secure facility. 

> However, there are some short articles that probably do belong in the
> wiki that could be poached from the manual, i.e. installation and
> configuration procedures, etc.

Sure, and some of these exist (Richard's "Getting Started With Lustre" page). Definitely there could be a lot more effort out into this, however. 

> Ned

From aik at fnal.gov  Wed Nov 14 19:47:21 2012
From: aik at fnal.gov (Alex Kulyavtsev)
Date: Wed, 14 Nov 2012 13:47:21 -0600
Subject: [Lustre-devel] Document Database Re: [Lustre-discuss] [wc-discuss]
	Seeking contributors for Lustre User Manual
In-Reply-To: <20121113212642.GN31700@llnl.gov>
References: <CCC2E817.24EA%andreas.dilger@intel.com>
	<39D7B2E1-4DEE-4E13-9F35-A4E0B2C149C6@xyratex.com>
	<20121113212642.GN31700@llnl.gov>
Message-ID: <F6F6CD0F-3F76-4798-B7F2-90692BAB6AF5@fnal.gov>


On Nov 13, 2012, at 3:26 PM, Ned Bass wrote:

> On Tue, Nov 13, 2012 at 11:48:35AM -0800, Nathan Rutman wrote:
>> Would it be easier to move the manual back to a Wiki?  The low hassle
>> factor of wikis has always been a draw for contribution.  The openSFS
>> site is up and running with MediaWiki now (wiki.opensfs.org).
> 
> Easier? Yes, probably. Better? I personally don't think so.  Wikis are
> great collaboration tools for informally sharing information, but I
> don't think the paradigm scales well for documents of this size and
> complexity. And a wiki isn't the right tool for producing a formal
> professional-quality document, which is what I think the Lustre manual
> should strive to be.
> 
> True, we would lower the bar for contributions, but for that we would
> sacrifice the following features that I consider essential.
> 
> - Ability to export to multiple formats (pdf, html, epub) from one source

http://www.docbook.org ?

> - Consistency of formatting and navigation elements
> - A review process for proposed changes that assures a high standard of quality
- ability to track changes between document versions to incrementally update 'higher level' documents

> 
> However, there are some short articles that probably do belong in the
> wiki that could be poached from the manual, i.e. installation and
> configuration procedures, etc.
Right. 
And also the other way around: detailed articles on wiki written by developers can be later 'harvested' by professional writer into manual chapter, referencing to wiki for details. 
Lowering entry bar is vital to encourage developers to write or update documentation.

DB:
In addition to wiki and "manual" it will be nice to have Document Database, where conference reports, RFCs, RFP, HLD, DLD, ... can be committed, updated and later searched. 
Something like DocDB
	http://sourceforge.net/projects/docdb-v/
Document format can be any.
DocDB has been created to keep track of documentation in large collaboration - BTeV experiment -  and then used by several others. DocDB has ability to manage access rights to some documents. 

I think we need all three - wiki, DocDB and manuals, they serve different purpose.

KB:
Right now lustre support tips and hints are living on lustre-discuss list. It is tedious to search emails (no tags,no links), and when the answer found, there is no guarantee it is still relevant.
It can be useful to accumulate tips and best practices in Knowledge Base and have mechanisms to update it, e.g. instead of answering directly to the list create entry in KB and post the ref. to the list.

Alex.

> 
> Ned
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss

From C.J.Walker at qmul.ac.uk  Wed Nov 14 23:51:49 2012
From: C.J.Walker at qmul.ac.uk (Christopher J. Walker)
Date: Wed, 14 Nov 2012 23:51:49 +0000
Subject: [Lustre-devel] [Lustre-discuss] [wc-discuss] Seeking
 contributors for Lustre User Manual
In-Reply-To: <20121113212642.GN31700@llnl.gov>
References: <CCC2E817.24EA%andreas.dilger@intel.com>	<39D7B2E1-4DEE-4E13-9F35-A4E0B2C149C6@xyratex.com>
	<20121113212642.GN31700@llnl.gov>
Message-ID: <50A42E95.6030602@qmul.ac.uk>

On 13/11/12 21:26, Ned Bass wrote:
> - A review process for proposed changes that assures a high standard of quality

That's important. I've spotted cases where it's clear there's an issue, 
but am not sure what the correct answer is.

A quick way of reporting issues in the manual would be useful too - 
whilst I did go to the effort of registering in order to submit typos, 
that's rather heavyweight. Just being able to send an e-mail would be 
easier - but it's true that someone needs to sort them.

Chris

From andreas.dilger at intel.com  Thu Nov 15 00:12:40 2012
From: andreas.dilger at intel.com (Dilger, Andreas)
Date: Thu, 15 Nov 2012 00:12:40 +0000
Subject: [Lustre-devel] [Lustre-discuss] [wc-discuss] Seeking
 contributors for Lustre User Manual
In-Reply-To: <50A42E95.6030602@qmul.ac.uk>
Message-ID: <CCC97D94.2646%andreas.dilger@intel.com>

On 11/14/12 4:51 PM, "Christopher J. Walker" <C.J.Walker at qmul.ac.uk> wrote:
>On 13/11/12 21:26, Ned Bass wrote:
>> - A review process for proposed changes that assures a high standard of
>>quality
>
>That's important. I've spotted cases where it's clear there's an issue,
>but am not sure what the correct answer is.
>
>A quick way of reporting issues in the manual would be useful too -
>whilst I did go to the effort of registering in order to submit typos,
>that's rather heavyweight. Just being able to send an e-mail would be
>easier - but it's true that someone needs to sort them.

I think that while email may be easier for the initial report to be
submitted, it places more burden on the recipient to track the issue (or
risk losing/forgetting the email over time), and the issue is only visible
to the recipient (and cannot easily be seen by others or transferred to
someone else for resolution).  If we really want to scale the process of
updating the manual, then pushing the work to turn the emails into
tickets/fixes is not moving in the right direction.

I would prefer to see a fix immediately rather than someone filing a
ticket to describe the fix, since the documentation fix should be
self-describing.  However, if there is a problem that isn't immediately
resolved then a Jira ticket should be submitted in order to track the
defect and allow assigning the work to someone.  The effort of setting up
Jira and/or Gerrit accounts is a one-time thing, and is shared with normal
bug reporting, so hopefully not a huge burden (no worse than having to get
an account for the wiki, which would also be a requirement to submit, to
avoid wiki spam).

Hopefully this will not be a huge obstacle for contributions in the end.

Cheers, Andreas

From bass6 at llnl.gov  Thu Nov 15 01:26:53 2012
From: bass6 at llnl.gov (Ned Bass)
Date: Wed, 14 Nov 2012 17:26:53 -0800
Subject: [Lustre-devel] [Lustre-discuss] [wc-discuss] Seeking
 contributors for Lustre User Manual
In-Reply-To: <CCC97D94.2646%andreas.dilger@intel.com>
References: <50A42E95.6030602@qmul.ac.uk>
	<CCC97D94.2646%andreas.dilger@intel.com>
Message-ID: <20121115012653.GP31700@llnl.gov>

On Thu, Nov 15, 2012 at 12:12:40AM +0000, Dilger, Andreas wrote:
>
> I would prefer to see a fix immediately rather than someone filing a
> ticket to describe the fix, since the documentation fix should be
> self-describing.  However, if there is a problem that isn't immediately
> resolved then a Jira ticket should be submitted in order to track the
> defect and allow assigning the work to someone.

LUDOC-11 seems to be a catch-all issue for submitting fixes to minor
problems like typos.  However it sounds like you're saying we can bypass
Jira altogether for such patches.  That would be nice; linking to a
ticket with no useful content doesn't serve any purpose that I can see.

The "Making changes to the Lustre Manual source" article currently
instructs the reader to "file an LUDOC bug for change tracking in Jira"
as the first step.  To avoid discouraging submission of minor fixes,
perhaps a more lightweight process for that case should be covered
first.  In particular, say either that minor fixes should reference
LUDOC-11 in the summary, or just omit the Jira reference altogether,
whichever is appropriate.

Ned

From rpwagner at sdsc.edu  Wed Nov 14 20:01:48 2012
From: rpwagner at sdsc.edu (Richard P Wagner)
Date: Wed, 14 Nov 2012 20:01:48 +0000
Subject: [Lustre-devel] Document Database Re: [Lustre-discuss]
 [wc-discuss] Seeking contributors for Lustre User Manual
In-Reply-To: <F6F6CD0F-3F76-4798-B7F2-90692BAB6AF5@fnal.gov>
References: <CCC2E817.24EA%andreas.dilger@intel.com>
	<39D7B2E1-4DEE-4E13-9F35-A4E0B2C149C6@xyratex.com>
	<20121113212642.GN31700@llnl.gov>,
	<F6F6CD0F-3F76-4798-B7F2-90692BAB6AF5@fnal.gov>
Message-ID: <91DB696E4F98904491C1D176DD6F8FDC1493DF35@XMAIL-MBX-BH2.AD.UCSD.EDU>


________________________________________
From: Alex Kulyavtsev [aik at fnal.gov]
Sent: Wednesday, November 14, 2012 11:47 AM
To: Ned Bass
Cc: Nathan Rutman; <wc-discuss at whamcloud.com>; <lustre-discuss at lists.lustre.org>; lustre-devel at lists.lustre.org
Subject: Document Database Re: [Lustre-discuss] [wc-discuss] Seeking contributors for Lustre User Manual

On Nov 13, 2012, at 3:26 PM, Ned Bass wrote:

> On Tue, Nov 13, 2012 at 11:48:35AM -0800, Nathan Rutman wrote:
>> Would it be easier to move the manual back to a Wiki?  The low hassle
>> factor of wikis has always been a draw for contribution.  The openSFS
>> site is up and running with MediaWiki now (wiki.opensfs.org).
>
> Easier? Yes, probably. Better? I personally don't think so.  Wikis are
> great collaboration tools for informally sharing information, but I
> don't think the paradigm scales well for documents of this size and
> complexity. And a wiki isn't the right tool for producing a formal
> professional-quality document, which is what I think the Lustre manual
> should strive to be.
>
> True, we would lower the bar for contributions, but for that we would
> sacrifice the following features that I consider essential.
>
> - Ability to export to multiple formats (pdf, html, epub) from one source

http://www.docbook.org ?

Sphinx http://sphinx-doc.org/?

All the multiple format benefits, with none of the XML. And I agree wholeheartedly that Wikis are not the best home for complex documentation.

--Rick

> - Consistency of formatting and navigation elements
> - A review process for proposed changes that assures a high standard of quality
- ability to track changes between document versions to incrementally update 'higher level' documents

>
> However, there are some short articles that probably do belong in the
> wiki that could be poached from the manual, i.e. installation and
> configuration procedures, etc.
Right.
And also the other way around: detailed articles on wiki written by developers can be later 'harvested' by professional writer into manual chapter, referencing to wiki for details.
Lowering entry bar is vital to encourage developers to write or update documentation.

DB:
In addition to wiki and "manual" it will be nice to have Document Database, where conference reports, RFCs, RFP, HLD, DLD, ... can be committed, updated and later searched.
Something like DocDB
        http://sourceforge.net/projects/docdb-v/
Document format can be any.
DocDB has been created to keep track of documentation in large collaboration - BTeV experiment -  and then used by several others. DocDB has ability to manage access rights to some documents.

I think we need all three - wiki, DocDB and manuals, they serve different purpose.

KB:
Right now lustre support tips and hints are living on lustre-discuss list. It is tedious to search emails (no tags,no links), and when the answer found, there is no guarantee it is still relevant.
It can be useful to accumulate tips and best practices in Knowledge Base and have mechanisms to update it, e.g. instead of answering directly to the list create entry in KB and post the ref. to the list.

Alex.

>
> Ned
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss


From richard.henwood at intel.com  Fri Nov 16 18:23:24 2012
From: richard.henwood at intel.com (Henwood, Richard)
Date: Fri, 16 Nov 2012 18:23:24 +0000
Subject: [Lustre-devel] [Lustre-discuss] [wc-discuss] Seeking
 contributors for Lustre User Manual
In-Reply-To: <20121115012653.GP31700@llnl.gov>
References: <50A42E95.6030602@qmul.ac.uk>
	<CCC97D94.2646%andreas.dilger@intel.com>
	<20121115012653.GP31700@llnl.gov>
Message-ID: <1353090203.30909.12.camel@rhworkstation.tacc.utexas.edu>

On Wed, 2012-11-14 at 17:26 -0800, Ned Bass wrote:
> On Thu, Nov 15, 2012 at 12:12:40AM +0000, Dilger, Andreas wrote:
> >
> > I would prefer to see a fix immediately rather than someone filing a
> > ticket to describe the fix, since the documentation fix should be
> > self-describing.  However, if there is a problem that isn't immediately
> > resolved then a Jira ticket should be submitted in order to track the
> > defect and allow assigning the work to someone.
> 
> LUDOC-11 seems to be a catch-all issue for submitting fixes to minor
> problems like typos.  However it sounds like you're saying we can bypass
> Jira altogether for such patches.  That would be nice; linking to a
> ticket with no useful content doesn't serve any purpose that I can see.
> 
> The "Making changes to the Lustre Manual source" article currently
> instructs the reader to "file an LUDOC bug for change tracking in Jira"
> as the first step.  To avoid discouraging submission of minor fixes,
> perhaps a more lightweight process for that case should be covered
> first.  In particular, say either that minor fixes should reference
> LUDOC-11 in the summary, or just omit the Jira reference altogether,
> whichever is appropriate.
> 

I've introduced a procedure for minor changes on the page:
http://wiki.whamcloud.com/display/PUB/Making+changes+to+the+Lustre
+Manual+source

This includes a cut-and-paste commit message to get people started.

My reasons including an arguably redundant issue (LUDOC-11) are: 
- it creates consistent submission policy across the different projects.
- it gives an opportunity to easily measure minor changes

cheers,
Richard
-- 
Richard.Henwood at intel.com
High Performance Data Division