From nikita.danilov at clusterstor.com  Tue Dec  1 15:00:39 2009
From: nikita.danilov at clusterstor.com (Nikita Danilov)
Date: Tue, 1 Dec 2009 18:00:39 +0300
Subject: [Lustre-devel] Fwd: Disk rebuild
Message-ID: <8acda98c0912010700r684bfcf6h94ec347cd6190990@mail.gmail.com>

Hello,

On 03 Feb 2009 Jody McIntyre wrote:
 > It is possible to significantly reduce resync (but not recovery) times
 > using bitmaps, but these have been shown to hurt performance
 > significantly.  Another approach, journal-guided resynchronization, was
 > studied in a 2005 paper but has never been merged into the kernel.  The
 > paper shows improvements in resync times from 254 seconds to 0.21
 > seconds (for a 1 GB test array) with under 5% performance impact.  This
 > is an option if we're willing to develop and maintain the patches to do
 > it.

what is the status of this? Is ext3 guided resync code (RHEL 5 version
was posted on lkml in October) used by Lustre?

 >
 > Cheers,
 > Jody

Thank you,
Nikita.

From scjody at sun.com  Tue Dec  1 23:57:56 2009
From: scjody at sun.com (Jody McIntyre)
Date: Tue, 01 Dec 2009 18:57:56 -0500
Subject: [Lustre-devel] Fwd: Disk rebuild
In-Reply-To: <8acda98c0912010700r684bfcf6h94ec347cd6190990@mail.gmail.com>
References: <8acda98c0912010700r684bfcf6h94ec347cd6190990@mail.gmail.com>
Message-ID: <20091201235754.GC4508@clouds>

Hi Nikita,

On Tue, Dec 01, 2009 at 06:00:39PM +0300, Nikita Danilov wrote:

> what is the status of this? Is ext3 guided resync code (RHEL 5 version
> was posted on lkml in October) used by Lustre?

This is covered by bug 19932.

Cheers,
Jody

> 
>  >
>  > Cheers,
>  > Jody
> 
> Thank you,
> Nikita.

From nikita.danilov at clusterstor.com  Wed Dec  2 14:13:24 2009
From: nikita.danilov at clusterstor.com (Nikita Danilov)
Date: Wed, 2 Dec 2009 17:13:24 +0300
Subject: [Lustre-devel] Fwd: Disk rebuild
In-Reply-To: <20091201235754.GC4508@clouds>
References: <8acda98c0912010700r684bfcf6h94ec347cd6190990@mail.gmail.com>
	<20091201235754.GC4508@clouds>
Message-ID: <8acda98c0912020613l4cee1a43s3be7ec4f52383304@mail.gmail.com>

2009/12/2 Jody McIntyre <scjody at sun.com>:
> Hi Nikita,

Hello Jody,

>
> On Tue, Dec 01, 2009 at 06:00:39PM +0300, Nikita Danilov wrote:
>
>> what is the status of this? Is ext3 guided resync code (RHEL 5 version
>> was posted on lkml in October) used by Lustre?
>
> This is covered by bug 19932.

The last (43rd) comment there is rather intriguing. Can you elaborate
on why guided resync cannot work with the Lustre IO stack?

>
> Cheers,
> Jody

Thank you,
Nikita.

From adilger at sun.com  Wed Dec  2 19:43:04 2009
From: adilger at sun.com (Andreas Dilger)
Date: Wed, 02 Dec 2009 12:43:04 -0700
Subject: [Lustre-devel] Fwd: Disk rebuild
In-Reply-To: <8acda98c0912020613l4cee1a43s3be7ec4f52383304@mail.gmail.com>
References: <8acda98c0912010700r684bfcf6h94ec347cd6190990@mail.gmail.com>
	<20091201235754.GC4508@clouds>
	<8acda98c0912020613l4cee1a43s3be7ec4f52383304@mail.gmail.com>
Message-ID: <E9666E71-8BE9-400D-9EA4-445011BB8961@sun.com>

Hi Nikita!

On 2009-12-02, at 07:13, Nikita Danilov wrote:
>> On Tue, Dec 01, 2009 at 06:00:39PM +0300, Nikita Danilov wrote:
>>> what is the status of this? Is ext3 guided resync code (RHEL 5  
>>> version was posted on lkml in October) used by Lustre?
>>
>> This is covered by bug 19932.
>
> The last (43rd) comment there is rather intriguing. Can you elaborate
> on why guided resync cannot work with the Lustre IO stack?


The problem lies in the way that obdfilter submits IO.  Since it is  
not using the normal buffer cache to track "data=ordered" (or in the  
case of this patch "data=declared") mode the bio_submit() will likely  
start modifying the MD device before the corresponding declare blocks  
are committed to the journal.

This breaks the whole validity of declared mode in case of a crash,  
since we can no longer be certain that the declare blocks contain all  
of the locations in the MD RAID that may need to have parity rebuilt.

It would be possible to fix this by having the OST use the normal VFS  
methods to order the IO to disk, but I'm sure you're well aware of the  
performance impact of this.  It wouldn't be so bad with older versions  
of Lustre, where we had to wait for the journal commit before  
returning to the client anyway, but in 1.8.2 there is a (disabled by  
default) async journal commit option that allows the client to get RPC  
replies before the bulk IO is committed.

In order to accommodate declared mode it mean that we need to  
implement full write-cached IO on the OST, which wouldn't be  
impossible given that 1.8 already uses the page cache for reading, but  
given the amount of change and risk this would introduce it wasn't  
thought worthwhile to implement for the short lifespan it would have.   
It wouldn't be practical to introduce such a major change any sooner  
than the DMU OSD in the 2.1 release, at which point it is largely  
obsolete.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.


From adilger at sun.com  Wed Dec  2 22:48:45 2009
From: adilger at sun.com (Andreas Dilger)
Date: Wed, 02 Dec 2009 15:48:45 -0700
Subject: [Lustre-devel] Fwd: Disk rebuild
In-Reply-To: <8acda98c0912021258j140748b1wb2cdb0d2036a864a@mail.gmail.com>
References: <8acda98c0912010700r684bfcf6h94ec347cd6190990@mail.gmail.com>
	<20091201235754.GC4508@clouds>
	<8acda98c0912020613l4cee1a43s3be7ec4f52383304@mail.gmail.com>
	<E9666E71-8BE9-400D-9EA4-445011BB8961@sun.com>
	<8acda98c0912021258j140748b1wb2cdb0d2036a864a@mail.gmail.com>
Message-ID: <23BF8704-265B-476D-AAF7-47F79A131553@sun.com>

On 2009-12-02, at 13:58, Nikita Danilov wrote:
>> The problem lies in the way that obdfilter submits IO.  Since it is  
>> not
>> using the normal buffer cache to track "data=ordered" (or in the  
>> case of
>> this patch "data=declared") mode the bio_submit() will likely start
>> modifying the MD device before the corresponding declare blocks are
>> committed to the journal.
>
> Thank you for the detailed explanation, data-path completely escaped
> my mind. Still, on the mdt side, osd goes through the normal VFS paths
> and data=declared should work, right?

Yes, though in general the MDT is a lot smaller than the OSTs, fails  
less often, has RAID-1 instead of RAID-6 so the rebuild goes  
considerably faster, has metadata journaling for everything (so  
doesn't get inconsistent in the first place).

There would likely be some improvement, but we haven't benchmarked it  
- the main concern was for the OSTs.


Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.


From nikita.danilov at clusterstor.com  Wed Dec  2 20:58:39 2009
From: nikita.danilov at clusterstor.com (Nikita Danilov)
Date: Wed, 2 Dec 2009 23:58:39 +0300
Subject: [Lustre-devel] Fwd: Disk rebuild
In-Reply-To: <E9666E71-8BE9-400D-9EA4-445011BB8961@sun.com>
References: <8acda98c0912010700r684bfcf6h94ec347cd6190990@mail.gmail.com>
	<20091201235754.GC4508@clouds>
	<8acda98c0912020613l4cee1a43s3be7ec4f52383304@mail.gmail.com>
	<E9666E71-8BE9-400D-9EA4-445011BB8961@sun.com>
Message-ID: <8acda98c0912021258j140748b1wb2cdb0d2036a864a@mail.gmail.com>

2009/12/2 Andreas Dilger <adilger at sun.com>:
> Hi Nikita!

Hello Andreas!

>
> On 2009-12-02, at 07:13, Nikita Danilov wrote:
>>>
>>> On Tue, Dec 01, 2009 at 06:00:39PM +0300, Nikita Danilov wrote:
>>>>
>>>> what is the status of this? Is ext3 guided resync code (RHEL 5 version
>>>> was posted on lkml in October) used by Lustre?
>>>
>>> This is covered by bug 19932.
>>
>> The last (43rd) comment there is rather intriguing. Can you elaborate
>> on why guided resync cannot work with the Lustre IO stack?
>
>
> The problem lies in the way that obdfilter submits IO.  Since it is not
> using the normal buffer cache to track "data=ordered" (or in the case of
> this patch "data=declared") mode the bio_submit() will likely start
> modifying the MD device before the corresponding declare blocks are
> committed to the journal.

Thank you for the detailed explanation, data-path completely escaped
my mind. Still, on the mdt side, osd goes through the normal VFS paths
and data=declared should work, right?

[...]

>
> Cheers, Andreas
> --
> Andreas Dilger
> Sr. Staff Engineer, Lustre Group
> Sun Microsystems of Canada, Inc.
>
>

Thank you,
Nikita.

From eeb at sun.com  Thu Dec  3 15:00:39 2009
From: eeb at sun.com (Eric Barton)
Date: Thu, 03 Dec 2009 15:00:39 +0000
Subject: [Lustre-devel] client-side reply handling
Message-ID: <027901ca7429$60703d10$2150b730$@com>

Edited from IRC...

> <maxim> eeb_: Liang: btw, shadow complained recently that it's not
>       great that both 'real' and early replies are being stored in
>       the same place. having in mind possible reordering of packets
>       -just a side note - not sure if it's applicable to your
>       discussion
> 
> <eeb_> the idea was to minimize the # of attach operations since
>        that was considered expensive
> 
> <eeb_> also, we'd need another portal or additional matchbits for
>        early replies if we register them separately
> 
> <eeb_> also, the current way of doing it allows the server to send
>        multiple early replies if it wants to
> 
> <eeb_> I'm not so worried about early replies sharing the same ME/MD
>        with the real reply provided....
> 
> <eeb_> 1. The client detects a protocol error if the "early" reply
>           overlaps the "real" reply part of the reply buffer or if
>           the "real" reply overlaps the "early" part of the reply
>           buffer
> 
> <eeb_> 2. Interpretation of replies (both early and real) is safe
>           against the network overwriting them - e.g. early replies
>           are copied out of the buffer before being interpreted and
>           the whole reply buffer is unlinked before the real reply
>           is interpreted in-place
> 
> <Liang> eeb_: do you mean, we can be 100% sure it's safe to unpack
>         in-place only when the buffer is unlinked? so it is better
>         to unregister reply buffer before calling into
>         after_reply()->unpack_reply()?

Yes, I think so.  While the reply buffer remains attached, it's
possible to overwrite it at any time.  This could happen if...

a) The server is buggy or malign

b) The request is re-sent and the same reply matchbits are used,
   which is what I think happens currently for non-bulk reqs.

...so unlikely, but possible.

    Cheers,
              Eric







From adilger at sun.com  Thu Dec  3 21:17:51 2009
From: adilger at sun.com (Andreas Dilger)
Date: Thu, 03 Dec 2009 14:17:51 -0700
Subject: [Lustre-devel] client-side reply handling
In-Reply-To: <027901ca7429$60703d10$2150b730$@com>
References: <027901ca7429$60703d10$2150b730$@com>
Message-ID: <5B2F74A3-0AA9-488B-BE30-312C68B70E60@sun.com>

On 2009-12-03, at 08:00, Eric Barton wrote:
> Edited from IRC...
>>
>> <Liang> eeb_: do you mean, we can be 100% sure it's safe to unpack
>>        in-place only when the buffer is unlinked? so it is better
>>        to unregister reply buffer before calling into
>>        after_reply()->unpack_reply()?
>
> Yes, I think so.  While the reply buffer remains attached, it's
> possible to overwrite it at any time.  This could happen if...
>
> a) The server is buggy or malign

Let's hope we never have to worry about malicious server nodes...

> b) The request is re-sent and the same reply matchbits are used,
>   which is what I think happens currently for non-bulk reqs.

In theory, the reply to the re-sent request should be identical due
to reply reconstruction, so it shouldn't matter if it happens to
overwrite the same buffer.

> ...so unlikely, but possible.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.


From rread at sun.com  Thu Dec  3 22:23:25 2009
From: rread at sun.com (Robert Read)
Date: Thu, 03 Dec 2009 14:23:25 -0800
Subject: [Lustre-devel] client-side reply handling
In-Reply-To: <5B2F74A3-0AA9-488B-BE30-312C68B70E60@sun.com>
References: <027901ca7429$60703d10$2150b730$@com>
	<5B2F74A3-0AA9-488B-BE30-312C68B70E60@sun.com>
Message-ID: <4EDD2ECF-5FC1-4410-9F70-09D07EF1C4FF@sun.com>


On Dec 3, 2009, at 13:17 , Andreas Dilger wrote:

> On 2009-12-03, at 08:00, Eric Barton wrote:
>> Edited from IRC...
>>> 
>>> <Liang> eeb_: do you mean, we can be 100% sure it's safe to unpack
>>>       in-place only when the buffer is unlinked? so it is better
>>>       to unregister reply buffer before calling into
>>>       after_reply()->unpack_reply()?
>> 
>> Yes, I think so.  While the reply buffer remains attached, it's
>> possible to overwrite it at any time.  This could happen if...
>> 
>> a) The server is buggy or malign
> 
> Let's hope we never have to worry about malicious server nodes...
> 
>> b) The request is re-sent and the same reply matchbits are used,
>>  which is what I think happens currently for non-bulk reqs.
> 
> In theory, the reply to the re-sent request should be identical due
> to reply reconstruction, so it shouldn't matter if it happens to
> overwrite the same buffer.

That's fine as long as the buffer is unlinked from the net before it gets swabbed, but hopefully that's the case already. 

robert


From He.Huang at Sun.COM  Fri Dec  4 03:14:20 2009
From: He.Huang at Sun.COM (Isaac Huang)
Date: Thu, 03 Dec 2009 22:14:20 -0500
Subject: [Lustre-devel] client-side reply handling
In-Reply-To: <4EDD2ECF-5FC1-4410-9F70-09D07EF1C4FF@sun.com>
References: <027901ca7429$60703d10$2150b730$@com>
	<5B2F74A3-0AA9-488B-BE30-312C68B70E60@sun.com>
	<4EDD2ECF-5FC1-4410-9F70-09D07EF1C4FF@sun.com>
Message-ID: <20091204031420.GC21620@sun.com>

On Thu, Dec 03, 2009 at 02:23:25PM -0800, Robert Read wrote:
> 
> On Dec 3, 2009, at 13:17 , Andreas Dilger wrote:
> 
> > On 2009-12-03, at 08:00, Eric Barton wrote:
> >> Edited from IRC...
> >>> 
> >>> <Liang> eeb_: do you mean, we can be 100% sure it's safe to unpack
> >>>       in-place only when the buffer is unlinked? so it is better
> >>>       to unregister reply buffer before calling into
> >>>       after_reply()->unpack_reply()?
> >> 
> >> Yes, I think so.  While the reply buffer remains attached, it's
> >> possible to overwrite it at any time.  This could happen if...
> >> 
> >> a) The server is buggy or malign
> > 
> > Let's hope we never have to worry about malicious server nodes...
> > 
> >> b) The request is re-sent and the same reply matchbits are used,
> >>  which is what I think happens currently for non-bulk reqs.
> > 
> > In theory, the reply to the re-sent request should be identical due
> > to reply reconstruction, so it shouldn't matter if it happens to
> > overwrite the same buffer.
> 
> That's fine as long as the buffer is unlinked from the net before it gets swabbed, but hopefully that's the case already. 

I think in-place unpack without unlinking the MD is OK if
LNET_MD_MANAGE_REMOTE is not set for the MD - e.g. request buffers -
because MD offset is maintained locally. However, all reply buffers
seem to have LNET_MD_MANAGE_REMOTE enabled.

Isaac

From Zhen.Liang at Sun.COM  Fri Dec  4 03:30:13 2009
From: Zhen.Liang at Sun.COM (Liang Zhen)
Date: Fri, 04 Dec 2009 11:30:13 +0800
Subject: [Lustre-devel] client-side reply handling
In-Reply-To: <4EDD2ECF-5FC1-4410-9F70-09D07EF1C4FF@sun.com>
References: <027901ca7429$60703d10$2150b730$@com>
	<5B2F74A3-0AA9-488B-BE30-312C68B70E60@sun.com>
	<4EDD2ECF-5FC1-4410-9F70-09D07EF1C4FF@sun.com>
Message-ID: <4B188245.7030503@sun.com>

Robert Read wrote:
>>
>>     
>>> b) The request is re-sent and the same reply matchbits are used,
>>>  which is what I think happens currently for non-bulk reqs.
>>>       
>> In theory, the reply to the re-sent request should be identical due
>> to reply reconstruction, so it shouldn't matter if it happens to
>> overwrite the same buffer.
>>     
>
> That's fine as long as the buffer is unlinked from the net before it gets swabbed, but hopefully that's the case already. 
>   

I'm afraid it's not the case in ptlrpc_check_set(), we begin to swab 
reply when the request is marked as replied and unregister(unlink) reply 
buffer after that, it's the reason Eric brought the topic...

Liang

> robert
>
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-devel
>   


From eeb at sun.com  Fri Dec  4 11:07:42 2009
From: eeb at sun.com (Eric Barton)
Date: Fri, 04 Dec 2009 11:07:42 +0000
Subject: [Lustre-devel] client-side reply handling
In-Reply-To: <5B2F74A3-0AA9-488B-BE30-312C68B70E60@sun.com>
References: <027901ca7429$60703d10$2150b730$@com>
	<5B2F74A3-0AA9-488B-BE30-312C68B70E60@sun.com>
Message-ID: <038601ca74d2$006e1910$014a4b30$@com>

> On 2009-12-03, at 08:00, Eric Barton wrote:
> > Edited from IRC...
> >>
> >> <Liang> eeb_: do you mean, we can be 100% sure it's safe to unpack
> >>        in-place only when the buffer is unlinked? so it is better
> >>        to unregister reply buffer before calling into
> >>        after_reply()->unpack_reply()?
> >
> > Yes, I think so.  While the reply buffer remains attached, it's
> > possible to overwrite it at any time.  This could happen if...
> >
> > a) The server is buggy or malign
> 
> Let's hope we never have to worry about malicious server nodes...

We do.

> > b) The request is re-sent and the same reply matchbits are used,
> >   which is what I think happens currently for non-bulk reqs.
> 
> In theory, the reply to the re-sent request should be identical due
> to reply reconstruction, so it shouldn't matter if it happens to
> overwrite the same buffer.

We're at the mercy of the sender here.  The only robust option is never
to interpret volatile buffers.



From nikita.danilov at clusterstor.com  Fri Dec  4 18:19:27 2009
From: nikita.danilov at clusterstor.com (Nikita Danilov)
Date: Fri, 4 Dec 2009 21:19:27 +0300
Subject: [Lustre-devel] Lustre client I/O
In-Reply-To: <4B1873C7.3060600@sun.com>
References: <8acda98c0912031432j5747b1aaxeb49ec27885fbeaf@mail.gmail.com>
	<4250440D-BD4C-43D8-B619-86AA0F59ED2D@Sun.COM>
	<4B1873C7.3060600@sun.com>
Message-ID: <8acda98c0912041019m7bec896cp71e582325cc9066b@mail.gmail.com>

[Lustre-devel copied.]

Hello,

see comments below.

2009/12/4 jay <Jinshan.Xiong at sun.com>:
> Oleg Drokin wrote:
>>
>> Hello!
>>
>> On Dec 3, 2009, at 5:32 PM, Nikita Danilov wrote:
>>
>>
>>>
>>> Hello Oleg,
>>>
>>> Peter Braam told me you pointed him to some issues with CLIO. Could
>>> you provide details? I am probably owing an explanation of why things
>>> were done in this or that way. May be on lustre-devel?
>>>
>>
>>   Actually I did not dig too much into details myself yet.
>> Jay and WangDi are the best people to ask about specific deficiencies,
>> I think I remember there were mentioned some missed state machine states,
>> then clio takes locks in random order for the same file, so it was
>> deadlock prone.
>> See bug 19906 for some of that. There are many more.
>>
>
> Mostly implementation defects. For 19906, it turns out to be lack of error
> handling at some point, for example, if lock wait fails, we have to
> de-referenced the enqueued sublocks.
>
>> I did some measurements and it is very slow too, over 30% slower than b1_8
>> code
>> for the case of local access.
>>
>
> We've never done any performance tune against clio.
>>
>> Certain questions are raised by Eric Barton if we ever need another cache
>> for compound locks if we already have ldlm cache for locks, since that
>> other
>> cache adds a lot of complications too. And I sort of agree the current
>> clio code
>> looks like a total overkill and very complicated.
>>
>
> Yes, the two-level cached lock mechanism makes cl_lock extremely complex and
> fragile - it can be simplified by removing the top level cache.

I agree that caching top- and bottom- locks adds considerable
complexity. Performance advantages of caching alone probably cannot
justify it. On the other hand, removing top-lock caching only makes
sense when layering is fixed: in a general IO stack layers must have
uniform caching behaviour.

>
> CLIO is still in its childhood, it has never been verified at customer's
> side. It's quite common that we meet new problems when running a new test
> suite. As a matter of fact, most clio bugs were found in new test suites -
> we don't have new issues in these days, because no new test suite is
> introduced.
>
> Before diving into the good and bad side of clio, let's check up what's the
> initiatives of having clio, and whether we have reached those targets or not
> in current implementation. We don't need to focus on those defects in
> implementation.
> This is the design goals of clio:
>
> clear layering;
> controlled state sharing;
> simplified layer interface;
> real stacking;
> reuse of mdt server layering;
> improving portability.

Those are mostly achieved. "Mostly" because few things, like
read-ahead, are still at a wrong level. As an example of advantages of
improved layering, implementation of lock-less IO in CLIO re-uses the
same code paths as a normal caching IO: DLM interaction details are
encapsulated within osc and it is possible to substitute "surrogate
locks" there without rest of the stack noticing. (As a note, CLIO
lock-less IO doesn't currently handle sub-page lockless read as
efficiently as 1.8, because it fetches the whole page from the server
first, but this is easy to fix or maybe it is already fixed.)

> support for:
>  SNS;
>  read-only p2p caching;
>  lock-less IO and ost intents;
>
>> Feel free to add lustre-devel at any point.

One of the more confusing parts of CLIO is its file and stripe lock
implementation, even putting top-lock caching aside for a moment.
There are, I think, two main reasons for this:

    * locks are implemented as non-blocking state machines. This is
rather unusual and definitely not common programming style looking
somewhat inside-out-ish at first sight. The justification for this is
support for a "parallel IO", i.e., a mode where a write, instead of
blocking on a full per-OST cache, proceeds to the next stripe. The
upside is that once non-blocking infrastructure is in place in cl_lock
and cl_io, many other interesting things, like concurrent copy_*_user
and "lock-ahead", are easier to do;

    * concurrency control for cl_lock is fiendishly difficult. I now
think it was a mistake to strive for finer-grained locking. Eric
Barton noted that a lock on a top-object could protect state of all
locks on the object and its sub-objects. This would greatly simplify
things without intolerable decrease in concurrency.

>>
>> Bye,
>>    Oleg
>>

Thank you,
Nikita.

>
>

From yuriy.umanets at clusterstor.com  Sat Dec  5 13:21:47 2009
From: yuriy.umanets at clusterstor.com (Yuriy Umanets)
Date: Sat, 5 Dec 2009 15:21:47 +0200
Subject: [Lustre-devel] loadgen improvements
Message-ID: <3ed212d00912050521l530cefc4oc6c44afce48faaa@mail.gmail.com>

hello Lustre hackers,

ClusterStor would like to start doing real contributions to Lustre stability
and popularity. First off, we would like to improve loadgen utility, written
by Nathan Rutman. We think loadgen is a really good tool for testing and
what is not less important, it is implemented right way from architecture
point of view.

However, these days it has number of issues:
1. Wrong stack size for threads, that results in segfault (find the patch in
attachment);
2. Little locking issues (push_kid() function);
3. Absence of striping functionality, it only may create load on one
OST/ECHO server.

We plan to add striping functionality to loadgen. General idea is to use LOV
module, which already has all the needed code. Loadgen itself and
echo_client also have related code, so it seems this functionality was
planned anyway, though it's not working currently. Currently an attempt to
attach to a LOV results in assertion failures.

Let's discuss these matters. The way we're going to implement this may be
roughly expressed as follows:

1. Attach to LOV device in loadgen, using "device" command. To do so we need
to construct new LOV instance, used by loadgen only, as we cannot use LOV
instance used by LLITE. This requires changes to handling function for
command "device". It should accept more than one OST target;

2. Stripe size and stripe count of new LOV instance should also be specified
while constructing it using "device" command;

3. Setup of virtual clients should take into account that LOV may be used as
a target. In this case attaching dedicated OSCs to created echo_client's is
not needed.

4. In order to be able to run thousands of threads, we need the following:
 - reasonable stack size for threads;
 - change 8192 OBD number limit to something more reasonable.

5. Load itself is created in the way being currently used - via obd_brw().

Thanks.

-- 
umka
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20091205/3f9cc0b6/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: loadgen-stacksize.patch
Type: application/octet-stream
Size: 1194 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20091205/3f9cc0b6/attachment.bin>

From Brian.Reitz at Sun.COM  Tue Dec  8 00:18:25 2009
From: Brian.Reitz at Sun.COM (Brian Reitz)
Date: Mon, 07 Dec 2009 18:18:25 -0600
Subject: [Lustre-devel] Flag Day : Lustre ( b1_6,
 b1_8 and HEAD ) transition to GIT on Dec 14th, 2009
Message-ID: <0EDE0A63-71C6-4757-96E0-7DA104F022C2@Sun.COM>

What is happening?

The Lustre source repository is moving from CVS, where it has lived for the last 7+ years to Git.  This will simplify ongoing development of Lustre in a number of ways, including tracking of changes and history, improved merging of development branches into releases, and offline development.

Unfortunately, due to the lengthy use and abuse of CVS to manage the Lustre code, it is not possible to do a straight import of the entire CVS repository into Git using any existing tools.  Rather than simply importing the current state of the code into Git and lose all of the history, we've worked to import the major development branches along with their full history into a single Git repository to maximize its usefulness.

When is this happening?

Monday December 14 2009 is the target date for switching all Lustre development from CVS to Git. 

All CVS Repositories will transition to read-only state on Friday December 11, 2009 at 9:00 am CST. The CVS repository will remain in this read-only state for the duration of the conversion process.  Please make your plans accordingly.

The conversion to Git will be done over the weekend (12/11/2009 - 12/13/2009).  The resulting Git repository will be available on Monday December 14 2009 at 12:00 PM CST.  The CVS tree will remain in a read-only state from this point on.  All subsequent code changes must be done via Git.

Instructions on using Git with the Lustre code base can be found here: 

http://wiki.lustre.org/index.php/Accessing_Lustre_Code
http://wiki.lustre.org/index.php/Applying_Lustre_Patches_to_a_Kernel
http://wiki.lustre.org/index.php/Submitting_Patches

What is supported?

Following Branches are migrated to GIT

	b1_6
 	b1_8
 	b_release_1_6_7
 	b_release_1_8_0
 	b_release_1_8_1
 	master aka HEAD represents 2.0 work

A limited number of tags are also supported in the git repository.  Only v* tags post 1.6.7 are supported.  

What is not supported?

	b1_4 branch will not be migrated and will be maintained under cvs.

Developer Migration Notes

To make your migration to git easier you should update any of your development CVS branches from the base, and land outstanding work in those development branches before Friday December 11, 2009.  Working directories that contain uncommitted changes should also be updated, but can be done after the migration while the CVS repository is in a read-only mode.
 
Tags will be placed in the CVS repository to mark the point were the conversion was done:

        B1_6_GIT_BASE
        B1_8_GIT_BASE
        B_RELEASE_1_6_7_GIT_BASE
        B_RELEASE_1_8_0_GIT_BASE
        B_RELEASE_1_8_1_GIT_BASE
        B2_0_GIT_BASE

In addition tags will also be in place in the Git repository marking the spot of the conversion:

        GIT_EPOCH_B1_6
        GIT_EPOCH_B1_8
        GIT_EPOCH_B_RELEASE_1_6_7
        GIT_EPOCH_B_RELEASE_1_8_0
        GIT_EPOCH_B_RELEASE_1_8_1
        GIT_EPOCH_MASTER

Support

lustre-devel at sun.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20091207/f7050d4e/attachment.html>

From yuriy.umanets at clusterstor.com  Tue Dec  8 12:27:29 2009
From: yuriy.umanets at clusterstor.com (Yuriy Umanets)
Date: Tue, 8 Dec 2009 14:27:29 +0200
Subject: [Lustre-devel] loadgen improvements
Message-ID: <3ed212d00912080427t184eecf0wf49373cfab555548@mail.gmail.com>

hello Lustre hackers,

ClusterStor would like to start doing real contributions to Lustre stability
and popularity. First off, we would like to improve loadgen utility, written
by Nathan Rutman. We think loadgen is a really good tool for testing and
what is not less important, it is implemented right way from architecture
point of view.

However, these days it has number of issues:
1. Wrong stack size for threads, that results in segfault (find the patch in
attachment);
2. Little locking issues (push_kid() function);
3. Absence of striping functionality, it only may create load on one
OST/ECHO server.

We plan to add striping functionality to loadgen. General idea is to use LOV
module, which already has all the needed code. Loadgen itself and
echo_client also have related code, so it seems this functionality was
planned anyway, though it's not working currently. Currently an attempt to
attach to a LOV results in assertion failures.

Let's discuss these matters. The way we're going to implement this may be
roughly expressed as follows:

1. Attach to LOV device in loadgen, using "device" command. To do so we need
to construct new LOV instance, used by loadgen only, as we cannot use LOV
instance used by LLITE. This requires changes to handling function for
command "device". It should accept more than one OST target;

2. Stripe size and stripe count of new LOV instance should also be specified
while constructing it using "device" command;

3. Setup of virtual clients should take into account that LOV may be used as
a target. In this case attaching dedicated OSCs to created echo_client's is
not needed.

4. In order to be able to run thousands of threads, we need the following:
 - reasonable stack size for threads;
 - change 8192 OBD number limit to something more reasonable.

5. Load itself is created in the way being currently used - via obd_brw().

Thanks.


-- 
umka
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20091208/a730c324/attachment.html>

From yuriy.umanets at clusterstor.com  Tue Dec  8 15:55:40 2009
From: yuriy.umanets at clusterstor.com (Yuriy Umanets)
Date: Tue, 8 Dec 2009 17:55:40 +0200
Subject: [Lustre-devel] test - please ignore this email
Message-ID: <3ed212d00912080755ib2be0e0u4b0aa1298caa9cf0@mail.gmail.com>

-- 
umka
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20091208/18135442/attachment.html>

From rread at sun.com  Tue Dec  8 18:38:01 2009
From: rread at sun.com (Robert Read)
Date: Tue, 08 Dec 2009 10:38:01 -0800
Subject: [Lustre-devel] loadgen improvements
In-Reply-To: <3ed212d00912050521l530cefc4oc6c44afce48faaa@mail.gmail.com>
References: <3ed212d00912050521l530cefc4oc6c44afce48faaa@mail.gmail.com>
Message-ID: <21235BEC-1B37-49E5-ADB2-885B154DCB4C@sun.com>

Hi Yury,

Sounds interesting; I'm happy to see loadgen will be improved.  Please create a bug in bugzilla.lustre.org and attach your patch so we can add it to the queue. 

thanks,
robert

On Dec 5, 2009, at 05:21 , Yuriy Umanets wrote:

> hello Lustre hackers,
> 
> ClusterStor would like to start doing real contributions to Lustre stability and popularity. First off, we would like to improve loadgen utility, written by Nathan Rutman. We think loadgen is a really good tool for testing and what is not less important, it is implemented right way from architecture point of view.
> 
> However, these days it has number of issues:
> 1. Wrong stack size for threads, that results in segfault (find the patch in attachment);
> 2. Little locking issues (push_kid() function);
> 3. Absence of striping functionality, it only may create load on one OST/ECHO server.
> 
> We plan to add striping functionality to loadgen. General idea is to use LOV module, which already has all the needed code. Loadgen itself and echo_client also have related code, so it seems this functionality was planned anyway, though it's not working currently. Currently an attempt to attach to a LOV results in assertion failures.
> 
> Let's discuss these matters. The way we're going to implement this may be roughly expressed as follows:
> 
> 1. Attach to LOV device in loadgen, using "device" command. To do so we need to construct new LOV instance, used by loadgen only, as we cannot use LOV instance used by LLITE. This requires changes to handling function for command "device". It should accept more than one OST target;
> 
> 2. Stripe size and stripe count of new LOV instance should also be specified while constructing it using "device" command;
> 
> 3. Setup of virtual clients should take into account that LOV may be used as a target. In this case attaching dedicated OSCs to created echo_client's is not needed.
> 
> 4. In order to be able to run thousands of threads, we need the following:
>  - reasonable stack size for threads;
>  - change 8192 OBD number limit to something more reasonable.
> 
> 5. Load itself is created in the way being currently used - via obd_brw().
> 
> Thanks.
> 
> -- 
> umka
> <loadgen-stacksize.patch>_______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-devel


From adilger at sun.com  Tue Dec  8 19:10:52 2009
From: adilger at sun.com (Andreas Dilger)
Date: Tue, 08 Dec 2009 12:10:52 -0700
Subject: [Lustre-devel] loadgen improvements
In-Reply-To: <3ed212d00912050521l530cefc4oc6c44afce48faaa@mail.gmail.com>
References: <3ed212d00912050521l530cefc4oc6c44afce48faaa@mail.gmail.com>
Message-ID: <5250C925-EDB4-48FA-AD3C-AC37B212BB2B@sun.com>

Hello Yuriy,

On 2009-12-05, at 06:21, Yuriy Umanets wrote:
> ClusterStor would like to start doing real contributions to Lustre  
> stability and popularity. First off, we would like to improve  
> loadgen utility, written by Nathan Rutman. We think loadgen is a  
> really good tool for testing and what is not less important, it is  
> implemented right way from architecture point of view.

Great.  We look forward to working with you again.

> However, these days it has number of issues:
> 1. Wrong stack size for threads, that results in segfault (find the  
> patch in attachment);
> 2. Little locking issues (push_kid() function);
> 3. Absence of striping functionality, it only may create load on one  
> OST/ECHO server.

Can you please file a bug for this, and attach this patch and later  
ones there.  That will ensure that it follows the proper inspection  
and testing process.

> We plan to add striping functionality to loadgen. General idea is to  
> use LOV module, which already has all the needed code. Loadgen  
> itself and echo_client also have related code, so it seems this  
> functionality was planned anyway, though it's not working currently.  
> Currently an attempt to attach to a LOV results in assertion failures.

Right - Eric and I worked on that some years ago, but it was never  
used fully.  Can you please ensure that you include a test in  
sanity.sh that exercises the functionality you are adding to loadgen,  
as with echo_client in test 180.  More complex load tests should go  
into their own script.

> Let's discuss these matters. The way we're going to implement this  
> may be roughly expressed as follows:
>
> 1. Attach to LOV device in loadgen, using "device" command. To do so  
> we need to construct new LOV instance, used by loadgen only, as we  
> cannot use LOV instance used by LLITE. This requires changes to  
> handling function for command "device". It should accept more than  
> one OST target;

This sounds reasonable.  It might be useful to support the wildcard  
specification of OSTs like "lustre-OST00[0-30]" or "lustre- 
OST00[0,3,6,9].  Some of that functionality already exists in lustre/ 
utils/nidlist.c.

> 2. Stripe size and stripe count of new LOV instance should also be  
> specified while constructing it using "device" command;

It probably makes sense to have this specified with a separate  
command, so that these parameters can be changed without having to  
tear down the devices and recreate them just to change the striping,  
and it avoids overloading the "device" command (which will soon become  
much more complex by allowing many OSTs to be specified).

> 3. Setup of virtual clients should take into account that LOV may be  
> used as a target. In this case attaching dedicated OSCs to created  
> echo_client's is not needed.
>
> 4. In order to be able to run thousands of threads, we need the  
> following:
>  - reasonable stack size for threads;
>  - change 8192 OBD number limit to something more reasonable.

If you are going beyond 8192 OBD devices just for the loadgen usage  
(presumably you need a new LOV device for each thread?) then it would  
make sense to have the obd_devs array be dynamically allocated and  
remove any upper limit on the number of devices.

Most of this work has already been done - all of the access to  
obd_devs[] is localized in lustre/obdclass/genops.c, with the  
exception of the array initialization, and an old  
EXPORT_SYMBOL(obd_devs), which isn't used anymore. The obd_devs[]  
declaration can be made static to genops.c and the initialization  
moved to obd_init_caches(), and then the actual representation of  
obd_devs[] can be changed without affecting the interface.

Note that the index number of the device is still exported to  
userspace, so it isn't possible/practical to simply go with a linked- 
list implementation.  The actual obd_device structures are already  
allocated and freed dynamically, just the master obd_devs[] array is  
static.  Probably a master array of pointers to pages that hold  
segments of the the obd_devs[] array in order is sufficient.  The  
master array will be quite small (1 pointer per 512 obd_devices) and  
can be reallocated under a rwlock, since nothing references it  
directly.  There is no need to be able to shrink the master array.

It would be trivial to also make the obd_types and obd_types_lock  
declarations static to genops.c, and the initialization in  
obd_init_caches() to properly isolate that code as well.

> 5. Load itself is created in the way being currently used - via  
> obd_brw().


Actually, the real code does not use obd_brw() directly anymore, but  
rather obd_brw_async().  I think Nic Henke submitted a patch to change  
that, I'm not sure of the current status of that patch.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.


From nic at cray.com  Tue Dec  8 20:01:51 2009
From: nic at cray.com (Nic Henke)
Date: Tue, 08 Dec 2009 14:01:51 -0600
Subject: [Lustre-devel] loadgen improvements
In-Reply-To: <5250C925-EDB4-48FA-AD3C-AC37B212BB2B@sun.com>
References: <3ed212d00912050521l530cefc4oc6c44afce48faaa@mail.gmail.com>
	<5250C925-EDB4-48FA-AD3C-AC37B212BB2B@sun.com>
Message-ID: <4B1EB0AF.4060601@cray.com>

Andreas Dilger wrote:
> Hello Yuriy,
>
>   
>> 5. Load itself is created in the way being currently used - via  
>> obd_brw().
>>     
>
>
> Actually, the real code does not use obd_brw() directly anymore, but  
> rather obd_brw_async().  I think Nic Henke submitted a patch to change  
> that, I'm not sure of the current status of that patch.
>
>   
This is Bug 18225 - looks like it was landed to 1.6.8+ and 1.8.1+, but 
was dropped from HEAD as there is more work that'd need to be done there.

Nic

From adilger at sun.com  Wed Dec  9 06:35:32 2009
From: adilger at sun.com (Andreas Dilger)
Date: Tue, 08 Dec 2009 23:35:32 -0700
Subject: [Lustre-devel] loadgen improvements
In-Reply-To: <1260335244.9521.114.camel@berloga.shadowland>
References: <3ed212d00912050521l530cefc4oc6c44afce48faaa@mail.gmail.com>
	<5250C925-EDB4-48FA-AD3C-AC37B212BB2B@sun.com>
	<1260335244.9521.114.camel@berloga.shadowland>
Message-ID: <20C3E301-9A42-47D9-A1A7-BA06814D396F@sun.com>

On 2009-12-08, at 22:07, Alexey Lyashkov wrote:
> This can be something similar as
>>
>     pattern $name $pool_name $operation [$stripe_size [$stripe_count]]
>
> where
> name - is pattern name
> pool_name - is name of OST pools assigned to this pattern
> operation - is one of READ or WRITE.
> stripe_size - if set, workload need to setup LOV and add linkage  
> between
> LOV and OSC targets, and separate single operation to stripes.
> stripe_count - if set, workload should be use for only part of OST  
> pool
> to each echo object.

Please do not use positional parameters.  That makes it nearly  
impossible to change the input format in the future, or to skip some  
fields and leave them as defaults.  This needs to use getopt() and  
named parameters:

  --pool=pool_name, --stripe_size=N,  --stripe_count=N

> next stage is prepare clients.
> In this stage we should be send obd_connect to echo client and wait
> until all osc targets is connected (via send obd_statfs() or something
> similar).
> To start clients we can uses:
>>
>     clients $name $count $workload_name [shared]
> where
> name - is client group name
> count - is client's count in that group
> workload_name - is name of workload parameters.
> shared - if set, clients in this pattern is share single OSC or LOV
> target. if don't set - each worker have own lustre stack.
> At this stage we also create echo objects with requested LSM for each
> client.

Same as above - please use named parameters instead of positional ones.

It probably also makes sense to look at the functions provided by LST  
(LNET self test) so that we might have these two tools operate in a  
similar manner. LST testing at the LNET level, loadgen testing at the  
OBD/CLIO level.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.


From yuriy.umanets at clusterstor.com  Wed Dec  9 13:28:40 2009
From: yuriy.umanets at clusterstor.com (Yuriy Umanets)
Date: Wed, 9 Dec 2009 15:28:40 +0200
Subject: [Lustre-devel] loadgen improvements
In-Reply-To: <5250C925-EDB4-48FA-AD3C-AC37B212BB2B@sun.com>
References: <3ed212d00912050521l530cefc4oc6c44afce48faaa@mail.gmail.com>
	<5250C925-EDB4-48FA-AD3C-AC37B212BB2B@sun.com>
Message-ID: <3ed212d00912090528j75a99bdby77d8e293bb58def@mail.gmail.com>

On Tue, Dec 8, 2009 at 9:10 PM, Andreas Dilger <adilger at sun.com> wrote:

> Hello Yuriy,
>
>
> On 2009-12-05, at 06:21, Yuriy Umanets wrote:
>
>> ClusterStor would like to start doing real contributions to Lustre
>> stability and popularity. First off, we would like to improve loadgen
>> utility, written by Nathan Rutman. We think loadgen is a really good tool
>> for testing and what is not less important, it is implemented right way from
>> architecture point of view.
>>
>
> Great.  We look forward to working with you again.
>
> Thank you all guys.

>
>  However, these days it has number of issues:
>> 1. Wrong stack size for threads, that results in segfault (find the patch
>> in attachment);
>> 2. Little locking issues (push_kid() function);
>> 3. Absence of striping functionality, it only may create load on one
>> OST/ECHO server.
>>
>
> Can you please file a bug for this, and attach this patch and later ones
> there.  That will ensure that it follows the proper inspection and testing
> process.


Sure, will do.

>
>
>  We plan to add striping functionality to loadgen. General idea is to use
>> LOV module, which already has all the needed code. Loadgen itself and
>> echo_client also have related code, so it seems this functionality was
>> planned anyway, though it's not working currently. Currently an attempt to
>> attach to a LOV results in assertion failures.
>>
>
> Right - Eric and I worked on that some years ago, but it was never used
> fully.  Can you please ensure that you include a test in sanity.sh that
> exercises the functionality you are adding to loadgen, as with echo_client
> in test 180.  More complex load tests should go into their own script.
>
> Ok

>
>  Let's discuss these matters. The way we're going to implement this may be
>> roughly expressed as follows:
>>
>> 1. Attach to LOV device in loadgen, using "device" command. To do so we
>> need to construct new LOV instance, used by loadgen only, as we cannot use
>> LOV instance used by LLITE. This requires changes to handling function for
>> command "device". It should accept more than one OST target;
>>
>
> This sounds reasonable.  It might be useful to support the wildcard
> specification of OSTs like "lustre-OST00[0-30]" or "lustre-OST00[0,3,6,9].
>  Some of that functionality already exists in lustre/utils/nidlist.c.


Looks reasonable. We can even use "dl" command output to validate the names
and prevent using not existing local obd names.

>
>
>  2. Stripe size and stripe count of new LOV instance should also be
>> specified while constructing it using "device" command;
>>
>
> It probably makes sense to have this specified with a separate command, so
> that these parameters can be changed without having to tear down the devices
> and recreate them just to change the striping, and it avoids overloading the
> "device" command (which will soon become much more complex by allowing many
> OSTs to be specified).


Tend to agree with Shadow's idea to use something like a "pool" command
instead of --stripe_count to minimize changes.

>
>
>  3. Setup of virtual clients should take into account that LOV may be used
>> as a target. In this case attaching dedicated OSCs to created echo_client's
>> is not needed.
>>
>> 4. In order to be able to run thousands of threads, we need the following:
>>  - reasonable stack size for threads;
>>  - change 8192 OBD number limit to something more reasonable.
>>
>
> If you are going beyond 8192 OBD devices just for the loadgen usage
> (presumably you need a new LOV device for each thread?) then it would make
> sense to have the obd_devs array be dynamically allocated and remove any
> upper limit on the number of devices.
>
> Event for current functionality, which we want to preserve, when we have 2
OBDs per thread (echo_client and its OSC) this limit will not allow us to
start more than ˜4K threads. I think our goal should be at least 100K of
them.


> Most of this work has already been done - all of the access to obd_devs[]
> is localized in lustre/obdclass/genops.c, with the exception of the array
> initialization, and an old EXPORT_SYMBOL(obd_devs), which isn't used
> anymore. The obd_devs[] declaration can be made static to genops.c and the
> initialization moved to obd_init_caches(), and then the actual
> representation of obd_devs[] can be changed without affecting the interface.
>
Agreed.

>
> Note that the index number of the device is still exported to userspace, so
> it isn't possible/practical to simply go with a linked-list implementation.
>  The actual obd_device structures are already allocated and freed
> dynamically, just the master obd_devs[] array is static.  Probably a master
> array of pointers to pages that hold segments of the the obd_devs[] array in
> order is sufficient.  The master array will be quite small (1 pointer per
> 512 obd_devices) and can be reallocated under a rwlock, since nothing
> references it directly.  There is no need to be able to shrink the master
> array.
>
> It would be trivial to also make the obd_types and obd_types_lock
> declarations static to genops.c, and the initialization in obd_init_caches()
> to properly isolate that code as well.
>
> Ok

>
>  5. Load itself is created in the way being currently used - via obd_brw().
>>
>
>
> Actually, the real code does not use obd_brw() directly anymore, but rather
> obd_brw_async().  I think Nic Henke submitted a patch to change that, I'm
> not sure of the current status of that patch.
>
> Ok.

Thanks!

> Cheers, Andreas
> --
> Andreas Dilger
> Sr. Staff Engineer, Lustre Group
> Sun Microsystems of Canada, Inc.
>
>


-- 
umka
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20091209/844d67ad/attachment.html>

From yuriy.umanets at clusterstor.com  Wed Dec  9 13:42:53 2009
From: yuriy.umanets at clusterstor.com (Yuriy Umanets)
Date: Wed, 9 Dec 2009 15:42:53 +0200
Subject: [Lustre-devel] loadgen improvements
In-Reply-To: <5250C925-EDB4-48FA-AD3C-AC37B212BB2B@sun.com>
References: <3ed212d00912050521l530cefc4oc6c44afce48faaa@mail.gmail.com>
	<5250C925-EDB4-48FA-AD3C-AC37B212BB2B@sun.com>
Message-ID: <3ed212d00912090542t1bca852er135ed80238921699@mail.gmail.com>

On Tue, Dec 8, 2009 at 9:10 PM, Andreas Dilger <adilger at sun.com> wrote:

> Hello Yuriy,
>


> Can you please file a bug for this, and attach this patch and later ones
> there.  That will ensure that it follows the proper inspection and testing
> process.
>
>
> Bug https://bugzilla.lustre.org/show_bug.cgi?id=21571 is created. Please
attach it to right tracker. I'm not aware of any valid these days.

Nathan, Andreas, I requested your inspections there. Hope you don't mind.

Thanks.


>
>> --
umka
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20091209/a5305147/attachment.html>

From adilger at sun.com  Thu Dec 10 07:33:25 2009
From: adilger at sun.com (Andreas Dilger)
Date: Thu, 10 Dec 2009 00:33:25 -0700
Subject: [Lustre-devel] loadgen improvements
In-Reply-To: <1260341830.9521.132.camel@berloga.shadowland>
References: <3ed212d00912050521l530cefc4oc6c44afce48faaa@mail.gmail.com>
	<5250C925-EDB4-48FA-AD3C-AC37B212BB2B@sun.com>
	<1260335244.9521.114.camel@berloga.shadowland>
	<20C3E301-9A42-47D9-A1A7-BA06814D396F@sun.com>
	<1260341830.9521.132.camel@berloga.shadowland>
Message-ID: <5F06E9CF-6386-4809-A6B1-1DA61502EC20@sun.com>

On 2009-12-08, at 23:57, Alexey Lyashkov wrote:
> On Tue, 2009-12-08 at 23:35 -0700, Andreas Dilger wrote:
>> On 2009-12-08, at 22:07, Alexey Lyashkov wrote:
>>> This can be something similar as
>>
>>>
>>>    pattern $name $pool_name $operation [$stripe_size  
>>> [$stripe_count]]
>>
>> Please do not use positional parameters.  That makes it nearly
>> impossible to change the input format in the future, or to skip some
>> fields and leave them as defaults.  This needs to use getopt() and
>> named parameters:
>>
>>  --pool=pool_name, --stripe_size=N,  --stripe_count=N
>
> This possible, but unlikely. because need will be save current pool
> name, instead have parsed as single command.
>
> And I don't have plan to rewrite loadgen to use getopt, instead of
> current Parser() function which is used in lctl, lfs, and other.

Note that the existing parser function passes argc, argv to the helper  
function, which is free to use getopt() or getopt_long() internally as  
desired.  See, for example, lustre/utils/lfs.c::lfs_setstripe(), which  
handles the parsing of "lfs setstripe --count N --pool {pool} ...".

> How about this style
>>
> pool $name { targets:"OST1, OST2" }
> pattern $pat { stripe_count:X stripe_size:Y block_size:Z };
> clients $pattern { count:XX pattern: pat shared }


I have no objection to splitting up the functionality into multiple  
sub-commands.  Allowing declarations of pools and patterns that are re- 
used for each client definition makes a lot of sense.

What I dislike is the introduction of a completely new input format  
and parsing engine for no apparent benefit.  Why not change the above  
to:

loadgen> pool --targets="OST1,OST2,..." NAME
loadgen> pattern --stripe_count=X --stripe_size=Y --block_size=Z PATTERN
loadgen> clients --count=XX --pattern=PATTERN ...

and use well-known input conventions?

It also makes sense to look at how LNET Self Test handles the setup of  
testing loads, and try and make the loadgen usage similar to that, as  
much as possible?

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.


From alexey.lyashkov at clusterstor.com  Wed Dec  9 05:16:49 2009
From: alexey.lyashkov at clusterstor.com (Alexey Lyashkov)
Date: Wed, 09 Dec 2009 08:16:49 +0300
Subject: [Lustre-devel] loadgen improvements
In-Reply-To: <5250C925-EDB4-48FA-AD3C-AC37B212BB2B@sun.com>
References: <3ed212d00912050521l530cefc4oc6c44afce48faaa@mail.gmail.com>
	<5250C925-EDB4-48FA-AD3C-AC37B212BB2B@sun.com>
Message-ID: <1260335809.9521.115.camel@berloga.shadowland>

Hi Andreas,

> 
> > However, these days it has number of issues:
> > 1. Wrong stack size for threads, that results in segfault (find
the  
> > patch in attachment);
> > 2. Little locking issues (push_kid() function);
> > 3. Absence of striping functionality, it only may create load on
one  
> > OST/ECHO server.
> 
> Can you please file a bug for this, and attach this patch and later  
> ones there.  That will ensure that it follows the proper inspection  
> and testing process.
Sure, but looks this will be complete rewrite loadgen.
Reasons for that - loadgen should be use jt functions to set,get lprocfs
data, should be add LOV targets into LOV instance.


> > Let's discuss these matters. The way we're going to implement this  
> > may be roughly expressed as follows:
> >
> > 1. Attach to LOV device in loadgen, using "device" command. To do
so  
> > we need to construct new LOV instance, used by loadgen only, as we  
> > cannot use LOV instance used by LLITE. This requires changes to  
> > handling function for command "device". It should accept more than  
> > one OST target;
> 
> This sounds reasonable.  It might be useful to support the wildcard  
> specification of OSTs like "lustre-OST00[0-30]" or "lustre- 
> OST00[0,3,6,9].  Some of that functionality already exists in lustre/ 
> utils/nidlist.c.
As we need fully setup lustre stack to testing, we can't use willcard in
'device' command, because we don't have access to MGS at this case and
don't have info about all OST tagets in cluster.
But we can use this in 'pool' command.

> > 2. Stripe size and stripe count of new LOV instance should also be  
> > specified while constructing it using "device" command;
> 
> It probably makes sense to have this specified with a separate  
> command, so that these parameters can be changed without having to  
> tear down the devices and recreate them just to change the striping,  
> and it avoids overloading the "device" command (which will soon
become  
> much more complex by allowing many OSTs to be specified).


in first we need call many "device" commands - which add linkage between
UUID and NID.

second step we define OST pool for each workload pattern, this can be
done via new added command 
>>>> pool $name $coma_sepated_OST_uuid
this command used to make LOV_ADD_TARGET commands, but later can be
translated into real OST pool, if need. Main goals of this, add
possibility to use different OST's in different LOV instance.


next step is define workload pattern. currently LoadGen is support only
write, or read command - this translated to obd_brw() or obd_brw_async.
But echo client have support for 3 types of IO.
1) simulate obd_brw/obd_brw_async.
2) simulate obd_queue_async_page()
3) looks direct connect to obdfilter to use obd_prep() & obd_commit()
calls.

Also we need extend this pattern to use OST pool and striping
parameters.
This can be something similar as
>>> pattern $name $pool_name $operation [$stripe_size [$stripe_count]]
where 
name - is pattern name
pool_name - is name of OST pools assigned to this pattern
operation - is one of READ or WRITE.
stripe_size - if set, workload need to setup LOV and add linkage between
LOV and OSC targets, and separate single operation to stripes.
stripe_count - if set, workload should be use for only part of OST pool
to each echo object.

next stage is prepare clients.
In this stage we should be send obd_connect to echo client and wait
until all osc targets is connected (via send obd_statfs() or something
similar).
To start clients we can uses:
>>>>  clients $name $count $workload_name [shared]
where 
name - is client group name
count - is client's count in that group
workload_name - is name of workload parameters.
shared - if set, clients in this pattern is share single OSC or LOV
target. if don't set - each worker have own lustre stack.  
At this stage we also create echo objects with requested LSM for each
client.

final step is spawn one thread for one client and client read own
pattern to call echo client obd IOCTL to start load.


-- 
Alexey Lyashkov <alexey.lyashkov at clusterstor.com>
ClusterStor



From alexey.lyashkov at clusterstor.com  Wed Dec  9 06:57:10 2009
From: alexey.lyashkov at clusterstor.com (Alexey Lyashkov)
Date: Wed, 09 Dec 2009 09:57:10 +0300
Subject: [Lustre-devel] loadgen improvements
In-Reply-To: <20C3E301-9A42-47D9-A1A7-BA06814D396F@sun.com>
References: <3ed212d00912050521l530cefc4oc6c44afce48faaa@mail.gmail.com>
	<5250C925-EDB4-48FA-AD3C-AC37B212BB2B@sun.com>
	<1260335244.9521.114.camel@berloga.shadowland>
	<20C3E301-9A42-47D9-A1A7-BA06814D396F@sun.com>
Message-ID: <1260341830.9521.132.camel@berloga.shadowland>

On Tue, 2009-12-08 at 23:35 -0700, Andreas Dilger wrote:
> On 2009-12-08, at 22:07, Alexey Lyashkov wrote:
> > This can be something similar as
> >>
> >     pattern $name $pool_name $operation [$stripe_size [$stripe_count]]
> >
> > where
> > name - is pattern name
> > pool_name - is name of OST pools assigned to this pattern
> > operation - is one of READ or WRITE.
> > stripe_size - if set, workload need to setup LOV and add linkage  
> > between
> > LOV and OSC targets, and separate single operation to stripes.
> > stripe_count - if set, workload should be use for only part of OST  
> > pool
> > to each echo object.
> 
> Please do not use positional parameters.  That makes it nearly  
> impossible to change the input format in the future, or to skip some  
> fields and leave them as defaults.  This needs to use getopt() and  
> named parameters:
> 
>   --pool=pool_name, --stripe_size=N,  --stripe_count=N
This possible, but unlikely. because need will be save current pool
name, instead have parsed as single command.

And I don't have plan to rewrite loadgen to use getopt, instead of
current Parser() function which is used in lctl, lfs, and other.
But i agree about avoid position depended parameters.

How about this style
>>> pool $name { targets:"OST1, OST2" }
and
>>> pattern $pat { stripe_count:X stripe_size:Y block_size:Z };
and 
>>> clients $pattern { count:XX pattern: pat shared }
?


-- 
Alexey Lyashkov <alexey.lyashkov at clusterstor.com>
ClusterStor


From peter.braam at clusterstor.com  Wed Dec  9 15:05:03 2009
From: peter.braam at clusterstor.com (Peter Braam)
Date: Wed, 9 Dec 2009 08:05:03 -0700
Subject: [Lustre-devel] client side reply handling
Message-ID: <2b20c7140912090705s47667951w26fe6c95e81f2c8f@mail.gmail.com>

Hi,

I wonder if LNET doesn't have an atomic operation that unlinks the packet
from the delivery process upon receiving a packet.   Iirc it does.  Why
wouldn't one use that?

Peter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20091209/27c0c4d8/attachment.html>

From peter.braam at clusterstor.com  Thu Dec 10 17:06:13 2009
From: peter.braam at clusterstor.com (Peter Braam)
Date: Thu, 10 Dec 2009 10:06:13 -0700
Subject: [Lustre-devel] loadgen improvements
In-Reply-To: <5F06E9CF-6386-4809-A6B1-1DA61502EC20@sun.com>
References: <3ed212d00912050521l530cefc4oc6c44afce48faaa@mail.gmail.com>
	<5250C925-EDB4-48FA-AD3C-AC37B212BB2B@sun.com>
	<1260335244.9521.114.camel@berloga.shadowland>
	<20C3E301-9A42-47D9-A1A7-BA06814D396F@sun.com>
	<1260341830.9521.132.camel@berloga.shadowland>
	<5F06E9CF-6386-4809-A6B1-1DA61502EC20@sun.com>
Message-ID: <2b20c7140912100906r133305e1p350b95b9497bc5cf@mail.gmail.com>

My message to lustre-devel will be held by the moderator for ever it seems
(13 days ago I sent the first one).

The "standard" conventions have led to an extremely limited set of
benchmarks being executed, of questionable value.

We need expressiveness that is sufficient to also describe realistic
scenarios, like out of order or biased delivery of requests (such was shown
to be critical at ORNL), mixing in smaller and bigger I/O.  I believe what
Umka and Alexey are showing is the beginning of the language we are
developing for this.

peter

On Thu, Dec 10, 2009 at 12:33 AM, Andreas Dilger <adilger at sun.com> wrote:

> On 2009-12-08, at 23:57, Alexey Lyashkov wrote:
> > On Tue, 2009-12-08 at 23:35 -0700, Andreas Dilger wrote:
> >> On 2009-12-08, at 22:07, Alexey Lyashkov wrote:
> >>> This can be something similar as
> >>
> >>>
> >>>    pattern $name $pool_name $operation [$stripe_size
> >>> [$stripe_count]]
> >>
> >> Please do not use positional parameters.  That makes it nearly
> >> impossible to change the input format in the future, or to skip some
> >> fields and leave them as defaults.  This needs to use getopt() and
> >> named parameters:
> >>
> >>  --pool=pool_name, --stripe_size=N,  --stripe_count=N
> >
> > This possible, but unlikely. because need will be save current pool
> > name, instead have parsed as single command.
> >
> > And I don't have plan to rewrite loadgen to use getopt, instead of
> > current Parser() function which is used in lctl, lfs, and other.
>
> Note that the existing parser function passes argc, argv to the helper
> function, which is free to use getopt() or getopt_long() internally as
> desired.  See, for example, lustre/utils/lfs.c::lfs_setstripe(), which
> handles the parsing of "lfs setstripe --count N --pool {pool} ...".
>
> > How about this style
> >>
> > pool $name { targets:"OST1, OST2" }
> > pattern $pat { stripe_count:X stripe_size:Y block_size:Z };
> > clients $pattern { count:XX pattern: pat shared }
>
>
> I have no objection to splitting up the functionality into multiple
> sub-commands.  Allowing declarations of pools and patterns that are re-
> used for each client definition makes a lot of sense.
>
> What I dislike is the introduction of a completely new input format
> and parsing engine for no apparent benefit.  Why not change the above
> to:
>
> loadgen> pool --targets="OST1,OST2,..." NAME
> loadgen> pattern --stripe_count=X --stripe_size=Y --block_size=Z PATTERN
> loadgen> clients --count=XX --pattern=PATTERN ...
>
> and use well-known input conventions?
>
> It also makes sense to look at how LNET Self Test handles the setup of
> testing loads, and try and make the loadgen usage similar to that, as
> much as possible?
>
> Cheers, Andreas
> --
> Andreas Dilger
> Sr. Staff Engineer, Lustre Group
> Sun Microsystems of Canada, Inc.
>
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20091210/b9b18e1b/attachment.html>

From jsimmons at infradead.org  Fri Dec 11 19:27:14 2009
From: jsimmons at infradead.org (James Simmons)
Date: Fri, 11 Dec 2009 19:27:14 +0000 (GMT)
Subject: [Lustre-devel] no reformating for test suite
Message-ID: <alpine.LFD.2.00.0912111926080.2668@casper.infradead.org>


Is there a way to stop the lustre test suite from reformating the disk 
when we run our test?

From adilger at sun.com  Fri Dec 11 23:21:10 2009
From: adilger at sun.com (Andreas Dilger)
Date: Fri, 11 Dec 2009 16:21:10 -0700
Subject: [Lustre-devel] no reformating for test suite
In-Reply-To: <alpine.LFD.2.00.0912111926080.2668@casper.infradead.org>
References: <alpine.LFD.2.00.0912111926080.2668@casper.infradead.org>
Message-ID: <9CA5FD93-CDC1-4394-B57B-E56A99DA09C8@sun.com>

On 2009-12-11, at 12:27, James Simmons wrote:
> Is there a way to stop the lustre test suite from reformating the disk
> when we run our test?


What version of Lustre are you testing?  In recent releases (the last  
9-12 months or so) it doesn't reformat the filesystem by default.

I _think_ (but don't recall for sure anymore) that it is also possible  
to specify REFORMAT="" so that it will not reformat.

Some tests, like conf-sanity.sh depend on being able to reformat, and  
can be run with a local config that just uses loopback files.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.


From peter.braam at clusterstor.com  Sun Dec 13 18:47:37 2009
From: peter.braam at clusterstor.com (Peter Braam)
Date: Sun, 13 Dec 2009 11:47:37 -0700
Subject: [Lustre-devel] Fwd: The Lustre community and GIT
In-Reply-To: <2b20c7140912130831t658c2790hb38c3752c18ec128@mail.gmail.com>
References: <2b20c7140912130831t658c2790hb38c3752c18ec128@mail.gmail.com>
Message-ID: <2b20c7140912131047qf0417f1mf3f3d9a8fc751dc0@mail.gmail.com>

Brian Reitz sent an email about GIT, that raises some questions, and I would
like to initiate some discussion here how we can reach the best
organization.

1. We need a public repository, where non Sun community members can commit.

a.  There are Lustre users than have their own branches.  LLNL has probably
the best tested branch among all users.  DDN's customers have a version of
Lustre that includes patches not yet in Sun's releases.  It would be very
valuable if these kind of releases could be collected in a publicly
accessible GIT repository.

I doubt that Sun will give commit rights to external entities (this is not
unreasonable, Sun needs to control what code enters its repositories).
 Hence I think that the community would be better served with a GIT
repository in a public place, like github, that can give such access.

My group at ClusterStor has reserved the "lustre" project at Github and we
will give keys to any and all organizations that wish to make serious
contributions to Lustre.  I was in particular hoping that LLNL would be
willing to commit their releases to that public place.

b.  Everyone is uncertain about what Sun will do with Lustre (proprietary
Solaris / ZFS server releases and discontinued support for Linux servers
have been mentioned to me several times now).  A public repository with the
open code will be valuable for the community and promote continued
development and access.


2. We need MUCH more in the repository than Sun is putting into it.

There are many development branches and sometimes abandoned projects that
still have a lot of value.  For example, there is a nearly complete OS X
client port - what if someone wanted to pick that up?  Similarly, there are
projects like size on MDS or the network request scheduler that may need
help from the community to get finished or re-prioritized.

It is unclear to me if these kind of branches can be found in the publicly
available CVS.  If they can, a collection of relevant branches, broadly
along the lines of what I mention above, should be placed in the public GIT
repository.

3. Brian's email message seems to indicate that Sun's git repository will be
used for Sun development.  In the past there were two CVS repositories - a
read-only one that was publicly accessible and when I last controlled the
group it was updated very frequently with all open source code (presently,
it seems to only contain releases, not the development stream of commits).
It is unclear how Sun can manage development with this git repository given
that parts of its work are proprietary (like the Windows port) or unreleased
(like the ZFS work).  Can we get more insight in what Brian is alluding to?


I am sure other community members have further suggestions, and I hope we
can discuss these here.  If there is support for github as a good location,
please let me know and we will give participants the keys.  In particular I
would like to publicly request that community members like LLNL that have
quite possibly the highest quality releases, place them in github or publish
another location where these can be found.

Thank you very much for your thoughts.

Peter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20091213/e10f165a/attachment.html>

From jsimmons at infradead.org  Mon Dec 14 20:18:20 2009
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 14 Dec 2009 20:18:20 +0000 (GMT)
Subject: [Lustre-devel] no reformating for test suite
In-Reply-To: <9CA5FD93-CDC1-4394-B57B-E56A99DA09C8@sun.com>
References: <alpine.LFD.2.00.0912111926080.2668@casper.infradead.org>
	<9CA5FD93-CDC1-4394-B57B-E56A99DA09C8@sun.com>
Message-ID: <alpine.LFD.2.00.0912142017220.28627@casper.infradead.org>


> On 2009-12-11, at 12:27, James Simmons wrote:
> > Is there a way to stop the lustre test suite from reformating the disk
> > when we run our test?
> 
> 
> What version of Lustre are you testing?  

b1_8

> In recent releases (the last 9-12
> months or so) it doesn't reformat the filesystem by default.
>
> I _think_ (but don't recall for sure anymore) that it is also possible to
> specify REFORMAT="" so that it will not reformat.

But llmount.sh doesn't use REFORMAT. Instead it uses NOFORMAT instead.
 
> Some tests, like conf-sanity.sh depend on being able to reformat, and can be
> run with a local config that just uses loopback files.

Yes I noticed.
 

From Brian.Reitz at Sun.COM  Mon Dec 14 20:23:32 2009
From: Brian.Reitz at Sun.COM (Brian Reitz)
Date: Mon, 14 Dec 2009 14:23:32 -0600
Subject: [Lustre-devel] Lustre transition to Git has been completed.
Message-ID: <4AAC4879-9522-4CCF-AE96-EA95DE84947E@sun.com>


The Lustre transition to Git has been completed.  You can find more information about Lustre and Git in the email below.

Included as an attachment to this email is a torrent from which you can download a tarball of the lustre git repository.  To ease initial demands on the repository, please use the .torrent.

Brian





> What is happening?
> 
> The Lustre source repository is moving from CVS, where it has lived for the last 7+ years to Git.  This will simplify ongoing development of Lustre in a number of ways, including tracking of changes and history, improved merging of development branches into releases, and offline development.
> 
> Unfortunately, due to the lengthy use and abuse of CVS to manage the Lustre code, it is not possible to do a straight import of the entire CVS repository into Git using any existing tools.  Rather than simply importing the current state of the code into Git and lose all of the history, we've worked to import the major development branches along with their full history into a single Git repository to maximize its usefulness.
> 
> When is this happening?
> 
> Monday December 14 2009 is the target date for switching all Lustre development from CVS to Git. 
> 
> All CVS Repositories will transition to read-only state on Friday December 11, 2009 at 9:00 am CST. The CVS repository will remain in this read-only state for the duration of the conversion process.  Please make your plans accordingly.
> 
> The conversion to Git will be done over the weekend (12/11/2009 - 12/13/2009).  The resulting Git repository will be available on Monday December 14 2009 at 12:00 PM CST.  The CVS tree will remain in a read-only state from this point on.  All subsequent code changes must be done via Git.
> 
> Instructions on using Git with the Lustre code base can be found here: 
> 
> http://wiki.lustre.org/index.php/Accessing_Lustre_Code
> http://wiki.lustre.org/index.php/Applying_Lustre_Patches_to_a_Kernel
> http://wiki.lustre.org/index.php/Submitting_Patches
> 
> What is supported?
> 
> Following Branches are migrated to GIT
> 
> 	b1_6
>  	b1_8
>  	b_release_1_6_7
>  	b_release_1_8_0
>  	b_release_1_8_1
>  	master aka HEAD represents 2.0 work
> 
> A limited number of tags are also supported in the git repository.  Only v* tags post 1.6.7 are supported.  
> 
> What is not supported?
> 
> 	b1_4 branch will not be migrated and will be maintained under cvs.
> 
> Developer Migration Notes
> 
> To make your migration to git easier you should update any of your development CVS branches from the base, and land outstanding work in those development branches before Friday December 11, 2009.  Working directories that contain uncommitted changes should also be updated, but can be done after the migration while the CVS repository is in a read-only mode.
>  
> Tags will be placed in the CVS repository to mark the point were the conversion was done:
> 
>         B1_6_GIT_BASE
>         B1_8_GIT_BASE
>         B_RELEASE_1_6_7_GIT_BASE
>         B_RELEASE_1_8_0_GIT_BASE
>         B_RELEASE_1_8_1_GIT_BASE
>         B2_0_GIT_BASE
> 
> In addition tags will also be in place in the Git repository marking the spot of the conversion:
> 
>         GIT_EPOCH_B1_6
>         GIT_EPOCH_B1_8
>         GIT_EPOCH_B_RELEASE_1_6_7
>         GIT_EPOCH_B_RELEASE_1_8_0
>         GIT_EPOCH_B_RELEASE_1_8_1
>         GIT_EPOCH_MASTER
> 
> Support
> 
> lustre-devel at sun.com
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20091214/8e16409c/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Lustre-prime-2009-12-14.tar.gz.torrent
Type: application/x-bittorrent
Size: 19401 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20091214/8e16409c/attachment.torrent>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20091214/8e16409c/attachment-0001.html>

From jsimmons at infradead.org  Mon Dec 14 20:25:44 2009
From: jsimmons at infradead.org (James Simmons)
Date: Mon, 14 Dec 2009 20:25:44 +0000 (GMT)
Subject: [Lustre-devel] Flag Day : Lustre ( b1_6,
 b1_8 and HEAD ) transition to GIT on Dec 14th, 2009
In-Reply-To: <0EDE0A63-71C6-4757-96E0-7DA104F022C2@Sun.COM>
References: <0EDE0A63-71C6-4757-96E0-7DA104F022C2@Sun.COM>
Message-ID: <alpine.LFD.2.00.0912142024590.28627@casper.infradead.org>


Attempted to do a checkout.

git clone git://git.lustre.org/prime/lustre.git
Initialized empty Git repository in /home/ffrwfr/lustre/.git/
git.lustre.org[0: 192.18.100.206]: errno=Connection timed out
fatal: unable to connect a socket (Connection timed out)

Is it ready yet?




From morrone2 at llnl.gov  Tue Dec 15 01:00:31 2009
From: morrone2 at llnl.gov (Christopher J. Morrone)
Date: Mon, 14 Dec 2009 17:00:31 -0800
Subject: [Lustre-devel] Fwd: The Lustre community and GIT
In-Reply-To: <2b20c7140912131047qf0417f1mf3f3d9a8fc751dc0@mail.gmail.com>
References: <2b20c7140912130831t658c2790hb38c3752c18ec128@mail.gmail.com>
	<2b20c7140912131047qf0417f1mf3f3d9a8fc751dc0@mail.gmail.com>
Message-ID: <4B26DFAF.2020906@llnl.gov>

Peter Braam wrote:

> My group at ClusterStor has reserved the "lustre" project at Github and we
> will give keys to any and all organizations that wish to make serious
> contributions to Lustre.  I was in particular hoping that LLNL would be
> willing to commit their releases to that public place.

First, I'd like to say thanks to Sun.  We are thrilled that they went 
through the effort of switching the repository over to git.

Yes, we at LLNL are happy to make our tree more public.  We have wanted 
to do this for a while, but hadn't spent the time to figure out a 
reasonable method of distribution.  Even with the move to git, it is not 
immediately clear how we should share our branch of Lustre given our 
current work flow.

For a long time, we used a script to create a tarball from Sun's CVS 
repo, and then imported that tree into a branch of our local Subversion 
repo.  All of our changes and maintained a quilt stack of patches.

For a number of reasons, when we started planning our 1.8.2-based 
release a couple of months ago, I transitioned us from Subversion to 
git.  We currently maintain all of our patches using topgit, which 
maintains each "patch" as a separate real "topic" branch in git.

topgit is nice in some ways, but one of its down sides is that it makes 
the git history pretty ugly, with dozens and dozens of merges.  topgit 
allows a DAG of topic branch dependencies, but I decided to keep it 
simple with a stack-like arrangement of patches.

But to make a long story short, I doubt that anyone wants to see our 
topgit noise in a public repo.  So how would we handle an llnl branch in 
a repo on github?

We are willing to change our workflow, and even give up using topgit, if 
someone can suggest a better way.  To some extent, we can try to improve 
our patch development process.  Instead of creating a new quilt 
patch/topgit branch, we could just:

  1) Create git temporary branch
  2) Develop fix, commit, commit, commit...
  3) Condense development branch into one clean commit on the llnl branch.

One issue will be that in parallel, we are going to be submitting this 
patch to Sun via bugzilla.  It is often a month or more before this 
patch is revised and landed upstream.  And if a different version IS 
landed upstream, merging later could be problematic.  At the very least, 
there will be conflicts, but those are easily dealt with.

More worrying is when Sun comes up with a better fix that touches other 
code that our fix.  When we merge from Sun, there is no obvious clue 
that we now have two fixes for the same bug, and while it may not have 
conflicted from git's point of view, the code is now probably broken.

So what is the best way for us to carry our long-term patches in git? 
Any suggestions?

Chris

From adilger at sun.com  Tue Dec 15 01:06:09 2009
From: adilger at sun.com (Andreas Dilger)
Date: Mon, 14 Dec 2009 18:06:09 -0700
Subject: [Lustre-devel] Flag Day : Lustre ( b1_6,
 b1_8 and HEAD ) transition to GIT on Dec 14th, 2009
In-Reply-To: <alpine.LFD.2.00.0912142024590.28627@casper.infradead.org>
References: <0EDE0A63-71C6-4757-96E0-7DA104F022C2@Sun.COM>
	<alpine.LFD.2.00.0912142024590.28627@casper.infradead.org>
Message-ID: <01075A95-285A-41EE-8457-4A7E1DE4D84C@sun.com>

On 2009-12-14, at 13:25, James Simmons wrote:
> Attempted to do a checkout.
>
> git clone git://git.lustre.org/prime/lustre.git
> Initialized empty Git repository in /home/ffrwfr/lustre/.git/
> git.lustre.org[0: 192.18.100.206]: errno=Connection timed out
> fatal: unable to connect a socket (Connection timed out)
>
> Is it ready yet?


James, were you able to do a clone?   I don't know if someone already  
contacted you, but I tested this a little while ago and it was working  
OK.  We also have a torrent set up for the initial repository clone,  
since it is about 120MB and we weren't sure how much load there was  
initially going to be on the repository.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.


From jsimmons at infradead.org  Tue Dec 15 12:36:16 2009
From: jsimmons at infradead.org (James Simmons)
Date: Tue, 15 Dec 2009 12:36:16 +0000 (GMT)
Subject: [Lustre-devel] Flag Day : Lustre ( b1_6,
 b1_8 and HEAD ) transition to GIT on Dec 14th, 2009
In-Reply-To: <01075A95-285A-41EE-8457-4A7E1DE4D84C@sun.com>
References: <0EDE0A63-71C6-4757-96E0-7DA104F022C2@Sun.COM>
	<alpine.LFD.2.00.0912142024590.28627@casper.infradead.org>
	<01075A95-285A-41EE-8457-4A7E1DE4D84C@sun.com>
Message-ID: <alpine.LFD.2.00.0912151235400.30505@casper.infradead.org>


> On 2009-12-14, at 13:25, James Simmons wrote:
> > Attempted to do a checkout.
> > 
> > git clone git://git.lustre.org/prime/lustre.git
> > Initialized empty Git repository in /home/ffrwfr/lustre/.git/
> > git.lustre.org[0: 192.18.100.206]: errno=Connection timed out
> > fatal: unable to connect a socket (Connection timed out)
> > 
> > Is it ready yet?
> 
> 
> James, were you able to do a clone?   I don't know if someone already
> contacted you, but I tested this a little while ago and it was working OK.  We
> also have a torrent set up for the initial repository clone, since it is about
> 120MB and we weren't sure how much load there was initially going to be on the
> repository.

It's firewall issues. 

From yuriy.umanets at clusterstor.com  Tue Dec 15 16:18:17 2009
From: yuriy.umanets at clusterstor.com (Yuriy Umanets)
Date: Tue, 15 Dec 2009 18:18:17 +0200
Subject: [Lustre-devel] First loadgen striping support patch
Message-ID: <3ed212d00912150818p68951ebkd5502502d59294f2@mail.gmail.com>

hi all,

We've attached loadgen striping support patch in bug 21571. Please inspect,
comment. We want to make sure we're are on same page with it. It is in line
with what was discussed earlier on the list.

Two more things coming soon:
1. Dedicated LOV instance for each thread/virtual client;
2. OBD devices limit fixes.

Thanks.

-- 
umka
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20091215/43d49a38/attachment.html>

From yuriy.umanets at clusterstor.com  Tue Dec 15 16:28:55 2009
From: yuriy.umanets at clusterstor.com (Yuriy Umanets)
Date: Tue, 15 Dec 2009 18:28:55 +0200
Subject: [Lustre-devel] client side reply handling
In-Reply-To: <2b20c7140912090705s47667951w26fe6c95e81f2c8f@mail.gmail.com>
References: <2b20c7140912090705s47667951w26fe6c95e81f2c8f@mail.gmail.com>
Message-ID: <3ed212d00912150828u2fc58cf2v541c95989d220a85@mail.gmail.com>

2009/12/9 Peter Braam <peter.braam at clusterstor.com>

> Hi,
>
> hi Peter,


> I wonder if LNET doesn't have an atomic operation that unlinks the packet
> from the delivery process upon receiving a packet.   Iirc it does.  Why
> wouldn't one use that?
>
That is probably because it still may be in use by upper layers (ptlrpc,
etc), i.e. it is up to upper layer to decide should its buffers be still
linked for access or not.

Thanks.

>
> Peter
>
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-devel
>
>


-- 
umka
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20091215/fd7d522c/attachment.html>

From adilger at sun.com  Tue Dec 15 21:05:14 2009
From: adilger at sun.com (Andreas Dilger)
Date: Tue, 15 Dec 2009 14:05:14 -0700
Subject: [Lustre-devel] Fwd: The Lustre community and GIT
In-Reply-To: <2b20c7140912131047qf0417f1mf3f3d9a8fc751dc0@mail.gmail.com>
References: <2b20c7140912130831t658c2790hb38c3752c18ec128@mail.gmail.com>
	<2b20c7140912131047qf0417f1mf3f3d9a8fc751dc0@mail.gmail.com>
Message-ID: <5F225623-87E0-4A91-988C-8B789B63E46D@sun.com>

Hello Peter,

As previously announced, we have already made the initial Lustre  
repository available to everyone this week at git.lustre.org.  This  
repository contains all of the main development and release branches  
and their history.  It's where all of the Sun developers and  
maintaners will get their sources from, and it will continue to be  
available under the GPL v2 license as it always has been.

Sun's policy of prioritizing stability, not just for releases but also
to underpin development, makes safeguarding the quality of this
repository paramount.  Everyone contributing to the Lustre sources -
not just internal Sun engineers but also external contributors like
LLNL, CEA, ORNL and DDN - is therefore subject to the same gatekeeping
procedures and must follow the same patch submission, testing and
landing processes.  These are further detailed in the wiki pages that
were in the original announcement:

http://wiki.lustre.org/index.php/Accessing_Lustre_Code
http://wiki.lustre.org/index.php/Submitting_Patches


For people not familiar with Git, it should be clarified that limiting  
commits to the Sun Prime repository does not in any way restrict  
access to the Lustre code, or the ability of non-Sun developers to  
share their own Git clone with the Lustre community.  Developers will  
be able to host their own Lustre clones (essentially full  
repositories) as and where they wish.  The major advantage of Git is  
that anyone can pull from any repository, or in fact from a number of  
different repositories at the same time - this choice is left to the  
user.

This is the same model used by the Linux kernel, and has proven to  
work well with Git.  Each kernel developer hosts their own clone(s) at  
git.kernel.org, or at some other site like repo.or.cz, or github.com,  
and when they have changes to submit upstream they send an email  
(usually containing both the patch, and the git URL to pull from) to  
the subsystem maintainer, or to Linus directly, requesting that he  
pull their changes.  This pull request is normally only sent after the  
patch has been reviewed, possibly multiple times,  by the appropriate  
subsystem maintainers.  Each developer has full control over their own  
clone, and in fact it is rare that more than one person has push  
access to a clone.

The fact that there are many different clones (some more important,  
and others less so) in no way implies that the kernel is being forked,  
but rather that this is the standard way in which to use Git to  
exchange changes.  The location at which a clone is hosted also has no  
bearing on its usefulness.



To answer your specific concerns in more detail,

On 2009-12-13, at 11:47, Peter Braam wrote:
> 1. We need a public repository, where non Sun community members can  
> commit.
>
> a.  There are Lustre users than have their own branches.  LLNL has  
> probably the best tested branch among all users.  DDN's customers  
> have a version of Lustre that includes patches not yet in Sun's  
> releases.  It would be very valuable if these kind of releases could  
> be collected in a publicly accessible GIT repository.

That is true even today - most of the patches that are in the DDN  
branches were initially developed by Sun and are already in the  
publicly-available Lustre CVS, and many of the LLNL patches have or  
are being landed for the 1.8.2 release.

As you know, there will always be a delta of patches that are not in  
an official release, and will be available in the next one.  With the  
increase in testing to improve the quality of new releases compared to  
the past, releases are by necessity less frequent.  Should anyone have  
a desire for more bleeding-edge code, they have always been able fetch  
the current branch before its release, regardless of whether this is  
Git or CVS.  We maintain a number of different branches (b1_8 for  
incremental development, b_release_1_8_1 for critical fixes on the  
1.8.1 release, etc) these are already available to the public.

> I doubt that Sun will give commit rights to external entities (this  
> is not unreasonable, Sun needs to control what code enters its  
> repositories).  Hence I think that the community would be better  
> served with a GIT repository in a public place, like github, that  
> can give such access.

While CVS was very restrictive in terms of managing external commit  
permissions due to its monolithic repository, some external  
contributors have had commit access to CVS prior our migration to Git,  
based on need.  With the migration to Git there is no need to manage  
such access ourselves, as developers are able to host their clones  
wherever they want.  The git.lustre.org repository will remain the  
canonical source for Sun releases, and we will be happy to pull fixes  
into this repository that meet the quality guidelines stated above.  I  
for one would welcome external inspectors on patches, and we continue  
to work with external sites to do scale testing of Lustre.

> My group at ClusterStor has reserved the "lustre" project at Github  
> and we will give keys to any and all organizations that wish to make  
> serious contributions to Lustre.  I was in particular hoping that  
> LLNL would be willing to commit their releases to that public place.


Note that with Git and github.com there is no need to give keys to  
anyone, and in fact that would seem to be detrimental to ClusterStor,  
because the "lustre" clone you have reserved is within your company's  
private hosting space (i.e. http://github.com/clusterstor/lustre).   
With Github (or Git in general) it is possible for anyone to make  
their own clone or mirror of a repository at any time, no keys or  
permission required, and it will appear ashttp://github.com/{user}/ 
lustre or whatever they want to call it.

> b.  Everyone is uncertain about what Sun will do with Lustre  
> (proprietary Solaris / ZFS server releases and discontinued support  
> for Linux servers have been mentioned to me several times now).  A  
> public repository with the open code will be valuable for the  
> community and promote continued development and access.

I agree that a public repository is important, and we will continue to  
host one at git.lustre.org as we have in the past for CVS, and it can  
and should be cloned as needed.  As you are hopefully aware, with Git  
every checkout has a full copy of the repository, including all  
history, so the code is already very "open" and "public".  Lustre is,  
and will continue to be, available as GPL software.

We definitely welcome external contributions to Lustre, which have in  
the past been done by a small number of people outside of Sun.  I  
don't think the choice of CVS or Git or hosting site has ever been a  
limiting factor in this regard.  We look forward to contributions of  
fixes, designs, and features, from ClusterStor, as we would with any  
Lustre contributor.

> 2. We need MUCH more in the repository than Sun is putting into it.
>
> There are many development branches and sometimes abandoned projects  
> that still have a lot of value.  For example, there is a nearly  
> complete OS X client port - what if someone wanted to pick that up?   
> Similarly, there are projects like size on MDS or the network  
> request scheduler that may need help from the community to get  
> finished or re-prioritized.

All of the Lustre history is still available in CVS, as it has always  
been.  As far as I know (I've never checked it out myself) even the OS/ 
X client port is publicly available today, which was not true a few  
years ago.

Due to the convoluted branch, repository, and tag abuse that was used  
to manage the Lustre code in CVS, there is a non-zero effort required  
to migrate any of the branches in CVS to Git.  Rather than bring all  
of the old detritus along into Git, only the main development/ 
production branches are being automatically migrated initially.

Now that the Git migration is complete, the Sun maintainers of non- 
release features (HSM, SOM, CMD, NRS, etc) will be creating Git clones  
and landing their work into them as time permits.

If anyone wants to import one of the other CVS branches (e.g. OS/X),  
all they will need to do is create a patch from that branch and then  
commit this into their own Git clone.

> It is unclear to me if these kind of branches can be found in the  
> publicly available CVS.

Yes, they can, as they always have been.  The CVS repository will  
remain available indefinitely for spelunking expeditions should the  
need arise.  Note that the full history that leads to the current  
release branches (b1_6, b1_8, HEAD) is available in Git, so there is  
at least a trail of breadcrumbs leading to where we are today.

> If they can, a collection of relevant branches, broadly along the  
> lines of what I mention above, should be placed in the public GIT  
> repository.

For currently-active branches this was already our plan.  For  
historical and inactive branches, and we welcome any contributions  
that bring these ancient CVS branches back to life.  Nikita is  
probably best positioned to know what needs to be imported from the OS/ 
X branch in CVS, and if there are other particular branches that  
contain important work we'll be happy to discuss them with you.  It  
will of course be possible to create Git clones for these old branches  
as needed.

> 3. Brian's email message seems to indicate that Sun's git repository  
> will be used for Sun development.  In the past there were two CVS  
> repositories - a read-only one that was publicly accessible and when  
> I last controlled the group it was updated very frequently with all  
> open source code (presently, it seems to only contain releases, not  
> the development stream of commits).

That's not correct.  The public cvs.lustre.org repository in fact  
contains all of the Lustre commits and all of the history.  I just did  
a checkout and there are in fact tags and branches for the pre-release  
1.8.2 builds, lprocfs rewrite, CMD, etc. that are very much works in  
progress.

One of the very last commits on CVS HEAD was on Thursday before CVS  
went read-only, from a patch submitted by LLNL:

revision 1.593
date: 2009/12/10 13:52:51;  author: dzogin;  state: Exp;  lines: +5 -0
Branch HEAD
b=21259
i=andrew.perepechko
i=alexey.lyashkov
----------------------------------------------------------------------
Description: Allow non-root access for "lfs check".
Details    : Added a check in obd_class_ioctl() for OBD_IOC_PING_TARGET.


> It is unclear how Sun can manage development with this git  
> repository given that parts of its work are proprietary (like the  
> Windows port) or unreleased (like the ZFS work).  Can we get more  
> insight in what Brian is alluding to?

I'm not sure what "alluding" you are alluding to?

Of course, if there is any proprietary code developed it will just not  
be pushed to the public git.lustre.org repository.  I expect that any  
proprietary or unreleased code that ClusterStor is already or will  
develop will similarly not be posted publicly until you are ready to  
do so.

Sun's current in-progress code will very likely reside in separate  
clones at git.lustre.org, but will not be merged into the Sun Prime  
repository for an official release until it is inspected, tested, and  
has permission to land.  Pulls into the Sun Prime repository will be  
done by the release manager for each branch, as previously stated.   
The Lustre engineers at ClusterStor are already familiar with this  
process and have already had their first patch pass inspections and it  
is ready for landing.

That is one of the major benefits of Git over CVS - that there ISN'T a  
single repository, and in fact every clone has a full copy of the  
history and can be used by anyone.  There is no need to have all of  
the development done in branches on a single repository, but rather to  
keep projects in their own clones.

That allows developers to do local commits on their machines, to push  
it to their public clone if they want to share their work and/or make  
an offsite backup, and people are free to choose which clone one they  
use.  Sun's Prime repository used for releases will only contain  
stable Lustre code.  This is the Git usage model for the Linux kernel,  
and think it is a good one to follow for Lustre as well.  You are free  
to manage your clones in some other way.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.

From morrone2 at llnl.gov  Tue Dec 15 22:19:53 2009
From: morrone2 at llnl.gov (Christopher J. Morrone)
Date: Tue, 15 Dec 2009 14:19:53 -0800
Subject: [Lustre-devel] Fwd: The Lustre community and GIT
In-Reply-To: <5F225623-87E0-4A91-988C-8B789B63E46D@sun.com>
References: <2b20c7140912130831t658c2790hb38c3752c18ec128@mail.gmail.com>	<2b20c7140912131047qf0417f1mf3f3d9a8fc751dc0@mail.gmail.com>
	<5F225623-87E0-4A91-988C-8B789B63E46D@sun.com>
Message-ID: <4B280B89.3010303@llnl.gov>

Andreas Dilger wrote:

> For people not familiar with Git, it should be clarified that limiting  
> commits to the Sun Prime repository does not in any way restrict  
> access to the Lustre code, or the ability of non-Sun developers to  
> share their own Git clone with the Lustre community.  Developers will  
> be able to host their own Lustre clones (essentially full  
> repositories) as and where they wish.

Ah, right, good point.  We would probably just have an llnl lustre repo 
on github (if not one for each developer...), once we figure out how to 
make our branch more presentable.

> This is the same model used by the Linux kernel, and has proven to  
> work well with Git.  Each kernel developer hosts their own clone(s) at  
> git.kernel.org, or at some other site like repo.or.cz, or github.com,  
> and when they have changes to submit upstream they send an email  
> (usually containing both the patch, and the git URL to pull from) to  
> the subsystem maintainer, or to Linus directly, requesting that he  
> pull their changes.  This pull request is normally only sent after the  
> patch has been reviewed, possibly multiple times,  by the appropriate  
> subsystem maintainers.  Each developer has full control over their own  
> clone, and in fact it is rare that more than one person has push  
> access to a clone.

Has Sun given any thought to how git might be more integrated into the 
Bugzilla patch review process?  In particular, for external developers? 
  I see there is an internal "Lustre GIT page" that I can't access, and 
external contributors are just told to follow the "submitting patches" 
guidelines.

We would love to be able to tell Sun to pull from a branch to get our 
unreviewed patches, and vice versa.

Chris

From adilger at sun.com  Tue Dec 15 23:25:07 2009
From: adilger at sun.com (Andreas Dilger)
Date: Tue, 15 Dec 2009 16:25:07 -0700
Subject: [Lustre-devel] Fwd: The Lustre community and GIT
In-Reply-To: <4B280B89.3010303@llnl.gov>
References: <2b20c7140912130831t658c2790hb38c3752c18ec128@mail.gmail.com>
	<2b20c7140912131047qf0417f1mf3f3d9a8fc751dc0@mail.gmail.com>
	<5F225623-87E0-4A91-988C-8B789B63E46D@sun.com>
	<4B280B89.3010303@llnl.gov>
Message-ID: <F661C7A0-E3C8-4B69-A275-F23844DDFCAB@sun.com>

On 2009-12-15, at 15:19, Christopher J. Morrone wrote:
> Andreas Dilger wrote:
>> For people not familiar with Git, it should be clarified that  
>> limiting
>> commits to the Sun Prime repository does not in any way restrict
>> access to the Lustre code, or the ability of non-Sun developers to
>> share their own Git clone with the Lustre community.  Developers will
>> be able to host their own Lustre clones (essentially full
>> repositories) as and where they wish.
>
> Ah, right, good point.  We would probably just have an llnl lustre  
> repo
> on github (if not one for each developer...), once we figure out how  
> to
> make our branch more presentable.

You would should rebase your repository against the Lustre git repo,  
but that should be fairly straight forward since topgit keeps each  
patch in a separate branch.  In general, users can use git rebase for  
keeping their patches updated against a remote tree, but since you  
started with a completely different git repo it probably will fail  
horribly.

>> This is the same model used by the Linux kernel, and has proven to
>> work well with Git.  Each kernel developer hosts their own clone(s)  
>> at
>> git.kernel.org, or at some other site like repo.or.cz, or github.com,
>> and when they have changes to submit upstream they send an email
>> (usually containing both the patch, and the git URL to pull from) to
>> the subsystem maintainer, or to Linus directly, requesting that he
>> pull their changes.  This pull request is normally only sent after  
>> the
>> patch has been reviewed, possibly multiple times,  by the appropriate
>> subsystem maintainers.  Each developer has full control over their  
>> own
>> clone, and in fact it is rare that more than one person has push
>> access to a clone.
>
> Has Sun given any thought to how git might be more integrated into the
> Bugzilla patch review process?  In particular, for external  
> developers?

Actually, Robert found a few tools that allow attaching patches to  
bugzilla directly from git that seem very useful.  I don't have the  
names offhand, but it was very simple, like "git bz send-patch  
[bugno]" or similar.  Ideally, the gatekeeper would also be able to  
automatically mark the patch as landed on the appropriate release  
branch, but I suspect that would need a bit more scripting.  There has  
also been some investigation of "Review Board", but I don't know where  
that is today.

>  I see there is an internal "Lustre GIT page" that I can't access, and
> external contributors are just told to follow the "submitting patches"
> guidelines.
>
> We would love to be able to tell Sun to pull from a branch to get our
> unreviewed patches, and vice versa.


I agree that doing git pulls instead of patch submission is cleaner,  
but it definitely isn't a good way to do patch review.  For the kernel  
all of the patch review is done via email, but that has the drawback  
that the discussion about a given patch isn't kept in a single place,  
and it isn't clear who has agreed with or objected to a patch without  
trolling the whole thread each time.

We're still investigating how to improve the patch workflow, and  
welcome input.  The first priority was to make a Git repository  
available, and then we can look at improving how patches are handled  
(tracking, inspections, testing, landing).  I didn't want to entangle  
those two major process changes, as it would have delayed the Git  
transition even further.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.


From morrone2 at llnl.gov  Wed Dec 16 01:18:47 2009
From: morrone2 at llnl.gov (Christopher J. Morrone)
Date: Tue, 15 Dec 2009 17:18:47 -0800
Subject: [Lustre-devel] Fwd: The Lustre community and GIT
In-Reply-To: <F661C7A0-E3C8-4B69-A275-F23844DDFCAB@sun.com>
References: <2b20c7140912130831t658c2790hb38c3752c18ec128@mail.gmail.com>
	<2b20c7140912131047qf0417f1mf3f3d9a8fc751dc0@mail.gmail.com>
	<5F225623-87E0-4A91-988C-8B789B63E46D@sun.com>
	<4B280B89.3010303@llnl.gov>
	<F661C7A0-E3C8-4B69-A275-F23844DDFCAB@sun.com>
Message-ID: <4B283577.3070905@llnl.gov>

Andreas Dilger wrote:
> On 2009-12-15, at 15:19, Christopher J. Morrone wrote:
>> Andreas Dilger wrote:
>>> For people not familiar with Git, it should be clarified that limiting
>>> commits to the Sun Prime repository does not in any way restrict
>>> access to the Lustre code, or the ability of non-Sun developers to
>>> share their own Git clone with the Lustre community.  Developers will
>>> be able to host their own Lustre clones (essentially full
>>> repositories) as and where they wish.
>>
>> Ah, right, good point.  We would probably just have an llnl lustre repo
>> on github (if not one for each developer...), once we figure out how to
>> make our branch more presentable.
> 
> You would should rebase your repository against the Lustre git repo, but 
> that should be fairly straight forward since topgit keeps each patch in 
> a separate branch.  In general, users can use git rebase for keeping 
> their patches updated against a remote tree, but since you started with 
> a completely different git repo it probably will fail horribly.

I'm not really clear what you mean here.  I am not worried about merging 
in the latest from Sun's repo, that can be done a number of ways easily 
enough.

Lets say that I DON'T use topgit, because it is just going to be 
confusing for anyone else who tries to look at our repo.  And lets say 
that I have a branch of lustre that has 50 "patches" against the 
upstream code (our 1.6.6 branch had 175 patches! lets hope we don't 
diverge that much again...)

How do I keep track of the logical set of patches?  I am not worried 
that I will lose them, I'm worried that cruft will accumulate, and that 
their will be hidden conflicts.  I am worried that llnl will fix the 
problem one way, and then Sun will fix it another, and now we have two 
fixes that don't get along in our tree.  Its entirely possible that 
there will be no contextual conflicts, even thought the fixes don't work 
together.

Also, if we commit all of our work to one tree, how do we keep track of 
which commits belong to the same "bug"?

Maybe I am just overthinking the problem.  I'm hoping that I am.

Sun is never going to perform a full "pull" from the "llnl" branch.  And 
rightly so, that would just be too many disparate fixes to digest.  So 
(at llnl) have two somewhat conflicting goals:

1) Keep every logical change separate (so we can interact with Sun sanely)
2) Keep all changes together (so we can tag and install it :))

Previously we solved the problem with quilt, and now we solve this with 
topgit.  As long as we do not publish our repo, using topgit is probably 
going to be fine.  But I am worried that if we publish our repo, the 
history will just look like nonsense to other (because of topgit), and 
not be terribly useful beyond those folks that just want to blindly use 
our latest tag.

Chris


From peter.braam at clusterstor.com  Wed Dec 16 02:01:33 2009
From: peter.braam at clusterstor.com (Peter Braam)
Date: Tue, 15 Dec 2009 19:01:33 -0700
Subject: [Lustre-devel] client side reply handling
In-Reply-To: <3ed212d00912150828u2fc58cf2v541c95989d220a85@mail.gmail.com>
References: <2b20c7140912090705s47667951w26fe6c95e81f2c8f@mail.gmail.com>
	<3ed212d00912150828u2fc58cf2v541c95989d220a85@mail.gmail.com>
Message-ID: <2b20c7140912151801m78871060pd20e5136c105b507@mail.gmail.com>

No - buffers can and probably should (because, for example, re-delivery may
leaver buffer state undefined) be unlinked from the match list before
passing the buffer up to any other layer.  Portals/LNET certainly can do
this.

The question is why isn't this done, Eric, Isaac, Andreas?

Peter

On Tue, Dec 15, 2009 at 9:28 AM, Yuriy Umanets <
yuriy.umanets at clusterstor.com> wrote:

>
>
> 2009/12/9 Peter Braam <peter.braam at clusterstor.com>
>
>> Hi,
>>
>> hi Peter,
>
>
>> I wonder if LNET doesn't have an atomic operation that unlinks the packet
>> from the delivery process upon receiving a packet.   Iirc it does.  Why
>> wouldn't one use that?
>>
> That is probably because it still may be in use by upper layers (ptlrpc,
> etc), i.e. it is up to upper layer to decide should its buffers be still
> linked for access or not.
>
> Thanks.
>
>>
>> Peter
>>
>> _______________________________________________
>> Lustre-devel mailing list
>> Lustre-devel at lists.lustre.org
>> http://lists.lustre.org/mailman/listinfo/lustre-devel
>>
>>
>
>
> --
> umka
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20091215/21a6c9bb/attachment.html>

From He.Huang at Sun.COM  Wed Dec 16 02:48:18 2009
From: He.Huang at Sun.COM (Isaac Huang)
Date: Tue, 15 Dec 2009 21:48:18 -0500
Subject: [Lustre-devel] client side reply handling
In-Reply-To: <2b20c7140912151801m78871060pd20e5136c105b507@mail.gmail.com>
References: <2b20c7140912090705s47667951w26fe6c95e81f2c8f@mail.gmail.com>
	<3ed212d00912150828u2fc58cf2v541c95989d220a85@mail.gmail.com>
	<2b20c7140912151801m78871060pd20e5136c105b507@mail.gmail.com>
Message-ID: <20091216024818.GA1196@sun.com>

On Tue, Dec 15, 2009 at 07:01:33PM -0700, Peter Braam wrote:
>    No - buffers can and probably should (because, for example, re-delivery
>    may leaver buffer state undefined) be unlinked from the match list
>    before passing the buffer up to any other layer.  Portals/LNET
>    certainly can do this.

Hi Peter, LNet does have a MD option to automatically unlink the
buffer once it's been exhausted. Duplicated delivery shouldn't cause any
problem because MD offset is increased locally (i.e. a dup would go
into a different offset), unless the MD has enabled peers to manage
offset remotely, which is now true for PTLRPC reply buffers (because
now early replies and real reply share a same reply buffer).

Isaac

From He.Huang at Sun.COM  Wed Dec 16 02:51:42 2009
From: He.Huang at Sun.COM (Isaac Huang)
Date: Tue, 15 Dec 2009 21:51:42 -0500
Subject: [Lustre-devel] client side reply handling
In-Reply-To: <20091216024818.GA1196@sun.com>
References: <2b20c7140912090705s47667951w26fe6c95e81f2c8f@mail.gmail.com>
	<3ed212d00912150828u2fc58cf2v541c95989d220a85@mail.gmail.com>
	<2b20c7140912151801m78871060pd20e5136c105b507@mail.gmail.com>
	<20091216024818.GA1196@sun.com>
Message-ID: <20091216025142.GB1196@sun.com>

On Tue, Dec 15, 2009 at 09:48:18PM -0500, Isaac Huang wrote:
> On Tue, Dec 15, 2009 at 07:01:33PM -0700, Peter Braam wrote:
> >    No - buffers can and probably should (because, for example, re-delivery
> >    may leaver buffer state undefined) be unlinked from the match list
> >    before passing the buffer up to any other layer.  Portals/LNET
> >    certainly can do this.
> 
> Hi Peter, LNet does have a MD option to automatically unlink the
> buffer once it's been exhausted. Duplicated delivery shouldn't cause any
> problem because MD offset is increased locally (i.e. a dup would go
> into a different offset), unless the MD has enabled peers to manage
> offset remotely, which is now true for PTLRPC reply buffers (because
> now early replies and real reply share a same reply buffer).

There was a recent discussion on this topic, under a similar subject:
http://lists.lustre.org/pipermail/lustre-devel/2009-December/003231.html

Isaac

From Zhen.Liang at Sun.COM  Wed Dec 16 02:55:11 2009
From: Zhen.Liang at Sun.COM (Liang Zhen)
Date: Wed, 16 Dec 2009 10:55:11 +0800
Subject: [Lustre-devel] client side reply handling
In-Reply-To: <2b20c7140912151801m78871060pd20e5136c105b507@mail.gmail.com>
References: <2b20c7140912090705s47667951w26fe6c95e81f2c8f@mail.gmail.com>
	<3ed212d00912150828u2fc58cf2v541c95989d220a85@mail.gmail.com>
	<2b20c7140912151801m78871060pd20e5136c105b507@mail.gmail.com>
Message-ID: <4B284C0F.3080303@sun.com>

Peter,

It depends on how buffers are attached to portal, ptlrpc has three cases 
of this:
1. If buffer is for incoming request (not for RDMA, so it's not unique 
match)
    the MD is constructed with lnet_md_t::threshold = LNET_MD_THRESH_INF,
    so it will be unlinked only for two cases:
    a) user call LNetMDUnlink explicitly
    b) the buffer is exhausted (lnet_md_exhausted() return 1 in LNet)
    Once LNet get a request for that buffer, it will change offset of 
the buffer so
it wouldn't be overwritten by the next match, it's possible that upper 
layer is handing
the request, and the buffer is still on match list

2. If buffer is for bulk (it's unique)
    the MD is constructed with lnet_md_t::threshold = 1, and
    LNetMDAttach(...LNET_UNLINK...)
    so LNet will auto-unlink the buffer from match list once it got a 
match, I think
this is the case you want

3. If the buffer is for reply (it's unique as well)
    the MD is constructed with lnet_md_t::threshold = LNET_MD_THRESH_INF,
so it's same as 1), the buffer wouldn't be unlinked unless user call  
LNetMDUnlink
or LNetMEUnlink, so safety of reply buffer can be guaranteed only if user is
sure the buffer is unlinked.

Regards
Liang


Peter Braam wrote:
> No - buffers can and probably should (because, for example, 
> re-delivery may leaver buffer state undefined) be unlinked from the 
> match list before passing the buffer up to any other layer. 
>  Portals/LNET certainly can do this.  
>
> The question is why isn't this done, Eric, Isaac, Andreas?
>
> Peter
>
> On Tue, Dec 15, 2009 at 9:28 AM, Yuriy Umanets 
> <yuriy.umanets at clusterstor.com <mailto:yuriy.umanets at clusterstor.com>> 
> wrote:
>
>
>
>     2009/12/9 Peter Braam <peter.braam at clusterstor.com
>     <mailto:peter.braam at clusterstor.com>>
>
>         Hi, 
>
>     hi Peter,
>      
>
>         I wonder if LNET doesn't have an atomic operation that unlinks
>         the packet from the delivery process upon receiving a packet.
>           Iirc it does.  Why wouldn't one use that?
>
>     That is probably because it still may be in use by upper layers
>     (ptlrpc, etc), i.e. it is up to upper layer to decide should its
>     buffers be still linked for access or not.
>
>     Thanks.
>
>
>         Peter
>
>         _______________________________________________
>         Lustre-devel mailing list
>         Lustre-devel at lists.lustre.org
>         <mailto:Lustre-devel at lists.lustre.org>
>         http://lists.lustre.org/mailman/listinfo/lustre-devel
>
>
>
>
>     -- 
>     umka
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-devel
>   


From adilger at sun.com  Wed Dec 16 07:34:55 2009
From: adilger at sun.com (Andreas Dilger)
Date: Wed, 16 Dec 2009 00:34:55 -0700
Subject: [Lustre-devel] Fwd: The Lustre community and GIT
In-Reply-To: <4B283577.3070905@llnl.gov>
References: <2b20c7140912130831t658c2790hb38c3752c18ec128@mail.gmail.com>
	<2b20c7140912131047qf0417f1mf3f3d9a8fc751dc0@mail.gmail.com>
	<5F225623-87E0-4A91-988C-8B789B63E46D@sun.com>
	<4B280B89.3010303@llnl.gov>
	<F661C7A0-E3C8-4B69-A275-F23844DDFCAB@sun.com>
	<4B283577.3070905@llnl.gov>
Message-ID: <9784AD2F-B1E4-4A48-BDB5-6B932437EBD9@sun.com>

On 2009-12-15, at 18:18, Christopher J. Morrone wrote:
> Lets say that I DON'T use topgit, because it is just going to be  
> confusing for anyone else who tries to look at our repo.  And lets  
> say that I have a branch of lustre that has 50 "patches" against the  
> upstream code (our 1.6.6 branch had 175 patches! lets hope we don't  
> diverge that much again...)
>
> How do I keep track of the logical set of patches?  I am not worried  
> that I will lose them, I'm worried that cruft will accumulate, and  
> that their will be hidden conflicts.  I am worried that llnl will  
> fix the problem one way, and then Sun will fix it another, and now  
> we have two fixes that don't get along in our tree.  Its entirely  
> possible that there will be no contextual conflicts, even thought  
> the fixes don't work together.

I know that Brian Behlendorf and Ricardo investigated a number of  
different tools for managing patches, and topgit was the one they  
chose.  Another tool in this vein is "guilt", which manages a patch  
series in a way similar to "quilt".

For developers that aren't keeping a huge number of patches, and  
especially not a large number of dependent patches, "git rebase" is  
probably sufficient. That will refresh the patches against changes in  
the upstream repository.

As for tracking hidden semantic conflicts that do not have contextual  
conflicts, that is of course more difficult.  I think one important  
way to track this is that our own commit message policy will be to  
include the bug number into the "short" (one line) patch description,  
and if you do a "git rebase" against the updated repo and your patch  
for that bug does not disappear due to the upstream commit for that  
bug, the patch should be reviewed.

I believe there are scripts that can automatically generate "shortlog"  
summaries of git commits, and if LLNL's commit policy is also to  
include our bug number along with some LLNL-specific marker into the  
short description then a simple "sort" should make it fairly simple to  
detect automatically where there are LLNL-applied and Sun patches in  
your repo for the same bug.

> Also, if we commit all of our work to one tree, how do we keep track  
> of which commits belong to the same "bug"?

Via the commit message, as we are doing, is probably the easiest.

> Sun is never going to perform a full "pull" from the "llnl" branch.   
> And rightly so, that would just be too many disparate fixes to  
> digest.  So (at llnl) have two somewhat conflicting goals:
>
> 1) Keep every logical change separate (so we can interact with Sun  
> sanely)
> 2) Keep all changes together (so we can tag and install it :))
>
> Previously we solved the problem with quilt, and now we solve this  
> with topgit.  As long as we do not publish our repo, using topgit is  
> probably going to be fine.  But I am worried that if we publish our  
> repo, the history will just look like nonsense to other (because of  
> topgit), and not be terribly useful beyond those folks that just  
> want to blindly use our latest tag.

I think that depends on how topgit is doing the updates against the  
upstream repository.  If it is doing a "rebase" each time, then there  
should only be a single git branch (each with a single commit) for  
each patch that you have applied.  If it does this via a "merge" then  
there will be new commits for each pull that you do from upstream that  
is resolving conflicts, and the repo will become a mess.

Topgit is actually reasonably well suited to this, because it keeps  
every patch in a separate branch, and it would be possible (I think)  
to pull from just that branch, or possibly to make a clone and then  
delete all of the other branches, and then pull from that.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.


From yuriy.umanets at clusterstor.com  Wed Dec 16 17:24:39 2009
From: yuriy.umanets at clusterstor.com (Yuriy Umanets)
Date: Wed, 16 Dec 2009 19:24:39 +0200
Subject: [Lustre-devel] Fwd: The Lustre community and GIT
In-Reply-To: <5F225623-87E0-4A91-988C-8B789B63E46D@sun.com>
References: <2b20c7140912130831t658c2790hb38c3752c18ec128@mail.gmail.com>
	<2b20c7140912131047qf0417f1mf3f3d9a8fc751dc0@mail.gmail.com>
	<5F225623-87E0-4A91-988C-8B789B63E46D@sun.com>
Message-ID: <3ed212d00912160924w5c823207h35378e814e0a6c23@mail.gmail.com>

hi Andreas,

On Tue, Dec 15, 2009 at 11:05 PM, Andreas Dilger <adilger at sun.com> wrote:

> Hello Peter,
>
>
> > 2. We need MUCH more in the repository than Sun is putting into it.
> >
> > There are many development branches and sometimes abandoned projects
> > that still have a lot of value.  For example, there is a nearly
> > complete OS X client port - what if someone wanted to pick that up?
> > Similarly, there are projects like size on MDS or the network
> > request scheduler that may need help from the community to get
> > finished or re-prioritized.
>
> All of the Lustre history is still available in CVS, as it has always
> been.  As far as I know (I've never checked it out myself) even the OS/
> X client port is publicly available today, which was not true a few
> years ago.
>

I would say, some time ago publicly available cvs repository "did not feel
well". I could not find cmd2 (seems to be not available todays too), gns or
even cmd3 code and any attempt to convert it to git or do anything that
requires "cvs rlog" resulted in this:

:pserver:anonymous at cvs.lustre.org:/lustre lustre-core
password file:
cvsroot: :pserver:anonymous at cvs.lustre.org:2401/lustre
pass: A
I LOVE YOU
cvs rlog: Logging lustre-core
cvs rlog: Logging lustre-core/asobd
cvs rlog: Logging lustre-core/autoconf
cvs rlog: Logging lustre-core/cmm
cvs rlog: Logging lustre-core/cmobd
cvs rlog: Logging lustre-core/cobd
cvs rlog: Logging lustre-core/conf
cvs rlog: Logging lustre-core/contrib
cvs rlog: Logging lustre-core/demos
cvs [rlog aborted]: unexpected '\x0' reading revision number in RCS
file /lustre/lustre-core/demos/Attic/snaprest.sh,v
exit: 1

Obviously demos/snaprest.sh,v contained garbage. Now it seems to be fixed.

I think this is why Peter said it is "not clear" if all valuable branches
are available in public cvs.

Thanks.

-- 
umka
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20091216/1119305f/attachment.html>

From adilger at sun.com  Wed Dec 16 22:47:29 2009
From: adilger at sun.com (Andreas Dilger)
Date: Wed, 16 Dec 2009 15:47:29 -0700
Subject: [Lustre-devel] Fwd: The Lustre community and GIT
In-Reply-To: <3ed212d00912160924w5c823207h35378e814e0a6c23@mail.gmail.com>
References: <2b20c7140912130831t658c2790hb38c3752c18ec128@mail.gmail.com>
	<2b20c7140912131047qf0417f1mf3f3d9a8fc751dc0@mail.gmail.com>
	<5F225623-87E0-4A91-988C-8B789B63E46D@sun.com>
	<3ed212d00912160924w5c823207h35378e814e0a6c23@mail.gmail.com>
Message-ID: <999B10A5-95F8-47B5-8A87-17E1ED18F260@sun.com>

On 2009-12-16, at 10:24, Yuriy Umanets wrote:
> Andreas Dilger wrote:
>> Peter Braam wrote:
>> > 2. We need MUCH more in the repository than Sun is putting into it.
>> >
>> > There are many development branches and sometimes abandoned  
>> projects
>> > that still have a lot of value.  For example, there is a nearly
>> > complete OS X client port - what if someone wanted to pick that up?
>> > Similarly, there are projects like size on MDS or the network
>> > request scheduler that may need help from the community to get
>> > finished or re-prioritized.
>>
>> All of the Lustre history is still available in CVS, as it has always
>> been.  As far as I know (I've never checked it out myself) even the  
>> OS/
>> X client port is publicly available today, which was not true a few
>> years ago.
>
> I would say, some time ago publicly available cvs repository "did  
> not feel well".

Was this from a checkout, or only via CVS scraping tools like cvs2git?

> I could not find cmd2 (seems to be not available todays too), gns or  
> even cmd3 code

Doing a "cvs log lustre/ChangeLog" shows that tags for all of those  
projects exist on that file.  I haven't tried to do a checkout of one  
of those tags, but I'd expect them to be there.  I know that ages ago  
some branches were deleted (in particular b_devel), but I don't think  
there is anything of interest there.

> and any attempt to convert it to git or do anything that requires  
> "cvs rlog" resulted in this:
>
> cvs [rlog aborted]: unexpected '\x0' reading revision number in RCS
> file /lustre/lustre-core/demos/Attic/snaprest.sh,v
> exit: 1
>
> Obviously demos/snaprest.sh,v contained garbage. Now it seems to be  
> fixed.

This file was corrupted at some point in the past (binary zeros  
appended to the file), as was documented in bugs 17080 and 18564.   
That was fixed this past summer when we started our own Git conversion  
process and noticed the same problem.

That was never a problem for checking out useful branches either  
internally or externally, because I don't think snaprest.sh was used  
since early 2000 when Peter and I were working on snapfs, and it was  
deleted several years ago due to being completely obsolete.  It only  
was noticeable when trying to scrape the entire repository, including  
all of the history.

> I think this is why Peter said it is "not clear" if all valuable  
> branches are available in public cvs.


It's unfortunate that incorrect conclusions were reached with  
incomplete data.

The problem isn't whether all of the valuable branches are available  
in CVS, but rather which branches are the valuable ones?  I just  
checked, and there are 7638 tags in CVS, including 453 branches and  
490 release tags.  If there are some useful bits in there that you are  
aware of, I'd be happy to see them extracted from the rest and brought  
back to life.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.


From morrone2 at llnl.gov  Wed Dec 16 23:01:23 2009
From: morrone2 at llnl.gov (Christopher J. Morrone)
Date: Wed, 16 Dec 2009 15:01:23 -0800
Subject: [Lustre-devel] Fwd: The Lustre community and GIT
In-Reply-To: <9784AD2F-B1E4-4A48-BDB5-6B932437EBD9@sun.com>
References: <2b20c7140912130831t658c2790hb38c3752c18ec128@mail.gmail.com>
	<2b20c7140912131047qf0417f1mf3f3d9a8fc751dc0@mail.gmail.com>
	<5F225623-87E0-4A91-988C-8B789B63E46D@sun.com>
	<4B280B89.3010303@llnl.gov>
	<F661C7A0-E3C8-4B69-A275-F23844DDFCAB@sun.com>
	<4B283577.3070905@llnl.gov>
	<9784AD2F-B1E4-4A48-BDB5-6B932437EBD9@sun.com>
Message-ID: <4B2966C3.1050707@llnl.gov>

Andreas Dilger wrote:

> I know that Brian Behlendorf and Ricardo investigated a number of 
> different tools for managing patches, and topgit was the one they 
> chose.  Another tool in this vein is "guilt", which manages a patch 
> series in a way similar to "quilt".

Yes, I have discussed this at length with Brian.

The main problem with guilt is that seems to be designed with the single 
developer in mind, and not really well suited for sharing.  The major 
problem being that the patches and series information are not stored in 
git.  You need to come up with your own method of storing and keeping 
track of the patch set.

With topgit, anyone can simply clone our repo and checkout the "t18/top" 
branch of lustre, and have a tree that contains all of our patches. 
They don't really even need to use topgit.  And, in fact, I think Brian 
is the only person using topgit directly on the project that you are 
referring to.  The others usually just branch off one of his topgit 
branches and pass patches back to Brian.

> For developers that aren't keeping a huge number of patches, and 
> especially not a large number of dependent patches, "git rebase" is 
> probably sufficient. That will refresh the patches against changes in 
> the upstream repository.

Except that "git rebase" is a destructive operation, right?  It is fine 
for the single developer to do that, but you basically can't rebase a 
branch that you have published.  Here is the kind of comment that I see 
associate with git rebase over and over on the web:

"You should only use git rebase on your local-only branches. Its purpose 
is to keep your local, invisible changes up-to-date so that when you 
publish them they'll be more relevant and easy to understand for others."

So rebase is off limits to us when managing our published "llnl" branch. 
  (This is also why topgit uses "git merge" internally, not "git rebase".)

> As for tracking hidden semantic conflicts that do not have contextual 
> conflicts, that is of course more difficult.  I think one important way 
> to track this is that our own commit message policy will be to include 
> the bug number into the "short" (one line) patch description, and if you 
> do a "git rebase" against the updated repo and your patch for that bug 
> does not disappear due to the upstream commit for that bug, the patch 
> should be reviewed.

Yes, that might work.  I am hoping that it will work, but I am not 
entirely convinced yet.  If we were using git rebase, our history would 
be shorter and easer to follow.  But we can't use rebase.  So I'm not 
sure how easy it will be to track things using the log with repeated merges.

Maybe we just need to try it and see how it goes.

> I think that depends on how topgit is doing the updates against the 
> upstream repository.  If it is doing a "rebase" each time, then there 
> should only be a single git branch (each with a single commit) for each 
> patch that you have applied.  If it does this via a "merge" then there 
> will be new commits for each pull that you do from upstream that is 
> resolving conflicts, and the repo will become a mess.

It uses "git merge", because "git rebase" would basically make the whole 
thing unsharable.  Many of the merges turn into fast-forwards, but not 
all of them.

Chris

From yuriy.umanets at clusterstor.com  Thu Dec 17 12:12:04 2009
From: yuriy.umanets at clusterstor.com (Yuriy Umanets)
Date: Thu, 17 Dec 2009 14:12:04 +0200
Subject: [Lustre-devel] Fwd: The Lustre community and GIT
In-Reply-To: <999B10A5-95F8-47B5-8A87-17E1ED18F260@sun.com>
References: <2b20c7140912130831t658c2790hb38c3752c18ec128@mail.gmail.com>
	<2b20c7140912131047qf0417f1mf3f3d9a8fc751dc0@mail.gmail.com>
	<5F225623-87E0-4A91-988C-8B789B63E46D@sun.com>
	<3ed212d00912160924w5c823207h35378e814e0a6c23@mail.gmail.com>
	<999B10A5-95F8-47B5-8A87-17E1ED18F260@sun.com>
Message-ID: <3ed212d00912170412q793c9cfja032bc41a712f0df@mail.gmail.com>

On Thu, Dec 17, 2009 at 12:47 AM, Andreas Dilger <adilger at sun.com> wrote:

> On 2009-12-16, at 10:24, Yuriy Umanets wrote:
>
>> Andreas Dilger wrote:
>>
>>  Peter Braam wrote:
>>> > 2. We need MUCH more in the repository than Sun is putting into it.
>>> >
>>> > There are many development branches and sometimes abandoned projects
>>> > that still have a lot of value.  For example, there is a nearly
>>> > complete OS X client port - what if someone wanted to pick that up?
>>> > Similarly, there are projects like size on MDS or the network
>>> > request scheduler that may need help from the community to get
>>> > finished or re-prioritized.
>>>
>>> All of the Lustre history is still available in CVS, as it has always
>>> been.  As far as I know (I've never checked it out myself) even the OS/
>>> X client port is publicly available today, which was not true a few
>>> years ago.
>>>
>>
>> I would say, some time ago publicly available cvs repository "did not feel
>> well".
>>
>
> Was this from a checkout, or only via CVS scraping tools like cvs2git?


For example, git cvsimport. Additionally, cvs contained *.*,v files in wrong
format

>
>
>  I could not find cmd2 (seems to be not available todays too), gns or even
>> cmd3 code
>>
>
> Doing a "cvs log lustre/ChangeLog" shows that tags for all of those
> projects exist on that file.  I haven't tried to do a checkout of one of
> those tags, but I'd expect them to be there.  I know that ages ago some
> branches were deleted (in particular b_devel), but I don't think there is
> anything of interest there.
>

Cannot recall exactly, seems that I could not checkout specific
tags/branches while tags them selves were existing. Is it possible that I
happen to do that in the middle of some cvs migration process? I know there
were couple of them lastly.

>
>  and any attempt to convert it to git or do anything that requires "cvs
>> rlog" resulted in this:
>>
>> cvs [rlog aborted]: unexpected '\x0' reading revision number in RCS
>> file /lustre/lustre-core/demos/Attic/snaprest.sh,v
>> exit: 1
>>
>> Obviously demos/snaprest.sh,v contained garbage. Now it seems to be fixed.
>>
>
> This file was corrupted at some point in the past (binary zeros appended to
> the file), as was documented in bugs 17080 and 18564.  That was fixed this
> past summer when we started our own Git conversion process and noticed the
> same problem.
>
Sorry to contradict but it was not fixed at Sep 21, the date I tried it last
time.


>
> That was never a problem for checking out useful branches either internally
> or externally, because I don't think snaprest.sh was used since early 2000
> when Peter and I were working on snapfs, and it was deleted several years
> ago due to being completely obsolete.  It only was noticeable when trying to
> scrape the entire repository, including all of the history.

Yeah, unfortunately it was in "demos" directory which is one of first, so
that, it failed very quickly.

>
>
>  I think this is why Peter said it is "not clear" if all valuable branches
>> are available in public cvs.
>>
>
>
> It's unfortunate that incorrect conclusions were reached with incomplete
> data.
>
> The problem isn't whether all of the valuable branches are available in
> CVS, but rather which branches are the valuable ones?  I just checked, and
> there are 7638 tags in CVS, including 453 branches and 490 release tags.  If
> there are some useful bits in there that you are aware of, I'd be happy to
> see them extracted from the rest and brought back to life.

Sure I can name some of them but only those I participated or aware of.
Little % of those thousands tags:

mac os client, cobd + cmobd + mds on tmpfs, gns, cmd2, cmd3 branches, som,
crow (may be), hendrix project tests for cmd2 and cmd3

Thanks.


>
>
> Cheers, Andreas
> --
> Andreas Dilger
> Sr. Staff Engineer, Lustre Group
> Sun Microsystems of Canada, Inc.
>
>


-- 
umka
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20091217/d9bb45f8/attachment.html>

From dillowda at ornl.gov  Thu Dec 17 02:53:13 2009
From: dillowda at ornl.gov (David Dillow)
Date: Wed, 16 Dec 2009 21:53:13 -0500
Subject: [Lustre-devel] Fwd: The Lustre community and GIT
In-Reply-To: <4B2966C3.1050707@llnl.gov>
References: <2b20c7140912130831t658c2790hb38c3752c18ec128@mail.gmail.com>
	<2b20c7140912131047qf0417f1mf3f3d9a8fc751dc0@mail.gmail.com>
	<5F225623-87E0-4A91-988C-8B789B63E46D@sun.com>
	<4B280B89.3010303@llnl.gov>
	<F661C7A0-E3C8-4B69-A275-F23844DDFCAB@sun.com>
	<4B283577.3070905@llnl.gov>
	<9784AD2F-B1E4-4A48-BDB5-6B932437EBD9@sun.com>
	<4B2966C3.1050707@llnl.gov>
Message-ID: <1261018393.30644.53.camel@obelisk.thedillows.org>

On Wed, 2009-12-16 at 15:01 -0800, Christopher J. Morrone wrote:
> Andreas Dilger wrote:
> > For developers that aren't keeping a huge number of patches, and 
> > especially not a large number of dependent patches, "git rebase" is 
> > probably sufficient. That will refresh the patches against changes in 
> > the upstream repository.
> 
> Except that "git rebase" is a destructive operation, right?  It is fine 
> for the single developer to do that, but you basically can't rebase a 
> branch that you have published.  Here is the kind of comment that I see 
> associate with git rebase over and over on the web:
> 
> "You should only use git rebase on your local-only branches. Its purpose 
> is to keep your local, invisible changes up-to-date so that when you 
> publish them they'll be more relevant and easy to understand for others."
> 
> So rebase is off limits to us when managing our published "llnl" branch. 
>   (This is also why topgit uses "git merge" internally, not "git rebase".)

It's not a rule written in stone. If you have branches on which you
expect others to base their work, rebasing is painful for them as they
have to keep rebasing their work on top of the ones you do. But, this
can be OK if this is communicated to those that would follow your work.
Dave Miller used to rebase often with the net-2.6 tree, but stopped due
to the pain it was causing people working with him; it does occasionally
get rebased (and an email sent to netdev) if there is a problem that
would result in intolerable ugliness in the history.

linux-next is essentially rebased every day, but keeps tags to the
previous heads so that they don't get lost. Ie, if one wants to develop
against linux-next, they should use a tag such as next-20091216 to get a
stable base as master gets rewound daily. Similarly, the "pu" branch of
git is often rewound, and this fact is well advertised.

Certainly, if your changes are to go upstream via git pull and not via
an email patch series, then I think rebasing -- at least before the send
-- is a good idea, just to keep the history clean of distracting commits
like "Oops, forgot to add this file."

-- 
Dave Dillow
National Center for Computational Science
Oak Ridge National Laboratory
(865) 241-6602 office


From jackyfriendly at sohu.com  Tue Dec 22 09:52:32 2009
From: jackyfriendly at sohu.com (=?gb2312?B?wc7U2dGn?=)
Date: Tue, 22 Dec 2009 17:52:32 +0800
Subject: [Lustre-devel] lustre configure error when compile
Message-ID: <200912221752285466212@sohu.com>

Hello All
	checking for /usr/src/kernels/linux-2.6.18-128.1.14/include/linux/xattr_acl.h... no
	checking for /usr/src/kernels/linux-2.6.18-128.1.14/include/linux/posix_acl_xattr.h... yes
	checking if linux/posix_acl_xattr.h can be compiled... yes
	checking calc acl size... configure: error: ACL size can't computed
this is configure results,please help me to resolve it
thanks
Best Regards


From erblichs at earthlink.net  Tue Dec 22 10:41:18 2009
From: erblichs at earthlink.net (Mitchell Erblich)
Date: Tue, 22 Dec 2009 02:41:18 -0800
Subject: [Lustre-devel] lustre configure error when compile
In-Reply-To: <200912221752285466212@sohu.com>
References: <200912221752285466212@sohu.com>
Message-ID: <B20C2B25-CD9E-4F83-89AD-2E24F5001690@earthlink.net>

Group,

		This is probably wrong as it is the fastest answer.

		Identify what name / release. Redhat, SuSE, SELES, etc..

		Goto kernel.org..

		See if xattr_acl.h is present and if so, copy it in and rerun  
configure.

		Report results..

		Mitchell Erblich
		=================


On Dec 22, 2009, at 1:52 AM, 廖再学 wrote:

> Hello All
> 	checking for /usr/src/kernels/linux-2.6.18-128.1.14/include/linux/ 
> xattr_acl.h... no
> 	checking for /usr/src/kernels/linux-2.6.18-128.1.14/include/linux/ 
> posix_acl_xattr.h... yes
> 	checking if linux/posix_acl_xattr.h can be compiled... yes
> 	checking calc acl size... configure: error: ACL size can't computed
> this is configure results,please help me to resolve it
> thanks
> Best Regards
>
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-devel


From yuriy.umanets at clusterstor.com  Tue Dec 22 12:27:53 2009
From: yuriy.umanets at clusterstor.com (Yuriy Umanets)
Date: Tue, 22 Dec 2009 14:27:53 +0200
Subject: [Lustre-devel] lustre configure error when compile
In-Reply-To: <B20C2B25-CD9E-4F83-89AD-2E24F5001690@earthlink.net>
References: <200912221752285466212@sohu.com>
	<B20C2B25-CD9E-4F83-89AD-2E24F5001690@earthlink.net>
Message-ID: <3ed212d00912220427kb81090bo96a7530afbdb7e1@mail.gmail.com>

It would really help if we could see configure logs. Usually configure
script generates small program with using include we're checking for and
then tries to build it. Failed build means this include file cannot be used
but seeing what exactly gcc did say would be just great.

Thanks.

On Tue, Dec 22, 2009 at 12:41 PM, Mitchell Erblich
<erblichs at earthlink.net>wrote:

> Group,
>
>                This is probably wrong as it is the fastest answer.
>
>                Identify what name / release. Redhat, SuSE, SELES, etc..
>
>                Goto kernel.org..
>
>                See if xattr_acl.h is present and if so, copy it in and
> rerun
> configure.
>
>                Report results..
>
>                Mitchell Erblich
>                =================
>
>
> On Dec 22, 2009, at 1:52 AM, 廖再学 wrote:
>
> > Hello All
> >       checking for /usr/src/kernels/linux-2.6.18-128.1.14/include/linux/
> > xattr_acl.h... no
> >       checking for /usr/src/kernels/linux-2.6.18-128.1.14/include/linux/
> > posix_acl_xattr.h... yes
> >       checking if linux/posix_acl_xattr.h can be compiled... yes
> >       checking calc acl size... configure: error: ACL size can't computed
> > this is configure results,please help me to resolve it
> > thanks
> > Best Regards
> >
> > _______________________________________________
> > Lustre-devel mailing list
> > Lustre-devel at lists.lustre.org
> > http://lists.lustre.org/mailman/listinfo/lustre-devel
>
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-devel
>



-- 
umka
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20091222/1f6a9f2e/attachment.html>

From adilger at sun.com  Tue Dec 22 23:49:51 2009
From: adilger at sun.com (Andreas Dilger)
Date: Tue, 22 Dec 2009 16:49:51 -0700
Subject: [Lustre-devel] lustre configure error when compile
In-Reply-To: <B20C2B25-CD9E-4F83-89AD-2E24F5001690@earthlink.net>
References: <200912221752285466212@sohu.com>
	<B20C2B25-CD9E-4F83-89AD-2E24F5001690@earthlink.net>
Message-ID: <09190CAE-4B0A-4770-BD31-4C3EBAF7C6DC@sun.com>

On 2009-12-22, at 03:41, Mitchell Erblich wrote:
> 		This is probably wrong as it is the fastest answer.
> 		Identify what name / release. Redhat, SuSE, SELES, etc..
> 		Goto kernel.org..
> 		See if xattr_acl.h is present and if so, copy it in and 		rerun  
> configure.

Sorry, but this is indeed the completely wrong answer.  The fact that  
xattr_acl.h is missing is not in itself a problem.

> On Dec 22, 2009, at 1:52 AM, 廖再学 wrote:
>> 	checking for /usr/src/kernels/linux-2.6.18-128.1.14/include/linux/
>> xattr_acl.h... no
>> 	checking for /usr/src/kernels/linux-2.6.18-128.1.14/include/linux/
>> posix_acl_xattr.h... yes
>> 	checking if linux/posix_acl_xattr.h can be compiled... yes
>> 	checking calc acl size... configure: error: ACL size can't computed


What version of Lustre are you trying to compile?

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.


From jackyfriendly at sohu.com  Wed Dec 23 00:57:37 2009
From: jackyfriendly at sohu.com (=?utf-8?B?5buW5YaN5a2m?=)
Date: Wed, 23 Dec 2009 08:57:37 +0800
Subject: [Lustre-devel] =?utf-8?q?lustre_configure_error_when_compile?=
References: <200912221752285466212@sohu.com>,
	<B20C2B25-CD9E-4F83-89AD-2E24F5001690@earthlink.net>,
	<3ed212d00912220427kb81090bo96a7530afbdb7e1@mail.gmail.com>
Message-ID: <200912230857362033753@sohu.com>

Hi
Thanks your help,now i resolved it.i run autogen.sh again and this error is not appeared.
Thanks 
Best Regards
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20091223/ecad660a/attachment.html>

From peter.braam at clusterstor.com  Sat Dec 26 15:01:31 2009
From: peter.braam at clusterstor.com (Peter Braam)
Date: Sat, 26 Dec 2009 08:01:31 -0700
Subject: [Lustre-devel] Fwd: The Lustre community and GIT
In-Reply-To: <5F225623-87E0-4A91-988C-8B789B63E46D@sun.com>
References: <2b20c7140912130831t658c2790hb38c3752c18ec128@mail.gmail.com>
	<2b20c7140912131047qf0417f1mf3f3d9a8fc751dc0@mail.gmail.com>
	<5F225623-87E0-4A91-988C-8B789B63E46D@sun.com>
Message-ID: <0BF036E2-E2A0-4612-B567-4C78794B0C45@clusterstor.com>



Sent from my iPhone

On Dec 15, 2009, at 14:05, Andreas Dilger <adilger at sun.com> wrote:

> Hello Peter,
>
> As previously announced, we have already made the initial Lustre  
> repository available to everyone this week at git.lustre.org.  This  
> repository contains all of the main development and release branches  
> and their history.  It's where all of the Sun developers and  
> maintaners will get their sources from, and it will continue to be  
> available under the GPL v2 license as it always has been.
>
> Sun's policy of prioritizing stability, not just for releases but also
> to underpin development, makes safeguarding the quality of this
> repository paramount.  Everyone contributing to the Lustre sources -
> not just internal Sun engineers but also external contributors like
> LLNL, CEA, ORNL and DDN - is therefore subject to the same gatekeeping
> procedures and must follow the same patch submission, testing and
> landing processes.  These are further detailed in the wiki pages that
> were in the original announcement:
>
> http://wiki.lustre.org/index.php/Accessing_Lustre_Code
> http://wiki.lustre.org/index.php/Submitting_Patches
>
>

Sun is no longer supporting the Lustre community (storage OEMs and  
others have been refused further support).  The model you describe  
won't apply much longer I think.  People will be looking for different  
arrangements.

Peter



> For people not familiar with Git, it should be clarified that  
> limiting commits to the Sun Prime repository does not in any way  
> restrict access to the Lustre code, or the ability of non-Sun  
> developers to share their own Git clone with the Lustre community.   
> Developers will be able to host their own Lustre clones (essentially  
> full repositories) as and where they wish.  The major advantage of  
> Git is that anyone can pull from any repository, or in fact from a  
> number of different repositories at the same time - this choice is  
> left to the user.
>
> This is the same model used by the Linux kernel, and has proven to  
> work well with Git.  Each kernel developer hosts their own clone(s)  
> at git.kernel.org, or at some other site like repo.or.cz, or github.com 
> , and when they have changes to submit upstream they send an email  
> (usually containing both the patch, and the git URL to pull from) to  
> the subsystem maintainer, or to Linus directly, requesting that he  
> pull their changes.  This pull request is normally only sent after  
> the patch has been reviewed, possibly multiple times,  by the  
> appropriate subsystem maintainers.  Each developer has full control  
> over their own clone, and in fact it is rare that more than one  
> person has push access to a clone.
>
> The fact that there are many different clones (some more important,  
> and others less so) in no way implies that the kernel is being  
> forked, but rather that this is the standard way in which to use Git  
> to exchange changes.  The location at which a clone is hosted also  
> has no bearing on its usefulness.
>
>
>
> To answer your specific concerns in more detail,
>
> On 2009-12-13, at 11:47, Peter Braam wrote:
>> 1. We need a public repository, where non Sun community members can  
>> commit.
>>
>> a.  There are Lustre users than have their own branches.  LLNL has  
>> probably the best tested branch among all users.  DDN's customers  
>> have a version of Lustre that includes patches not yet in Sun's  
>> releases.  It would be very valuable if these kind of releases  
>> could be collected in a publicly accessible GIT repository.
>
> That is true even today - most of the patches that are in the DDN  
> branches were initially developed by Sun and are already in the  
> publicly-available Lustre CVS, and many of the LLNL patches have or  
> are being landed for the 1.8.2 release.
>
> As you know, there will always be a delta of patches that are not in  
> an official release, and will be available in the next one.  With  
> the increase in testing to improve the quality of new releases  
> compared to the past, releases are by necessity less frequent.   
> Should anyone have a desire for more bleeding-edge code, they have  
> always been able fetch the current branch before its release,  
> regardless of whether this is Git or CVS.  We maintain a number of  
> different branches (b1_8 for incremental development,  
> b_release_1_8_1 for critical fixes on the 1.8.1 release, etc) these  
> are already available to the public.
>
>> I doubt that Sun will give commit rights to external entities (this  
>> is not unreasonable, Sun needs to control what code enters its  
>> repositories).  Hence I think that the community would be better  
>> served with a GIT repository in a public place, like github, that  
>> can give such access.
>
> While CVS was very restrictive in terms of managing external commit  
> permissions due to its monolithic repository, some external  
> contributors have had commit access to CVS prior our migration to  
> Git, based on need.  With the migration to Git there is no need to  
> manage such access ourselves, as developers are able to host their  
> clones wherever they want.  The git.lustre.org repository will  
> remain the canonical source for Sun releases, and we will be happy  
> to pull fixes into this repository that meet the quality guidelines  
> stated above.  I for one would welcome external inspectors on  
> patches, and we continue to work with external sites to do scale  
> testing of Lustre.
>
>> My group at ClusterStor has reserved the "lustre" project at Github  
>> and we will give keys to any and all organizations that wish to  
>> make serious contributions to Lustre.  I was in particular hoping  
>> that LLNL would be willing to commit their releases to that public  
>> place.
>
>
> Note that with Git and github.com there is no need to give keys to  
> anyone, and in fact that would seem to be detrimental to  
> ClusterStor, because the "lustre" clone you have reserved is within  
> your company's private hosting space (i.e. http://github.com/clusterstor/lustre 
> ).  With Github (or Git in general) it is possible for anyone to  
> make their own clone or mirror of a repository at any time, no keys  
> or permission required, and it will appear ashttp://github.com/ 
> {user}/lustre or whatever they want to call it.
>
>> b.  Everyone is uncertain about what Sun will do with Lustre  
>> (proprietary Solaris / ZFS server releases and discontinued support  
>> for Linux servers have been mentioned to me several times now).  A  
>> public repository with the open code will be valuable for the  
>> community and promote continued development and access.
>
> I agree that a public repository is important, and we will continue  
> to host one at git.lustre.org as we have in the past for CVS, and it  
> can and should be cloned as needed.  As you are hopefully aware,  
> with Git every checkout has a full copy of the repository, including  
> all history, so the code is already very "open" and "public".   
> Lustre is, and will continue to be, available as GPL software.
>
> We definitely welcome external contributions to Lustre, which have  
> in the past been done by a small number of people outside of Sun.  I  
> don't think the choice of CVS or Git or hosting site has ever been a  
> limiting factor in this regard.  We look forward to contributions of  
> fixes, designs, and features, from ClusterStor, as we would with any  
> Lustre contributor.
>
>> 2. We need MUCH more in the repository than Sun is putting into it.
>>
>> There are many development branches and sometimes abandoned  
>> projects that still have a lot of value.  For example, there is a  
>> nearly complete OS X client port - what if someone wanted to pick  
>> that up?  Similarly, there are projects like size on MDS or the  
>> network request scheduler that may need help from the community to  
>> get finished or re-prioritized.
>
> All of the Lustre history is still available in CVS, as it has  
> always been.  As far as I know (I've never checked it out myself)  
> even the OS/X client port is publicly available today, which was not  
> true a few years ago.
>
> Due to the convoluted branch, repository, and tag abuse that was  
> used to manage the Lustre code in CVS, there is a non-zero effort  
> required to migrate any of the branches in CVS to Git.  Rather than  
> bring all of the old detritus along into Git, only the main  
> development/production branches are being automatically migrated  
> initially.
>
> Now that the Git migration is complete, the Sun maintainers of non- 
> release features (HSM, SOM, CMD, NRS, etc) will be creating Git  
> clones and landing their work into them as time permits.
>
> If anyone wants to import one of the other CVS branches (e.g. OS/X),  
> all they will need to do is create a patch from that branch and then  
> commit this into their own Git clone.
>
>> It is unclear to me if these kind of branches can be found in the  
>> publicly available CVS.
>
> Yes, they can, as they always have been.  The CVS repository will  
> remain available indefinitely for spelunking expeditions should the  
> need arise.  Note that the full history that leads to the current  
> release branches (b1_6, b1_8, HEAD) is available in Git, so there is  
> at least a trail of breadcrumbs leading to where we are today.
>
>> If they can, a collection of relevant branches, broadly along the  
>> lines of what I mention above, should be placed in the public GIT  
>> repository.
>
> For currently-active branches this was already our plan.  For  
> historical and inactive branches, and we welcome any contributions  
> that bring these ancient CVS branches back to life.  Nikita is  
> probably best positioned to know what needs to be imported from the  
> OS/X branch in CVS, and if there are other particular branches that  
> contain important work we'll be happy to discuss them with you.  It  
> will of course be possible to create Git clones for these old  
> branches as needed.
>
>> 3. Brian's email message seems to indicate that Sun's git  
>> repository will be used for Sun development.  In the past there  
>> were two CVS repositories - a read-only one that was publicly  
>> accessible and when I last controlled the group it was updated very  
>> frequently with all open source code (presently, it seems to only  
>> contain releases, not the development stream of commits).
>
> That's not correct.  The public cvs.lustre.org repository in fact  
> contains all of the Lustre commits and all of the history.  I just  
> did a checkout and there are in fact tags and branches for the pre- 
> release 1.8.2 builds, lprocfs rewrite, CMD, etc. that are very much  
> works in progress.
>
> One of the very last commits on CVS HEAD was on Thursday before CVS  
> went read-only, from a patch submitted by LLNL:
>
> revision 1.593
> date: 2009/12/10 13:52:51;  author: dzogin;  state: Exp;  lines: +5 -0
> Branch HEAD
> b=21259
> i=andrew.perepechko
> i=alexey.lyashkov
> ----------------------------------------------------------------------
> Description: Allow non-root access for "lfs check".
> Details    : Added a check in obd_class_ioctl() for  
> OBD_IOC_PING_TARGET.
>
>
>> It is unclear how Sun can manage development with this git  
>> repository given that parts of its work are proprietary (like the  
>> Windows port) or unreleased (like the ZFS work).  Can we get more  
>> insight in what Brian is alluding to?
>
> I'm not sure what "alluding" you are alluding to?
>
> Of course, if there is any proprietary code developed it will just  
> not be pushed to the public git.lustre.org repository.  I expect  
> that any proprietary or unreleased code that ClusterStor is already  
> or will develop will similarly not be posted publicly until you are  
> ready to do so.
>
> Sun's current in-progress code will very likely reside in separate  
> clones at git.lustre.org, but will not be merged into the Sun Prime  
> repository for an official release until it is inspected, tested,  
> and has permission to land.  Pulls into the Sun Prime repository  
> will be done by the release manager for each branch, as previously  
> stated.  The Lustre engineers at ClusterStor are already familiar  
> with this process and have already had their first patch pass  
> inspections and it is ready for landing.
>
> That is one of the major benefits of Git over CVS - that there ISN'T  
> a single repository, and in fact every clone has a full copy of the  
> history and can be used by anyone.  There is no need to have all of  
> the development done in branches on a single repository, but rather  
> to keep projects in their own clones.
>
> That allows developers to do local commits on their machines, to  
> push it to their public clone if they want to share their work and/ 
> or make an offsite backup, and people are free to choose which clone  
> one they use.  Sun's Prime repository used for releases will only  
> contain stable Lustre code.  This is the Git usage model for the  
> Linux kernel, and think it is a good one to follow for Lustre as  
> well.  You are free to manage your clones in some other way.
>
> Cheers, Andreas
> --
> Andreas Dilger
> Sr. Staff Engineer, Lustre Group
> Sun Microsystems of Canada, Inc.

From magawake at gmail.com  Mon Dec 28 00:49:01 2009
From: magawake at gmail.com (Mag Gam)
Date: Sun, 27 Dec 2009 19:49:01 -0500
Subject: [Lustre-devel] Fwd: The Lustre community and GIT
In-Reply-To: <0BF036E2-E2A0-4612-B567-4C78794B0C45@clusterstor.com>
References: <2b20c7140912130831t658c2790hb38c3752c18ec128@mail.gmail.com>
	<2b20c7140912131047qf0417f1mf3f3d9a8fc751dc0@mail.gmail.com>
	<5F225623-87E0-4A91-988C-8B789B63E46D@sun.com>
	<0BF036E2-E2A0-4612-B567-4C78794B0C45@clusterstor.com>
Message-ID: <1cbd6f830912271649n188095dcp9c65d96bd16eb71e@mail.gmail.com>

I though majority of the lustre development and support was from Sun.
This is news to us.

Nevertheless I am glad Sun did this conversion.

Couple of questions:

1) What will happen to the Wiki? It still has a lot of Sun stuff on it (logo)
2) Will we still keep bugzilla.lustre.org?
3) Will there ever be a git over http? This is important for people
who are behind firewalls.

Also, it would be nice to have the RPMs on the Wiki instead of
registering with Sun.




On Sat, Dec 26, 2009 at 10:01 AM, Peter Braam
<peter.braam at clusterstor.com> wrote:
>
>
> Sent from my iPhone
>
> On Dec 15, 2009, at 14:05, Andreas Dilger <adilger at sun.com> wrote:
>
>> Hello Peter,
>>
>> As previously announced, we have already made the initial Lustre
>> repository available to everyone this week at git.lustre.org.  This
>> repository contains all of the main development and release branches
>> and their history.  It's where all of the Sun developers and
>> maintaners will get their sources from, and it will continue to be
>> available under the GPL v2 license as it always has been.
>>
>> Sun's policy of prioritizing stability, not just for releases but also
>> to underpin development, makes safeguarding the quality of this
>> repository paramount.  Everyone contributing to the Lustre sources -
>> not just internal Sun engineers but also external contributors like
>> LLNL, CEA, ORNL and DDN - is therefore subject to the same gatekeeping
>> procedures and must follow the same patch submission, testing and
>> landing processes.  These are further detailed in the wiki pages that
>> were in the original announcement:
>>
>> http://wiki.lustre.org/index.php/Accessing_Lustre_Code
>> http://wiki.lustre.org/index.php/Submitting_Patches
>>
>>
>
> Sun is no longer supporting the Lustre community (storage OEMs and
> others have been refused further support).  The model you describe
> won't apply much longer I think.  People will be looking for different
> arrangements.
>
> Peter
>
>
>
>> For people not familiar with Git, it should be clarified that
>> limiting commits to the Sun Prime repository does not in any way
>> restrict access to the Lustre code, or the ability of non-Sun
>> developers to share their own Git clone with the Lustre community.
>> Developers will be able to host their own Lustre clones (essentially
>> full repositories) as and where they wish.  The major advantage of
>> Git is that anyone can pull from any repository, or in fact from a
>> number of different repositories at the same time - this choice is
>> left to the user.
>>
>> This is the same model used by the Linux kernel, and has proven to
>> work well with Git.  Each kernel developer hosts their own clone(s)
>> at git.kernel.org, or at some other site like repo.or.cz, or github.com
>> , and when they have changes to submit upstream they send an email
>> (usually containing both the patch, and the git URL to pull from) to
>> the subsystem maintainer, or to Linus directly, requesting that he
>> pull their changes.  This pull request is normally only sent after
>> the patch has been reviewed, possibly multiple times,  by the
>> appropriate subsystem maintainers.  Each developer has full control
>> over their own clone, and in fact it is rare that more than one
>> person has push access to a clone.
>>
>> The fact that there are many different clones (some more important,
>> and others less so) in no way implies that the kernel is being
>> forked, but rather that this is the standard way in which to use Git
>> to exchange changes.  The location at which a clone is hosted also
>> has no bearing on its usefulness.
>>
>>
>>
>> To answer your specific concerns in more detail,
>>
>> On 2009-12-13, at 11:47, Peter Braam wrote:
>>> 1. We need a public repository, where non Sun community members can
>>> commit.
>>>
>>> a.  There are Lustre users than have their own branches.  LLNL has
>>> probably the best tested branch among all users.  DDN's customers
>>> have a version of Lustre that includes patches not yet in Sun's
>>> releases.  It would be very valuable if these kind of releases
>>> could be collected in a publicly accessible GIT repository.
>>
>> That is true even today - most of the patches that are in the DDN
>> branches were initially developed by Sun and are already in the
>> publicly-available Lustre CVS, and many of the LLNL patches have or
>> are being landed for the 1.8.2 release.
>>
>> As you know, there will always be a delta of patches that are not in
>> an official release, and will be available in the next one.  With
>> the increase in testing to improve the quality of new releases
>> compared to the past, releases are by necessity less frequent.
>> Should anyone have a desire for more bleeding-edge code, they have
>> always been able fetch the current branch before its release,
>> regardless of whether this is Git or CVS.  We maintain a number of
>> different branches (b1_8 for incremental development,
>> b_release_1_8_1 for critical fixes on the 1.8.1 release, etc) these
>> are already available to the public.
>>
>>> I doubt that Sun will give commit rights to external entities (this
>>> is not unreasonable, Sun needs to control what code enters its
>>> repositories).  Hence I think that the community would be better
>>> served with a GIT repository in a public place, like github, that
>>> can give such access.
>>
>> While CVS was very restrictive in terms of managing external commit
>> permissions due to its monolithic repository, some external
>> contributors have had commit access to CVS prior our migration to
>> Git, based on need.  With the migration to Git there is no need to
>> manage such access ourselves, as developers are able to host their
>> clones wherever they want.  The git.lustre.org repository will
>> remain the canonical source for Sun releases, and we will be happy
>> to pull fixes into this repository that meet the quality guidelines
>> stated above.  I for one would welcome external inspectors on
>> patches, and we continue to work with external sites to do scale
>> testing of Lustre.
>>
>>> My group at ClusterStor has reserved the "lustre" project at Github
>>> and we will give keys to any and all organizations that wish to
>>> make serious contributions to Lustre.  I was in particular hoping
>>> that LLNL would be willing to commit their releases to that public
>>> place.
>>
>>
>> Note that with Git and github.com there is no need to give keys to
>> anyone, and in fact that would seem to be detrimental to
>> ClusterStor, because the "lustre" clone you have reserved is within
>> your company's private hosting space (i.e. http://github.com/clusterstor/lustre
>> ).  With Github (or Git in general) it is possible for anyone to
>> make their own clone or mirror of a repository at any time, no keys
>> or permission required, and it will appear ashttp://github.com/
>> {user}/lustre or whatever they want to call it.
>>
>>> b.  Everyone is uncertain about what Sun will do with Lustre
>>> (proprietary Solaris / ZFS server releases and discontinued support
>>> for Linux servers have been mentioned to me several times now).  A
>>> public repository with the open code will be valuable for the
>>> community and promote continued development and access.
>>
>> I agree that a public repository is important, and we will continue
>> to host one at git.lustre.org as we have in the past for CVS, and it
>> can and should be cloned as needed.  As you are hopefully aware,
>> with Git every checkout has a full copy of the repository, including
>> all history, so the code is already very "open" and "public".
>> Lustre is, and will continue to be, available as GPL software.
>>
>> We definitely welcome external contributions to Lustre, which have
>> in the past been done by a small number of people outside of Sun.  I
>> don't think the choice of CVS or Git or hosting site has ever been a
>> limiting factor in this regard.  We look forward to contributions of
>> fixes, designs, and features, from ClusterStor, as we would with any
>> Lustre contributor.
>>
>>> 2. We need MUCH more in the repository than Sun is putting into it.
>>>
>>> There are many development branches and sometimes abandoned
>>> projects that still have a lot of value.  For example, there is a
>>> nearly complete OS X client port - what if someone wanted to pick
>>> that up?  Similarly, there are projects like size on MDS or the
>>> network request scheduler that may need help from the community to
>>> get finished or re-prioritized.
>>
>> All of the Lustre history is still available in CVS, as it has
>> always been.  As far as I know (I've never checked it out myself)
>> even the OS/X client port is publicly available today, which was not
>> true a few years ago.
>>
>> Due to the convoluted branch, repository, and tag abuse that was
>> used to manage the Lustre code in CVS, there is a non-zero effort
>> required to migrate any of the branches in CVS to Git.  Rather than
>> bring all of the old detritus along into Git, only the main
>> development/production branches are being automatically migrated
>> initially.
>>
>> Now that the Git migration is complete, the Sun maintainers of non-
>> release features (HSM, SOM, CMD, NRS, etc) will be creating Git
>> clones and landing their work into them as time permits.
>>
>> If anyone wants to import one of the other CVS branches (e.g. OS/X),
>> all they will need to do is create a patch from that branch and then
>> commit this into their own Git clone.
>>
>>> It is unclear to me if these kind of branches can be found in the
>>> publicly available CVS.
>>
>> Yes, they can, as they always have been.  The CVS repository will
>> remain available indefinitely for spelunking expeditions should the
>> need arise.  Note that the full history that leads to the current
>> release branches (b1_6, b1_8, HEAD) is available in Git, so there is
>> at least a trail of breadcrumbs leading to where we are today.
>>
>>> If they can, a collection of relevant branches, broadly along the
>>> lines of what I mention above, should be placed in the public GIT
>>> repository.
>>
>> For currently-active branches this was already our plan.  For
>> historical and inactive branches, and we welcome any contributions
>> that bring these ancient CVS branches back to life.  Nikita is
>> probably best positioned to know what needs to be imported from the
>> OS/X branch in CVS, and if there are other particular branches that
>> contain important work we'll be happy to discuss them with you.  It
>> will of course be possible to create Git clones for these old
>> branches as needed.
>>
>>> 3. Brian's email message seems to indicate that Sun's git
>>> repository will be used for Sun development.  In the past there
>>> were two CVS repositories - a read-only one that was publicly
>>> accessible and when I last controlled the group it was updated very
>>> frequently with all open source code (presently, it seems to only
>>> contain releases, not the development stream of commits).
>>
>> That's not correct.  The public cvs.lustre.org repository in fact
>> contains all of the Lustre commits and all of the history.  I just
>> did a checkout and there are in fact tags and branches for the pre-
>> release 1.8.2 builds, lprocfs rewrite, CMD, etc. that are very much
>> works in progress.
>>
>> One of the very last commits on CVS HEAD was on Thursday before CVS
>> went read-only, from a patch submitted by LLNL:
>>
>> revision 1.593
>> date: 2009/12/10 13:52:51;  author: dzogin;  state: Exp;  lines: +5 -0
>> Branch HEAD
>> b=21259
>> i=andrew.perepechko
>> i=alexey.lyashkov
>> ----------------------------------------------------------------------
>> Description: Allow non-root access for "lfs check".
>> Details    : Added a check in obd_class_ioctl() for
>> OBD_IOC_PING_TARGET.
>>
>>
>>> It is unclear how Sun can manage development with this git
>>> repository given that parts of its work are proprietary (like the
>>> Windows port) or unreleased (like the ZFS work).  Can we get more
>>> insight in what Brian is alluding to?
>>
>> I'm not sure what "alluding" you are alluding to?
>>
>> Of course, if there is any proprietary code developed it will just
>> not be pushed to the public git.lustre.org repository.  I expect
>> that any proprietary or unreleased code that ClusterStor is already
>> or will develop will similarly not be posted publicly until you are
>> ready to do so.
>>
>> Sun's current in-progress code will very likely reside in separate
>> clones at git.lustre.org, but will not be merged into the Sun Prime
>> repository for an official release until it is inspected, tested,
>> and has permission to land.  Pulls into the Sun Prime repository
>> will be done by the release manager for each branch, as previously
>> stated.  The Lustre engineers at ClusterStor are already familiar
>> with this process and have already had their first patch pass
>> inspections and it is ready for landing.
>>
>> That is one of the major benefits of Git over CVS - that there ISN'T
>> a single repository, and in fact every clone has a full copy of the
>> history and can be used by anyone.  There is no need to have all of
>> the development done in branches on a single repository, but rather
>> to keep projects in their own clones.
>>
>> That allows developers to do local commits on their machines, to
>> push it to their public clone if they want to share their work and/
>> or make an offsite backup, and people are free to choose which clone
>> one they use.  Sun's Prime repository used for releases will only
>> contain stable Lustre code.  This is the Git usage model for the
>> Linux kernel, and think it is a good one to follow for Lustre as
>> well.  You are free to manage your clones in some other way.
>>
>> Cheers, Andreas
>> --
>> Andreas Dilger
>> Sr. Staff Engineer, Lustre Group
>> Sun Microsystems of Canada, Inc.
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-devel
>

From adilger at sun.com  Wed Dec 30 23:12:03 2009
From: adilger at sun.com (Andreas Dilger)
Date: Wed, 30 Dec 2009 16:12:03 -0700
Subject: [Lustre-devel] Fwd: The Lustre community and GIT
In-Reply-To: <1cbd6f830912271649n188095dcp9c65d96bd16eb71e@mail.gmail.com>
References: <2b20c7140912130831t658c2790hb38c3752c18ec128@mail.gmail.com>
	<2b20c7140912131047qf0417f1mf3f3d9a8fc751dc0@mail.gmail.com>
	<5F225623-87E0-4A91-988C-8B789B63E46D@sun.com>
	<0BF036E2-E2A0-4612-B567-4C78794B0C45@clusterstor.com>
	<1cbd6f830912271649n188095dcp9c65d96bd16eb71e@mail.gmail.com>
Message-ID: <18F59523-4EFC-4FA5-9F46-29D2D3AB2C6A@sun.com>

On 2009-12-27, at 17:49, Mag Gam wrote:
> I though majority of the lustre development and support was from Sun.
> This is news to us.

Peter has his own opinions and motivations, and does not in any way  
speak for Sun.  His comments should in no way be confused with any  
statement of direction for Lustre.  Sun will continue Lustre  
development and support in the future.

> Nevertheless I am glad Sun did this conversion.

Thanks.  We worked hard to make sure the transition to Git was  
successful.

> Couple of questions:
>
> 1) What will happen to the Wiki? It still has a lot of Sun stuff on  
> it (logo)

The wiki is continuing to be improved by our documentation team.

> 2) Will we still keep bugzilla.lustre.org?

Yes.

> 3) Will there ever be a git over http? This is important for people
> who are behind firewalls.

I'll pass that question on to our Git administration folks.

> Also, it would be nice to have the RPMs on the Wiki instead of
> registering with Sun.

I don't think that will be changing in the near future, but our  
management is aware of this issue.

> On Sat, Dec 26, 2009 at 10:01 AM, Peter Braam
> <peter.braam at clusterstor.com> wrote:
>> Sent from my iPhone
>>
>> On Dec 15, 2009, at 14:05, Andreas Dilger <adilger at sun.com> wrote:
>>
>>> Hello Peter,
>>>
>>> As previously announced, we have already made the initial Lustre
>>> repository available to everyone this week at git.lustre.org.  This
>>> repository contains all of the main development and release branches
>>> and their history.  It's where all of the Sun developers and
>>> maintaners will get their sources from, and it will continue to be
>>> available under the GPL v2 license as it always has been.
>>>
>>> Sun's policy of prioritizing stability, not just for releases but  
>>> also
>>> to underpin development, makes safeguarding the quality of this
>>> repository paramount.  Everyone contributing to the Lustre sources -
>>> not just internal Sun engineers but also external contributors like
>>> LLNL, CEA, ORNL and DDN - is therefore subject to the same  
>>> gatekeeping
>>> procedures and must follow the same patch submission, testing and
>>> landing processes.  These are further detailed in the wiki pages  
>>> that
>>> were in the original announcement:
>>>
>>> http://wiki.lustre.org/index.php/Accessing_Lustre_Code
>>> http://wiki.lustre.org/index.php/Submitting_Patches
>>>
>>>
>>
>> Sun is no longer supporting the Lustre community (storage OEMs and
>> others have been refused further support).  The model you describe
>> won't apply much longer I think.  People will be looking for  
>> different
>> arrangements.
>>
>> Peter
>>
>>
>>
>>> For people not familiar with Git, it should be clarified that
>>> limiting commits to the Sun Prime repository does not in any way
>>> restrict access to the Lustre code, or the ability of non-Sun
>>> developers to share their own Git clone with the Lustre community.
>>> Developers will be able to host their own Lustre clones (essentially
>>> full repositories) as and where they wish.  The major advantage of
>>> Git is that anyone can pull from any repository, or in fact from a
>>> number of different repositories at the same time - this choice is
>>> left to the user.
>>>
>>> This is the same model used by the Linux kernel, and has proven to
>>> work well with Git.  Each kernel developer hosts their own clone(s)
>>> at git.kernel.org, or at some other site like repo.or.cz, or  
>>> github.com
>>> , and when they have changes to submit upstream they send an email
>>> (usually containing both the patch, and the git URL to pull from) to
>>> the subsystem maintainer, or to Linus directly, requesting that he
>>> pull their changes.  This pull request is normally only sent after
>>> the patch has been reviewed, possibly multiple times,  by the
>>> appropriate subsystem maintainers.  Each developer has full control
>>> over their own clone, and in fact it is rare that more than one
>>> person has push access to a clone.
>>>
>>> The fact that there are many different clones (some more important,
>>> and others less so) in no way implies that the kernel is being
>>> forked, but rather that this is the standard way in which to use Git
>>> to exchange changes.  The location at which a clone is hosted also
>>> has no bearing on its usefulness.
>>>
>>>
>>>
>>> To answer your specific concerns in more detail,
>>>
>>> On 2009-12-13, at 11:47, Peter Braam wrote:
>>>> 1. We need a public repository, where non Sun community members can
>>>> commit.
>>>>
>>>> a.  There are Lustre users than have their own branches.  LLNL has
>>>> probably the best tested branch among all users.  DDN's customers
>>>> have a version of Lustre that includes patches not yet in Sun's
>>>> releases.  It would be very valuable if these kind of releases
>>>> could be collected in a publicly accessible GIT repository.
>>>
>>> That is true even today - most of the patches that are in the DDN
>>> branches were initially developed by Sun and are already in the
>>> publicly-available Lustre CVS, and many of the LLNL patches have or
>>> are being landed for the 1.8.2 release.
>>>
>>> As you know, there will always be a delta of patches that are not in
>>> an official release, and will be available in the next one.  With
>>> the increase in testing to improve the quality of new releases
>>> compared to the past, releases are by necessity less frequent.
>>> Should anyone have a desire for more bleeding-edge code, they have
>>> always been able fetch the current branch before its release,
>>> regardless of whether this is Git or CVS.  We maintain a number of
>>> different branches (b1_8 for incremental development,
>>> b_release_1_8_1 for critical fixes on the 1.8.1 release, etc) these
>>> are already available to the public.
>>>
>>>> I doubt that Sun will give commit rights to external entities (this
>>>> is not unreasonable, Sun needs to control what code enters its
>>>> repositories).  Hence I think that the community would be better
>>>> served with a GIT repository in a public place, like github, that
>>>> can give such access.
>>>
>>> While CVS was very restrictive in terms of managing external commit
>>> permissions due to its monolithic repository, some external
>>> contributors have had commit access to CVS prior our migration to
>>> Git, based on need.  With the migration to Git there is no need to
>>> manage such access ourselves, as developers are able to host their
>>> clones wherever they want.  The git.lustre.org repository will
>>> remain the canonical source for Sun releases, and we will be happy
>>> to pull fixes into this repository that meet the quality guidelines
>>> stated above.  I for one would welcome external inspectors on
>>> patches, and we continue to work with external sites to do scale
>>> testing of Lustre.
>>>
>>>> My group at ClusterStor has reserved the "lustre" project at Github
>>>> and we will give keys to any and all organizations that wish to
>>>> make serious contributions to Lustre.  I was in particular hoping
>>>> that LLNL would be willing to commit their releases to that public
>>>> place.
>>>
>>>
>>> Note that with Git and github.com there is no need to give keys to
>>> anyone, and in fact that would seem to be detrimental to
>>> ClusterStor, because the "lustre" clone you have reserved is within
>>> your company's private hosting space (i.e. http://github.com/clusterstor/lustre
>>> ).  With Github (or Git in general) it is possible for anyone to
>>> make their own clone or mirror of a repository at any time, no keys
>>> or permission required, and it will appear ashttp://github.com/
>>> {user}/lustre or whatever they want to call it.
>>>
>>>> b.  Everyone is uncertain about what Sun will do with Lustre
>>>> (proprietary Solaris / ZFS server releases and discontinued support
>>>> for Linux servers have been mentioned to me several times now).  A
>>>> public repository with the open code will be valuable for the
>>>> community and promote continued development and access.
>>>
>>> I agree that a public repository is important, and we will continue
>>> to host one at git.lustre.org as we have in the past for CVS, and it
>>> can and should be cloned as needed.  As you are hopefully aware,
>>> with Git every checkout has a full copy of the repository, including
>>> all history, so the code is already very "open" and "public".
>>> Lustre is, and will continue to be, available as GPL software.
>>>
>>> We definitely welcome external contributions to Lustre, which have
>>> in the past been done by a small number of people outside of Sun.  I
>>> don't think the choice of CVS or Git or hosting site has ever been a
>>> limiting factor in this regard.  We look forward to contributions of
>>> fixes, designs, and features, from ClusterStor, as we would with any
>>> Lustre contributor.
>>>
>>>> 2. We need MUCH more in the repository than Sun is putting into it.
>>>>
>>>> There are many development branches and sometimes abandoned
>>>> projects that still have a lot of value.  For example, there is a
>>>> nearly complete OS X client port - what if someone wanted to pick
>>>> that up?  Similarly, there are projects like size on MDS or the
>>>> network request scheduler that may need help from the community to
>>>> get finished or re-prioritized.
>>>
>>> All of the Lustre history is still available in CVS, as it has
>>> always been.  As far as I know (I've never checked it out myself)
>>> even the OS/X client port is publicly available today, which was not
>>> true a few years ago.
>>>
>>> Due to the convoluted branch, repository, and tag abuse that was
>>> used to manage the Lustre code in CVS, there is a non-zero effort
>>> required to migrate any of the branches in CVS to Git.  Rather than
>>> bring all of the old detritus along into Git, only the main
>>> development/production branches are being automatically migrated
>>> initially.
>>>
>>> Now that the Git migration is complete, the Sun maintainers of non-
>>> release features (HSM, SOM, CMD, NRS, etc) will be creating Git
>>> clones and landing their work into them as time permits.
>>>
>>> If anyone wants to import one of the other CVS branches (e.g. OS/X),
>>> all they will need to do is create a patch from that branch and then
>>> commit this into their own Git clone.
>>>
>>>> It is unclear to me if these kind of branches can be found in the
>>>> publicly available CVS.
>>>
>>> Yes, they can, as they always have been.  The CVS repository will
>>> remain available indefinitely for spelunking expeditions should the
>>> need arise.  Note that the full history that leads to the current
>>> release branches (b1_6, b1_8, HEAD) is available in Git, so there is
>>> at least a trail of breadcrumbs leading to where we are today.
>>>
>>>> If they can, a collection of relevant branches, broadly along the
>>>> lines of what I mention above, should be placed in the public GIT
>>>> repository.
>>>
>>> For currently-active branches this was already our plan.  For
>>> historical and inactive branches, and we welcome any contributions
>>> that bring these ancient CVS branches back to life.  Nikita is
>>> probably best positioned to know what needs to be imported from the
>>> OS/X branch in CVS, and if there are other particular branches that
>>> contain important work we'll be happy to discuss them with you.  It
>>> will of course be possible to create Git clones for these old
>>> branches as needed.
>>>
>>>> 3. Brian's email message seems to indicate that Sun's git
>>>> repository will be used for Sun development.  In the past there
>>>> were two CVS repositories - a read-only one that was publicly
>>>> accessible and when I last controlled the group it was updated very
>>>> frequently with all open source code (presently, it seems to only
>>>> contain releases, not the development stream of commits).
>>>
>>> That's not correct.  The public cvs.lustre.org repository in fact
>>> contains all of the Lustre commits and all of the history.  I just
>>> did a checkout and there are in fact tags and branches for the pre-
>>> release 1.8.2 builds, lprocfs rewrite, CMD, etc. that are very much
>>> works in progress.
>>>
>>> One of the very last commits on CVS HEAD was on Thursday before CVS
>>> went read-only, from a patch submitted by LLNL:
>>>
>>> revision 1.593
>>> date: 2009/12/10 13:52:51;  author: dzogin;  state: Exp;  lines:  
>>> +5 -0
>>> Branch HEAD
>>> b=21259
>>> i=andrew.perepechko
>>> i=alexey.lyashkov
>>> ----------------------------------------------------------------------
>>> Description: Allow non-root access for "lfs check".
>>> Details    : Added a check in obd_class_ioctl() for
>>> OBD_IOC_PING_TARGET.
>>>
>>>
>>>> It is unclear how Sun can manage development with this git
>>>> repository given that parts of its work are proprietary (like the
>>>> Windows port) or unreleased (like the ZFS work).  Can we get more
>>>> insight in what Brian is alluding to?
>>>
>>> I'm not sure what "alluding" you are alluding to?
>>>
>>> Of course, if there is any proprietary code developed it will just
>>> not be pushed to the public git.lustre.org repository.  I expect
>>> that any proprietary or unreleased code that ClusterStor is already
>>> or will develop will similarly not be posted publicly until you are
>>> ready to do so.
>>>
>>> Sun's current in-progress code will very likely reside in separate
>>> clones at git.lustre.org, but will not be merged into the Sun Prime
>>> repository for an official release until it is inspected, tested,
>>> and has permission to land.  Pulls into the Sun Prime repository
>>> will be done by the release manager for each branch, as previously
>>> stated.  The Lustre engineers at ClusterStor are already familiar
>>> with this process and have already had their first patch pass
>>> inspections and it is ready for landing.
>>>
>>> That is one of the major benefits of Git over CVS - that there ISN'T
>>> a single repository, and in fact every clone has a full copy of the
>>> history and can be used by anyone.  There is no need to have all of
>>> the development done in branches on a single repository, but rather
>>> to keep projects in their own clones.
>>>
>>> That allows developers to do local commits on their machines, to
>>> push it to their public clone if they want to share their work and/
>>> or make an offsite backup, and people are free to choose which clone
>>> one they use.  Sun's Prime repository used for releases will only
>>> contain stable Lustre code.  This is the Git usage model for the
>>> Linux kernel, and think it is a good one to follow for Lustre as
>>> well.  You are free to manage your clones in some other way.
>>>
>>> Cheers, Andreas
>>> --
>>> Andreas Dilger
>>> Sr. Staff Engineer, Lustre Group
>>> Sun Microsystems of Canada, Inc.
>> _______________________________________________
>> Lustre-devel mailing list
>> Lustre-devel at lists.lustre.org
>> http://lists.lustre.org/mailman/listinfo/lustre-devel
>>


Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.


From peter.braam at clusterstor.com  Thu Dec 31 02:20:56 2009
From: peter.braam at clusterstor.com (Peter Braam)
Date: Thu, 31 Dec 2009 02:20:56 +0000
Subject: [Lustre-devel] Fwd: The Lustre community and GIT
In-Reply-To: <18F59523-4EFC-4FA5-9F46-29D2D3AB2C6A@sun.com>
References: <2b20c7140912130831t658c2790hb38c3752c18ec128@mail.gmail.com>
	<2b20c7140912131047qf0417f1mf3f3d9a8fc751dc0@mail.gmail.com>
	<5F225623-87E0-4A91-988C-8B789B63E46D@sun.com>
	<0BF036E2-E2A0-4612-B567-4C78794B0C45@clusterstor.com>
	<1cbd6f830912271649n188095dcp9c65d96bd16eb71e@mail.gmail.com>
	<18F59523-4EFC-4FA5-9F46-29D2D3AB2C6A@sun.com>
Message-ID: <2b20c7140912301820k72f100c2q22d7e482ac560da5@mail.gmail.com>

On Wed, Dec 30, 2009 at 11:12 PM, Andreas Dilger <adilger at sun.com> wrote:

> On 2009-12-27, at 17:49, Mag Gam wrote:
> > I though majority of the lustre development and support was from Sun.
> > This is news to us.
>
> Peter has his own opinions and motivations, and does not in any way
> speak for Sun.  His comments should in no way be confused with any
> statement of direction for Lustre.  Sun will continue Lustre
> development and support in the future.
>
>
Indeed.  Let's see if Sun's VP of HPC Marc Hamilton can clarify what he told
someone who works for me.  Hopefully the plans have changed.  I haven't met
anybody who is informed about the facts, and doesn't share my concerns,
including many senior staff in the Lustre team.

As for my own motivations, I would like the Lustre community to continue to
aid high-end HPC.

Peter

> > Nevertheless I am glad Sun did this conversion.
>
> Thanks.  We worked hard to make sure the transition to Git was
> successful.
>
> > Couple of questions:
> >
> > 1) What will happen to the Wiki? It still has a lot of Sun stuff on
> > it (logo)
>
> The wiki is continuing to be improved by our documentation team.
>
> > 2) Will we still keep bugzilla.lustre.org?
>
> Yes.
>
> > 3) Will there ever be a git over http? This is important for people
> > who are behind firewalls.
>
> I'll pass that question on to our Git administration folks.
>
> > Also, it would be nice to have the RPMs on the Wiki instead of
> > registering with Sun.
>
> I don't think that will be changing in the near future, but our
> management is aware of this issue.
>
> > On Sat, Dec 26, 2009 at 10:01 AM, Peter Braam
> > <peter.braam at clusterstor.com> wrote:
> >> Sent from my iPhone
> >>
> >> On Dec 15, 2009, at 14:05, Andreas Dilger <adilger at sun.com> wrote:
> >>
> >>> Hello Peter,
> >>>
> >>> As previously announced, we have already made the initial Lustre
> >>> repository available to everyone this week at git.lustre.org.  This
> >>> repository contains all of the main development and release branches
> >>> and their history.  It's where all of the Sun developers and
> >>> maintaners will get their sources from, and it will continue to be
> >>> available under the GPL v2 license as it always has been.
> >>>
> >>> Sun's policy of prioritizing stability, not just for releases but
> >>> also
> >>> to underpin development, makes safeguarding the quality of this
> >>> repository paramount.  Everyone contributing to the Lustre sources -
> >>> not just internal Sun engineers but also external contributors like
> >>> LLNL, CEA, ORNL and DDN - is therefore subject to the same
> >>> gatekeeping
> >>> procedures and must follow the same patch submission, testing and
> >>> landing processes.  These are further detailed in the wiki pages
> >>> that
> >>> were in the original announcement:
> >>>
> >>> http://wiki.lustre.org/index.php/Accessing_Lustre_Code
> >>> http://wiki.lustre.org/index.php/Submitting_Patches
> >>>
> >>>
> >>
> >> Sun is no longer supporting the Lustre community (storage OEMs and
> >> others have been refused further support).  The model you describe
> >> won't apply much longer I think.  People will be looking for
> >> different
> >> arrangements.
> >>
> >> Peter
> >>
> >>
> >>
> >>> For people not familiar with Git, it should be clarified that
> >>> limiting commits to the Sun Prime repository does not in any way
> >>> restrict access to the Lustre code, or the ability of non-Sun
> >>> developers to share their own Git clone with the Lustre community.
> >>> Developers will be able to host their own Lustre clones (essentially
> >>> full repositories) as and where they wish.  The major advantage of
> >>> Git is that anyone can pull from any repository, or in fact from a
> >>> number of different repositories at the same time - this choice is
> >>> left to the user.
> >>>
> >>> This is the same model used by the Linux kernel, and has proven to
> >>> work well with Git.  Each kernel developer hosts their own clone(s)
> >>> at git.kernel.org, or at some other site like repo.or.cz, or
> >>> github.com
> >>> , and when they have changes to submit upstream they send an email
> >>> (usually containing both the patch, and the git URL to pull from) to
> >>> the subsystem maintainer, or to Linus directly, requesting that he
> >>> pull their changes.  This pull request is normally only sent after
> >>> the patch has been reviewed, possibly multiple times,  by the
> >>> appropriate subsystem maintainers.  Each developer has full control
> >>> over their own clone, and in fact it is rare that more than one
> >>> person has push access to a clone.
> >>>
> >>> The fact that there are many different clones (some more important,
> >>> and others less so) in no way implies that the kernel is being
> >>> forked, but rather that this is the standard way in which to use Git
> >>> to exchange changes.  The location at which a clone is hosted also
> >>> has no bearing on its usefulness.
> >>>
> >>>
> >>>
> >>> To answer your specific concerns in more detail,
> >>>
> >>> On 2009-12-13, at 11:47, Peter Braam wrote:
> >>>> 1. We need a public repository, where non Sun community members can
> >>>> commit.
> >>>>
> >>>> a.  There are Lustre users than have their own branches.  LLNL has
> >>>> probably the best tested branch among all users.  DDN's customers
> >>>> have a version of Lustre that includes patches not yet in Sun's
> >>>> releases.  It would be very valuable if these kind of releases
> >>>> could be collected in a publicly accessible GIT repository.
> >>>
> >>> That is true even today - most of the patches that are in the DDN
> >>> branches were initially developed by Sun and are already in the
> >>> publicly-available Lustre CVS, and many of the LLNL patches have or
> >>> are being landed for the 1.8.2 release.
> >>>
> >>> As you know, there will always be a delta of patches that are not in
> >>> an official release, and will be available in the next one.  With
> >>> the increase in testing to improve the quality of new releases
> >>> compared to the past, releases are by necessity less frequent.
> >>> Should anyone have a desire for more bleeding-edge code, they have
> >>> always been able fetch the current branch before its release,
> >>> regardless of whether this is Git or CVS.  We maintain a number of
> >>> different branches (b1_8 for incremental development,
> >>> b_release_1_8_1 for critical fixes on the 1.8.1 release, etc) these
> >>> are already available to the public.
> >>>
> >>>> I doubt that Sun will give commit rights to external entities (this
> >>>> is not unreasonable, Sun needs to control what code enters its
> >>>> repositories).  Hence I think that the community would be better
> >>>> served with a GIT repository in a public place, like github, that
> >>>> can give such access.
> >>>
> >>> While CVS was very restrictive in terms of managing external commit
> >>> permissions due to its monolithic repository, some external
> >>> contributors have had commit access to CVS prior our migration to
> >>> Git, based on need.  With the migration to Git there is no need to
> >>> manage such access ourselves, as developers are able to host their
> >>> clones wherever they want.  The git.lustre.org repository will
> >>> remain the canonical source for Sun releases, and we will be happy
> >>> to pull fixes into this repository that meet the quality guidelines
> >>> stated above.  I for one would welcome external inspectors on
> >>> patches, and we continue to work with external sites to do scale
> >>> testing of Lustre.
> >>>
> >>>> My group at ClusterStor has reserved the "lustre" project at Github
> >>>> and we will give keys to any and all organizations that wish to
> >>>> make serious contributions to Lustre.  I was in particular hoping
> >>>> that LLNL would be willing to commit their releases to that public
> >>>> place.
> >>>
> >>>
> >>> Note that with Git and github.com there is no need to give keys to
> >>> anyone, and in fact that would seem to be detrimental to
> >>> ClusterStor, because the "lustre" clone you have reserved is within
> >>> your company's private hosting space (i.e.
> http://github.com/clusterstor/lustre
> >>> ).  With Github (or Git in general) it is possible for anyone to
> >>> make their own clone or mirror of a repository at any time, no keys
> >>> or permission required, and it will appear ashttp://github.com/
> >>> {user}/lustre or whatever they want to call it.
> >>>
> >>>> b.  Everyone is uncertain about what Sun will do with Lustre
> >>>> (proprietary Solaris / ZFS server releases and discontinued support
> >>>> for Linux servers have been mentioned to me several times now).  A
> >>>> public repository with the open code will be valuable for the
> >>>> community and promote continued development and access.
> >>>
> >>> I agree that a public repository is important, and we will continue
> >>> to host one at git.lustre.org as we have in the past for CVS, and it
> >>> can and should be cloned as needed.  As you are hopefully aware,
> >>> with Git every checkout has a full copy of the repository, including
> >>> all history, so the code is already very "open" and "public".
> >>> Lustre is, and will continue to be, available as GPL software.
> >>>
> >>> We definitely welcome external contributions to Lustre, which have
> >>> in the past been done by a small number of people outside of Sun.  I
> >>> don't think the choice of CVS or Git or hosting site has ever been a
> >>> limiting factor in this regard.  We look forward to contributions of
> >>> fixes, designs, and features, from ClusterStor, as we would with any
> >>> Lustre contributor.
> >>>
> >>>> 2. We need MUCH more in the repository than Sun is putting into it.
> >>>>
> >>>> There are many development branches and sometimes abandoned
> >>>> projects that still have a lot of value.  For example, there is a
> >>>> nearly complete OS X client port - what if someone wanted to pick
> >>>> that up?  Similarly, there are projects like size on MDS or the
> >>>> network request scheduler that may need help from the community to
> >>>> get finished or re-prioritized.
> >>>
> >>> All of the Lustre history is still available in CVS, as it has
> >>> always been.  As far as I know (I've never checked it out myself)
> >>> even the OS/X client port is publicly available today, which was not
> >>> true a few years ago.
> >>>
> >>> Due to the convoluted branch, repository, and tag abuse that was
> >>> used to manage the Lustre code in CVS, there is a non-zero effort
> >>> required to migrate any of the branches in CVS to Git.  Rather than
> >>> bring all of the old detritus along into Git, only the main
> >>> development/production branches are being automatically migrated
> >>> initially.
> >>>
> >>> Now that the Git migration is complete, the Sun maintainers of non-
> >>> release features (HSM, SOM, CMD, NRS, etc) will be creating Git
> >>> clones and landing their work into them as time permits.
> >>>
> >>> If anyone wants to import one of the other CVS branches (e.g. OS/X),
> >>> all they will need to do is create a patch from that branch and then
> >>> commit this into their own Git clone.
> >>>
> >>>> It is unclear to me if these kind of branches can be found in the
> >>>> publicly available CVS.
> >>>
> >>> Yes, they can, as they always have been.  The CVS repository will
> >>> remain available indefinitely for spelunking expeditions should the
> >>> need arise.  Note that the full history that leads to the current
> >>> release branches (b1_6, b1_8, HEAD) is available in Git, so there is
> >>> at least a trail of breadcrumbs leading to where we are today.
> >>>
> >>>> If they can, a collection of relevant branches, broadly along the
> >>>> lines of what I mention above, should be placed in the public GIT
> >>>> repository.
> >>>
> >>> For currently-active branches this was already our plan.  For
> >>> historical and inactive branches, and we welcome any contributions
> >>> that bring these ancient CVS branches back to life.  Nikita is
> >>> probably best positioned to know what needs to be imported from the
> >>> OS/X branch in CVS, and if there are other particular branches that
> >>> contain important work we'll be happy to discuss them with you.  It
> >>> will of course be possible to create Git clones for these old
> >>> branches as needed.
> >>>
> >>>> 3. Brian's email message seems to indicate that Sun's git
> >>>> repository will be used for Sun development.  In the past there
> >>>> were two CVS repositories - a read-only one that was publicly
> >>>> accessible and when I last controlled the group it was updated very
> >>>> frequently with all open source code (presently, it seems to only
> >>>> contain releases, not the development stream of commits).
> >>>
> >>> That's not correct.  The public cvs.lustre.org repository in fact
> >>> contains all of the Lustre commits and all of the history.  I just
> >>> did a checkout and there are in fact tags and branches for the pre-
> >>> release 1.8.2 builds, lprocfs rewrite, CMD, etc. that are very much
> >>> works in progress.
> >>>
> >>> One of the very last commits on CVS HEAD was on Thursday before CVS
> >>> went read-only, from a patch submitted by LLNL:
> >>>
> >>> revision 1.593
> >>> date: 2009/12/10 13:52:51;  author: dzogin;  state: Exp;  lines:
> >>> +5 -0
> >>> Branch HEAD
> >>> b=21259
> >>> i=andrew.perepechko
> >>> i=alexey.lyashkov
> >>> ----------------------------------------------------------------------
> >>> Description: Allow non-root access for "lfs check".
> >>> Details    : Added a check in obd_class_ioctl() for
> >>> OBD_IOC_PING_TARGET.
> >>>
> >>>
> >>>> It is unclear how Sun can manage development with this git
> >>>> repository given that parts of its work are proprietary (like the
> >>>> Windows port) or unreleased (like the ZFS work).  Can we get more
> >>>> insight in what Brian is alluding to?
> >>>
> >>> I'm not sure what "alluding" you are alluding to?
> >>>
> >>> Of course, if there is any proprietary code developed it will just
> >>> not be pushed to the public git.lustre.org repository.  I expect
> >>> that any proprietary or unreleased code that ClusterStor is already
> >>> or will develop will similarly not be posted publicly until you are
> >>> ready to do so.
> >>>
> >>> Sun's current in-progress code will very likely reside in separate
> >>> clones at git.lustre.org, but will not be merged into the Sun Prime
> >>> repository for an official release until it is inspected, tested,
> >>> and has permission to land.  Pulls into the Sun Prime repository
> >>> will be done by the release manager for each branch, as previously
> >>> stated.  The Lustre engineers at ClusterStor are already familiar
> >>> with this process and have already had their first patch pass
> >>> inspections and it is ready for landing.
> >>>
> >>> That is one of the major benefits of Git over CVS - that there ISN'T
> >>> a single repository, and in fact every clone has a full copy of the
> >>> history and can be used by anyone.  There is no need to have all of
> >>> the development done in branches on a single repository, but rather
> >>> to keep projects in their own clones.
> >>>
> >>> That allows developers to do local commits on their machines, to
> >>> push it to their public clone if they want to share their work and/
> >>> or make an offsite backup, and people are free to choose which clone
> >>> one they use.  Sun's Prime repository used for releases will only
> >>> contain stable Lustre code.  This is the Git usage model for the
> >>> Linux kernel, and think it is a good one to follow for Lustre as
> >>> well.  You are free to manage your clones in some other way.
> >>>
> >>> Cheers, Andreas
> >>> --
> >>> Andreas Dilger
> >>> Sr. Staff Engineer, Lustre Group
> >>> Sun Microsystems of Canada, Inc.
> >> _______________________________________________
> >> Lustre-devel mailing list
> >> Lustre-devel at lists.lustre.org
> >> http://lists.lustre.org/mailman/listinfo/lustre-devel
> >>
>
>
> Cheers, Andreas
> --
> Andreas Dilger
> Sr. Staff Engineer, Lustre Group
> Sun Microsystems of Canada, Inc.
>
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel_lists.lustre.org/attachments/20091231/a0c7ef8d/attachment.html>