[Lustre-discuss] OST node filling up and aborting write

[Nick Jennings]

minh diep wrote:
> On 02/28/09 06:22, Nick Jennings wrote:
>> Brian J. Murrell wrote:
>>> On Sat, 2009-02-28 at 02:34 +0100, Nick Jennings wrote:
>>>>   I've got 4 OSTs (each 2gigs in size) on one lustre file system. I 
>>>> dd a 4 gig file to the filesystem and after the first OST fills up, 
>>>> the write fails (not enough space on device):
>>> Writes to do not "cascade" over to another OST when one fills up.
>>
>> I see. I guess I have a misunderstanding of the way striping works.
>>
>> If you set the stripesize=1MB, and stripecount=-1 - Then I would 
>> assume this means: Split each write process into 1MB chunks, stripe 
>> across all OSTs. By write process I mean 1 single file being written 
>> to disk. I've read over Chapter 25 as well but it doesn't seem to 
>> clarify this for me (I'm probably letting something fly over my head).
>>
>>
>>>> I initially thought this could be solved by enabling striping, but 
>>>> from HowTo (which doesn't say much on the subject admittedly) I 
>>>> gathered striping was already enabled?
>>> No.  By default, stripesize == 1.  In order to get a single file onto
>>> multiple OSTs you will need to explicitly set a striping policy either
>>> on the file you are going to write into or the directory the file is in.
>>
>> Then what is stripesize=-1 used for? (when specified for the 
>> filesystem, and not a file or a directory). Can you give me an example?
> You can only setstripe on a directory, not a file.
> 
> You could try this
> 1. rm -f /mnt/testfs/testfile1
> 2. lfs setstripe -c -1 /mnt/testfs
> 3. dd if=/dev/zero of=/mnt/testfs/testfile1 bs=4096 count=614400
> 4. df -h

Ok, that makes sense in itself, but I'm still confused as to what effect 
the stripe settings in /proc/fs/lustre/lov/ have on new files that don't 
have special lfs settings.

Thanks,
-Nick