[Lustre-discuss] OST node filling up and aborting write

[minh diep]

On 02/28/09 06:22, Nick Jennings wrote:
> Hi Brian,
> 
>   (Thanks for pointing out the -1 as opposed to 1, I missed that)
> 
> Brian J. Murrell wrote:
>> On Sat, 2009-02-28 at 02:34 +0100, Nick Jennings wrote:
>>> Hi Everyone,
>> Hi Nick,
>>
>>>   I've got 4 OSTs (each 2gigs in size) on one lustre file system. I dd a 
>>> 4 gig file to the filesystem and after the first OST fills up, the write 
>>> fails (not enough space on device):
>> Writes to do not "cascade" over to another OST when one fills up.
> 
> I see. I guess I have a misunderstanding of the way striping works.
> 
> If you set the stripesize=1MB, and stripecount=-1 - Then I would assume 
> this means: Split each write process into 1MB chunks, stripe across all 
> OSTs. By write process I mean 1 single file being written to disk. I've 
> read over Chapter 25 as well but it doesn't seem to clarify this for me 
> (I'm probably letting something fly over my head).
> 
> 
>>> I initially thought this could be solved by enabling striping, but from 
>>> HowTo (which doesn't say much on the subject admittedly) I gathered 
>>> striping was already enabled?
>> No.  By default, stripesize == 1.  In order to get a single file onto
>> multiple OSTs you will need to explicitly set a striping policy either
>> on the file you are going to write into or the directory the file is in.
> 
> Then what is stripesize=-1 used for? (when specified for the filesystem, 
> and not a file or a directory). Can you give me an example?
You can only setstripe on a directory, not a file.

You could try this
1. rm -f /mnt/testfs/testfile1
2. lfs setstripe -c -1 /mnt/testfs
3. dd if=/dev/zero of=/mnt/testfs/testfile1 bs=4096 count=614400
4. df -h

Thanks
-Minh
> 
> 
> 
> --
>   Write Test #2
> --
> 
> # lctl conf_param testfs-MDT0000.lov.stripecount=-1
> 
> /proc/fs/lustre/lov/testfs-clilov-c464c000/stripecount:-1
> /proc/fs/lustre/lov/testfs-clilov-c464c000/stripeoffset:0
> /proc/fs/lustre/lov/testfs-clilov-c464c000/stripesize:1048576
> /proc/fs/lustre/lov/testfs-clilov-c464c000/stripetype:1
> /proc/fs/lustre/lov/testfs-mdtlov/stripecount:-1
> /proc/fs/lustre/lov/testfs-mdtlov/stripeoffset:0
> /proc/fs/lustre/lov/testfs-mdtlov/stripesize:1048576
> /proc/fs/lustre/lov/testfs-mdtlov/stripetype:1
> 
> # dd if=/dev/zero of=/mnt/testfs/testfile1 bs=4096 count=614400
> dd: writing `/mnt/testfs/testfile1': No space left on device
> 437506+0 records in
> 437505+0 records out
> 1792020480 bytes (1.8 GB) copied, 52.5727 seconds, 34.1 MB/s
> 
> # df -h
> Filesystem            Size  Used Avail Use% Mounted on
> /dev/hda1              15G  7.7G  5.9G  57% /
> tmpfs                 252M     0  252M   0% /dev/shm
> /dev/hda5             4.1G  198M  3.7G   6% /mnt/lustre/mdt
> /dev/hda6             1.9G  1.8G   68K 100% /mnt/lustre/ost0
> /dev/hda7             1.9G   80M  1.7G   5% /mnt/lustre/ost1
> /dev/hda8             1.9G   80M  1.7G   5% /mnt/lustre/ost2
> /dev/hda9             1.9G   80M  1.7G   5% /mnt/lustre/ost3
> 192.168.0.149 at tcp0:/testfs
>                        7.4G  2.0G  5.1G  29% /mnt/testfs
> 
> 
> 
> Thanks for your help,
> -Nick
> 
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss