[lustre-discuss] problem getting high performance output to single file

[John Bauer]

David

You note that you write a 6GB file.  I suspect that your Linux systems 
have significantly more memory than 6GB meaning your file will end being 
cached in the system buffers.  It wont matter how many OSTs you use as 
you probably are not measuring the speed to the OST's, but rather, you 
are measuring the memory copy speed.
What transfer rate are you seeing?

John

On 5/19/2015 10:40 AM, Schneider, David A. wrote:
> I am trying to get good performance with parallel writing to one file through MPI. Our cluster has high performance when I write to separate files, but when I use one file - I see very little performance increase.
>
> As I understand, our cluster defaults to use one OST per file. There are many OST's though, which is how we get good performance when writing to multiple files. I have been using the command
>
>   lfs setstripe
>
> to change the stripe count and block size. I can see that this works, when I do lfs getstripe, I see the output file is striped, but I'm getting very little I/O performance when I create the striped file.
>
> When working from hdf5 and mpi, I have seen a number of references about tuning parameters, I haven't dug into this yet. I first want to make sure lustre has the high output performance at a basic level. I tried to write a C program uses simple POSIX calls (open and looping over writes) but I don't see much increase in performance (I've tried 8 and 19 OST's, 1MB and 4MB chunks, I write a 6GB file).
>
> Does anyone know if this should work? What is the simplest C program I could write to see an increase in output performance after I stripe? Do I need separate processes/threads with separate file handles? I am on linux red hat 5. I'm not sure what version of lustre this is. I have skimmed through a 450 page pdf of lustre documentation, I saw references to destructive testing one does in the beginning, but I'm not sure what I can do now. I think this is the first work we've done to get high performance when writing a single file, so I'm worried there is something buried in the lustre configuration that needs to be changed. I can run /usr/sbin/lcntl, maybe there are certain parameters I should check?
>
> best,
>
> David Schneider
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

-- 
I/O Doctors, LLC
507-766-0378
bauerj at iodoctors.com