<html>
<head>
<meta http-equiv="content-type" content="text/html;
charset=windows-1252">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<p>All</p>
<p>I am running a<font face="Courier New, Courier, monospace"> 4
rank</font> MPI job where all the ranks do an open of the file,
attempt to set the striping with ioctl() and then do a small
write. Intermittently, I get errors on the write() and ioctl().
This is a synthetic test case, boiled down from a much larger real
world job. Note that I set the stripe_count to rank+1 so I can
tell which of the ranks actually set the striping.</p>
<p>I have determined that I only get the write failure when the
ioctl also failed with "No data available". It also strikes me
that at most, only one rank reports "File exists". With a 4 rank
job, I would think that normal behavior would be 1 rank would work
as expected ( no error ) and the other 3 would report file exists.</p>
<p>Is this expected behavior?<br>
</p>
<p><font face="Courier New, Courier, monospace" color="#ff0000">rank=1
doIO() -1=ioctl(fd=9) No data available<br>
rank=1 doIO() -1=write(fd=9) Bad file descriptor<br>
rank=3 doIO() -1=ioctl(fd=9) File exists</font><br>
<br>
</p>
<p>oflags = O_CREAT|O_TRUNC|O_RDWR</p>
<p><font face="Courier New, Courier, monospace">void<br>
doIO(const char *fileName, int rank){<br>
int status ;<br>
int fd=open(fileName,
O_RDWR|O_TRUNC|O_CREAT|O_LOV_DELAY_CREATE, 0640 ) ;<br>
if( fd < 0 ) return ;<br>
<br>
struct lov_user_md opts = {0};<br>
opts.lmm_magic = LOV_USER_MAGIC;<br>
opts.lmm_stripe_size = 1048576;<br>
opts.lmm_stripe_offset = -1 ;<br>
opts.lmm_stripe_count = rank+1 ;<br>
opts.lmm_pattern = 0 ;<br>
<br>
status = ioctl ( fd , LL_IOC_LOV_SETSTRIPE, &opts);<br>
if(status<0)fprintf(stderr,"rank=%d %s() %d=ioctl(fd=%d)
%s\n",rank,__func__,status,fd,strerror(errno));<br>
<br>
char *string = "this is it\n" ;<br>
int nc = strlen(string) ;<br>
status = write( fd, string, nc ) ;<br>
if( status != nc ) fprintf(stderr,"rank=%d %s()
%d=write(fd=%d)
%s\n",rank,__func__,status,fd,status<0?strerror(errno):"");<br>
status = close(fd) ;<br>
if(status<0)fprintf(stderr,"rank=%d %s() %d=close(fd=%d)
%s\n",rank,__func__,status,fd,strerror(errno));<br>
}<br>
</font><br>
</p>
<pre class="moz-signature" cols="72">--
I/O Doctors, LLC
507-766-0378
<a class="moz-txt-link-abbreviated" href="mailto:bauerj@iodoctors.com">bauerj@iodoctors.com</a></pre>
</body>
</html>