<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
On Jun 28, 2022, at 21:51, Karan Singh via lustre-discuss <<a href="mailto:lustre-discuss@lists.lustre.org" class="">lustre-discuss@lists.lustre.org</a>> wrote:<br class="">
<div>
<blockquote type="cite" class=""><br class="Apple-interchange-newline">
<div class="">
<div dir="ltr" class="">Hi team
<div class=""><br class="">
</div>
<div class="">Below are the details :</div>
<div class="">Using 40 lustre docker clients running on 4 x Dell R750 with each lustre docker client running the below mentioned command </div>
<div class=""><br class="">
</div>
<div class=""><span style="color:rgb(23,43,77);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen,Ubuntu,"Fira Sans","Droid Sans","Helvetica Neue",sans-serif;font-size:14px;background-color:rgb(183,183,183)" class="">${xfersize} = </span><span style="background-color:rgb(183,183,183);color:rgb(23,43,77);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen,Ubuntu,"Fira Sans","Droid Sans","Helvetica Neue",sans-serif;font-size:14px" class="">${blksize}</span><span style="background-color:rgb(183,183,183);color:rgb(23,43,77);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen,Ubuntu,"Fira Sans","Droid Sans","Helvetica Neue",sans-serif;font-size:14px" class=""> </span></div>
<div class=""><span style="background-color:rgb(183,183,183);color:rgb(23,43,77);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen,Ubuntu,"Fira Sans","Droid Sans","Helvetica Neue",sans-serif;font-size:14px" class="">${blksize} = </span><span style="color:rgb(23,43,77);font-family:Roboto,RobotoDraft,Helvetica,Arial,sans-serif;font-size:14px" class="">128M,
64M, 32M, 16M, 8M, 4M, 2M, 1M</span><br class="">
</div>
<div class=""><span style="color:rgb(23,43,77);font-family:Roboto,RobotoDraft,Helvetica,Arial,sans-serif;font-size:14px" class=""><br class="">
</span></div>
<div class=""><span style="color:rgb(23,43,77);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen,Ubuntu,"Fira Sans","Droid Sans","Helvetica Neue",sans-serif;font-size:14px;background-color:rgb(183,183,183)" class="">mpirun -hostfile nodefile
--map-by node -np 16 /usr/local/bin/ior -a POSIX --posix.odirect -v -i 50 -g -F -e -k -o /mnt/beegfs/test.ior</span></div>
</div>
</div>
</blockquote>
<div><br class="">
</div>
<div>I think the problem is pretty clear right here - you are using BeeGFS for your testing, not Lustre... :-)</div>
<br class="">
<blockquote type="cite" class="">
<div class="">
<div dir="ltr" class="">
<div class=""><span style="color:rgb(23,43,77);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen,Ubuntu,"Fira Sans","Droid Sans","Helvetica Neue",sans-serif;font-size:14px;background-color:rgb(183,183,183)" class="">-r -t ${xfersize} -b
${blksize} -O summaryFile=./${TESTDIR}/</span><span style="color:rgb(23,43,77);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen,Ubuntu,"Fira Sans","Droid Sans","Helvetica Neue",sans-serif;font-size:14px;background-color:rgb(183,183,183)" class="">iorresult_SeqWrite_p${</span><span style="color:rgb(23,43,77);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen,Ubuntu,"Fira Sans","Droid Sans","Helvetica Neue",sans-serif;font-size:14px;background-color:rgb(183,183,183)" class="">numthreads}_bs${blksize}_tf${</span><span style="color:rgb(23,43,77);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen,Ubuntu,"Fira Sans","Droid Sans","Helvetica Neue",sans-serif;font-size:14px;background-color:rgb(183,183,183)" class="">xfersize}.json,summaryFormat=</span><span style="color:rgb(23,43,77);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen,Ubuntu,"Fira Sans","Droid Sans","Helvetica Neue",sans-serif;font-size:14px;background-color:rgb(183,183,183)" class="">JSON</span><br class="">
</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class=""><span id="cid:ii_l4z1z8dt0"><image.png></span><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">please let me know what is the bottleneck in the above setup ?</div>
<div class="">Also if you need more info , will be provide it right away .</div>
</div>
</div>
</blockquote>
</div>
<br class="">
<div class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div>Cheers, Andreas</div>
<div>--</div>
<div>Andreas Dilger</div>
<div>Lustre Principal Architect</div>
<div>Whamcloud</div>
<div><br class="">
</div>
<div><br class="">
</div>
<div><br class="">
</div>
</div>
</div>
</div>
</div>
</div>
<br class="Apple-interchange-newline">
</div>
<br class="Apple-interchange-newline">
<br class="Apple-interchange-newline">
</div>
<br class="">
</body>
</html>