[Lustre-discuss] Slow NFS/CIFS when exporting Lustre

Dan dan at nerp.net
Thu Sep 22 08:31:47 PDT 2011


Hi,

After researching this in the archives it seems 85% of Lustre's native
performance is reasonable via NFS.  Over GigE I'm seeing 114 MB/sec
read/write via Lustre native.  NFS writes at 44 MB/sec via NFS (about
38%) but reads at 95 MB/sec.  CIFS is lucky to see 18 MB/sec writes
after tuning Samba (12 MB/sec prior).  I've tried tuning the Lustre
client and NFS client and server but nothing impacts the performance
except setting noatime on NFS, which gets another 1.5 MB/sec on writes.
What  are some of you seeing in this configuration?  Suggestions?


My configuration is  a separate MDS and OSS with one OST running Lustre
1.8.6 on GigE only.  Testing is currently simple using dd.
Obdfilter-survey results below (single Areca 1680 w/ 10 3TB disks in
RAID 6, no stripe/stride tuning or external journal, using the stock
kernel driver w/ latest firmware):

Wed Sep 21 20:07:02 PDT 2011 Obdfilter-survey for case=disk from
localhost.localdomain
ost  1 sz 16777216K rsz 1024K obj    1 thr    1 write  183.91 [ 126.86,
241.73] rewrite  202.90 [ 163.57, 241.73] read  753.34
[ 152.83,3440.19] 
ost  1 sz 16777216K rsz 1024K obj    1 thr    2 write  384.32 [ 255.70,
486.45] rewrite  516.70 [ 290.68, 598.34] read 1570.43
[ 376.59,3589.96] 
ost  1 sz 16777216K rsz 1024K obj    1 thr    4 write  415.34 [ 332.63,
488.45] rewrite  653.74 [  38.96, 877.01] read 1915.92
[ 554.28,4865.86] 
ost  1 sz 16777216K rsz 1024K obj    1 thr    8 write  401.46 [ 305.66,
541.39] rewrite  664.63 [ 383.57,1030.43] read 2081.41
[ 803.67,5404.30] 
ost  1 sz 16777216K rsz 1024K obj    1 thr   16 write  406.12 [ 290.68,
541.39] rewrite  700.57 [   0.00,1112.77] read 1563.54 [
63.93,1936.96] 
ost  1 sz 16777216K rsz 1024K obj    2 thr    2 write  373.83 [ 293.66,
458.50] rewrite  569.01 [  60.99, 806.09] read 1380.53
[ 307.19,3737.88] 
ost  1 sz 16777216K rsz 1024K obj    2 thr    4 write  443.58 [ 351.64,
517.43] rewrite  591.88 [ 334.62, 852.04] read 1633.55 [
7.99,4560.89] 
ost  1 sz 16777216K rsz 1024K obj    2 thr    8 write  459.65 [ 374.58,
532.95] rewrite  602.29 [ 358.60,1039.84] read 1416.61 [
37.96,3275.32] 
ost  1 sz 16777216K rsz 1024K obj    2 thr   16 write  461.84 [ 366.59,
517.42] rewrite  663.96 [ 262.71,1056.77] read 1366.43 [
51.94,2379.27] 
ost  1 sz 16777216K rsz 1024K obj    4 thr    4 write  434.83 [ 281.67,
518.41] rewrite  561.85 [ 326.63, 859.03] read 1225.48 [
4.99,3352.31] 
ost  1 sz 16777216K rsz 1024K obj    4 thr    8 write  481.20 [ 378.57,
566.37] rewrite  602.96 [ 143.84, 989.18] read 1478.49 [
14.98,4548.44] 
ost  1 sz 16777216K rsz 1024K obj    4 thr   16 write  493.07 [ 373.58,
583.34] rewrite  628.67 [ 304.66,1044.05] read 1475.33 [
57.93,3221.92] 
ost  1 sz 16777216K rsz 1024K obj    8 thr    8 write  436.75 [   4.00,
535.33] rewrite  556.52 [ 249.72,1003.87] read 1473.04 [
20.98,3419.10] 
ost  1 sz 16777216K rsz 1024K obj    8 thr   16 write  483.68 [ 383.57,
588.34] rewrite  627.79 [ 226.74,1194.59] read 1490.97 [
41.95,4202.74] 
ost  1 sz 16777216K rsz 1024K obj   16 thr   16 write  469.65 [ 377.57,
552.84] rewrite  544.70 [ 257.71,1216.66] read 1375.23 [
31.96,1133.43] 
done!

Thank you,

Dan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20110922/83b52192/attachment.htm>


More information about the lustre-discuss mailing list