[Lustre-discuss] Slow NFS/CIFS when exporting Lustre
Dan
dan at nerp.net
Thu Sep 22 08:31:47 PDT 2011
Hi,
After researching this in the archives it seems 85% of Lustre's native
performance is reasonable via NFS. Over GigE I'm seeing 114 MB/sec
read/write via Lustre native. NFS writes at 44 MB/sec via NFS (about
38%) but reads at 95 MB/sec. CIFS is lucky to see 18 MB/sec writes
after tuning Samba (12 MB/sec prior). I've tried tuning the Lustre
client and NFS client and server but nothing impacts the performance
except setting noatime on NFS, which gets another 1.5 MB/sec on writes.
What are some of you seeing in this configuration? Suggestions?
My configuration is a separate MDS and OSS with one OST running Lustre
1.8.6 on GigE only. Testing is currently simple using dd.
Obdfilter-survey results below (single Areca 1680 w/ 10 3TB disks in
RAID 6, no stripe/stride tuning or external journal, using the stock
kernel driver w/ latest firmware):
Wed Sep 21 20:07:02 PDT 2011 Obdfilter-survey for case=disk from
localhost.localdomain
ost 1 sz 16777216K rsz 1024K obj 1 thr 1 write 183.91 [ 126.86,
241.73] rewrite 202.90 [ 163.57, 241.73] read 753.34
[ 152.83,3440.19]
ost 1 sz 16777216K rsz 1024K obj 1 thr 2 write 384.32 [ 255.70,
486.45] rewrite 516.70 [ 290.68, 598.34] read 1570.43
[ 376.59,3589.96]
ost 1 sz 16777216K rsz 1024K obj 1 thr 4 write 415.34 [ 332.63,
488.45] rewrite 653.74 [ 38.96, 877.01] read 1915.92
[ 554.28,4865.86]
ost 1 sz 16777216K rsz 1024K obj 1 thr 8 write 401.46 [ 305.66,
541.39] rewrite 664.63 [ 383.57,1030.43] read 2081.41
[ 803.67,5404.30]
ost 1 sz 16777216K rsz 1024K obj 1 thr 16 write 406.12 [ 290.68,
541.39] rewrite 700.57 [ 0.00,1112.77] read 1563.54 [
63.93,1936.96]
ost 1 sz 16777216K rsz 1024K obj 2 thr 2 write 373.83 [ 293.66,
458.50] rewrite 569.01 [ 60.99, 806.09] read 1380.53
[ 307.19,3737.88]
ost 1 sz 16777216K rsz 1024K obj 2 thr 4 write 443.58 [ 351.64,
517.43] rewrite 591.88 [ 334.62, 852.04] read 1633.55 [
7.99,4560.89]
ost 1 sz 16777216K rsz 1024K obj 2 thr 8 write 459.65 [ 374.58,
532.95] rewrite 602.29 [ 358.60,1039.84] read 1416.61 [
37.96,3275.32]
ost 1 sz 16777216K rsz 1024K obj 2 thr 16 write 461.84 [ 366.59,
517.42] rewrite 663.96 [ 262.71,1056.77] read 1366.43 [
51.94,2379.27]
ost 1 sz 16777216K rsz 1024K obj 4 thr 4 write 434.83 [ 281.67,
518.41] rewrite 561.85 [ 326.63, 859.03] read 1225.48 [
4.99,3352.31]
ost 1 sz 16777216K rsz 1024K obj 4 thr 8 write 481.20 [ 378.57,
566.37] rewrite 602.96 [ 143.84, 989.18] read 1478.49 [
14.98,4548.44]
ost 1 sz 16777216K rsz 1024K obj 4 thr 16 write 493.07 [ 373.58,
583.34] rewrite 628.67 [ 304.66,1044.05] read 1475.33 [
57.93,3221.92]
ost 1 sz 16777216K rsz 1024K obj 8 thr 8 write 436.75 [ 4.00,
535.33] rewrite 556.52 [ 249.72,1003.87] read 1473.04 [
20.98,3419.10]
ost 1 sz 16777216K rsz 1024K obj 8 thr 16 write 483.68 [ 383.57,
588.34] rewrite 627.79 [ 226.74,1194.59] read 1490.97 [
41.95,4202.74]
ost 1 sz 16777216K rsz 1024K obj 16 thr 16 write 469.65 [ 377.57,
552.84] rewrite 544.70 [ 257.71,1216.66] read 1375.23 [
31.96,1133.43]
done!
Thank you,
Dan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20110922/83b52192/attachment.htm>
More information about the lustre-discuss
mailing list