<div class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 11pt; COLOR: #1f497d; mso-ansi-language: EN-US"><font face="Calibri" color="#000000">Hi All,</font></span></div>
<div class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 11pt; COLOR: #1f497d; mso-ansi-language: EN-US"><font face="Calibri" color="#000000"></font></span> </div>
<div class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 11pt; COLOR: #1f497d; mso-ansi-language: EN-US"><font face="Calibri" color="#000000">I am running lustre servers on 1.8.5 (recently upgraded from 1.8.2). Clients are still on 1.8.2 . </font></span></div>
<div class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 11pt; COLOR: #1f497d; mso-ansi-language: EN-US"><font face="Calibri" color="#000000"></font></span> </div>
<div class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 11pt; COLOR: #1f497d; mso-ansi-language: EN-US"><font face="Calibri"><font color="#000000">I am getting the error "<span lang="EN-US" style="FONT-SIZE: 11pt; FONT-FAMILY: 'Calibri','sans-serif'; mso-ansi-language: EN-US; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin; mso-bidi-font-family: 'Times New Roman'; mso-fareast-language: EN-GB; mso-bidi-language: AR-SA">ost_write operation failed with -28" in the clients. Due to this i am getting error message as "No space left on the device" oftenly. As per lfs df -h output all the OSTs are occupied around 55% only. </span></font></font></span></div>
<div class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 11pt; COLOR: #1f497d; mso-ansi-language: EN-US"><font face="Calibri"><font color="#000000"><span lang="EN-US" style="FONT-SIZE: 11pt; FONT-FAMILY: 'Calibri','sans-serif'; mso-ansi-language: EN-US; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin; mso-bidi-font-family: 'Times New Roman'; mso-fareast-language: EN-GB; mso-bidi-language: AR-SA"></span></font></font></span> </div>
<div class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 11pt; COLOR: #1f497d; mso-ansi-language: EN-US"><font face="Calibri"><font color="#000000"><span lang="EN-US" style="FONT-SIZE: 11pt; FONT-FAMILY: 'Calibri','sans-serif'; mso-ansi-language: EN-US; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin; mso-bidi-font-family: 'Times New Roman'; mso-fareast-language: EN-GB; mso-bidi-language: AR-SA">lfs df -h<br>
UUID bytes Used Available Use% Mounted on<br>lustre-MDT0000_UUID 52.3G 4.2G 48.1G 8% /opt/lustre[MDT:0]<br>lustre-OST0000_UUID 442.9G 245.6G 197.3G 55% /opt/lustre[OST:0]<br>
lustre-OST0001_UUID 442.9G 238.7G 204.3G 53% /opt/lustre[OST:1]<br>lustre-OST0002_UUID 442.9G 243.2G 199.7G 54% /opt/lustre[OST:2]<br>lustre-OST0003_UUID 442.9G 236.5G 206.5G 53% /opt/lustre[OST:3]<br>
lustre-OST0004_UUID 442.9G 234.8G 208.1G 53% /opt/lustre[OST:4]<br>lustre-OST0005_UUID 442.9G 239.7G 203.3G 54% /opt/lustre[OST:5]<br>lustre-OST0006_UUID 442.9G 237.2G 205.7G 53% /opt/lustre[OST:6]<br>
lustre-OST0007_UUID 442.9G 227.9G 215.0G 51% /opt/lustre[OST:7]</span></font></font></span></div>
<div class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 11pt; COLOR: #1f497d; mso-ansi-language: EN-US"><font face="Calibri"><font color="#000000"><span lang="EN-US" style="FONT-SIZE: 11pt; FONT-FAMILY: 'Calibri','sans-serif'; mso-ansi-language: EN-US; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin; mso-bidi-font-family: 'Times New Roman'; mso-fareast-language: EN-GB; mso-bidi-language: AR-SA">filesystem summary: 3.5T 1.9T 1.6T 53% /opt/lustre<br>
</span></font></font></span></div>
<div class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 11pt; COLOR: #1f497d; mso-ansi-language: EN-US"><font face="Calibri"><font color="#000000"><span lang="EN-US" style="FONT-SIZE: 11pt; FONT-FAMILY: 'Calibri','sans-serif'; mso-ansi-language: EN-US; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin; mso-bidi-font-family: 'Times New Roman'; mso-fareast-language: EN-GB; mso-bidi-language: AR-SA">As per the below bugzilla, i have upgraded one of the lustre client verstion to 1.8.5 but still the issue persist in that client.</span></font></font></span></div>
<div class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 11pt; COLOR: #1f497d; mso-ansi-language: EN-US"><font face="Calibri" color="#000000"><span lang="EN-US" style="FONT-SIZE: 11pt; FONT-FAMILY: 'Calibri','sans-serif'; mso-ansi-language: EN-US; mso-fareast-font-family: Calibri; mso-fareast-theme-font: minor-latin; mso-bidi-font-family: 'Times New Roman'; mso-fareast-language: EN-GB; mso-bidi-language: AR-SA"></span></font></span> </div>
<div class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 11pt; COLOR: #1f497d; mso-ansi-language: EN-US"><font face="Calibri" color="#000000"> </font><a href="https://bugzilla.lustre.org/show_bug.cgi?id=22755"><font face="Calibri" color="#000000">https://bugzilla.lustre.org/show_bug.cgi?id=22755</font></a></span></div>
<div class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 11pt; COLOR: #1f497d; mso-ansi-language: EN-US"><font color="#000000"></font></span> </div>
<div class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 11pt; COLOR: #1f497d; mso-ansi-language: EN-US"><font color="#000000">Lustre clients are on Suse linux 10.1 . In order to install lustre client packages of 1.8.5, i have upgraded the Suse kernel also.</font></span></div>
<div class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 11pt; COLOR: #1f497d; mso-ansi-language: EN-US"><font color="#000000"></font></span> </div>
<div class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 11pt; COLOR: #1f497d; mso-ansi-language: EN-US"><font color="#000000">I have also checked and found that no quota are enabled in the clients.</font></span></div>
<div class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 11pt; COLOR: #1f497d; mso-ansi-language: EN-US"><font color="#000000"></font></span> </div>
<div class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 11pt; COLOR: #1f497d; mso-ansi-language: EN-US"><font color="#000000">lfs quota -u 36401 /opt/lustre<br>Disk quotas for user 36401 (uid 36401):<br>
Filesystem kbytes quota limit grace files quota limit grace<br> /opt/lustre 127315748 0 0 - 1001083 0 0 -<br></font></span></div>
<div class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 11pt; COLOR: #1f497d; mso-ansi-language: EN-US"><font color="#000000">Below are the lustre client packages i have installed.</font></span></div>
<div class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 11pt; COLOR: #1f497d; mso-ansi-language: EN-US"><font color="#000000"></font></span> </div>
<div class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 11pt; COLOR: #1f497d; mso-ansi-language: EN-US">
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="mso-ansi-language: EN-US"><font face="Calibri"><font color="#000000">lustre-client-modules-1.8.5-2.6.16_60_0.69.1_lustre.1.8.5_smp</font></font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="mso-ansi-language: EN-US"><font face="Calibri" color="#000000">lustre-client-1.8.5-2.6.16_60_0.69.1_lustre.1.8.5_smp</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="mso-ansi-language: EN-US"><font face="Calibri" color="#000000"></font></span> </p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="mso-ansi-language: EN-US"><font face="Calibri" color="#000000">Suse kernel packages installed:</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="mso-ansi-language: EN-US"><font face="Calibri" color="#000000"></font></span> </p><span lang="EN-US" style="mso-ansi-language: EN-US"><font face="Calibri" color="#000000">
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="mso-ansi-language: EN-US">kernel-default-2.6.16.60-0.69.1</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="mso-ansi-language: EN-US">kernel-source-2.6.16.60-0.69.1</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="mso-ansi-language: EN-US">kernel-smp-2.6.16.60-0.69.1</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="mso-ansi-language: EN-US">kernel-syms-2.6.16.60-0.69.1</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p></font></span>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="mso-ansi-language: EN-US"><font face="Calibri" color="#000000">Error:</font></span></p><span lang="EN-US" style="mso-ansi-language: EN-US">
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="mso-ansi-language: EN-US"><font face="Calibri" color="#000000">Apr 29 15:35:55 hostname kernel: LustreError: 11-0: an error occurred while communicating with </font><a href="mailto:172.16.x.x@tcp"><font face="Calibri" color="#000000">172.16.x.x@tcp</font></a><font face="Calibri"><font color="#000000">. The ost_write operation failed with -28</font></font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="mso-ansi-language: EN-US"><font face="Calibri"><font color="#000000">Apr 29 15:35:55 hostname kernel: LustreError: Skipped 9657 previous similar messages</font></font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="mso-ansi-language: EN-US"><font face="Calibri" color="#000000">Apr 29 15:38:03 hostname kernel: LustreError: 11-0: an error occurred while communicating with </font><a href="mailto:172.16.x.x@tcp"><font face="Calibri" color="#000000">172.16.x.x@tcp</font></a><font face="Calibri"><font color="#000000">. The ost_write operation failed with -28</font></font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><font color="#000000"></font></p></span>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="mso-ansi-language: EN-US"><font face="Calibri" color="#000000">Kindly suggest.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="mso-ansi-language: EN-US"><font face="Calibri" color="#000000"></font></span> </p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="mso-ansi-language: EN-US"><font face="Calibri" color="#000000">Regards,</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="mso-ansi-language: EN-US"><font face="Calibri" color="#000000">Prasad</font></span></p></span></div>