<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Hi Folks,<div><br class="webkit-block-placeholder"></div><div>My lustre and kernel version: 2.6.9-55.0.9.EL_lustre.1.6.3smp<br><br><div><br class="webkit-block-placeholder"></div><div>I am experiencing few problems with lustre quotas.</div><div><br class="webkit-block-placeholder"></div><div>1)</div><div>When I run: "lfs quotacheck -ug /home" I can see in syslog lots of LustreError messages</div><div>on mds</div><div><div>Nov 15 13:30:35 mds01.beowulf.cluster kernel: LustreError: 22395:0:(ldlm_lib.c:1629:target_handle_dqacq_callback()) dqacq failed! (rc:-5) </div></div><div><br class="webkit-block-placeholder"></div><div>on oss</div><div>Nov 15 13:30:35 storage07.beowulf.cluster kernel: LustreError: 15905:0:(quota_context.c:482:dqacq_completion()) acquire qunit got error! (rc:-5)</div><div><br class="webkit-block-placeholder"></div><div>Is there someone who can explain reason for these errors?</div><div><br class="webkit-block-placeholder"></div><div>2) </div><div>I set only hard limit for each user. The limit is 1TB</div><div>When I run lfs quota -u <username> /home I can see that some users have exceeded their per OST limit even if they total usage if being far from limit. Also per OST limit seems to be not displayed correctly. If I add up per OST limits it will not equal total limit (see below for example)</div><div><br class="webkit-block-placeholder"></div><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica">Disk quotas for user se282 (uid 694):</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica"> Filesystem kbytes quota limit grace files quota limit grace</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica"> /home 36479512 0 1000000000 886 0 0 </font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica">ddn_home-MDT0000_UUID</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica"> 120 0 102400 886 0 0 </font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica">ddn_home-OST0000_UUID</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica"> 13217456* 0 11366400 </font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica">ddn_home-OST0001_UUID</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica"> 9115004* 0 7884800 </font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica">ddn_home-OST0002_UUID</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica"> 8324440 0 8396800 </font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica">ddn_home-OST0003_UUID</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica"> 5822492 0 5939200 </font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">3) Is the MAX quota hard limit 2TB ?</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">I set hard limit per user to 15TB. </div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">lfs setquota -u se282 0 15000000000 0 0 /home</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Then I check user quota:</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><div>lfs quota -u se282 /home</div><div><br class="webkit-block-placeholder"></div><div>Disk quotas for user se282 (uid 694):</div><div> Filesystem kbytes quota limit grace files quota limit grace</div><div> /home 36479512 0 2115098112 886 0 0 </div><div>ddn_home-MDT0000_UUID</div><div> 120 0 102400 886 0 0 </div><div>ddn_home-OST0000_UUID</div><div> 13217456 0 13312000 </div><div>ddn_home-OST0001_UUID</div><div> 9114996 0 9216000 </div><div>ddn_home-OST0002_UUID</div><div> 8324440 0 8396800 </div><div>ddn_home-OST0003_UUID</div><div> 5822500 0 5939200 </div></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><br class="webkit-block-placeholder"></div> As you can see hard limit is 2115098112 instead of 15000000000</div><div><br></div><div><br class="webkit-block-placeholder"></div><div>Thanks,</div><div><br class="webkit-block-placeholder"></div><div>Wojciech Turek</div><div><div><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0; "><div><br class="webkit-block-placeholder"></div></span><br class="Apple-interchange-newline"></span></span> </div><br></div></div></body></html>