<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Hi Folks,<div><br class="webkit-block-placeholder"></div><div>My lustre and kernel version:2.6.9-55.0.9.EL_lustre.1.6.3smp<br><br><div><br class="webkit-block-placeholder"></div><div>I am experiencing few problems with lustre quotas.</div><div><br class="webkit-block-placeholder"></div><div>1)</div><div>When I run: "lfs quotacheck -ug /home" I can see in syslog lots of LustreError messages</div><div>on mds</div><div><div>Nov 15 13:30:35 mds01.beowulf.cluster kernel: LustreError: 22395:0:(ldlm_lib.c:1629:target_handle_dqacq_callback()) dqacq failed! (rc:-5)</div></div><div><br class="webkit-block-placeholder"></div><div>on oss</div><div>Nov 15 13:30:35 storage07.beowulf.cluster kernel: LustreError: 15905:0:(quota_context.c:482:dqacq_completion()) acquire qunit got error! (rc:-5)</div><div><br class="webkit-block-placeholder"></div><div>Is there someone who can explain reason for these errors?</div><div><br class="webkit-block-placeholder"></div><div>2)</div><div>I set only hard limit for each user. The limit is 1TB</div><div>When I run lfs quota -u <username> /home I can see that some users haveexceededtheir per OST limit even if they total usage if being far from limit. Also per OST limit seems to be not displayed correctly. If I add up per OST limits it will not equal total limit (see below for example)</div><div><br class="webkit-block-placeholder"></div><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica">Disk quotas for user se282 (uid 694):</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica"> Filesystem kbytes quota limit grace files quota limit grace</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica"> /home 36479512 0 1000000000 886 0 0 </font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica">ddn_home-MDT0000_UUID</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica"> 120 0 102400 886 0 0 </font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica">ddn_home-OST0000_UUID</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica"> 13217456* 0 11366400 </font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica">ddn_home-OST0001_UUID</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica"> 9115004* 0 7884800 </font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica">ddn_home-OST0002_UUID</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica"> 8324440 0 8396800 </font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica">ddn_home-OST0003_UUID</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" style="font: 12.0px Helvetica"> 5822492 0 5939200 </font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">3) Is the MAX quota hard limit 2TB ?</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">I set hard limit per user to 15TB.</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">lfs setquota -u se282 0 15000000000 0 0 /home</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; ">Then I check user quota:</div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><div>lfs quota -u se282 /home</div><div><br class="webkit-block-placeholder"></div><div>Disk quotas for user se282 (uid 694):</div><div> Filesystem kbytes quota limit grace files quota limit grace</div><div> /home 36479512 02115098112 886 0 0 </div><div>ddn_home-MDT0000_UUID</div><div> 120 0 102400 886 0 0 </div><div>ddn_home-OST0000_UUID</div><div> 13217456 0 13312000 </div><div>ddn_home-OST0001_UUID</div><div> 9114996 0 9216000 </div><div>ddn_home-OST0002_UUID</div><div> 8324440 0 8396800 </div><div>ddn_home-OST0003_UUID</div><div> 5822500 0 5939200 </div></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><br class="webkit-block-placeholder"></div> As you can see hard limit is2115098112 instead of15000000000</div><div><br></div><div><br class="webkit-block-placeholder"></div><div>Thanks,</div><div><br class="webkit-block-placeholder"></div><div>Wojciech Turek</div><div><div><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0; "><div><br class="webkit-block-placeholder"></div></span><br class="Apple-interchange-newline"></span></span> </div><br></div></div></body></html>