Thank you very much for your reply.<div><br></div><div>So the problem was that the OST2 was inactive.</div><div><br></div><div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div>mds2# cat /proc/fs/lustre/lov/jtest1-mdtlov/target_obd</div>
<div><div>0: jtest1-OST0000_UUID ACTIVE</div><div>1: jtest1-OST0001_UUID ACTIVE</div><div>2: jtest1-OST0002_UUID INACTIVE</div><div>3: jtest1-OST0003_UUID ACTIVE</div></div><div><br></div></blockquote>And I fixed it by ::</div>
<div><br></div><div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div>mds2# lctl dl</div><div> 0 UP mgs MGS MGS 15</div><div> 1 UP mgc MGCXX.XX.XX.XX@o2ib 6cf8b24c-b023-4593-7c82-efb04c681aaa 5</div><div>
2 UP mdt MDS MDS_uuid 3</div><div> 3 UP lov jtest1-mdtlov jtest1-mdtlov_UUID 4</div><div> 4 UP mds jtest1-MDT0000 jtest1-MDT0000_UUID 11</div><div> 5 UP osc jtest1-OST0001-osc jtest1-mdtlov_UUID 5</div><div> 6 UP osc jtest1-OST0003-osc jtest1-mdtlov_UUID 5</div>
<div> 7 UP osc jtest1-OST0000-osc jtest1-mdtlov_UUID 5</div><div> 8 IN osc jtest1-OST0002-osc jtest1-mdtlov_UUID 5</div><div><br></div><div>mds2# lctl --device 8 activate</div><div>mds2# cat /proc/fs/lustre/lov/jtest1-mdtlov/target_obd</div>
<div>0: jtest1-OST0000_UUID ACTIVE</div><div>1: jtest1-OST0001_UUID ACTIVE</div><div>2: jtest1-OST0002_UUID ACTIVE</div><div>3: jtest1-OST0003_UUID ACTIVE</div><div><br></div></blockquote>And finally I had to make a quotacheck through a client in order to have the correct quotas.</div>
<div><br></div><div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div>clie1# lfs quotacheck -ug /lustre/jtest1/</div>clie1# lfs quota /lustre/jtest1/<br>.....<br></blockquote><br><div class="gmail_quote">
On Tue, Jan 22, 2013 at 10:26 AM, Niu, Yawei <span dir="ltr"><<a href="mailto:yawei.niu@intel.com" target="_blank">yawei.niu@intel.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="font-size:14px;font-family:Calibri,sans-serif;word-wrap:break-word">
<div>Looks your OST 2 is administratively disabled, you'd activate all your OSTs if you want to get accurate quota usage. Please refer to manual to see how activate/deactivate OSTs.</div>
<div><br>
</div>
<div>Thanks</div>
<div>- Niu</div>
<div><br>
</div>
<span>
<div style="border-right:medium none;padding-right:0in;padding-left:0in;padding-top:3pt;text-align:left;font-size:11pt;border-bottom:medium none;font-family:Calibri;border-top:#b5c4df 1pt solid;padding-bottom:0in;border-left:medium none">
<div class="im">
<span style="font-weight:bold">From: </span>???????? ????????? ???????? <<a href="mailto:theodoros.stylianos.kondylis@gmail.com" target="_blank">theodoros.stylianos.kondylis@gmail.com</a>><br>
</div><span style="font-weight:bold">Date: </span>Friday, January 18, 2013 5:40 PM<br>
<span style="font-weight:bold">To: </span>Yawei Niu <<a href="mailto:yawei.niu@intel.com" target="_blank">yawei.niu@intel.com</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [Lustre-discuss] Question about quotas.<br>
</div><div><div class="h5">
<div><br>
</div>
<blockquote style="BORDER-LEFT:#b5c4df 5 solid;PADDING:0 0 0 5;MARGIN:0 0 0 5">
<div>
<div>
<div>Thank you all for your information. </div>
<div><br>
</div>
<div>I tried your proposals, and I found the following ::</div>
<div>
<blockquote style="margin:0 0 0 40px;border:none;padding:0px">
<div><br>
</div>
<div>clie1# lfs check servers</div>
<div>jtest1-MDT0000-mdc-ffff88033d143400 active.</div>
<div>jtest1-OST0001-osc-ffff88033d143400 active.</div>
<div>jtest1-OST0003-osc-ffff88033d143400 active.</div>
<div>jtest1-OST0000-osc-ffff88033d143400 active.</div>
<div>jtest1-OST0002-osc-ffff88033d143400 active.</div>
<div><br>
</div>
</blockquote>
But when I do a ::</div>
<div><br>
</div>
<div>
<blockquote style="margin:0 0 0 40px;border:none;padding:0px">
<div>
<div># lfs quota /lustre/jtest1/</div>
<div>Disk quotas for user root (uid 0):</div>
<div> Filesystem kbytes quota limit grace files quota limit grace</div>
<div>/lustre/jtest1/ [15751753680] 30786325577728 31555983717171 - [1280] 10000 11000 -</div>
<div>Some errors happened when getting quota info. Some devices may be not working or deactivated. The data in "[]" is inaccurate.</div>
<div>Disk quotas for group root (gid 0):</div>
<div> Filesystem kbytes quota limit grace files quota limit grace</div>
<div>/lustre/jtest1/ [15751753680] 30786325577728 31555983717171 - [1280] 10000 11000 -</div>
<div>Some errors happened when getting quota info. Some devices may be not working or deactivated. The data in "[]" is inaccurate.</div>
</div>
<div><br>
</div>
</blockquote>
</div>
<div>At the same time in the mds I see ::</div>
<div><br>
</div>
<div>
<blockquote style="margin:0 0 0 40px;border:none;padding:0px">
<div>mds2# dmesg | tail</div>
<div>LustreError: 7116:0:(quota_ctl.c:463:lov_quota_ctl()) ost 2 is inactive</div>
<div>LustreError: 7116:0:(quota_ctl.c:463:lov_quota_ctl()) Skipped 1 previous similar message</div>
<div><br>
</div>
</blockquote>
But on the OSS1 (responsible for ost2) the OST2 md array is mounted.</div>
<div><br>
</div>
<div>
<blockquote style="margin:0 0 0 40px;border:none;padding:0px">How I can check further about the ost's status ? Since the OST's md_array is mounted, shouldn't that be active ?<br>
<br>
</blockquote>
I even try through heartbeat a failover from OSS1 to OSS2 but the ost2 remained inactive ( despite the ost2's md_array being mounted ).</div>
<div><br>
</div>
<div>If you have any idea or indication to lustre commands/files that would be very helpful.</div>
<br>
<div class="gmail_quote">On Thu, Jan 17, 2013 at 10:01 AM, Niu, Yawei <span dir="ltr">
<<a href="mailto:yawei.niu@intel.com" target="_blank">yawei.niu@intel.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="font-size:14px;font-family:Calibri,sans-serif;word-wrap:break-word">
<div>Perhaps quota wasn't able to turned on for some OSTs, did you see any error message (in syslog) when starting MDT & OSTs.</div>
<div><br>
</div>
<span>
<div style="border-right:medium none;padding-right:0in;padding-left:0in;padding-top:3pt;text-align:left;font-size:11pt;border-bottom:medium none;font-family:Calibri;border-top:#b5c4df 1pt solid;padding-bottom:0in;border-left:medium none">
<span style="font-weight:bold">From: </span>???????? ????????? ???????? <<a href="mailto:theodoros.stylianos.kondylis@gmail.com" target="_blank">theodoros.stylianos.kondylis@gmail.com</a>><br>
<span style="font-weight:bold">Date: </span>Wednesday, January 16, 2013 2:26 AM<br>
<span style="font-weight:bold">To: </span>"<a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.org</a>" <<a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.org</a>>
<div><br>
<span style="font-weight:bold">Subject: </span>[Lustre-discuss] Question about quotas.<br>
</div>
</div>
<div>
<div>
<div><br>
</div>
<blockquote style="BORDER-LEFT:#b5c4df 5 solid;PADDING:0 0 0 5;MARGIN:0 0 0 5">
<div>
<div>Hello to everyone.
<div><br>
</div>
<div>We have a lustre (v1.8.4) test cluster with 2 MDSs, 2OSSs and 4 Clients.</div>
<div><br>
</div>
<div>I am experimenting with the quotas but something seems to not work.</div>
<div><br>
</div>
<div>In first I did a </div>
<div><br>
</div>
<div>
<blockquote style="margin:0 0 0 40px;border:none;padding:0px">
<div>cli1# lfs quotacheck /lustre</div>
cli1# lfs quotaon -ug /lustre<br>
cli1# lfs quota /lustre
<div><br>
</div>
</blockquote>
To get ::</div>
<div>
<blockquote style="margin:0 0 0 40px;border:none;padding:0px">
<div>
<div>Disk quotas for user root (uid 0):</div>
<div> Filesystem kbytes quota limit grace files quota limit grace</div>
<div>/lustre/jtest1/ [0] 0 0 - [0] 0 0 -</div>
<div>Some errors happened when getting quota info. Some devices may be not working or deactivated. The data in "[]" is inaccurate.</div>
<div>Disk quotas for group root (gid 0):</div>
<div> Filesystem kbytes quota limit grace files quota limit grace</div>
<div>/lustre/jtest1/ [0] 0 0 - [0] 0 0 -</div>
<div>Some errors happened when getting quota info. Some devices may be not working or deactivated. The data in "[]" is inaccurate.</div>
</div>
<div><br>
</div>
</blockquote>
So I tried ::</div>
<div>
<blockquote style="margin:0 0 0 40px;border:none;padding:0px">
<div>mds2# cat /proc/fs/lustre/lquota/jtest1-MDT0000/quota_type</div>
<div>>> ug3</div>
<div>oss1# cat /proc/fs/lustre/lquota/jtest1-OST000{1,3}/quota_type</div>
>> ug3</blockquote>
</div>
<blockquote style="margin:0 0 0 40px;border:none;padding:0px">
<div>>> ug3</div>
</blockquote>
<div>
<blockquote style="margin:0 0 0 40px;border:none;padding:0px">
<div>oss2# cat /proc/fs/lustre/lquota/jtest1-OST000{0,2}/quota_type<br>
>> ug3</div>
>> ug3<br>
<br>
</blockquote>
Then I tried ::</div>
<blockquote style="margin:0 0 0 40px;border:none;padding:0px">
<div>
<div>cli1# lfs quotaon -ugf /lustre</div>
</div>
<div>
<div>cli1# lfs quota /lustre</div>
</div>
<div>
<div>>>user quotas are not enabled.</div>
<div>>>group quotas are not enabled.</div>
</div>
<div>
<div>mds2# cat /proc/fs/lustre/lquota/jtest1-MDT0000/quota_type<br>
>> 3<br>
oss1# cat /proc/fs/lustre/lquota/jtest1-OST000{1,3}/quota_type<br>
>> 3</div>
>> 3
<div>oss2# cat /proc/fs/lustre/lquota/jtest1-OST000{0,2}/quota_type<br>
>> 3<br>
>> 3</div>
</div>
<div><br>
</div>
</blockquote>
And then again ::
<div><br>
<div>
<blockquote style="margin:0 0 0 40px;border:none;padding:0px">
<div>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"></blockquote>
</div>
</blockquote>
<blockquote style="margin:0 0 0 40px;border:none;padding:0px">
<div>cli1# lfs quotaon -ug /lustre</div>
<div>cli1# lfs quota /lustre</div>
<div>
<div>
<div>>> Disk quotas for user root (uid 0):</div>
<div> Filesystem kbytes quota limit grace files quota limit grace</div>
<div>>> /lustre/jtest1/ [0] 0 0 - [0] 0 0 -</div>
<div>>> Some errors happened when getting quota info. Some devices may be not working or deactivated. The data in "[]" is inaccurate.</div>
<div>>> Disk quotas for group root (gid 0):</div>
<div> Filesystem kbytes quota limit grace files quota limit grace</div>
<div>>> /lustre/jtest1/ [0] 0 0 - [0] 0 0 -</div>
<div>>> Some errors happened when getting quota info. Some devices may be not working or deactivated. The data in "[]" is inaccurate.</div>
</div>
</div>
<div>
<div>mds2# cat /proc/fs/lustre/lquota/jtest1-MDT0000/quota_type<br>
>> ug3<br>
oss1# cat /proc/fs/lustre/lquota/jtest1-OST000{1,3}/quota_type<br>
>> ug3</div>
>> ug3
<div>oss2# cat /proc/fs/lustre/lquota/jtest1-OST000{0,2}/quota_type<br>
>> ug3<br>
>> ug3</div>
</div>
<div><br>
</div>
<div><br>
</div>
</blockquote>
So I would like to make a few questions in case anyone knows.</div>
</div>
<div><br>
</div>
<div>First of all is this a known issue, or am I doing something wrong here? I am forcing the quotaon and that reacts as if I did a quotaoff.</div>
<div><br>
</div>
<div>Furthermore I would like to ask about this number 3 in the quota_type files in case anyone knows what does it mean and why is this necessary.</div>
<div><br>
</div>
<div>Finally if I am not doing something wrong here, is there a way to fix this ?</div>
<div><br>
</div>
<div>Thank you in advance for your time and any replies/guidance/directions.</div>
<div><br>
</div>
<div>Stelios.</div>
</div>
</div>
</blockquote>
</div>
</div>
</span></div>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
</div></div></span>
</div>
</blockquote></div><br></div>