[Lustre-discuss] non-consecutive OST ordering

[Christopher Walker]

Thanks very much for your reply. I've tried remaking the mdsdb and all
of the ostdb's, but I still get the same error -- it checks the first 34
osts without a problem, but can't find the ostdb file for the 35th
(which has ost_idx 42):

...
lfsck: ost_idx 34: pass3 OK (676803 files total)
lfsck: can't find file for ost_idx 35
Files affected by missing ost info are : -
lfsck: can't find file for ost_idx 36
Files affected by missing ost info are : -
lfsck: can't find file for ost_idx 37
Files affected by missing ost info are : -
lfsck: can't find file for ost_idx 38
Files affected by missing ost info are : -
lfsck: can't find file for ost_idx 39
Files affected by missing ost info are : -
lfsck: can't find file for ost_idx 40
Files affected by missing ost info are : -
lfsck: can't find file for ost_idx 41
Files affected by missing ost info are : -
lfsck: can't find file for ost_idx 42
Files affected by missing ost info are : -
...

e2fsck claims to be making the ostdb without a problem:

Pass 6: Acquiring information for lfsck
OST: 'aegalfs-OST002a_UUID' ost idx 42: compat 0x2 rocomp 0 incomp 0x2
OST: num files = 676803
OST: last_id = 858163

and with the filesystem up I can see files on this OST:

[cwalker at iliadaccess04 P-Gadget3.3.1]$ lfs getstripe predict.c
OBDS:
0: aegalfs-OST0000_UUID ACTIVE
...
33: aegalfs-OST0021_UUID ACTIVE
42: aegalfs-OST002a_UUID ACTIVE
predict.c
obdidx objid objid group
42 10 0xa 0


lfsck identifies several hundred GB of orphan data that we'd like to
recover, so we'd really like to run lfsck on this array. We're willing
to forgo the recovery on the 35th ost, but I want to make sure that
running lfsck -l with the current configuration won't make things worse.

Thanks again for your reply; any further advice is very much appreciated!

Best,
Chris

On 11/10/10 12:10 AM, Wang Yibin wrote:
> The error message indicates that the UUID of OST #35 does not match between the live filesystem and the ostdb file.
> Is this ostdb obsolete?
>
> 在 2010-11-9，下午11:45， Christopher Walker 写道：
>
>>
>> For reasons that I can't recall, our OSTs are not in consecutive order 
>> -- we have 35 OSTs, which are numbered consecutively from
>> 0000-0021
>> and then there's one last OST at
>> 002a
>>
>> When I try to run lfsck on this array, it works fine for the first 34 
>> OSTs, but it can't seem to find the last OST db file:
>>
>> lfsck: ost_idx 34: pass3 OK (680045 files total)
>> lfsck: can't find file for ost_idx 35
>> Files affected by missing ost info are : -
>> lfsck: can't find file for ost_idx 36
>> Files affected by missing ost info are : -
>> lfsck: can't find file for ost_idx 37
>> Files affected by missing ost info are : -
>> lfsck: can't find file for ost_idx 38
>> Files affected by missing ost info are : -
>> lfsck: can't find file for ost_idx 39
>> Files affected by missing ost info are : -
>> lfsck: can't find file for ost_idx 40
>> Files affected by missing ost info are : -
>> lfsck: can't find file for ost_idx 41
>> Files affected by missing ost info are : -
>> lfsck: can't find file for ost_idx 42
>> Files affected by missing ost info are : -
>> /n/scratch/hernquist_lab/tcox/tests/SbSbhs_e_8/P-Gadget3.3.1/IdlSubfind/.svn/text-base/ReadSubhaloFromReshuffledSnapshot.pro.svn-base
>>
>> and then lists all of the files that live on OST 002a.  This db file 
>> definitely does exist -- it lives in the same directory as all of the 
>> other db files, and e2fsck for this OST ran without problems.
>>
>> Is there some way of forcing lfsck to recognize this OST db?  Or, 
>> failing that, is it dangerous to run lfsck on the first 34 OSTs only?
>>
>> We're using e2fsck 1.41.6.sun1 (30-May-2009)
>>
>> Thanks very much!
>>
>> Chris
>>
>>
>> _______________________________________________
>> Lustre-discuss mailing list
>> Lustre-discuss at lists.lustre.org
>> http://lists.lustre.org/mailman/listinfo/lustre-discuss