[Lustre-discuss] lfs find doesn't find files but lustre seems to be using them

[Ross, Justin]

Yes.  Actually that is exactly it.  As you say lfs getstripe works just fine.  So I am finding all of the files that way.

Thanks for the info on that one.  I was worried that things were very corrupted...  But I guess just a bug.

Thanks,
Justin

-----Original Message-----
From: Michael Barnes [mailto:Michael.Barnes at jlab.org] 
Sent: Monday, May 21, 2012 8:15 AM
To: Ross, Justin
Cc: lustre-discuss at lists.lustre.org
Subject: Re: [Lustre-discuss] lfs find doesn't find files but lustre seems to be using them


Justin,

Do you have holes in your OST list?

Meaning, is this OST something like:

lfs df for me shows:

...
lustre-OST001b_UUID  15378613900  8631157344  5966252368  59% /lustre[OST:27]
lustre-OST0029_UUID   7689826700  5611663868  1687542044  77% /lustre[OST:41]
...

Note, there are no OST:28,29,... and no OST001c,1d,...

I can lfs find files on lustre-OST001b_UUID but never on lustre-OST0029_UUID.  I did some reconfiguring of OSTs a while back and holes got into my lustre system, and the OST after hole is never found with lfs find.  lfs getstripe shows that the files are indeed there and on the OST indexed as 41.

-mb


On May 21, 2012, at 4:32 AM, Ross, Justin wrote:

> I am stuck on an issue with an OST.  I am in the process of replacing some disks in the Lustre file system under a few OSTs.  The process has been to drain the OSTs by marking the OST inactive, lfs find, copy to a new name, move to the old name.  It has been working very well for all of the OSTs.  I have one particular OST that has about 4.5TB of data on it and lfs find doesn't find any files on it at all.  My assumption was that Lustre doesn't know about any files on that OST.  I assumed a full fsck is in order.  But the real weird catch is that just watching the OST it is in use.  I have set it inactive so no new files are written to it.  There are a good bit of reads coming from that OST.  I watch the OSS with iostat and all of the other OSTs on that system are idle, already drained.  This one OST has plenty of read activity.  So it seems that Lustre does know about files on this OST, but just not with lfs find...  Any ideas how to find these files and move them?  I have mounted the OST with ldiskfs and everything looks ok.  There are plenty of files there.
> 
> I have file striping set to one OST.  I am running 1.8.5.  I will upgrade to 1.8.7 as soon as I can, but am curious if anyone else has seen anything like this.  I can't find any similar threads.
> 
> Thanks,
> Justin
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss

--
+-----------------------------------------------
| Michael Barnes
|
| Thomas Jefferson National Accelerator Facility Scientific Computing 
| Group
| 12000 Jefferson Ave.
| Newport News, VA 23606
| (757) 269-7634
+-----------------------------------------------