[Lustre-discuss] Find all files on a specific OST
Anselm Strauss
anselm.strauss at id.unibe.ch
Wed Jan 30 11:48:12 PST 2008
On 30.01.2008, at 18:46, Cliff White wrote:
> Anselm Strauss wrote:
>> On 30.01.2008, at 00:03, Andreas Dilger wrote:
>>> On Jan 29, 2008 19:57 +0100, Anselm Strauss wrote:
>>>> I want to know all files that are located on a specific OST, but
>>>> neither 'lfs find ...' nor 'lfs getstripe ...' can give me an
>>>> answer.
>>>> I do not use striping. This is what I tried:
>>>>
>>>> gnode30 ~ # lfs df -h
>>>> UUID bytes Used Available Use% Mounted on
>>>> scratch-MDT0000_UUID 710.0G 41.0G 668.9G 5% /mnt/
>>>> lustre[MDT:0]
>>>> scratch-OST0001_UUID 6.3T 327.3G 5.9T 5% /mnt/
>>>> lustre[OST:1]
>>>> scratch-OST0002_UUID 6.3T 327.3G 5.9T 5% /mnt/
>>>> lustre[OST:2]
>>>> scratch-OST0003_UUID 6.3T 328.3G 5.9T 5% /mnt/
>>>> lustre[OST:3]
>>>> scratch-OST0004_UUID 6.3T 327.3G 5.9T 5% /mnt/
>>>> lustre[OST:4]
>>>> scratch-OST0006_UUID 6.3T 327.3G 5.9T 5% /mnt/
>>>> lustre[OST:6]
>>>> scratch-OST0007_UUID 6.3T 326.3G 5.9T 5% /mnt/
>>>> lustre[OST:7]
>>>>
>>>> filesystem summary: 37.6T 1.9T 35.7T 5% /mnt/
>>>> lustre
>>>>
>>>> gnode30 lustre # lfs find -O scratch-OST0003_UUID /mnt/lustre
>>>> gnode30 lustre # lfs find -O 7 /mnt/lustre
>>>> gnode30 lustre # lfs getstripe -O scratch-OST0003_UUID /mnt/lustre
>>>> error: setup_obd_uuid: unknown obduuid: scratch-OST0003_UUID
>>>> /mnt/lustre/zero1
>>>> /mnt/lustre/zero2
>>>> /mnt/lustre/zero3
>>>> /mnt/lustre/zero4
>>>> /mnt/lustre/zero5
>>>> /mnt/lustre/zero6
>>> Try using just "lfs getstripe /mnt/lustre" as a starter. It will
>>> print
>>> out the list of OST UUIDs at the start. It would appear that
>>> your syntax
>>> is correct, so I'm not sure what is going wrong...
>>>
>>> $ lfs df
>>> UUID 1K-blocks Used Available Use% Mounted on
>>> mds-myth-0_UUID 9174328 677728 8496600 7% /myth[MDT:
>>> 0]
>>> ost-myth-0_UUID 292223856 247877024 44346832 84% /myth[OST:
>>> 0]
>>> ost-myth-1_UUID 94442984 91277176 3165808 96% /myth[OST:
>>> 1]
>>> ost-myth-2_UUID 487388376 332816808 154571568 68% /myth[OST:
>>> 2]
>>> ost-myth-3_UUID 487865304 347373400 140491904 71% /myth[OST:
>>> 3]
>>>
>>> filesystem summary: 1361920520 1019344408 342576112 74% /myth
>> gnode30 ~ # lfs df
>> UUID 1K-blocks Used Available Use% Mounted on
>> scratch-MDT0000_UUID 744455808 43024252 701431556 5% /mnt/
>> lustre[MDT:0]
>> scratch-OST0001_UUID 6726440480 342319020 6384121460 5% /mnt/
>> lustre[OST:1]
>> scratch-OST0002_UUID 6726440480 342319020 6384121460 5% /mnt/
>> lustre[OST:2]
>> scratch-OST0003_UUID 6726440480 342319020 6384121460 5% /mnt/
>> lustre[OST:3]
>> scratch-OST0004_UUID 6726440480 343343024 6383097456 5% /mnt/
>> lustre[OST:4]
>> scratch-OST0006_UUID 6726440480 342317996 6384122484 5% /mnt/
>> lustre[OST:6]
>> scratch-OST0007_UUID 6726440480 342317996 6384122484 5% /mnt/
>> lustre[OST:7]
>> filesystem summary: 40358642880 2054936076 38303706804 5% /mnt/
>> lustre
>>> $ cat /proc/fs/lustre/version
>>> lustre: 1.6.4.2
>>> kernel: 47
>>> build: b_release_1_6_4-20080123034511-
>>> CHANGED-.usr.src.linux-2.6.16.46-0.14
>> gnode30 ~ # cat /proc/fs/lustre/version
>> lustre: 1.6.4.2
>> kernel: patchless
>> build: 1.6.4.2-19700101010000-PRISTINE-.usr.src.linux-2.6.22-
>> gentoo- r9-ubelix-node-2007.1
>>> $ lfs getstripe /myth/tv/1017_20080129100000.mpg
>>> OBDS:
>>> 0: ost-myth-0_UUID ACTIVE
>>> 1: ost-myth-1_UUID ACTIVE
>>> 2: ost-myth-2_UUID ACTIVE
>>> 3: ost-myth-3_UUID ACTIVE
>>> /myth/tv/1017_20080129100000.mpg
>>> obdidx objid objid group
>>> 2 527023 0x80aaf 0
>>> 1 531093 0x81a95 0
>> gnode30 ~ # lfs getstripe /mnt/lustre
>> OBDS:
>> /mnt/lustre
>> default stripe_count: 6 stripe_size: 0 stripe_offset: 1
>> /mnt/lustre/zero2
>> obdidx objid objid group
>> 4 7 0x7 0
>> /mnt/lustre/zero
>> obdidx objid objid group
>> 1 75 0x4b 0
>> 2 12 0xc 0
>> 3 73 0x49 0
>> 4 10 0xa 0
>> 6 9 0x9 0
>> 7 70 0x46 0
>> I'm not seeing an OBDS list here, I assume this is not good ...
>>> $ lfs getstripe -O ost-myth-1_UUID /myth/tv | head
>>> /myth/tv/1017_20080129100000.mpg
>>> /myth/tv/1017_20071101143000.mpg.png
>>> /myth/tv/1037_20080119013000.mpg.png
>>> /myth/tv/1017_20071219103000.mpg
>>> /myth/tv/1008_20080102200000.mpg
>>> /myth/tv/1039_20080123131300.mpg.png
>>> /myth/tv/1014_20070905170000.mpg.png
>>> /myth/tv/1039_20070914160000.mpg.png
>>> /myth/tv/1014_20071009090000.mpg.png
>>> /myth/tv/1017_20070917180000.mpg
>> gnode30 ~ # lfs getstripe -O scratch-OST0001_UUID /mnt/lustre
>> error: setup_obd_uuid: unknown obduuid: scratch-OST0001_UUID
>> /mnt/lustre
>> default stripe_count: 6 stripe_size: 0 stripe_offset: 1
>> /mnt/lustre/zero2
>> /mnt/lustre/zero
>> By the way, is the 'lfs getstripe' command supposed to work if
>> striping is disabled with 'lfs setstripe -d /mnt/lustre'?
>
> Yes, it should. Are you running this from a client?
> cliffw
Yes, although it's a patchless client running on a 2.6.22 kernel. I
also tried it on the failover server that was not mounting any lustre
devices while I mounted the filesystem from the first server. It runs
a Vanilla 2.6.18.8 kernel with lustre patches and lustre version
1.6.4.2. I got the same error.
I also did an strace for the command on my patchless client:
...
open("/proc/fs/lustre/lov/scratch-clilov-ffff8100f64bd000/target_obd",
O_RDONLY) = 4
fstat(4, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0x2b163b43a000
read(4, "", 1024) = 0
close(4) = 0
munmap(0x2b163b43a000, 4096) = 0
write(2, "error: setup_obd_uuid: unknown o"..., 77error:
setup_obd_uuid: unknown obduuid: scratch-OST0001-osc-ffff8100f64bd000
) = 77
...
Should there be any content in /proc/fs/lustre/lov/.../target_obd,
mine is empty?
Anselm
>>> Cheers, Andreas
>> Thanks for your help,
>> Anselm
More information about the lustre-discuss
mailing list