[lustre-discuss] llapi_file_get_stripe() and /proc/fs/lustre/osc restated

[Read, Robert]

Hi John,

The initial string in the OSC name is the filesystem name and the long hexadecimal number at the end is the client ID. The client ID is specific to the mount point that “owns” the OSC, and typically changes each time the filesystem is mounted. It is there to differentiate devices  when the same filesystem is mounted multiple times.

You can retrieve both of the values by passing the mount directory to llapi_get_name(). This function returns a string with filesystem name and client ID in the format “FSNAME-ID". You can split the string on the ‘-‘ to extract those values, and then use them construct the right name for the current OSC being used for that file on that particular mountpoint.

Note, like the client ID, the OSC name is specific to that mount point. The actual OST name is just FSNAME-OST####.

robert

On Jul 18, 2016, at 15:22 , John Bauer <bauerj at iodoctors.com<mailto:bauerj at iodoctors.com>> wrote:


I will restate the problem I am having with Lustre.

With my I/O instrumentation library, I want to use llapi_file_get_stripe() to find the OSTs that a file of interest is striped on and then monitor only those OST's using files in the directory /proc/fs/lustre/osc.  This needs to be done programmatically, and in a general sense, with potentially only a relative path name.

llapi_file_get stripe() yields
lmm_stripe_count:   1
lmm_stripe_size:    1048576
lmm_pattern:        1
lmm_layout_gen:     0
lmm_stripe_offset:  12

lmm_oi.oi_fid       0x194aa

        obdidx           objid           objid           group
            12        31515860      0x1e0e4d4                0

If I use obdidx=12=0xc to find the OST in directory /proc/fs/lustre/osc, I get multiple OSTs as there are multiple file systems with an ost of index 12 ( note that obdidx is decimal and entries in /proc/fs/lustre/osc are hexadecimal, so we are looking for OST000c ).

% ls -ld /proc/fs/lustre/osc/*OST000c*
dr-xr-xr-x 2 root root 0 Jul 18 14:31 /proc/fs/lustre/osc/nbp1-OST000c-osc-ffff88090509f000
dr-xr-xr-x 2 root root 0 Jul 18 14:31 /proc/fs/lustre/osc/nbp2-OST000c-osc-ffff881038061c00
dr-xr-xr-x 2 root root 0 Jul 18 14:31 /proc/fs/lustre/osc/nbp6-OST000c-osc-ffff88084c405400
dr-xr-xr-x 2 root root 0 Jul 18 14:31 /proc/fs/lustre/osc/nbp7-OST000c-osc-ffff8807a8e1d400
dr-xr-xr-x 2 root root 0 Jul 18 14:31 /proc/fs/lustre/osc/nbp8-OST000c-osc-ffff88078339b800
dr-xr-xr-x 2 root root 0 Jul 18 14:31 /proc/fs/lustre/osc/nbp9-OST000c-osc-ffff8807833a5400

So I need to figure out which directory entry applies to the OST of my file of interest.

I looked at the inode for clues.  I did an stat() of the file to get

dev_t     st_dev=0xcc5d43c2
ino_t     st_ino=0x2000311ed0194aa

I notice the lov_user_md->oi_fid=0x0194aa,  populated by llapi_file_get_stripe(), is reflected in the lower part of stat.st<http://stat.st>_ino=0x2000311ed0194aa.  My question is, "Does the remainder of st_ino, 2000311ed, give me any clue as to which OST I should use out of /proc/fs/lustre/osc?"  The same question applies to the OST's objid=0x1e0e4d4 and the file's st_dev=0xcc5d43c2.

Because I know a priori that the file is in the lov nbp2, I know I need to find /proc/fs/lustre/osc/nbp2-OST000c-osc-ffff881038061c00.  What does the
ffff881038061c00 represent?  It is the same value for all OST's in a given lov, so I am guessing it is lov related.

There are over 1200 OST on the node, so I want to minimize the number that I instrument.
Any information that would shed some light on this would be greatly appreciated.
John

--
I/O Doctors, LLC
507-766-0378
bauerj at iodoctors.com<mailto:bauerj at iodoctors.com>

_______________________________________________
lustre-discuss mailing list
lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20160718/1eb17e96/attachment.htm>