[lustre-discuss] Access times for file (file heat)

[Andreas Dilger]

Anna,
for monitoring server storage access, the client-side file heat is not actually very useful, because (a) it is spread across all of the clients, and (b) it shows the client-side access patterns (which may be largely from cache) and not the actual server storage access which is what is important for tiering decisions.

Consider if a client reads a file once from the storage, and then accesses it 1000x from cache, the file may be considered "hot" by the client but it doesn't really matter what kind of storage the file is on since the storage only sees a single read.

Instead of the client-side file heat there is a different mechanism, the OST (lustre/utils/ofd_access_log_reader.c, ALR) that is a lightweight mechanism to aggregate all storage access into a producer/consumer circular log that is consumed by a userspace process. It is up to the userspace process to aggregate these log records across all of the OSTs to make decisions about which files are "hot" and which are "cold".  The ALR records are described in lustre/include/uapi/linux/lustre/lustre_access_log.h:

struct ofd_access_entry_v1 {
        struct lu_fid   oae_parent_fid; /* 16 */
        __u64           oae_begin; /* 24 */
        __u64           oae_end; /* 32 */
        __u64           oae_time; /* 40 */
        __u32           oae_size; /* 44 */
        __u32           oae_segment_count; /* 48 */
        __u32           oae_flags; /* 52 enum ofd_access_flags */
        __u32           oae_reserved1; /* 56 */
        __u32           oae_reserved2; /* 60 */
        __u32           oae_reserved3; /* 64 */
};

The records contain the MDT FID (to allow aggregation of IOs across multiple describe if it  is a read or write, and the start/end of the extent read/written so that the consumer can decide if the IOs are well-formed for the storage (e.g. large or sequential reads/writes on an HDD are OK vs. small/random reads/writes on an HDD are not OK).  The ALR queue is transient, so it is up to the consumer to make any decisions about the current IO patterns on files

Cheers, Andreas

On Feb 22, 2023, at 14:16, Anna Fuchs <anna.fuchs at uni-hamburg.de<mailto:anna.fuchs at uni-hamburg.de>> wrote:

Thank you.

Is there any documentation for the values?
Client-side means only statistics for the file remaining in the client cache? Not lifetime statistics?
Are there any plans to work further on this feature?

I think of several use cases when knowing these stats.
Cold data could be moved to archive like slow tape without relying on access time.
Hot blocks could be replicated or moved to faster caches and lot more optimizations.

Best regards
Anna



Am 18.02.2023 um 21:57 schrieb Andreas Dilger:

Anna, there was a client-side file heat mechanism added a few years ago, but I don't know if it is fully functional today.

lctl get_param llite.*.*heat*
llite.myth-ffff979380fc1800.file_heat=1
llite.myth-ffff979380fc1800.heat_decay_percentage=80
llite.myth-ffff979380fc1800.heat_period_second=60

And then "lfs heat_get <file>" to dump the file heat,  it there haven't been any good tools developed yet to list top heat files.

Cheers, Andreas



On Feb 7, 2023, at 08:56, Anna Fuchs via lustre-discuss <lustre-discuss at lists.lustre.org><mailto:lustre-discuss at lists.lustre.org> wrote:

Hello,

is there a way to see how many times a file has been accessed ever (like a heat map)?

Thanks
Anna

--
Anna Fuchs
Universität Hamburg
https://wr.informatik.uni-hamburg.de<https://wr.informatik.uni-hamburg.de/>

anna.fuchs at informatik.uni-hamburg.de<mailto:anna.fuchs at informatik.uni-hamburg.de>
https://wr.informatik.uni-hamburg.de/people/anna_fuchs



_______________________________________________
lustre-discuss mailing list
lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org



--
Anna Fuchs
Universität Hamburg
https://wr.informatik.uni-hamburg.de<https://wr.informatik.uni-hamburg.de/>

anna.fuchs at informatik.uni-hamburg.de<mailto:anna.fuchs at informatik.uni-hamburg.de>
https://wr.informatik.uni-hamburg.de/people/anna_fuchs

Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Whamcloud







-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20230226/35a13b8a/attachment.htm>