[lustre-devel] [PATCH 17/32] lustre: llog: Add LLOG_SKIP_PLAIN to skip llog plain
James Simmons
jsimmons at infradead.org
Wed Aug 3 18:38:02 PDT 2022
From: Etienne AUJAMES <etienne.aujames at cea.fr>
Add the catalog callback return LLOG_SKIP_PLAIN to conditionally skip
an entire llog plain.
This could speedup the catalog processing for specific usages when a
record need to be access in the "middle" of the catalog. This could
be useful for changelog with several users or HSM.
This patch modify chlg_read_cat_process_cb() to use LLOG_SKIP_PLAIN.
The main idea came from: d813c75d ("LU-14688 mdt: changelog purge
deletes plain llog")
**Performance test:**
* Environment:
2474195 changelogs record store on the mds0 (40 llog plain):
mds# lctl get_param -n mdd.lustrefs-MDT0000.changelog_users
current index: 2474195
ID index (idle seconds)
cl1 0 (3509)
* Test
Access to records at the end of the catalog (offset: 2474194):
client# time lfs changelog lustrefs-MDT0000 2474194 >/dev/null
* Results
- with the patch: real 0m0.592s
- without the patch: real 0m17.835s (x30)
WC-bug-id: https://jira.whamcloud.com/browse/LU-15481
Lustre-commit: aa22a6826ee521ab1 ("LU-15481 llog: Add LLOG_SKIP_PLAIN to skip llog plain")
Signed-off-by: Etienne AUJAMES <etienne.aujames at cea.fr>
Reviewed-on: https://review.whamcloud.com/46310
Reviewed-by: Alexander Boyko <alexander.boyko at hpe.com>
Reviewed-by: Andreas Dilger <adilger at whamcloud.com>
Reviewed-by: Oleg Drokin <green at whamcloud.com>
Signed-off-by: James Simmons <jsimmons at infradead.org>
---
fs/lustre/include/lustre_log.h | 18 +++++++++++++++++-
fs/lustre/mdc/mdc_changelog.c | 5 +++++
2 files changed, 22 insertions(+), 1 deletion(-)
diff --git a/fs/lustre/include/lustre_log.h b/fs/lustre/include/lustre_log.h
index 2e43d56..dbf3fd6 100644
--- a/fs/lustre/include/lustre_log.h
+++ b/fs/lustre/include/lustre_log.h
@@ -264,7 +264,7 @@ struct llog_ctxt {
};
#define LLOG_PROC_BREAK 0x0001
-#define LLOG_DEL_RECORD 0x0002
+#define LLOG_SKIP_PLAIN 0x0004
static inline int llog_handle2ops(struct llog_handle *loghandle,
const struct llog_operations **lop)
@@ -375,6 +375,22 @@ static inline int llog_next_block(const struct lu_env *env,
return rc;
}
+/* Determine if a llog plain of a catalog could be skiped based on record
+ * custom indexes.
+ * This assumes that indexes follow each other. The number of records to skip
+ * can be computed base on a starting offset and the index of the current
+ * record (in llog catalog callback).
+ */
+static inline int llog_is_plain_skipable(struct llog_log_hdr *lh,
+ struct llog_rec_hdr *rec,
+ u64 curr, u64 start)
+{
+ if (start == 0 || curr >= start)
+ return 0;
+
+ return (LLOG_HDR_BITMAP_SIZE(lh) - rec->lrh_index) < (start - curr);
+}
+
/* llog.c */
int lustre_process_log(struct super_block *sb, char *logname,
struct config_llog_instance *cfg);
diff --git a/fs/lustre/mdc/mdc_changelog.c b/fs/lustre/mdc/mdc_changelog.c
index 36d7fdd..cd2a610 100644
--- a/fs/lustre/mdc/mdc_changelog.c
+++ b/fs/lustre/mdc/mdc_changelog.c
@@ -225,6 +225,11 @@ static int chlg_read_cat_process_cb(const struct lu_env *env,
return rc;
}
+ /* Check if we can skip the entire llog plain */
+ if (llog_is_plain_skipable(llh->lgh_hdr, hdr, rec->cr.cr_index,
+ crs->crs_start_offset))
+ return LLOG_SKIP_PLAIN;
+
/* Skip undesired records */
if (rec->cr.cr_index < crs->crs_start_offset)
return 0;
--
1.8.3.1
More information about the lustre-devel
mailing list