From wszhang159 at gmail.com Tue Nov 1 02:15:20 2016 From: wszhang159 at gmail.com (W. Zhang) Date: Tue, 1 Nov 2016 10:15:20 +0800 Subject: [lustre-devel] Failed to build a Lustre Storage Client on OpenVZ Kernel ! Is this a bug? Message-ID: Hello, I'm trying to build a Lustre client on OpenVZ kernel (version: 042stab120.5, based on RHEL 6.8 kernel 2.6.32-642.6.1.el6), and some bugs occured: (1) When build version v2_8_59_0 in git tag repo of Lustre, I encountered the following errors: " LD [M] /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/fld/ fld.o CC [M] /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit e/lloop.o In file included from /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit e/lloop.c:109: /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit e/llite_internal.h:1469: error: conflicting types for 'iov_iter_iovec' include/linux/fs.h:765: note: previous definition of 'iov_iter_iovec' was here /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit e/llite_internal.h: In function 'iov_iter_iovec': /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit e/llite_internal.h:1472: error: 'const struct iov_iter' has no member named 'iov' /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit e/llite_internal.h:1473: error: 'const struct iov_iter' has no member named 'iov' cc1: warnings being treated as errors /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit e/llite_internal.h:1473: error: type defaults to 'int' in declaration of '_min2' ... " (2) When build old version v2_8_50_0 in git tag repo of Lustre, the following errors display: " CC [M] /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit e/vvp_lock.o CC [M] /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit e/vvp_io.o /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit e/vvp_io.c: In function 'vvp_mmap_locks': /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit e/vvp_io.c:423: error: 'struct iov_iter' has no member named 'iov' /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit e/vvp_io.c: In function 'vvp_io_advance': /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit e/vvp_io.c:514: error: 'struct iov_iter' has no member named 'iov' ... " Files with full error context are attached. So, How can I fix the problem? Is this a bug which need to be fixed? Thanks! Any suggestion is appreciated! Best Wishes! -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: log.zip Type: application/zip Size: 13180 bytes Desc: not available URL: From jsimmons at infradead.org Wed Nov 2 18:46:52 2016 From: jsimmons at infradead.org (James Simmons) Date: Wed, 2 Nov 2016 14:46:52 -0400 Subject: [lustre-devel] [PATCH v2] staging: lustre: mdc: manage number of modify RPCs in flight Message-ID: <1478112412-8200-1-git-send-email-jsimmons@infradead.org> From: Gregoire Pichon This patch is the main client part of a new feature that supports multiple modify metadata RPCs in parallel. Its goal is to improve metadata operations performance of a single client, while maintening the consistency of MDT reply reconstruction and MDT recovery mechanisms. It allows to manage the number of modify RPCs in flight within the client obd structure and to assign a virtual index (the tag) to each modify RPC to help server side cleaning of reply data. The mdc component uses this feature to send multiple modify RPCs in parallel. Signed-off-by: Gregoire Pichon Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-5319 Reviewed-on: http://review.whamcloud.com/14374 Reviewed-by: Andreas Dilger Reviewed-by: Alex Zhuravlev Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- drivers/staging/lustre/lustre/fid/fid_request.c | 4 - drivers/staging/lustre/lustre/include/lustre_mdc.h | 24 +++ drivers/staging/lustre/lustre/include/obd.h | 7 +- drivers/staging/lustre/lustre/include/obd_class.h | 6 + drivers/staging/lustre/lustre/ldlm/ldlm_lib.c | 37 +++++ drivers/staging/lustre/lustre/mdc/lproc_mdc.c | 23 +++ drivers/staging/lustre/lustre/mdc/mdc_locks.c | 13 +- drivers/staging/lustre/lustre/mdc/mdc_reint.c | 23 +-- drivers/staging/lustre/lustre/mdc/mdc_request.c | 50 ++----- drivers/staging/lustre/lustre/obdclass/genops.c | 171 +++++++++++++++++++- 10 files changed, 293 insertions(+), 65 deletions(-) diff --git a/drivers/staging/lustre/lustre/fid/fid_request.c b/drivers/staging/lustre/lustre/fid/fid_request.c index 1148b9a..999f250 100644 --- a/drivers/staging/lustre/lustre/fid/fid_request.c +++ b/drivers/staging/lustre/lustre/fid/fid_request.c @@ -112,11 +112,7 @@ static int seq_client_rpc(struct lu_client_seq *seq, ptlrpc_at_set_req_timeout(req); - if (opc != SEQ_ALLOC_SUPER && seq->lcs_type == LUSTRE_SEQ_METADATA) - mdc_get_rpc_lock(exp->exp_obd->u.cli.cl_rpc_lock, NULL); rc = ptlrpc_queue_wait(req); - if (opc != SEQ_ALLOC_SUPER && seq->lcs_type == LUSTRE_SEQ_METADATA) - mdc_put_rpc_lock(exp->exp_obd->u.cli.cl_rpc_lock, NULL); if (rc) goto out_req; diff --git a/drivers/staging/lustre/lustre/include/lustre_mdc.h b/drivers/staging/lustre/lustre/include/lustre_mdc.h index 92a5c0f..198ceb0 100644 --- a/drivers/staging/lustre/lustre/include/lustre_mdc.h +++ b/drivers/staging/lustre/lustre/include/lustre_mdc.h @@ -156,6 +156,30 @@ static inline void mdc_put_rpc_lock(struct mdc_rpc_lock *lck, mutex_unlock(&lck->rpcl_mutex); } +static inline void mdc_get_mod_rpc_slot(struct ptlrpc_request *req, + struct lookup_intent *it) +{ + struct client_obd *cli = &req->rq_import->imp_obd->u.cli; + u32 opc; + u16 tag; + + opc = lustre_msg_get_opc(req->rq_reqmsg); + tag = obd_get_mod_rpc_slot(cli, opc, it); + lustre_msg_set_tag(req->rq_reqmsg, tag); +} + +static inline void mdc_put_mod_rpc_slot(struct ptlrpc_request *req, + struct lookup_intent *it) +{ + struct client_obd *cli = &req->rq_import->imp_obd->u.cli; + u32 opc; + u16 tag; + + opc = lustre_msg_get_opc(req->rq_reqmsg); + tag = lustre_msg_get_tag(req->rq_reqmsg); + obd_put_mod_rpc_slot(cli, opc, it, tag); +} + /** * Update the maximum possible easize. * diff --git a/drivers/staging/lustre/lustre/include/obd.h b/drivers/staging/lustre/lustre/include/obd.h index c8a6e23..09e3e71 100644 --- a/drivers/staging/lustre/lustre/include/obd.h +++ b/drivers/staging/lustre/lustre/include/obd.h @@ -263,14 +263,17 @@ struct client_obd { wait_queue_head_t cl_destroy_waitq; struct mdc_rpc_lock *cl_rpc_lock; - struct mdc_rpc_lock *cl_close_lock; /* modify rpcs in flight * currently used for metadata only */ spinlock_t cl_mod_rpcs_lock; u16 cl_max_mod_rpcs_in_flight; - + u16 cl_mod_rpcs_in_flight; + u16 cl_close_rpcs_in_flight; + wait_queue_head_t cl_mod_rpcs_waitq; + unsigned long *cl_mod_tag_bitmap; + struct obd_histogram cl_mod_rpcs_hist; /* mgc datastruct */ atomic_t cl_mgc_refcount; diff --git a/drivers/staging/lustre/lustre/include/obd_class.h b/drivers/staging/lustre/lustre/include/obd_class.h index 70b355e..f79133c 100644 --- a/drivers/staging/lustre/lustre/include/obd_class.h +++ b/drivers/staging/lustre/lustre/include/obd_class.h @@ -101,6 +101,12 @@ struct obd_device *class_devices_in_group(struct obd_uuid *grp_uuid, __u32 obd_get_max_rpcs_in_flight(struct client_obd *cli); int obd_set_max_rpcs_in_flight(struct client_obd *cli, __u32 max); int obd_set_max_mod_rpcs_in_flight(struct client_obd *cli, u16 max); +int obd_mod_rpc_stats_seq_show(struct client_obd *cli, struct seq_file *seq); + +u16 obd_get_mod_rpc_slot(struct client_obd *cli, u32 opc, + struct lookup_intent *it); +void obd_put_mod_rpc_slot(struct client_obd *cli, u32 opc, + struct lookup_intent *it, u16 tag); struct llog_handle; struct llog_rec_hdr; diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c index 4f9480e..ee4bb56 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c @@ -372,6 +372,25 @@ int client_obd_setup(struct obd_device *obddev, struct lustre_cfg *lcfg) } else { cli->cl_max_rpcs_in_flight = OBD_MAX_RIF_DEFAULT; } + + spin_lock_init(&cli->cl_mod_rpcs_lock); + spin_lock_init(&cli->cl_mod_rpcs_hist.oh_lock); + cli->cl_max_mod_rpcs_in_flight = 0; + cli->cl_mod_rpcs_in_flight = 0; + cli->cl_close_rpcs_in_flight = 0; + init_waitqueue_head(&cli->cl_mod_rpcs_waitq); + cli->cl_mod_tag_bitmap = NULL; + + if (connect_op == MDS_CONNECT) { + cli->cl_max_mod_rpcs_in_flight = cli->cl_max_rpcs_in_flight - 1; + cli->cl_mod_tag_bitmap = kcalloc(BITS_TO_LONGS(OBD_MAX_RIF_MAX), + sizeof(long), GFP_NOFS); + if (!cli->cl_mod_tag_bitmap) { + rc = -ENOMEM; + goto err; + } + } + rc = ldlm_get_ref(); if (rc) { CERROR("ldlm_get_ref failed: %d\n", rc); @@ -431,12 +450,16 @@ int client_obd_setup(struct obd_device *obddev, struct lustre_cfg *lcfg) err_ldlm: ldlm_put_ref(); err: + kfree(cli->cl_mod_tag_bitmap); + cli->cl_mod_tag_bitmap = NULL; return rc; } EXPORT_SYMBOL(client_obd_setup); int client_obd_cleanup(struct obd_device *obddev) { + struct client_obd *cli = &obddev->u.cli; + ldlm_namespace_free_post(obddev->obd_namespace); obddev->obd_namespace = NULL; @@ -444,6 +467,10 @@ int client_obd_cleanup(struct obd_device *obddev) LASSERT(!obddev->u.cli.cl_import); ldlm_put_ref(); + + kfree(cli->cl_mod_tag_bitmap); + cli->cl_mod_tag_bitmap = NULL; + return 0; } EXPORT_SYMBOL(client_obd_cleanup); @@ -458,6 +485,7 @@ int client_connect_import(const struct lu_env *env, struct obd_import *imp = cli->cl_import; struct obd_connect_data *ocd; struct lustre_handle conn = { 0 }; + bool is_mdc = false; int rc; *exp = NULL; @@ -484,6 +512,10 @@ int client_connect_import(const struct lu_env *env, ocd = &imp->imp_connect_data; if (data) { *ocd = *data; + is_mdc = !strncmp(imp->imp_obd->obd_type->typ_name, + LUSTRE_MDC_NAME, 3); + if (is_mdc) + data->ocd_connect_flags |= OBD_CONNECT_MULTIMODRPCS; imp->imp_connect_flags_orig = data->ocd_connect_flags; } @@ -499,6 +531,11 @@ int client_connect_import(const struct lu_env *env, ocd->ocd_connect_flags, "old %#llx, new %#llx\n", data->ocd_connect_flags, ocd->ocd_connect_flags); data->ocd_connect_flags = ocd->ocd_connect_flags; + /* clear the flag as it was not set and is not known + * by upper layers + */ + if (is_mdc) + data->ocd_connect_flags &= ~OBD_CONNECT_MULTIMODRPCS; } ptlrpc_pinger_add_import(imp); diff --git a/drivers/staging/lustre/lustre/mdc/lproc_mdc.c b/drivers/staging/lustre/lustre/mdc/lproc_mdc.c index 76b9afc..9021c46 100644 --- a/drivers/staging/lustre/lustre/mdc/lproc_mdc.c +++ b/drivers/staging/lustre/lustre/mdc/lproc_mdc.c @@ -146,6 +146,27 @@ static ssize_t max_mod_rpcs_in_flight_store(struct kobject *kobj, } LUSTRE_RW_ATTR(max_mod_rpcs_in_flight); +static int mdc_rpc_stats_seq_show(struct seq_file *seq, void *v) +{ + struct obd_device *dev = seq->private; + + return obd_mod_rpc_stats_seq_show(&dev->u.cli, seq); +} + +static ssize_t mdc_rpc_stats_seq_write(struct file *file, + const char __user *buf, + size_t len, loff_t *off) +{ + struct seq_file *seq = file->private_data; + struct obd_device *dev = seq->private; + struct client_obd *cli = &dev->u.cli; + + lprocfs_oh_clear(&cli->cl_mod_rpcs_hist); + + return len; +} +LPROC_SEQ_FOPS(mdc_rpc_stats); + LPROC_SEQ_FOPS_WR_ONLY(mdc, ping); LPROC_SEQ_FOPS_RO_TYPE(mdc, connect_flags); @@ -185,6 +206,8 @@ static ssize_t max_pages_per_rpc_show(struct kobject *kobj, { "import", &mdc_import_fops, NULL, 0 }, { "state", &mdc_state_fops, NULL, 0 }, { "pinger_recov", &mdc_pinger_recov_fops, NULL, 0 }, + { .name = "rpc_stats", + .fops = &mdc_rpc_stats_fops }, { NULL } }; diff --git a/drivers/staging/lustre/lustre/mdc/mdc_locks.c b/drivers/staging/lustre/lustre/mdc/mdc_locks.c index b9ca140..42a128f 100644 --- a/drivers/staging/lustre/lustre/mdc/mdc_locks.c +++ b/drivers/staging/lustre/lustre/mdc/mdc_locks.c @@ -766,15 +766,16 @@ int mdc_enqueue(struct obd_export *exp, struct ldlm_enqueue_info *einfo, req->rq_sent = ktime_get_real_seconds() + resends; } - /* It is important to obtain rpc_lock first (if applicable), so that - * threads that are serialised with rpc_lock are not polluting our - * rpcs in flight counter. We do not do flock request limiting, though + /* It is important to obtain modify RPC slot first (if applicable), so + * that threads that are waiting for a modify RPC slot are not polluting + * our rpcs in flight counter. + * We do not do flock request limiting, though */ if (it) { - mdc_get_rpc_lock(obddev->u.cli.cl_rpc_lock, it); + mdc_get_mod_rpc_slot(req, it); rc = obd_get_request_slot(&obddev->u.cli); if (rc != 0) { - mdc_put_rpc_lock(obddev->u.cli.cl_rpc_lock, it); + mdc_put_mod_rpc_slot(req, it); mdc_clear_replay_flag(req, 0); ptlrpc_req_finished(req); return rc; @@ -801,7 +802,7 @@ int mdc_enqueue(struct obd_export *exp, struct ldlm_enqueue_info *einfo, } obd_put_request_slot(&obddev->u.cli); - mdc_put_rpc_lock(obddev->u.cli.cl_rpc_lock, it); + mdc_put_mod_rpc_slot(req, it); if (rc < 0) { CDEBUG(D_INFO, "%s: ldlm_cli_enqueue failed: rc = %d\n", diff --git a/drivers/staging/lustre/lustre/mdc/mdc_reint.c b/drivers/staging/lustre/lustre/mdc/mdc_reint.c index 1847e5a..b551c57 100644 --- a/drivers/staging/lustre/lustre/mdc/mdc_reint.c +++ b/drivers/staging/lustre/lustre/mdc/mdc_reint.c @@ -40,17 +40,15 @@ #include "../include/lustre_fid.h" /* mdc_setattr does its own semaphore handling */ -static int mdc_reint(struct ptlrpc_request *request, - struct mdc_rpc_lock *rpc_lock, - int level) +static int mdc_reint(struct ptlrpc_request *request, int level) { int rc; request->rq_send_state = level; - mdc_get_rpc_lock(rpc_lock, NULL); + mdc_get_mod_rpc_slot(request, NULL); rc = ptlrpc_queue_wait(request); - mdc_put_rpc_lock(rpc_lock, NULL); + mdc_put_mod_rpc_slot(request, NULL); if (rc) CDEBUG(D_INFO, "error in handling %d\n", rc); else if (!req_capsule_server_get(&request->rq_pill, &RMF_MDT_BODY)) @@ -103,8 +101,6 @@ int mdc_setattr(struct obd_export *exp, struct md_op_data *op_data, { LIST_HEAD(cancels); struct ptlrpc_request *req; - struct mdc_rpc_lock *rpc_lock; - struct obd_device *obd = exp->exp_obd; int count = 0, rc; __u64 bits; @@ -131,8 +127,6 @@ int mdc_setattr(struct obd_export *exp, struct md_op_data *op_data, return rc; } - rpc_lock = obd->u.cli.cl_rpc_lock; - if (op_data->op_attr.ia_valid & (ATTR_MTIME | ATTR_CTIME)) CDEBUG(D_INODE, "setting mtime %ld, ctime %ld\n", LTIME_S(op_data->op_attr.ia_mtime), @@ -141,7 +135,7 @@ int mdc_setattr(struct obd_export *exp, struct md_op_data *op_data, ptlrpc_request_set_replen(req); - rc = mdc_reint(req, rpc_lock, LUSTRE_IMP_FULL); + rc = mdc_reint(req, LUSTRE_IMP_FULL); if (rc == -ERESTARTSYS) rc = 0; @@ -220,7 +214,7 @@ int mdc_create(struct obd_export *exp, struct md_op_data *op_data, } level = LUSTRE_IMP_FULL; resend: - rc = mdc_reint(req, exp->exp_obd->u.cli.cl_rpc_lock, level); + rc = mdc_reint(req, level); /* Resend if we were told to. */ if (rc == -ERESTARTSYS) { @@ -292,7 +286,7 @@ int mdc_unlink(struct obd_export *exp, struct md_op_data *op_data, *request = req; - rc = mdc_reint(req, obd->u.cli.cl_rpc_lock, LUSTRE_IMP_FULL); + rc = mdc_reint(req, LUSTRE_IMP_FULL); if (rc == -ERESTARTSYS) rc = 0; return rc; @@ -302,7 +296,6 @@ int mdc_link(struct obd_export *exp, struct md_op_data *op_data, struct ptlrpc_request **request) { LIST_HEAD(cancels); - struct obd_device *obd = exp->exp_obd; struct ptlrpc_request *req; int count = 0, rc; @@ -334,7 +327,7 @@ int mdc_link(struct obd_export *exp, struct md_op_data *op_data, mdc_link_pack(req, op_data); ptlrpc_request_set_replen(req); - rc = mdc_reint(req, obd->u.cli.cl_rpc_lock, LUSTRE_IMP_FULL); + rc = mdc_reint(req, LUSTRE_IMP_FULL); *request = req; if (rc == -ERESTARTSYS) rc = 0; @@ -398,7 +391,7 @@ int mdc_rename(struct obd_export *exp, struct md_op_data *op_data, obd->u.cli.cl_default_mds_easize); ptlrpc_request_set_replen(req); - rc = mdc_reint(req, obd->u.cli.cl_rpc_lock, LUSTRE_IMP_FULL); + rc = mdc_reint(req, LUSTRE_IMP_FULL); *request = req; if (rc == -ERESTARTSYS) rc = 0; diff --git a/drivers/staging/lustre/lustre/mdc/mdc_request.c b/drivers/staging/lustre/lustre/mdc/mdc_request.c index c620f5c..e3a167a 100644 --- a/drivers/staging/lustre/lustre/mdc/mdc_request.c +++ b/drivers/staging/lustre/lustre/mdc/mdc_request.c @@ -327,12 +327,12 @@ static int mdc_xattr_common(struct obd_export *exp, /* make rpc */ if (opcode == MDS_REINT) - mdc_get_rpc_lock(exp->exp_obd->u.cli.cl_rpc_lock, NULL); + mdc_get_mod_rpc_slot(req, NULL); rc = ptlrpc_queue_wait(req); if (opcode == MDS_REINT) - mdc_put_rpc_lock(exp->exp_obd->u.cli.cl_rpc_lock, NULL); + mdc_put_mod_rpc_slot(req, NULL); if (rc) ptlrpc_req_finished(req); @@ -775,9 +775,9 @@ static int mdc_close(struct obd_export *exp, struct md_op_data *op_data, ptlrpc_request_set_replen(req); - mdc_get_rpc_lock(obd->u.cli.cl_close_lock, NULL); + mdc_get_mod_rpc_slot(req, NULL); rc = ptlrpc_queue_wait(req); - mdc_put_rpc_lock(obd->u.cli.cl_close_lock, NULL); + mdc_put_mod_rpc_slot(req, NULL); if (!req->rq_repmsg) { CDEBUG(D_RPCTRACE, "request failed to send: %p, %d\n", req, @@ -1490,9 +1490,9 @@ static int mdc_ioc_hsm_progress(struct obd_export *exp, ptlrpc_request_set_replen(req); - mdc_get_rpc_lock(exp->exp_obd->u.cli.cl_rpc_lock, NULL); + mdc_get_mod_rpc_slot(req, NULL); rc = ptlrpc_queue_wait(req); - mdc_put_rpc_lock(exp->exp_obd->u.cli.cl_rpc_lock, NULL); + mdc_put_mod_rpc_slot(req, NULL); out: ptlrpc_req_finished(req); return rc; @@ -1670,9 +1670,9 @@ static int mdc_ioc_hsm_state_set(struct obd_export *exp, ptlrpc_request_set_replen(req); - mdc_get_rpc_lock(exp->exp_obd->u.cli.cl_rpc_lock, NULL); + mdc_get_mod_rpc_slot(req, NULL); rc = ptlrpc_queue_wait(req); - mdc_put_rpc_lock(exp->exp_obd->u.cli.cl_rpc_lock, NULL); + mdc_put_mod_rpc_slot(req, NULL); out: ptlrpc_req_finished(req); return rc; @@ -1735,9 +1735,9 @@ static int mdc_ioc_hsm_request(struct obd_export *exp, ptlrpc_request_set_replen(req); - mdc_get_rpc_lock(exp->exp_obd->u.cli.cl_rpc_lock, NULL); + mdc_get_mod_rpc_slot(req, NULL); rc = ptlrpc_queue_wait(req); - mdc_put_rpc_lock(exp->exp_obd->u.cli.cl_rpc_lock, NULL); + mdc_put_mod_rpc_slot(req, NULL); out: ptlrpc_req_finished(req); return rc; @@ -2582,29 +2582,17 @@ static void mdc_llog_finish(struct obd_device *obd) static int mdc_setup(struct obd_device *obd, struct lustre_cfg *cfg) { - struct client_obd *cli = &obd->u.cli; struct lprocfs_static_vars lvars = { NULL }; int rc; - cli->cl_rpc_lock = kzalloc(sizeof(*cli->cl_rpc_lock), GFP_NOFS); - if (!cli->cl_rpc_lock) - return -ENOMEM; - mdc_init_rpc_lock(cli->cl_rpc_lock); - rc = ptlrpcd_addref(); if (rc < 0) - goto err_rpc_lock; - - cli->cl_close_lock = kzalloc(sizeof(*cli->cl_close_lock), GFP_NOFS); - if (!cli->cl_close_lock) { - rc = -ENOMEM; - goto err_ptlrpcd_decref; - } - mdc_init_rpc_lock(cli->cl_close_lock); + return rc; rc = client_obd_setup(obd, cfg); if (rc) - goto err_close_lock; + goto err_ptlrpcd_decref; + lprocfs_mdc_init_vars(&lvars); lprocfs_obd_setup(obd, lvars.obd_vars, lvars.sysfs_vars); sptlrpc_lprocfs_cliobd_attach(obd); @@ -2621,17 +2609,10 @@ static int mdc_setup(struct obd_device *obd, struct lustre_cfg *cfg) return rc; } - spin_lock_init(&cli->cl_mod_rpcs_lock); - cli->cl_max_mod_rpcs_in_flight = OBD_MAX_RIF_DEFAULT - 1; - return rc; -err_close_lock: - kfree(cli->cl_close_lock); err_ptlrpcd_decref: ptlrpcd_decref(); -err_rpc_lock: - kfree(cli->cl_rpc_lock); return rc; } @@ -2679,11 +2660,6 @@ static int mdc_precleanup(struct obd_device *obd, enum obd_cleanup_stage stage) static int mdc_cleanup(struct obd_device *obd) { - struct client_obd *cli = &obd->u.cli; - - kfree(cli->cl_rpc_lock); - kfree(cli->cl_close_lock); - ptlrpcd_decref(); return client_obd_cleanup(obd); diff --git a/drivers/staging/lustre/lustre/obdclass/genops.c b/drivers/staging/lustre/lustre/obdclass/genops.c index 62e6636..438d619 100644 --- a/drivers/staging/lustre/lustre/obdclass/genops.c +++ b/drivers/staging/lustre/lustre/obdclass/genops.c @@ -1461,6 +1461,7 @@ int obd_set_max_mod_rpcs_in_flight(struct client_obd *cli, __u16 max) { struct obd_connect_data *ocd; u16 maxmodrpcs; + u16 prev; if (max > OBD_MAX_RIF_MAX || max < 1) return -ERANGE; @@ -1486,10 +1487,178 @@ int obd_set_max_mod_rpcs_in_flight(struct client_obd *cli, __u16 max) return -ERANGE; } + spin_lock(&cli->cl_mod_rpcs_lock); + + prev = cli->cl_max_mod_rpcs_in_flight; cli->cl_max_mod_rpcs_in_flight = max; - /* will have to wakeup waiters if max has been increased */ + /* wakeup waiters if limit has been increased */ + if (cli->cl_max_mod_rpcs_in_flight > prev) + wake_up(&cli->cl_mod_rpcs_waitq); + + spin_unlock(&cli->cl_mod_rpcs_lock); return 0; } EXPORT_SYMBOL(obd_set_max_mod_rpcs_in_flight); + +#define pct(a, b) (b ? (a * 100) / b : 0) + +int obd_mod_rpc_stats_seq_show(struct client_obd *cli, struct seq_file *seq) +{ + unsigned long mod_tot = 0, mod_cum; + struct timespec64 now; + int i; + + ktime_get_real_ts64(&now); + + spin_lock(&cli->cl_mod_rpcs_lock); + + seq_printf(seq, "snapshot_time: %llu.%9lu (secs.nsecs)\n", + (s64)now.tv_sec, (unsigned long)now.tv_nsec); + seq_printf(seq, "modify_RPCs_in_flight: %hu\n", + cli->cl_mod_rpcs_in_flight); + + seq_puts(seq, "\n\t\t\tmodify\n"); + seq_puts(seq, "rpcs in flight rpcs %% cum %%\n"); + + mod_tot = lprocfs_oh_sum(&cli->cl_mod_rpcs_hist); + + mod_cum = 0; + for (i = 0; i < OBD_HIST_MAX; i++) { + unsigned long mod = cli->cl_mod_rpcs_hist.oh_buckets[i]; + + mod_cum += mod; + seq_printf(seq, "%d:\t\t%10lu %3lu %3lu\n", + i, mod, pct(mod, mod_tot), + pct(mod_cum, mod_tot)); + if (mod_cum == mod_tot) + break; + } + + spin_unlock(&cli->cl_mod_rpcs_lock); + + return 0; +} +EXPORT_SYMBOL(obd_mod_rpc_stats_seq_show); +#undef pct + +/* + * The number of modify RPCs sent in parallel is limited + * because the server has a finite number of slots per client to + * store request result and ensure reply reconstruction when needed. + * On the client, this limit is stored in cl_max_mod_rpcs_in_flight + * that takes into account server limit and cl_max_rpcs_in_flight + * value. + * On the MDC client, to avoid a potential deadlock (see Bugzilla 3462), + * one close request is allowed above the maximum. + */ +static inline bool obd_mod_rpc_slot_avail_locked(struct client_obd *cli, + bool close_req) +{ + bool avail; + + /* A slot is available if + * - number of modify RPCs in flight is less than the max + * - it's a close RPC and no other close request is in flight + */ + avail = cli->cl_mod_rpcs_in_flight < cli->cl_max_mod_rpcs_in_flight || + (close_req && !cli->cl_close_rpcs_in_flight); + + return avail; +} + +static inline bool obd_mod_rpc_slot_avail(struct client_obd *cli, + bool close_req) +{ + bool avail; + + spin_lock(&cli->cl_mod_rpcs_lock); + avail = obd_mod_rpc_slot_avail_locked(cli, close_req); + spin_unlock(&cli->cl_mod_rpcs_lock); + return avail; +} + +/* Get a modify RPC slot from the obd client @cli according + * to the kind of operation @opc that is going to be sent + * and the intent @it of the operation if it applies. + * If the maximum number of modify RPCs in flight is reached + * the thread is put to sleep. + * Returns the tag to be set in the request message. Tag 0 + * is reserved for non-modifying requests. + */ +u16 obd_get_mod_rpc_slot(struct client_obd *cli, __u32 opc, + struct lookup_intent *it) +{ + struct l_wait_info lwi = LWI_INTR(NULL, NULL); + bool close_req = false; + u16 i, max; + + /* read-only metadata RPCs don't consume a slot on MDT + * for reply reconstruction + */ + if (it && (it->it_op == IT_GETATTR || it->it_op == IT_LOOKUP || + it->it_op == IT_LAYOUT || it->it_op == IT_READDIR)) + return 0; + + if (opc == MDS_CLOSE) + close_req = true; + + do { + spin_lock(&cli->cl_mod_rpcs_lock); + max = cli->cl_max_mod_rpcs_in_flight; + if (obd_mod_rpc_slot_avail_locked(cli, close_req)) { + /* there is a slot available */ + cli->cl_mod_rpcs_in_flight++; + if (close_req) + cli->cl_close_rpcs_in_flight++; + lprocfs_oh_tally(&cli->cl_mod_rpcs_hist, + cli->cl_mod_rpcs_in_flight); + /* find a free tag */ + i = find_first_zero_bit(cli->cl_mod_tag_bitmap, + max + 1); + LASSERT(i < OBD_MAX_RIF_MAX); + LASSERT(!test_and_set_bit(i, cli->cl_mod_tag_bitmap)); + spin_unlock(&cli->cl_mod_rpcs_lock); + /* tag 0 is reserved for non-modify RPCs */ + return i + 1; + } + spin_unlock(&cli->cl_mod_rpcs_lock); + + CDEBUG(D_RPCTRACE, "%s: sleeping for a modify RPC slot opc %u, max %hu\n", + cli->cl_import->imp_obd->obd_name, opc, max); + + l_wait_event(cli->cl_mod_rpcs_waitq, + obd_mod_rpc_slot_avail(cli, close_req), &lwi); + } while (true); +} +EXPORT_SYMBOL(obd_get_mod_rpc_slot); + +/* + * Put a modify RPC slot from the obd client @cli according + * to the kind of operation @opc that has been sent and the + * intent @it of the operation if it applies. + */ +void obd_put_mod_rpc_slot(struct client_obd *cli, u32 opc, + struct lookup_intent *it, u16 tag) +{ + bool close_req = false; + + if (it && (it->it_op == IT_GETATTR || it->it_op == IT_LOOKUP || + it->it_op == IT_LAYOUT || it->it_op == IT_READDIR)) + return; + + if (opc == MDS_CLOSE) + close_req = true; + + spin_lock(&cli->cl_mod_rpcs_lock); + cli->cl_mod_rpcs_in_flight--; + if (close_req) + cli->cl_close_rpcs_in_flight--; + /* release the tag in the bitmap */ + LASSERT(tag - 1 < OBD_MAX_RIF_MAX); + LASSERT(test_and_clear_bit(tag - 1, cli->cl_mod_tag_bitmap) != 0); + spin_unlock(&cli->cl_mod_rpcs_lock); + wake_up(&cli->cl_mod_rpcs_waitq); +} +EXPORT_SYMBOL(obd_put_mod_rpc_slot); -- 1.7.1 From andreas.dilger at intel.com Wed Nov 2 23:05:29 2016 From: andreas.dilger at intel.com (Dilger, Andreas) Date: Wed, 2 Nov 2016 23:05:29 +0000 Subject: [lustre-devel] [PATCH 1/2] staging: lustre: replace uses of class_devno_max by MAX_OBD_DEVICES In-Reply-To: <20161025164730.GA27553@waves> References: <4ba436cdccbea6c6dbc631f8cf75cd80f4f9ddb4.1476739569.git.mahfouz.saif.elyazal@gmail.com> <20161025164730.GA27553@waves> Message-ID: <8FDBA22B-F6A6-4CA7-B997-41327603E8A9@intel.com> On Oct 25, 2016, at 10:47, Aya Mahfouz wrote: > > On Mon, Oct 17, 2016 at 10:38:31PM +0000, Dilger, Andreas wrote: >> On Oct 17, 2016, at 15:46, Aya Mahfouz wrote: >>> >>> class_devno_max is an inline function that returns >>> MAX_OBD_DEVICES. Replace all calls to the function >>> by MAX_OBD_DEVICES. >> >> Thanks for your patch, but unfortunately it can't be accepted. >> >> This function was added in preparation of being able to tune the maximum >> number of storage devices dynamically, rather than having to hard code it >> to the maximum possible number of servers that a client can possibly >> connect to. >> >> While the current maximum of 8192 servers has been enough for current >> filesystems, I'd rather move in the direction of dynamically handling this >> limit rather than re-introducing a hard-coded constant throughout the code. >> > Hello, > > I would like to proceed with implementing the function if possible. > Kindly direct me to some starting pointers. Hi Aya, thanks for offering to look into this. There are several ways to approach this problem to make the allocation of the obd_devs[] array dynamic. In most cases, there isn't any value to dynamically shrink this array, since the filesystem(s) will typically be mounted until the node is rebooted, and it is only in the tens of KB size range, so this will not affect ongoing operations, and that simplifies the implementation. The easiest way would be to have a dynamically-sized obd_devs[] array that is reallocated in class_newdev() in PAGE_SIZE chunks whenever the current array has no more free slots and copied to the new array, using obd_dev_lock to protect the array while it is being reallocated and copied. In most cases, this would save memory over the static array (not many filesystems have so many servers), but for the few sites that have 10000+ servers they don't need to change the source to handle this. Using libcfs_kvzalloc() would avoid issues with allocating large chunks of memory. There are a few places where obd_devs[] is accessed outside obd_dev_lock that would need to be fixed now that this array may be changed at runtime. A second approach that may scale better is to change obd_devs from an array to a doubly linked list (using standard list_head helpers). In many cases the whole list is seached linearly, and most of the uses of class_num2obd() are just used to walk that list in order, which could be replaced with list_for_each_entry() list traversal. The class_name2dev() function should be changed to return the pointer to the obd_device structure, and a new helper class_dev2num() would just return the obd_minor number from the obd_device struct for the one use in class_resolve_dev_name(). Using a linked list has the advantage that there is no need to search for free slots in the array, since devices would be removed from the list when it is freed. Cheers, Andreas >> One comment inline below, if you still want to submit a patch. >> >>> Signed-off-by: Aya Mahfouz >>> --- >>> drivers/staging/lustre/lustre/obdclass/class_obd.c | 6 +++--- >>> drivers/staging/lustre/lustre/obdclass/genops.c | 22 +++++++++++----------- >>> .../lustre/lustre/obdclass/linux/linux-module.c | 6 +++--- >>> 3 files changed, 17 insertions(+), 17 deletions(-) >>> >>> diff --git a/drivers/staging/lustre/lustre/obdclass/class_obd.c b/drivers/staging/lustre/lustre/obdclass/class_obd.c >>> index 2b21675..b775c74 100644 >>> --- a/drivers/staging/lustre/lustre/obdclass/class_obd.c >>> +++ b/drivers/staging/lustre/lustre/obdclass/class_obd.c >>> @@ -345,7 +345,7 @@ int class_handle_ioctl(unsigned int cmd, unsigned long arg) >>> goto out; >>> } >>> obd = class_name2obd(data->ioc_inlbuf4); >>> - } else if (data->ioc_dev < class_devno_max()) { >>> + } else if (data->ioc_dev < MAX_OBD_DEVICES) { >>> obd = class_num2obd(data->ioc_dev); >>> } else { >>> CERROR("OBD ioctl: No device\n"); >>> @@ -498,7 +498,7 @@ static int __init obdclass_init(void) >>> } >>> >>> /* This struct is already zeroed for us (static global) */ >>> - for (i = 0; i < class_devno_max(); i++) >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) >>> obd_devs[i] = NULL; >> >> This block can just be removed entirely. It used to do something useful, >> but through a series of changes it has become useless. >> >> Cheers, Andreas >> >>> /* Default the dirty page cache cap to 1/2 of system memory. >>> @@ -548,7 +548,7 @@ static void obdclass_exit(void) >>> lustre_unregister_fs(); >>> >>> misc_deregister(&obd_psdev); >>> - for (i = 0; i < class_devno_max(); i++) { >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) { >>> struct obd_device *obd = class_num2obd(i); >>> >>> if (obd && obd->obd_set_up && >>> diff --git a/drivers/staging/lustre/lustre/obdclass/genops.c b/drivers/staging/lustre/lustre/obdclass/genops.c >>> index 99c2da6..af4fc58 100644 >>> --- a/drivers/staging/lustre/lustre/obdclass/genops.c >>> +++ b/drivers/staging/lustre/lustre/obdclass/genops.c >>> @@ -290,7 +290,7 @@ struct obd_device *class_newdev(const char *type_name, const char *name) >>> LASSERT(newdev->obd_magic == OBD_DEVICE_MAGIC); >>> >>> write_lock(&obd_dev_lock); >>> - for (i = 0; i < class_devno_max(); i++) { >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) { >>> struct obd_device *obd = class_num2obd(i); >>> >>> if (obd && (strcmp(name, obd->obd_name) == 0)) { >>> @@ -322,9 +322,9 @@ struct obd_device *class_newdev(const char *type_name, const char *name) >>> } >>> write_unlock(&obd_dev_lock); >>> >>> - if (!result && i >= class_devno_max()) { >>> + if (!result && i >= MAX_OBD_DEVICES) { >>> CERROR("all %u OBD devices used, increase MAX_OBD_DEVICES\n", >>> - class_devno_max()); >>> + MAX_OBD_DEVICES); >>> result = ERR_PTR(-EOVERFLOW); >>> goto out; >>> } >>> @@ -372,7 +372,7 @@ int class_name2dev(const char *name) >>> return -1; >>> >>> read_lock(&obd_dev_lock); >>> - for (i = 0; i < class_devno_max(); i++) { >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) { >>> struct obd_device *obd = class_num2obd(i); >>> >>> if (obd && strcmp(name, obd->obd_name) == 0) { >>> @@ -397,7 +397,7 @@ struct obd_device *class_name2obd(const char *name) >>> { >>> int dev = class_name2dev(name); >>> >>> - if (dev < 0 || dev > class_devno_max()) >>> + if (dev < 0 || dev > MAX_OBD_DEVICES) >>> return NULL; >>> return class_num2obd(dev); >>> } >>> @@ -408,7 +408,7 @@ int class_uuid2dev(struct obd_uuid *uuid) >>> int i; >>> >>> read_lock(&obd_dev_lock); >>> - for (i = 0; i < class_devno_max(); i++) { >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) { >>> struct obd_device *obd = class_num2obd(i); >>> >>> if (obd && obd_uuid_equals(uuid, &obd->obd_uuid)) { >>> @@ -435,7 +435,7 @@ struct obd_device *class_num2obd(int num) >>> { >>> struct obd_device *obd = NULL; >>> >>> - if (num < class_devno_max()) { >>> + if (num < MAX_OBD_DEVICES) { >>> obd = obd_devs[num]; >>> if (!obd) >>> return NULL; >>> @@ -463,7 +463,7 @@ struct obd_device *class_find_client_obd(struct obd_uuid *tgt_uuid, >>> int i; >>> >>> read_lock(&obd_dev_lock); >>> - for (i = 0; i < class_devno_max(); i++) { >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) { >>> struct obd_device *obd = class_num2obd(i); >>> >>> if (!obd) >>> @@ -496,13 +496,13 @@ struct obd_device *class_devices_in_group(struct obd_uuid *grp_uuid, int *next) >>> >>> if (!next) >>> i = 0; >>> - else if (*next >= 0 && *next < class_devno_max()) >>> + else if (*next >= 0 && *next < MAX_OBD_DEVICES) >>> i = *next; >>> else >>> return NULL; >>> >>> read_lock(&obd_dev_lock); >>> - for (; i < class_devno_max(); i++) { >>> + for (; i < MAX_OBD_DEVICES; i++) { >>> struct obd_device *obd = class_num2obd(i); >>> >>> if (!obd) >>> @@ -533,7 +533,7 @@ int class_notify_sptlrpc_conf(const char *fsname, int namelen) >>> LASSERT(namelen > 0); >>> >>> read_lock(&obd_dev_lock); >>> - for (i = 0; i < class_devno_max(); i++) { >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) { >>> obd = class_num2obd(i); >>> >>> if (!obd || obd->obd_set_up == 0 || obd->obd_stopping) >>> diff --git a/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c b/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c >>> index 33342bf..ca5b466 100644 >>> --- a/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c >>> +++ b/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c >>> @@ -228,7 +228,7 @@ static ssize_t health_show(struct kobject *kobj, struct attribute *attr, >>> return sprintf(buf, "LBUG\n"); >>> >>> read_lock(&obd_dev_lock); >>> - for (i = 0; i < class_devno_max(); i++) { >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) { >>> struct obd_device *obd; >>> >>> obd = class_num2obd(i); >>> @@ -326,7 +326,7 @@ static struct attribute *lustre_attrs[] = { >>> >>> static void *obd_device_list_seq_start(struct seq_file *p, loff_t *pos) >>> { >>> - if (*pos >= class_devno_max()) >>> + if (*pos >= MAX_OBD_DEVICES) >>> return NULL; >>> >>> return pos; >>> @@ -339,7 +339,7 @@ static void obd_device_list_seq_stop(struct seq_file *p, void *v) >>> static void *obd_device_list_seq_next(struct seq_file *p, void *v, loff_t *pos) >>> { >>> ++*pos; >>> - if (*pos >= class_devno_max()) >>> + if (*pos >= MAX_OBD_DEVICES) >>> return NULL; >>> >>> return pos; >>> -- >>> 2.5.0 >>> >>> >>> -- >>> Kind Regards, >>> Aya Saif El-yazal Mahfouz >>> _______________________________________________ >>> lustre-devel mailing list >>> lustre-devel at lists.lustre.org >>> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org >> > > -- > Kind Regards, > Aya Saif El-yazal Mahfouz > _______________________________________________ > lustre-devel mailing list > lustre-devel at lists.lustre.org > http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org From green at linuxhacker.ru Thu Nov 3 01:24:52 2016 From: green at linuxhacker.ru (Oleg Drokin) Date: Wed, 2 Nov 2016 21:24:52 -0400 Subject: [lustre-devel] [PATCH 02/14] staging/lustre/ldlm: fix export reference problem In-Reply-To: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> References: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> Message-ID: <1478136304-867780-3-git-send-email-green@linuxhacker.ru> From: Hongchao Zhang 1, in client_import_del_conn, the export returned from class_conn2export is not released after using it. 2, in ptlrpc_connect_interpret, the export is not released if the connect_flags isn't compatible. Signed-off-by: Hongchao Zhang Reviewed-on: http://review.whamcloud.com/22031 Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-8500 Reviewed-by: James Simmons Reviewed-by: Bobi Jam Signed-off-by: Oleg Drokin --- drivers/staging/lustre/lustre/ldlm/ldlm_lib.c | 3 +++ drivers/staging/lustre/lustre/ptlrpc/import.c | 19 ++++++++++--------- 2 files changed, 13 insertions(+), 9 deletions(-) diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c index 4f9480e..06d3cc6 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c @@ -170,6 +170,9 @@ int client_import_del_conn(struct obd_import *imp, struct obd_uuid *uuid) ptlrpc_connection_put(dlmexp->exp_connection); dlmexp->exp_connection = NULL; } + + if (dlmexp) + class_export_put(dlmexp); } list_del(&imp_conn->oic_item); diff --git a/drivers/staging/lustre/lustre/ptlrpc/import.c b/drivers/staging/lustre/lustre/ptlrpc/import.c index 46ba5a4..05fd92d 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/import.c +++ b/drivers/staging/lustre/lustre/ptlrpc/import.c @@ -972,6 +972,16 @@ static int ptlrpc_connect_interpret(const struct lu_env *env, spin_unlock(&imp->imp_lock); + if (!exp) { + /* This could happen if export is cleaned during the + * connect attempt + */ + CERROR("%s: missing export after connect\n", + imp->imp_obd->obd_name); + rc = -ENODEV; + goto out; + } + /* check that server granted subset of flags we asked for. */ if ((ocd->ocd_connect_flags & imp->imp_connect_flags_orig) != ocd->ocd_connect_flags) { @@ -982,15 +992,6 @@ static int ptlrpc_connect_interpret(const struct lu_env *env, goto out; } - if (!exp) { - /* This could happen if export is cleaned during the - * connect attempt - */ - CERROR("%s: missing export after connect\n", - imp->imp_obd->obd_name); - rc = -ENODEV; - goto out; - } old_connect_flags = exp_connect_flags(exp); exp->exp_connect_data = *ocd; imp->imp_obd->obd_self_export->exp_connect_data = *ocd; -- 2.7.4 From green at linuxhacker.ru Thu Nov 3 01:24:53 2016 From: green at linuxhacker.ru (Oleg Drokin) Date: Wed, 2 Nov 2016 21:24:53 -0400 Subject: [lustre-devel] [PATCH 03/14] staging/lustre: conflicting PW & PR extent locks on a client In-Reply-To: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> References: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> Message-ID: <1478136304-867780-4-git-send-email-green@linuxhacker.ru> From: Andriy Skulysh PW lock isn't replayed once a lock is marked LDLM_FL_CANCELING and glimpse lock doesn't wait for conflicting locks on the client. So the server will grant a PR lock in response to the glimpse lock request, which conflicts with the PW lock in LDLM_FL_CANCELING state on the client. Lock in LDLM_FL_CANCELING state may still have pending IO, so it should be replayed until LDLM_FL_BL_DONE is set to avoid granted conflicting lock by a server. Seagate-bug-id: MRP-3311 Signed-off-by: Andriy Skulysh Reviewed-on: http://review.whamcloud.com/20345 Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-8175 Reviewed-by: Jinshan Xiong Signed-off-by: Oleg Drokin --- drivers/staging/lustre/lustre/include/obd_support.h | 3 +++ drivers/staging/lustre/lustre/ldlm/ldlm_extent.c | 20 ++++++++++++++++++++ drivers/staging/lustre/lustre/ldlm/ldlm_request.c | 4 ++-- drivers/staging/lustre/lustre/osc/osc_request.c | 1 + 4 files changed, 26 insertions(+), 2 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/obd_support.h b/drivers/staging/lustre/lustre/include/obd_support.h index 7f3f8cd..aaedec7 100644 --- a/drivers/staging/lustre/lustre/include/obd_support.h +++ b/drivers/staging/lustre/lustre/include/obd_support.h @@ -321,6 +321,8 @@ extern char obd_jobid_var[]; #define OBD_FAIL_LDLM_CP_CB_WAIT4 0x322 #define OBD_FAIL_LDLM_CP_CB_WAIT5 0x323 +#define OBD_FAIL_LDLM_GRANT_CHECK 0x32a + /* LOCKLESS IO */ #define OBD_FAIL_LDLM_SET_CONTENTION 0x385 @@ -343,6 +345,7 @@ extern char obd_jobid_var[]; #define OBD_FAIL_OSC_CP_ENQ_RACE 0x410 #define OBD_FAIL_OSC_NO_GRANT 0x411 #define OBD_FAIL_OSC_DELAY_SETTIME 0x412 +#define OBD_FAIL_OSC_DELAY_IO 0x414 #define OBD_FAIL_PTLRPC 0x500 #define OBD_FAIL_PTLRPC_ACK 0x501 diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c b/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c index ecf472e..a7b34e4 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c @@ -193,6 +193,26 @@ void ldlm_extent_add_lock(struct ldlm_resource *res, * add the locks into grant list, for debug purpose, .. */ ldlm_resource_add_lock(res, &res->lr_granted, lock); + + if (OBD_FAIL_CHECK(OBD_FAIL_LDLM_GRANT_CHECK)) { + struct ldlm_lock *lck; + + list_for_each_entry_reverse(lck, &res->lr_granted, + l_res_link) { + if (lck == lock) + continue; + if (lockmode_compat(lck->l_granted_mode, + lock->l_granted_mode)) + continue; + if (ldlm_extent_overlap(&lck->l_req_extent, + &lock->l_req_extent)) { + CDEBUG(D_ERROR, "granting conflicting lock %p %p\n", + lck, lock); + ldlm_resource_dump(D_ERROR, res); + LBUG(); + } + } + } } /** Remove cancelled lock from resource interval tree. */ diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c index 43856ff..6e704c7 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c @@ -1846,7 +1846,7 @@ static int ldlm_chain_lock_for_replay(struct ldlm_lock *lock, void *closure) * bug 17614: locks being actively cancelled. Get a reference * on a lock so that it does not disappear under us (e.g. due to cancel) */ - if (!(lock->l_flags & (LDLM_FL_FAILED | LDLM_FL_CANCELING))) { + if (!(lock->l_flags & (LDLM_FL_FAILED | LDLM_FL_BL_DONE))) { list_add(&lock->l_pending_chain, list); LDLM_LOCK_GET(lock); } @@ -1915,7 +1915,7 @@ static int replay_one_lock(struct obd_import *imp, struct ldlm_lock *lock) int flags; /* Bug 11974: Do not replay a lock which is actively being canceled */ - if (ldlm_is_canceling(lock)) { + if (ldlm_is_bl_done(lock)) { LDLM_DEBUG(lock, "Not replaying canceled lock:"); return 0; } diff --git a/drivers/staging/lustre/lustre/osc/osc_request.c b/drivers/staging/lustre/lustre/osc/osc_request.c index 091558e..8023561 100644 --- a/drivers/staging/lustre/lustre/osc/osc_request.c +++ b/drivers/staging/lustre/lustre/osc/osc_request.c @@ -1823,6 +1823,7 @@ int osc_build_rpc(const struct lu_env *env, struct client_obd *cli, DEBUG_REQ(D_INODE, req, "%d pages, aa %p. now %ur/%dw in flight", page_count, aa, cli->cl_r_in_flight, cli->cl_w_in_flight); + OBD_FAIL_TIMEOUT(OBD_FAIL_OSC_DELAY_IO, cfs_fail_val); ptlrpcd_add_req(req); rc = 0; -- 2.7.4 From green at linuxhacker.ru Thu Nov 3 01:24:55 2016 From: green at linuxhacker.ru (Oleg Drokin) Date: Wed, 2 Nov 2016 21:24:55 -0400 Subject: [lustre-devel] [PATCH 05/14] staging/lustre: Get rid of cl_env hash table In-Reply-To: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> References: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> Message-ID: <1478136304-867780-6-git-send-email-green@linuxhacker.ru> From: Jinshan Xiong cl_env hash table is under heavy contention when there are lots of processes doing IO at the same time; reduce lock contention by replacing cl_env cache with percpu array; remove cl_env_nested_get() and cl_env_nested_put(); remove cl_env_reenter() and cl_env_reexit(); Signed-off-by: Jinshan Xiong Reviewed-on: http://review.whamcloud.com/20254 Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-4257 Reviewed-by: Andreas Dilger Reviewed-by: Bobi Jam Signed-off-by: Oleg Drokin --- drivers/staging/lustre/lustre/include/cl_object.h | 22 -- drivers/staging/lustre/lustre/ldlm/ldlm_pool.c | 9 - drivers/staging/lustre/lustre/llite/file.c | 18 +- drivers/staging/lustre/lustre/llite/lcommon_cl.c | 11 +- drivers/staging/lustre/lustre/llite/lcommon_misc.c | 8 +- drivers/staging/lustre/lustre/llite/llite_mmap.c | 59 ++-- drivers/staging/lustre/lustre/llite/rw.c | 6 +- drivers/staging/lustre/lustre/llite/rw26.c | 16 +- drivers/staging/lustre/lustre/lov/lov_io.c | 5 - drivers/staging/lustre/lustre/lov/lov_object.c | 7 +- .../staging/lustre/lustre/obdclass/cl_internal.h | 23 -- drivers/staging/lustre/lustre/obdclass/cl_io.c | 1 - drivers/staging/lustre/lustre/obdclass/cl_object.c | 389 ++++----------------- drivers/staging/lustre/lustre/obdclass/lu_object.c | 4 - .../staging/lustre/lustre/obdecho/echo_client.c | 14 +- drivers/staging/lustre/lustre/osc/osc_cache.c | 6 +- drivers/staging/lustre/lustre/osc/osc_lock.c | 116 +++--- drivers/staging/lustre/lustre/osc/osc_page.c | 6 +- 18 files changed, 183 insertions(+), 537 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/cl_object.h b/drivers/staging/lustre/lustre/include/cl_object.h index 514d650..3fe26e7 100644 --- a/drivers/staging/lustre/lustre/include/cl_object.h +++ b/drivers/staging/lustre/lustre/include/cl_object.h @@ -2640,35 +2640,13 @@ void cl_sync_io_end(const struct lu_env *env, struct cl_sync_io *anchor); * - allocation and destruction of environment is amortized by caching no * longer used environments instead of destroying them; * - * - there is a notion of "current" environment, attached to the kernel - * data structure representing current thread Top-level lustre code - * allocates an environment and makes it current, then calls into - * non-lustre code, that in turn calls lustre back. Low-level lustre - * code thus called can fetch environment created by the top-level code - * and reuse it, avoiding additional environment allocation. - * Right now, three interfaces can attach the cl_env to running thread: - * - cl_env_get - * - cl_env_implant - * - cl_env_reexit(cl_env_reenter had to be called priorly) - * * \see lu_env, lu_context, lu_context_key * @{ */ -struct cl_env_nest { - int cen_refcheck; - void *cen_cookie; -}; - struct lu_env *cl_env_get(int *refcheck); struct lu_env *cl_env_alloc(int *refcheck, __u32 tags); -struct lu_env *cl_env_nested_get(struct cl_env_nest *nest); void cl_env_put(struct lu_env *env, int *refcheck); -void cl_env_nested_put(struct cl_env_nest *nest, struct lu_env *env); -void *cl_env_reenter(void); -void cl_env_reexit(void *cookie); -void cl_env_implant(struct lu_env *env, int *refcheck); -void cl_env_unplant(struct lu_env *env, int *refcheck); unsigned int cl_env_cache_purge(unsigned int nr); struct lu_env *cl_env_percpu_get(void); void cl_env_percpu_put(struct lu_env *env); diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c index b29c9561..19831c5 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c @@ -794,7 +794,6 @@ static unsigned long ldlm_pools_count(ldlm_side_t client, gfp_t gfp_mask) int nr_ns; struct ldlm_namespace *ns; struct ldlm_namespace *ns_old = NULL; /* loop detection */ - void *cookie; if (client == LDLM_NAMESPACE_CLIENT && !(gfp_mask & __GFP_FS)) return 0; @@ -802,8 +801,6 @@ static unsigned long ldlm_pools_count(ldlm_side_t client, gfp_t gfp_mask) CDEBUG(D_DLMTRACE, "Request to count %s locks from all pools\n", client == LDLM_NAMESPACE_CLIENT ? "client" : "server"); - cookie = cl_env_reenter(); - /* * Find out how many resources we may release. */ @@ -812,7 +809,6 @@ static unsigned long ldlm_pools_count(ldlm_side_t client, gfp_t gfp_mask) mutex_lock(ldlm_namespace_lock(client)); if (list_empty(ldlm_namespace_list(client))) { mutex_unlock(ldlm_namespace_lock(client)); - cl_env_reexit(cookie); return 0; } ns = ldlm_namespace_first_locked(client); @@ -838,7 +834,6 @@ static unsigned long ldlm_pools_count(ldlm_side_t client, gfp_t gfp_mask) ldlm_namespace_put(ns); } - cl_env_reexit(cookie); return total; } @@ -847,13 +842,10 @@ static unsigned long ldlm_pools_scan(ldlm_side_t client, int nr, gfp_t gfp_mask) unsigned long freed = 0; int tmp, nr_ns; struct ldlm_namespace *ns; - void *cookie; if (client == LDLM_NAMESPACE_CLIENT && !(gfp_mask & __GFP_FS)) return -1; - cookie = cl_env_reenter(); - /* * Shrink at least ldlm_namespace_nr_read(client) namespaces. */ @@ -883,7 +875,6 @@ static unsigned long ldlm_pools_scan(ldlm_side_t client, int nr, gfp_t gfp_mask) freed += ldlm_pool_shrink(&ns->ns_pool, cancel, gfp_mask); ldlm_namespace_put(ns); } - cl_env_reexit(cookie); /* * we only decrease the SLV in server pools shrinker, return * SHRINK_STOP to kernel to avoid needless loop. LU-1128 diff --git a/drivers/staging/lustre/lustre/llite/file.c b/drivers/staging/lustre/lustre/llite/file.c index 0ae78c7..c1c7551 100644 --- a/drivers/staging/lustre/lustre/llite/file.c +++ b/drivers/staging/lustre/lustre/llite/file.c @@ -1584,11 +1584,11 @@ int ll_data_version(struct inode *inode, __u64 *data_version, int flags) */ int ll_hsm_release(struct inode *inode) { - struct cl_env_nest nest; struct lu_env *env; struct obd_client_handle *och = NULL; __u64 data_version = 0; int rc; + int refcheck; CDEBUG(D_INODE, "%s: Releasing file "DFID".\n", ll_get_fsname(inode->i_sb, NULL, 0), @@ -1605,14 +1605,14 @@ int ll_hsm_release(struct inode *inode) if (rc != 0) goto out; - env = cl_env_nested_get(&nest); + env = cl_env_get(&refcheck); if (IS_ERR(env)) { rc = PTR_ERR(env); goto out; } ll_merge_attr(env, inode); - cl_env_nested_put(&nest, env); + cl_env_put(env, &refcheck); /* Release the file. * NB: lease lock handle is released in mdc_hsm_release_pack() because @@ -2268,17 +2268,17 @@ static int ll_flush(struct file *file, fl_owner_t id) int cl_sync_file_range(struct inode *inode, loff_t start, loff_t end, enum cl_fsync_mode mode, int ignore_layout) { - struct cl_env_nest nest; struct lu_env *env; struct cl_io *io; struct cl_fsync_io *fio; int result; + int refcheck; if (mode != CL_FSYNC_NONE && mode != CL_FSYNC_LOCAL && mode != CL_FSYNC_DISCARD && mode != CL_FSYNC_ALL) return -EINVAL; - env = cl_env_nested_get(&nest); + env = cl_env_get(&refcheck); if (IS_ERR(env)) return PTR_ERR(env); @@ -2301,7 +2301,7 @@ int cl_sync_file_range(struct inode *inode, loff_t start, loff_t end, if (result == 0) result = fio->fi_nr_written; cl_io_fini(env, io); - cl_env_nested_put(&nest, env); + cl_env_put(env, &refcheck); return result; } @@ -3149,14 +3149,14 @@ int ll_layout_conf(struct inode *inode, const struct cl_object_conf *conf) { struct ll_inode_info *lli = ll_i2info(inode); struct cl_object *obj = lli->lli_clob; - struct cl_env_nest nest; struct lu_env *env; int rc; + int refcheck; if (!obj) return 0; - env = cl_env_nested_get(&nest); + env = cl_env_get(&refcheck); if (IS_ERR(env)) return PTR_ERR(env); @@ -3190,7 +3190,7 @@ int ll_layout_conf(struct inode *inode, const struct cl_object_conf *conf) ll_layout_version_set(lli, cl.cl_layout_gen); } out: - cl_env_nested_put(&nest, env); + cl_env_put(env, &refcheck); return rc; } diff --git a/drivers/staging/lustre/lustre/llite/lcommon_cl.c b/drivers/staging/lustre/lustre/llite/lcommon_cl.c index 56e414b..dd1cfd8 100644 --- a/drivers/staging/lustre/lustre/llite/lcommon_cl.c +++ b/drivers/staging/lustre/lustre/llite/lcommon_cl.c @@ -246,15 +246,11 @@ void cl_inode_fini(struct inode *inode) int emergency; if (clob) { - void *cookie; - - cookie = cl_env_reenter(); env = cl_env_get(&refcheck); emergency = IS_ERR(env); if (emergency) { mutex_lock(&cl_inode_fini_guard); LASSERT(cl_inode_fini_env); - cl_env_implant(cl_inode_fini_env, &refcheck); env = cl_inode_fini_env; } /* @@ -266,13 +262,10 @@ void cl_inode_fini(struct inode *inode) lu_object_ref_del(&clob->co_lu, "inode", inode); cl_object_put_last(env, clob); lli->lli_clob = NULL; - if (emergency) { - cl_env_unplant(cl_inode_fini_env, &refcheck); + if (emergency) mutex_unlock(&cl_inode_fini_guard); - } else { + else cl_env_put(env, &refcheck); - } - cl_env_reexit(cookie); } } diff --git a/drivers/staging/lustre/lustre/llite/lcommon_misc.c b/drivers/staging/lustre/lustre/llite/lcommon_misc.c index 1558b55..f48660e 100644 --- a/drivers/staging/lustre/lustre/llite/lcommon_misc.c +++ b/drivers/staging/lustre/lustre/llite/lcommon_misc.c @@ -162,13 +162,11 @@ int cl_get_grouplock(struct cl_object *obj, unsigned long gid, int nonblock, return rc; } - cg->lg_env = cl_env_get(&refcheck); + cg->lg_env = env; cg->lg_io = io; cg->lg_lock = lock; cg->lg_gid = gid; - LASSERT(cg->lg_env == env); - cl_env_unplant(env, &refcheck); return 0; } @@ -177,14 +175,10 @@ void cl_put_grouplock(struct ll_grouplock *cg) struct lu_env *env = cg->lg_env; struct cl_io *io = cg->lg_io; struct cl_lock *lock = cg->lg_lock; - int refcheck; LASSERT(cg->lg_env); LASSERT(cg->lg_gid); - cl_env_implant(env, &refcheck); - cl_env_put(env, &refcheck); - cl_lock_release(env, lock); cl_io_fini(env, io); cl_env_put(env, NULL); diff --git a/drivers/staging/lustre/lustre/llite/llite_mmap.c b/drivers/staging/lustre/lustre/llite/llite_mmap.c index 4366918..ad2a699 100644 --- a/drivers/staging/lustre/lustre/llite/llite_mmap.c +++ b/drivers/staging/lustre/lustre/llite/llite_mmap.c @@ -80,43 +80,24 @@ struct vm_area_struct *our_vma(struct mm_struct *mm, unsigned long addr, * API independent part for page fault initialization. * \param vma - virtual memory area addressed to page fault * \param env - corespondent lu_env to processing - * \param nest - nested level * \param index - page index corespondent to fault. * \parm ra_flags - vma readahead flags. * - * \return allocated and initialized env for fault operation. - * \retval EINVAL if env can't allocated - * \return other error codes from cl_io_init. + * \return error codes from cl_io_init. */ static struct cl_io * -ll_fault_io_init(struct vm_area_struct *vma, struct lu_env **env_ret, - struct cl_env_nest *nest, pgoff_t index, - unsigned long *ra_flags) +ll_fault_io_init(struct lu_env *env, struct vm_area_struct *vma, + pgoff_t index, unsigned long *ra_flags) { struct file *file = vma->vm_file; struct inode *inode = file_inode(file); struct cl_io *io; struct cl_fault_io *fio; - struct lu_env *env; int rc; - *env_ret = NULL; if (ll_file_nolock(file)) return ERR_PTR(-EOPNOTSUPP); - /* - * page fault can be called when lustre IO is - * already active for the current thread, e.g., when doing read/write - * against user level buffer mapped from Lustre buffer. To avoid - * stomping on existing context, optionally force an allocation of a new - * one. - */ - env = cl_env_nested_get(nest); - if (IS_ERR(env)) - return ERR_PTR(-EINVAL); - - *env_ret = env; - restart: io = vvp_env_thread_io(env); io->ci_obj = ll_i2info(inode)->lli_clob; @@ -155,7 +136,6 @@ ll_fault_io_init(struct vm_area_struct *vma, struct lu_env **env_ret, if (io->ci_need_restart) goto restart; - cl_env_nested_put(nest, env); io = ERR_PTR(rc); } @@ -169,13 +149,17 @@ static int ll_page_mkwrite0(struct vm_area_struct *vma, struct page *vmpage, struct lu_env *env; struct cl_io *io; struct vvp_io *vio; - struct cl_env_nest nest; int result; + int refcheck; sigset_t set; struct inode *inode; struct ll_inode_info *lli; - io = ll_fault_io_init(vma, &env, &nest, vmpage->index, NULL); + env = cl_env_get(&refcheck); + if (IS_ERR(env)) + return PTR_ERR(env); + + io = ll_fault_io_init(env, vma, vmpage->index, NULL); if (IS_ERR(io)) { result = PTR_ERR(io); goto out; @@ -240,8 +224,8 @@ static int ll_page_mkwrite0(struct vm_area_struct *vma, struct page *vmpage, out_io: cl_io_fini(env, io); - cl_env_nested_put(&nest, env); out: + cl_env_put(env, &refcheck); CDEBUG(D_MMAP, "%s mkwrite with %d\n", current->comm, result); LASSERT(ergo(result == 0, PageLocked(vmpage))); @@ -285,13 +269,19 @@ static int ll_fault0(struct vm_area_struct *vma, struct vm_fault *vmf) struct vvp_io *vio = NULL; struct page *vmpage; unsigned long ra_flags; - struct cl_env_nest nest; - int result; + int result = 0; int fault_ret = 0; + int refcheck; - io = ll_fault_io_init(vma, &env, &nest, vmf->pgoff, &ra_flags); - if (IS_ERR(io)) - return to_fault_error(PTR_ERR(io)); + env = cl_env_get(&refcheck); + if (IS_ERR(env)) + return PTR_ERR(env); + + io = ll_fault_io_init(env, vma, vmf->pgoff, &ra_flags); + if (IS_ERR(io)) { + result = to_fault_error(PTR_ERR(io)); + goto out; + } result = io->ci_result; if (result == 0) { @@ -322,14 +312,15 @@ static int ll_fault0(struct vm_area_struct *vma, struct vm_fault *vmf) } } cl_io_fini(env, io); - cl_env_nested_put(&nest, env); vma->vm_flags |= ra_flags; + +out: + cl_env_put(env, &refcheck); if (result != 0 && !(fault_ret & VM_FAULT_RETRY)) fault_ret |= to_fault_error(result); - CDEBUG(D_MMAP, "%s fault %d/%d\n", - current->comm, fault_ret, result); + CDEBUG(D_MMAP, "%s fault %d/%d\n", current->comm, fault_ret, result); return fault_ret; } diff --git a/drivers/staging/lustre/lustre/llite/rw.c b/drivers/staging/lustre/lustre/llite/rw.c index 80cb8e0..d2515a8 100644 --- a/drivers/staging/lustre/lustre/llite/rw.c +++ b/drivers/staging/lustre/lustre/llite/rw.c @@ -896,17 +896,17 @@ int ll_writepage(struct page *vmpage, struct writeback_control *wbc) struct cl_io *io; struct cl_page *page; struct cl_object *clob; - struct cl_env_nest nest; bool redirtied = false; bool unlocked = false; int result; + int refcheck; LASSERT(PageLocked(vmpage)); LASSERT(!PageWriteback(vmpage)); LASSERT(ll_i2dtexp(inode)); - env = cl_env_nested_get(&nest); + env = cl_env_get(&refcheck); if (IS_ERR(env)) { result = PTR_ERR(env); goto out; @@ -971,7 +971,7 @@ int ll_writepage(struct page *vmpage, struct writeback_control *wbc) } } - cl_env_nested_put(&nest, env); + cl_env_put(env, &refcheck); goto out; out: diff --git a/drivers/staging/lustre/lustre/llite/rw26.c b/drivers/staging/lustre/lustre/llite/rw26.c index 67010be..1a08a9d 100644 --- a/drivers/staging/lustre/lustre/llite/rw26.c +++ b/drivers/staging/lustre/lustre/llite/rw26.c @@ -103,7 +103,6 @@ static void ll_invalidatepage(struct page *vmpage, unsigned int offset, static int ll_releasepage(struct page *vmpage, gfp_t gfp_mask) { struct lu_env *env; - void *cookie; struct cl_object *obj; struct cl_page *page; struct address_space *mapping; @@ -129,7 +128,6 @@ static int ll_releasepage(struct page *vmpage, gfp_t gfp_mask) if (!page) return 1; - cookie = cl_env_reenter(); env = cl_env_percpu_get(); LASSERT(!IS_ERR(env)); @@ -155,7 +153,6 @@ static int ll_releasepage(struct page *vmpage, gfp_t gfp_mask) cl_page_put(env, page); cl_env_percpu_put(env); - cl_env_reexit(cookie); return result; } @@ -340,7 +337,8 @@ static ssize_t ll_direct_IO_26_seg(const struct lu_env *env, struct cl_io *io, PAGE_SIZE) & ~(DT_MAX_BRW_SIZE - 1)) static ssize_t ll_direct_IO_26(struct kiocb *iocb, struct iov_iter *iter) { - struct lu_env *env; + struct ll_cl_context *lcc; + const struct lu_env *env; struct cl_io *io; struct file *file = iocb->ki_filp; struct inode *inode = file->f_mapping->host; @@ -348,7 +346,6 @@ static ssize_t ll_direct_IO_26(struct kiocb *iocb, struct iov_iter *iter) ssize_t count = iov_iter_count(iter); ssize_t tot_bytes = 0, result = 0; long size = MAX_DIO_SIZE; - int refcheck; /* FIXME: io smaller than PAGE_SIZE is broken on ia64 ??? */ if ((file_offset & ~PAGE_MASK) || (count & ~PAGE_MASK)) @@ -363,9 +360,13 @@ static ssize_t ll_direct_IO_26(struct kiocb *iocb, struct iov_iter *iter) if (iov_iter_alignment(iter) & ~PAGE_MASK) return -EINVAL; - env = cl_env_get(&refcheck); + lcc = ll_cl_find(file); + if (!lcc) + return -EIO; + + env = lcc->lcc_env; LASSERT(!IS_ERR(env)); - io = vvp_env_io(env)->vui_cl.cis_io; + io = lcc->lcc_io; LASSERT(io); while (iov_iter_count(iter)) { @@ -422,7 +423,6 @@ static ssize_t ll_direct_IO_26(struct kiocb *iocb, struct iov_iter *iter) vio->u.write.vui_written += tot_bytes; } - cl_env_put(env, &refcheck); return tot_bytes ? tot_bytes : result; } diff --git a/drivers/staging/lustre/lustre/lov/lov_io.c b/drivers/staging/lustre/lustre/lov/lov_io.c index 011ab0f..002326c 100644 --- a/drivers/staging/lustre/lustre/lov/lov_io.c +++ b/drivers/staging/lustre/lustre/lov/lov_io.c @@ -167,12 +167,7 @@ static int lov_io_sub_init(const struct lu_env *env, struct lov_io *lio, sub->sub_env = ld->ld_emrg[stripe]->emrg_env; sub->sub_borrowed = 1; } else { - void *cookie; - - /* obtain new environment */ - cookie = cl_env_reenter(); sub->sub_env = cl_env_get(&sub->sub_refcheck); - cl_env_reexit(cookie); if (IS_ERR(sub->sub_env)) result = PTR_ERR(sub->sub_env); diff --git a/drivers/staging/lustre/lustre/lov/lov_object.c b/drivers/staging/lustre/lustre/lov/lov_object.c index 711a3b9..317311c 100644 --- a/drivers/staging/lustre/lustre/lov/lov_object.c +++ b/drivers/staging/lustre/lustre/lov/lov_object.c @@ -729,19 +729,15 @@ static int lov_layout_change(const struct lu_env *unused, union lov_layout_state *state = &lov->u; const struct lov_layout_operations *old_ops; const struct lov_layout_operations *new_ops; - void *cookie; struct lu_env *env; int refcheck; int rc; LASSERT(0 <= lov->lo_type && lov->lo_type < ARRAY_SIZE(lov_dispatch)); - cookie = cl_env_reenter(); env = cl_env_get(&refcheck); - if (IS_ERR(env)) { - cl_env_reexit(cookie); + if (IS_ERR(env)) return PTR_ERR(env); - } LASSERT(0 <= llt && llt < ARRAY_SIZE(lov_dispatch)); @@ -782,7 +778,6 @@ static int lov_layout_change(const struct lu_env *unused, lov->lo_type = llt; out: cl_env_put(env, &refcheck); - cl_env_reexit(cookie); return rc; } diff --git a/drivers/staging/lustre/lustre/obdclass/cl_internal.h b/drivers/staging/lustre/lustre/obdclass/cl_internal.h index e866754..7b403fb 100644 --- a/drivers/staging/lustre/lustre/obdclass/cl_internal.h +++ b/drivers/staging/lustre/lustre/obdclass/cl_internal.h @@ -50,25 +50,6 @@ enum clt_nesting_level { }; /** - * Counters used to check correctness of cl_lock interface usage. - */ -struct cl_thread_counters { - /** - * Number of outstanding calls to cl_lock_mutex_get() made by the - * current thread. For debugging. - */ - int ctc_nr_locks_locked; - /** List of locked locks. */ - struct lu_ref ctc_locks_locked; - /** Number of outstanding holds on locks. */ - int ctc_nr_held; - /** Number of outstanding uses on locks. */ - int ctc_nr_used; - /** Number of held extent locks. */ - int ctc_nr_locks_acquired; -}; - -/** * Thread local state internal for generic cl-code. */ struct cl_thread_info { @@ -83,10 +64,6 @@ struct cl_thread_info { */ struct cl_lock_descr clt_descr; struct cl_page_list clt_list; - /** - * Counters for every level of lock nesting. - */ - struct cl_thread_counters clt_counters[CNL_NR]; /** @} debugging */ /* diff --git a/drivers/staging/lustre/lustre/obdclass/cl_io.c b/drivers/staging/lustre/lustre/obdclass/cl_io.c index c5621ad..af8e4d6 100644 --- a/drivers/staging/lustre/lustre/obdclass/cl_io.c +++ b/drivers/staging/lustre/lustre/obdclass/cl_io.c @@ -412,7 +412,6 @@ void cl_io_unlock(const struct lu_env *env, struct cl_io *io) scan->cis_iop->op[io->ci_type].cio_unlock(env, scan); } io->ci_state = CIS_UNLOCKED; - LASSERT(!cl_env_info(env)->clt_counters[CNL_TOP].ctc_nr_locks_acquired); } EXPORT_SYMBOL(cl_io_unlock); diff --git a/drivers/staging/lustre/lustre/obdclass/cl_object.c b/drivers/staging/lustre/lustre/obdclass/cl_object.c index b4600c4..f5d4e23 100644 --- a/drivers/staging/lustre/lustre/obdclass/cl_object.c +++ b/drivers/staging/lustre/lustre/obdclass/cl_object.c @@ -544,36 +544,20 @@ EXPORT_SYMBOL(cl_site_stats_print); * bz20044, bz22683. */ -static LIST_HEAD(cl_envs); -static unsigned int cl_envs_cached_nr; -static unsigned int cl_envs_cached_max = 128; /* XXX: prototype: arbitrary limit - * for now. - */ -static DEFINE_SPINLOCK(cl_envs_guard); +static unsigned int cl_envs_cached_max = 32; /* XXX: prototype: arbitrary limit + * for now. + */ +static struct cl_env_cache { + rwlock_t cec_guard; + unsigned int cec_count; + struct list_head cec_envs; +} *cl_envs = NULL; struct cl_env { void *ce_magic; struct lu_env ce_lu; struct lu_context ce_ses; - /** - * This allows cl_env to be entered into cl_env_hash which implements - * the current thread -> client environment lookup. - */ - struct hlist_node ce_node; - /** - * Owner for the current cl_env. - * - * If LL_TASK_CL_ENV is defined, this point to the owning current, - * only for debugging purpose ; - * Otherwise hash is used, and this is the key for cfs_hash. - * Now current thread pid is stored. Note using thread pointer would - * lead to unbalanced hash because of its specific allocation locality - * and could be varied for different platforms and OSes, even different - * OS versions. - */ - void *ce_owner; - /* * Linkage into global list of all client environments. Used for * garbage collection. @@ -597,122 +581,13 @@ static void cl_env_init0(struct cl_env *cle, void *debug) { LASSERT(cle->ce_ref == 0); LASSERT(cle->ce_magic == &cl_env_init0); - LASSERT(!cle->ce_debug && !cle->ce_owner); + LASSERT(!cle->ce_debug); cle->ce_ref = 1; cle->ce_debug = debug; CL_ENV_INC(busy); } -/* - * The implementation of using hash table to connect cl_env and thread - */ - -static struct cfs_hash *cl_env_hash; - -static unsigned cl_env_hops_hash(struct cfs_hash *lh, - const void *key, unsigned mask) -{ -#if BITS_PER_LONG == 64 - return cfs_hash_u64_hash((__u64)key, mask); -#else - return cfs_hash_u32_hash((__u32)key, mask); -#endif -} - -static void *cl_env_hops_obj(struct hlist_node *hn) -{ - struct cl_env *cle = hlist_entry(hn, struct cl_env, ce_node); - - LASSERT(cle->ce_magic == &cl_env_init0); - return (void *)cle; -} - -static int cl_env_hops_keycmp(const void *key, struct hlist_node *hn) -{ - struct cl_env *cle = cl_env_hops_obj(hn); - - LASSERT(cle->ce_owner); - return (key == cle->ce_owner); -} - -static void cl_env_hops_noop(struct cfs_hash *hs, struct hlist_node *hn) -{ - struct cl_env *cle = hlist_entry(hn, struct cl_env, ce_node); - - LASSERT(cle->ce_magic == &cl_env_init0); -} - -static struct cfs_hash_ops cl_env_hops = { - .hs_hash = cl_env_hops_hash, - .hs_key = cl_env_hops_obj, - .hs_keycmp = cl_env_hops_keycmp, - .hs_object = cl_env_hops_obj, - .hs_get = cl_env_hops_noop, - .hs_put_locked = cl_env_hops_noop, -}; - -static inline struct cl_env *cl_env_fetch(void) -{ - struct cl_env *cle; - - cle = cfs_hash_lookup(cl_env_hash, (void *)(long)current->pid); - LASSERT(ergo(cle, cle->ce_magic == &cl_env_init0)); - return cle; -} - -static inline void cl_env_attach(struct cl_env *cle) -{ - if (cle) { - int rc; - - LASSERT(!cle->ce_owner); - cle->ce_owner = (void *)(long)current->pid; - rc = cfs_hash_add_unique(cl_env_hash, cle->ce_owner, - &cle->ce_node); - LASSERT(rc == 0); - } -} - -static inline void cl_env_do_detach(struct cl_env *cle) -{ - void *cookie; - - LASSERT(cle->ce_owner == (void *)(long)current->pid); - cookie = cfs_hash_del(cl_env_hash, cle->ce_owner, - &cle->ce_node); - LASSERT(cookie == cle); - cle->ce_owner = NULL; -} - -static int cl_env_store_init(void) -{ - cl_env_hash = cfs_hash_create("cl_env", - HASH_CL_ENV_BITS, HASH_CL_ENV_BITS, - HASH_CL_ENV_BKT_BITS, 0, - CFS_HASH_MIN_THETA, - CFS_HASH_MAX_THETA, - &cl_env_hops, - CFS_HASH_RW_BKTLOCK); - return cl_env_hash ? 0 : -ENOMEM; -} - -static void cl_env_store_fini(void) -{ - cfs_hash_putref(cl_env_hash); -} - -static inline struct cl_env *cl_env_detach(struct cl_env *cle) -{ - if (!cle) - cle = cl_env_fetch(); - - if (cle && cle->ce_owner) - cl_env_do_detach(cle); - - return cle; -} - static struct lu_env *cl_env_new(__u32 ctx_tags, __u32 ses_tags, void *debug) { struct lu_env *env; @@ -762,16 +637,20 @@ static struct lu_env *cl_env_obtain(void *debug) { struct cl_env *cle; struct lu_env *env; + int cpu = get_cpu(); - spin_lock(&cl_envs_guard); - LASSERT(equi(cl_envs_cached_nr == 0, list_empty(&cl_envs))); - if (cl_envs_cached_nr > 0) { + read_lock(&cl_envs[cpu].cec_guard); + LASSERT(equi(cl_envs[cpu].cec_count == 0, + list_empty(&cl_envs[cpu].cec_envs))); + if (cl_envs[cpu].cec_count > 0) { int rc; - cle = container_of(cl_envs.next, struct cl_env, ce_linkage); + cle = container_of(cl_envs[cpu].cec_envs.next, struct cl_env, + ce_linkage); list_del_init(&cle->ce_linkage); - cl_envs_cached_nr--; - spin_unlock(&cl_envs_guard); + cl_envs[cpu].cec_count--; + read_unlock(&cl_envs[cpu].cec_guard); + put_cpu(); env = &cle->ce_lu; rc = lu_env_refill(env); @@ -784,7 +663,8 @@ static struct lu_env *cl_env_obtain(void *debug) env = ERR_PTR(rc); } } else { - spin_unlock(&cl_envs_guard); + read_unlock(&cl_envs[cpu].cec_guard); + put_cpu(); env = cl_env_new(lu_context_tags_default, lu_session_tags_default, debug); } @@ -796,27 +676,6 @@ static inline struct cl_env *cl_env_container(struct lu_env *env) return container_of(env, struct cl_env, ce_lu); } -static struct lu_env *cl_env_peek(int *refcheck) -{ - struct lu_env *env; - struct cl_env *cle; - - CL_ENV_INC(lookup); - - /* check that we don't go far from untrusted pointer */ - CLASSERT(offsetof(struct cl_env, ce_magic) == 0); - - env = NULL; - cle = cl_env_fetch(); - if (cle) { - CL_ENV_INC(hit); - env = &cle->ce_lu; - *refcheck = ++cle->ce_ref; - } - CDEBUG(D_OTHER, "%d@%p\n", cle ? cle->ce_ref : 0, cle); - return env; -} - /** * Returns lu_env: if there already is an environment associated with the * current thread, it is returned, otherwise, new environment is allocated. @@ -834,17 +693,13 @@ struct lu_env *cl_env_get(int *refcheck) { struct lu_env *env; - env = cl_env_peek(refcheck); - if (!env) { - env = cl_env_obtain(__builtin_return_address(0)); - if (!IS_ERR(env)) { - struct cl_env *cle; + env = cl_env_obtain(__builtin_return_address(0)); + if (!IS_ERR(env)) { + struct cl_env *cle; - cle = cl_env_container(env); - cl_env_attach(cle); - *refcheck = cle->ce_ref; - CDEBUG(D_OTHER, "%d@%p\n", cle->ce_ref, cle); - } + cle = cl_env_container(env); + *refcheck = cle->ce_ref; + CDEBUG(D_OTHER, "%d@%p\n", cle->ce_ref, cle); } return env; } @@ -859,7 +714,6 @@ struct lu_env *cl_env_alloc(int *refcheck, __u32 tags) { struct lu_env *env; - LASSERT(!cl_env_peek(refcheck)); env = cl_env_new(tags, tags, __builtin_return_address(0)); if (!IS_ERR(env)) { struct cl_env *cle; @@ -874,7 +728,6 @@ EXPORT_SYMBOL(cl_env_alloc); static void cl_env_exit(struct cl_env *cle) { - LASSERT(!cle->ce_owner); lu_context_exit(&cle->ce_lu.le_ctx); lu_context_exit(&cle->ce_ses); } @@ -887,20 +740,25 @@ static void cl_env_exit(struct cl_env *cle) unsigned int cl_env_cache_purge(unsigned int nr) { struct cl_env *cle; + unsigned int i; - spin_lock(&cl_envs_guard); - for (; !list_empty(&cl_envs) && nr > 0; --nr) { - cle = container_of(cl_envs.next, struct cl_env, ce_linkage); - list_del_init(&cle->ce_linkage); - LASSERT(cl_envs_cached_nr > 0); - cl_envs_cached_nr--; - spin_unlock(&cl_envs_guard); + for_each_possible_cpu(i) { + write_lock(&cl_envs[i].cec_guard); + for (; !list_empty(&cl_envs[i].cec_envs) && nr > 0; --nr) { + cle = container_of(cl_envs[i].cec_envs.next, + struct cl_env, ce_linkage); + list_del_init(&cle->ce_linkage); + LASSERT(cl_envs[i].cec_count > 0); + cl_envs[i].cec_count--; + write_unlock(&cl_envs[i].cec_guard); - cl_env_fini(cle); - spin_lock(&cl_envs_guard); + cl_env_fini(cle); + write_lock(&cl_envs[i].cec_guard); + } + LASSERT(equi(cl_envs[i].cec_count == 0, + list_empty(&cl_envs[i].cec_envs))); + write_unlock(&cl_envs[i].cec_guard); } - LASSERT(equi(cl_envs_cached_nr == 0, list_empty(&cl_envs))); - spin_unlock(&cl_envs_guard); return nr; } EXPORT_SYMBOL(cl_env_cache_purge); @@ -923,8 +781,9 @@ void cl_env_put(struct lu_env *env, int *refcheck) CDEBUG(D_OTHER, "%d@%p\n", cle->ce_ref, cle); if (--cle->ce_ref == 0) { + int cpu = get_cpu(); + CL_ENV_DEC(busy); - cl_env_detach(cle); cle->ce_debug = NULL; cl_env_exit(cle); /* @@ -933,107 +792,22 @@ void cl_env_put(struct lu_env *env, int *refcheck) * Return environment to the cache only when it was allocated * with the standard tags. */ - if (cl_envs_cached_nr < cl_envs_cached_max && + if (cl_envs[cpu].cec_count < cl_envs_cached_max && (env->le_ctx.lc_tags & ~LCT_HAS_EXIT) == LCT_CL_THREAD && (env->le_ses->lc_tags & ~LCT_HAS_EXIT) == LCT_SESSION) { - spin_lock(&cl_envs_guard); - list_add(&cle->ce_linkage, &cl_envs); - cl_envs_cached_nr++; - spin_unlock(&cl_envs_guard); + read_lock(&cl_envs[cpu].cec_guard); + list_add(&cle->ce_linkage, &cl_envs[cpu].cec_envs); + cl_envs[cpu].cec_count++; + read_unlock(&cl_envs[cpu].cec_guard); } else { cl_env_fini(cle); } + put_cpu(); } } EXPORT_SYMBOL(cl_env_put); /** - * Declares a point of re-entrancy. - * - * \see cl_env_reexit() - */ -void *cl_env_reenter(void) -{ - return cl_env_detach(NULL); -} -EXPORT_SYMBOL(cl_env_reenter); - -/** - * Exits re-entrancy. - */ -void cl_env_reexit(void *cookie) -{ - cl_env_detach(NULL); - cl_env_attach(cookie); -} -EXPORT_SYMBOL(cl_env_reexit); - -/** - * Setup user-supplied \a env as a current environment. This is to be used to - * guaranteed that environment exists even when cl_env_get() fails. It is up - * to user to ensure proper concurrency control. - * - * \see cl_env_unplant() - */ -void cl_env_implant(struct lu_env *env, int *refcheck) -{ - struct cl_env *cle = cl_env_container(env); - - LASSERT(cle->ce_ref > 0); - - cl_env_attach(cle); - cl_env_get(refcheck); - CDEBUG(D_OTHER, "%d@%p\n", cle->ce_ref, cle); -} -EXPORT_SYMBOL(cl_env_implant); - -/** - * Detach environment installed earlier by cl_env_implant(). - */ -void cl_env_unplant(struct lu_env *env, int *refcheck) -{ - struct cl_env *cle = cl_env_container(env); - - LASSERT(cle->ce_ref > 1); - - CDEBUG(D_OTHER, "%d@%p\n", cle->ce_ref, cle); - - cl_env_detach(cle); - cl_env_put(env, refcheck); -} -EXPORT_SYMBOL(cl_env_unplant); - -struct lu_env *cl_env_nested_get(struct cl_env_nest *nest) -{ - struct lu_env *env; - - nest->cen_cookie = NULL; - env = cl_env_peek(&nest->cen_refcheck); - if (env) { - if (!cl_io_is_going(env)) - return env; - cl_env_put(env, &nest->cen_refcheck); - nest->cen_cookie = cl_env_reenter(); - } - env = cl_env_get(&nest->cen_refcheck); - if (IS_ERR(env)) { - cl_env_reexit(nest->cen_cookie); - return env; - } - - LASSERT(!cl_io_is_going(env)); - return env; -} -EXPORT_SYMBOL(cl_env_nested_get); - -void cl_env_nested_put(struct cl_env_nest *nest, struct lu_env *env) -{ - cl_env_put(env, &nest->cen_refcheck); - cl_env_reexit(nest->cen_cookie); -} -EXPORT_SYMBOL(cl_env_nested_put); - -/** * Converts struct ost_lvb to struct cl_attr. * * \see cl_attr2lvb @@ -1060,6 +834,10 @@ static int cl_env_percpu_init(void) for_each_possible_cpu(i) { struct lu_env *env; + rwlock_init(&cl_envs[i].cec_guard); + INIT_LIST_HEAD(&cl_envs[i].cec_envs); + cl_envs[i].cec_count = 0; + cle = &cl_env_percpu[i]; env = &cle->ce_lu; @@ -1127,7 +905,6 @@ void cl_env_percpu_put(struct lu_env *env) LASSERT(cle->ce_ref == 0); CL_ENV_DEC(busy); - cl_env_detach(cle); cle->ce_debug = NULL; put_cpu(); @@ -1141,7 +918,6 @@ struct lu_env *cl_env_percpu_get(void) cle = &cl_env_percpu[get_cpu()]; cl_env_init0(cle, __builtin_return_address(0)); - cl_env_attach(cle); return &cle->ce_lu; } EXPORT_SYMBOL(cl_env_percpu_get); @@ -1205,51 +981,19 @@ LU_KEY_INIT_FINI(cl0, struct cl_thread_info); static void *cl_key_init(const struct lu_context *ctx, struct lu_context_key *key) { - struct cl_thread_info *info; - - info = cl0_key_init(ctx, key); - if (!IS_ERR(info)) { - size_t i; - - for (i = 0; i < ARRAY_SIZE(info->clt_counters); ++i) - lu_ref_init(&info->clt_counters[i].ctc_locks_locked); - } - return info; + return cl0_key_init(ctx, key); } static void cl_key_fini(const struct lu_context *ctx, struct lu_context_key *key, void *data) { - struct cl_thread_info *info; - size_t i; - - info = data; - for (i = 0; i < ARRAY_SIZE(info->clt_counters); ++i) - lu_ref_fini(&info->clt_counters[i].ctc_locks_locked); cl0_key_fini(ctx, key, data); } -static void cl_key_exit(const struct lu_context *ctx, - struct lu_context_key *key, void *data) -{ - struct cl_thread_info *info = data; - size_t i; - - for (i = 0; i < ARRAY_SIZE(info->clt_counters); ++i) { - LASSERT(info->clt_counters[i].ctc_nr_held == 0); - LASSERT(info->clt_counters[i].ctc_nr_used == 0); - LASSERT(info->clt_counters[i].ctc_nr_locks_acquired == 0); - LASSERT(info->clt_counters[i].ctc_nr_locks_locked == 0); - lu_ref_fini(&info->clt_counters[i].ctc_locks_locked); - lu_ref_init(&info->clt_counters[i].ctc_locks_locked); - } -} - static struct lu_context_key cl_key = { .lct_tags = LCT_CL_THREAD, .lct_init = cl_key_init, .lct_fini = cl_key_fini, - .lct_exit = cl_key_exit }; static struct lu_kmem_descr cl_object_caches[] = { @@ -1273,13 +1017,15 @@ int cl_global_init(void) { int result; - result = cl_env_store_init(); - if (result) - return result; + cl_envs = kzalloc(sizeof(*cl_envs) * num_possible_cpus(), GFP_KERNEL); + if (!cl_envs) { + result = -ENOMEM; + goto out; + } result = lu_kmem_init(cl_object_caches); if (result) - goto out_store; + goto out_envs; LU_CONTEXT_KEY_INIT(&cl_key); result = lu_context_key_register(&cl_key); @@ -1289,16 +1035,17 @@ int cl_global_init(void) result = cl_env_percpu_init(); if (result) /* no cl_env_percpu_fini on error */ - goto out_context; + goto out_keys; return 0; -out_context: +out_keys: lu_context_key_degister(&cl_key); out_kmem: lu_kmem_fini(cl_object_caches); -out_store: - cl_env_store_fini(); +out_envs: + kfree(cl_envs); +out: return result; } @@ -1310,5 +1057,5 @@ void cl_global_fini(void) cl_env_percpu_fini(); lu_context_key_degister(&cl_key); lu_kmem_fini(cl_object_caches); - cl_env_store_fini(); + kfree(cl_envs); } diff --git a/drivers/staging/lustre/lustre/obdclass/lu_object.c b/drivers/staging/lustre/lustre/obdclass/lu_object.c index e031fd2..14ca82d 100644 --- a/drivers/staging/lustre/lustre/obdclass/lu_object.c +++ b/drivers/staging/lustre/lustre/obdclass/lu_object.c @@ -1521,10 +1521,6 @@ void lu_context_key_quiesce(struct lu_context_key *key) if (!(key->lct_tags & LCT_QUIESCENT)) { /* - * XXX layering violation. - */ - cl_env_cache_purge(~0); - /* * XXX memory barrier has to go here. */ spin_lock(&lu_keys_guard); diff --git a/drivers/staging/lustre/lustre/obdecho/echo_client.c b/drivers/staging/lustre/lustre/obdecho/echo_client.c index 30caf5f..25ecfb3 100644 --- a/drivers/staging/lustre/lustre/obdecho/echo_client.c +++ b/drivers/staging/lustre/lustre/obdecho/echo_client.c @@ -563,16 +563,10 @@ static void echo_thread_key_fini(const struct lu_context *ctx, kmem_cache_free(echo_thread_kmem, info); } -static void echo_thread_key_exit(const struct lu_context *ctx, - struct lu_context_key *key, void *data) -{ -} - static struct lu_context_key echo_thread_key = { .lct_tags = LCT_CL_THREAD, .lct_init = echo_thread_key_init, .lct_fini = echo_thread_key_fini, - .lct_exit = echo_thread_key_exit }; static void *echo_session_key_init(const struct lu_context *ctx, @@ -594,16 +588,10 @@ static void echo_session_key_fini(const struct lu_context *ctx, kmem_cache_free(echo_session_kmem, session); } -static void echo_session_key_exit(const struct lu_context *ctx, - struct lu_context_key *key, void *data) -{ -} - static struct lu_context_key echo_session_key = { .lct_tags = LCT_SESSION, .lct_init = echo_session_key_init, .lct_fini = echo_session_key_fini, - .lct_exit = echo_session_key_exit }; LU_TYPE_INIT_FINI(echo, &echo_thread_key, &echo_session_key); @@ -787,6 +775,8 @@ static struct lu_device *echo_device_free(const struct lu_env *env, cl_device_fini(&ed->ed_cl); kfree(ed); + cl_env_cache_purge(~0); + return NULL; } diff --git a/drivers/staging/lustre/lustre/osc/osc_cache.c b/drivers/staging/lustre/lustre/osc/osc_cache.c index b645957..cbbe85e 100644 --- a/drivers/staging/lustre/lustre/osc/osc_cache.c +++ b/drivers/staging/lustre/lustre/osc/osc_cache.c @@ -977,7 +977,6 @@ static int osc_extent_wait(const struct lu_env *env, struct osc_extent *ext, static int osc_extent_truncate(struct osc_extent *ext, pgoff_t trunc_index, bool partial) { - struct cl_env_nest nest; struct lu_env *env; struct cl_io *io; struct osc_object *obj = ext->oe_obj; @@ -990,6 +989,7 @@ static int osc_extent_truncate(struct osc_extent *ext, pgoff_t trunc_index, int grants = 0; int nr_pages = 0; int rc = 0; + int refcheck; LASSERT(sanity_check(ext) == 0); EASSERT(ext->oe_state == OES_TRUNC, ext); @@ -999,7 +999,7 @@ static int osc_extent_truncate(struct osc_extent *ext, pgoff_t trunc_index, * We can't use that env from osc_cache_truncate_start() because * it's from lov_io_sub and not fully initialized. */ - env = cl_env_nested_get(&nest); + env = cl_env_get(&refcheck); io = &osc_env_info(env)->oti_io; io->ci_obj = cl_object_top(osc2cl(obj)); rc = cl_io_init(env, io, CIT_MISC, io->ci_obj); @@ -1085,7 +1085,7 @@ static int osc_extent_truncate(struct osc_extent *ext, pgoff_t trunc_index, out: cl_io_fini(env, io); - cl_env_nested_put(&nest, env); + cl_env_put(env, &refcheck); return rc; } diff --git a/drivers/staging/lustre/lustre/osc/osc_lock.c b/drivers/staging/lustre/lustre/osc/osc_lock.c index 45d5e6d..e337e87 100644 --- a/drivers/staging/lustre/lustre/osc/osc_lock.c +++ b/drivers/staging/lustre/lustre/osc/osc_lock.c @@ -294,10 +294,10 @@ static int osc_lock_upcall(void *cookie, struct lustre_handle *lockh, struct osc_lock *oscl = cookie; struct cl_lock_slice *slice = &oscl->ols_cl; struct lu_env *env; - struct cl_env_nest nest; int rc; + int refcheck; - env = cl_env_nested_get(&nest); + env = cl_env_get(&refcheck); /* should never happen, similar to osc_ldlm_blocking_ast(). */ LASSERT(!IS_ERR(env)); @@ -336,7 +336,7 @@ static int osc_lock_upcall(void *cookie, struct lustre_handle *lockh, if (oscl->ols_owner) cl_sync_io_note(env, oscl->ols_owner, rc); - cl_env_nested_put(&nest, env); + cl_env_put(env, &refcheck); return rc; } @@ -347,9 +347,9 @@ static int osc_lock_upcall_agl(void *cookie, struct lustre_handle *lockh, struct osc_object *osc = cookie; struct ldlm_lock *dlmlock; struct lu_env *env; - struct cl_env_nest nest; + int refcheck; - env = cl_env_nested_get(&nest); + env = cl_env_get(&refcheck); LASSERT(!IS_ERR(env)); if (errcode == ELDLM_LOCK_MATCHED) { @@ -374,7 +374,7 @@ static int osc_lock_upcall_agl(void *cookie, struct lustre_handle *lockh, out: cl_object_put(env, osc2cl(osc)); - cl_env_nested_put(&nest, env); + cl_env_put(env, &refcheck); return ldlm_error2errno(errcode); } @@ -382,11 +382,11 @@ static int osc_lock_flush(struct osc_object *obj, pgoff_t start, pgoff_t end, enum cl_lock_mode mode, int discard) { struct lu_env *env; - struct cl_env_nest nest; + int refcheck; int rc = 0; int rc2 = 0; - env = cl_env_nested_get(&nest); + env = cl_env_get(&refcheck); if (IS_ERR(env)) return PTR_ERR(env); @@ -404,7 +404,7 @@ static int osc_lock_flush(struct osc_object *obj, pgoff_t start, pgoff_t end, if (rc == 0 && rc2 < 0) rc = rc2; - cl_env_nested_put(&nest, env); + cl_env_put(env, &refcheck); return rc; } @@ -536,7 +536,7 @@ static int osc_ldlm_blocking_ast(struct ldlm_lock *dlmlock, } case LDLM_CB_CANCELING: { struct lu_env *env; - struct cl_env_nest nest; + int refcheck; /* * This can be called in the context of outer IO, e.g., @@ -549,14 +549,14 @@ static int osc_ldlm_blocking_ast(struct ldlm_lock *dlmlock, * new environment has to be created to not corrupt outer * context. */ - env = cl_env_nested_get(&nest); + env = cl_env_get(&refcheck); if (IS_ERR(env)) { result = PTR_ERR(env); break; } result = osc_dlm_blocking_ast0(env, dlmlock, data, flag); - cl_env_nested_put(&nest, env); + cl_env_put(env, &refcheck); break; } default: @@ -568,61 +568,61 @@ static int osc_ldlm_blocking_ast(struct ldlm_lock *dlmlock, static int osc_ldlm_glimpse_ast(struct ldlm_lock *dlmlock, void *data) { struct ptlrpc_request *req = data; - struct cl_env_nest nest; struct lu_env *env; struct ost_lvb *lvb; struct req_capsule *cap; + struct cl_object *obj = NULL; int result; + int refcheck; LASSERT(lustre_msg_get_opc(req->rq_reqmsg) == LDLM_GL_CALLBACK); - env = cl_env_nested_get(&nest); - if (!IS_ERR(env)) { - struct cl_object *obj = NULL; + env = cl_env_get(&refcheck); + if (IS_ERR(env)) { + result = PTR_ERR(env); + goto out; + } - lock_res_and_lock(dlmlock); - if (dlmlock->l_ast_data) { - obj = osc2cl(dlmlock->l_ast_data); - cl_object_get(obj); - } - unlock_res_and_lock(dlmlock); + lock_res_and_lock(dlmlock); + if (dlmlock->l_ast_data) { + obj = osc2cl(dlmlock->l_ast_data); + cl_object_get(obj); + } + unlock_res_and_lock(dlmlock); - if (obj) { - /* Do not grab the mutex of cl_lock for glimpse. - * See LU-1274 for details. - * BTW, it's okay for cl_lock to be cancelled during - * this period because server can handle this race. - * See ldlm_server_glimpse_ast() for details. - * cl_lock_mutex_get(env, lock); - */ - cap = &req->rq_pill; - req_capsule_extend(cap, &RQF_LDLM_GL_CALLBACK); - req_capsule_set_size(cap, &RMF_DLM_LVB, RCL_SERVER, - sizeof(*lvb)); - result = req_capsule_server_pack(cap); - if (result == 0) { - lvb = req_capsule_server_get(cap, &RMF_DLM_LVB); - result = cl_object_glimpse(env, obj, lvb); - } - if (!exp_connect_lvb_type(req->rq_export)) - req_capsule_shrink(&req->rq_pill, - &RMF_DLM_LVB, - sizeof(struct ost_lvb_v1), - RCL_SERVER); - cl_object_put(env, obj); - } else { - /* - * These errors are normal races, so we don't want to - * fill the console with messages by calling - * ptlrpc_error() - */ - lustre_pack_reply(req, 1, NULL, NULL); - result = -ELDLM_NO_LOCK_DATA; + if (obj) { + /* Do not grab the mutex of cl_lock for glimpse. + * See LU-1274 for details. + * BTW, it's okay for cl_lock to be cancelled during + * this period because server can handle this race. + * See ldlm_server_glimpse_ast() for details. + * cl_lock_mutex_get(env, lock); + */ + cap = &req->rq_pill; + req_capsule_extend(cap, &RQF_LDLM_GL_CALLBACK); + req_capsule_set_size(cap, &RMF_DLM_LVB, RCL_SERVER, + sizeof(*lvb)); + result = req_capsule_server_pack(cap); + if (result == 0) { + lvb = req_capsule_server_get(cap, &RMF_DLM_LVB); + result = cl_object_glimpse(env, obj, lvb); } - cl_env_nested_put(&nest, env); + if (!exp_connect_lvb_type(req->rq_export)) + req_capsule_shrink(&req->rq_pill, &RMF_DLM_LVB, + sizeof(struct ost_lvb_v1), RCL_SERVER); + cl_object_put(env, obj); } else { - result = PTR_ERR(env); + /* + * These errors are normal races, so we don't want to + * fill the console with messages by calling + * ptlrpc_error() + */ + lustre_pack_reply(req, 1, NULL, NULL); + result = -ELDLM_NO_LOCK_DATA; } + cl_env_put(env, &refcheck); + +out: req->rq_status = result; return result; } @@ -677,12 +677,12 @@ static unsigned long osc_lock_weight(const struct lu_env *env, */ unsigned long osc_ldlm_weigh_ast(struct ldlm_lock *dlmlock) { - struct cl_env_nest nest; struct lu_env *env; struct osc_object *obj; struct osc_lock *oscl; unsigned long weight; bool found = false; + int refcheck; might_sleep(); /* @@ -692,7 +692,7 @@ unsigned long osc_ldlm_weigh_ast(struct ldlm_lock *dlmlock) * the upper context because cl_lock_put don't modify environment * variables. But just in case .. */ - env = cl_env_nested_get(&nest); + env = cl_env_get(&refcheck); if (IS_ERR(env)) /* Mostly because lack of memory, do not eliminate this lock */ return 1; @@ -722,7 +722,7 @@ unsigned long osc_ldlm_weigh_ast(struct ldlm_lock *dlmlock) weight = osc_lock_weight(env, obj, &dlmlock->l_policy_data.l_extent); out: - cl_env_nested_put(&nest, env); + cl_env_put(env, &refcheck); return weight; } diff --git a/drivers/staging/lustre/lustre/osc/osc_page.c b/drivers/staging/lustre/lustre/osc/osc_page.c index 2fb0e53..93248d1 100644 --- a/drivers/staging/lustre/lustre/osc/osc_page.c +++ b/drivers/staging/lustre/lustre/osc/osc_page.c @@ -644,15 +644,15 @@ long osc_lru_shrink(const struct lu_env *env, struct client_obd *cli, long osc_lru_reclaim(struct client_obd *cli) { - struct cl_env_nest nest; struct lu_env *env; struct cl_client_cache *cache = cli->cl_cache; int max_scans; + int refcheck; long rc = 0; LASSERT(cache); - env = cl_env_nested_get(&nest); + env = cl_env_get(&refcheck); if (IS_ERR(env)) return 0; @@ -704,7 +704,7 @@ long osc_lru_reclaim(struct client_obd *cli) spin_unlock(&cache->ccc_lru_lock); out: - cl_env_nested_put(&nest, env); + cl_env_put(env, &refcheck); CDEBUG(D_CACHE, "%s: cli %p freed %ld pages.\n", cli->cl_import->imp_obd->obd_name, cli, rc); return rc; -- 2.7.4 From green at linuxhacker.ru Thu Nov 3 01:24:54 2016 From: green at linuxhacker.ru (Oleg Drokin) Date: Wed, 2 Nov 2016 21:24:54 -0400 Subject: [lustre-devel] [PATCH 04/14] staging/lustre/llite: clear inode timestamps after losing UPDATE lock In-Reply-To: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> References: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> Message-ID: <1478136304-867780-5-git-send-email-green@linuxhacker.ru> From: Niu Yawei Otherwise, those leftovers would interfere with new timestamps especially when the timestamps are set back in time on the other clients. Signed-off-by: Jinshan Xiong Signed-off-by: Niu Yawei Reviewed-on: http://review.whamcloud.com/22623 Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-8446 Reviewed-by: Bobi Jam Signed-off-by: Oleg Drokin --- drivers/staging/lustre/lustre/llite/namei.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/drivers/staging/lustre/lustre/llite/namei.c b/drivers/staging/lustre/lustre/llite/namei.c index 74d9b73..c268f32 100644 --- a/drivers/staging/lustre/lustre/llite/namei.c +++ b/drivers/staging/lustre/lustre/llite/namei.c @@ -251,6 +251,16 @@ int ll_md_blocking_ast(struct ldlm_lock *lock, struct ldlm_lock_desc *desc, PFID(ll_inode2fid(inode)), rc); } + if (bits & MDS_INODELOCK_UPDATE) { + struct ll_inode_info *lli = ll_i2info(inode); + + spin_lock(&lli->lli_lock); + LTIME_S(inode->i_mtime) = 0; + LTIME_S(inode->i_atime) = 0; + LTIME_S(inode->i_ctime) = 0; + spin_unlock(&lli->lli_lock); + } + if ((bits & MDS_INODELOCK_UPDATE) && S_ISDIR(inode->i_mode)) { struct ll_inode_info *lli = ll_i2info(inode); -- 2.7.4 From green at linuxhacker.ru Thu Nov 3 01:24:50 2016 From: green at linuxhacker.ru (Oleg Drokin) Date: Wed, 2 Nov 2016 21:24:50 -0400 Subject: [lustre-devel] [PATCH 00/14] Lustre fixes Message-ID: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> This batch of patches represents mostly recent fixes, also a couple of cleanups and a couple of changes that fixes depend on. Amir Shehata (1): staging/lustre/ptlrpc: Correctly calculate hrp->hrp_nthrs Andrew Perepechko (1): staging/lustre/llite: drop_caches hangs in cl_inode_fini() Andriy Skulysh (1): staging/lustre: conflicting PW & PR extent locks on a client Bobi Jam (2): staging/lustre/llite: protect from accessing NULL lli_clob staging/lustre/llite: update ras window correctly Hongchao Zhang (2): staging/lustre/ldlm: fix export reference problem staging/lustre/ldlm: engage ELC for all ldlm enqueue req Jinshan Xiong (2): staging/lustre: Get rid of cl_env hash table staging/lustre/llite: do not clear uptodate bit in page delete Niu Yawei (1): staging/lustre/llite: clear inode timestamps after losing UPDATE lock Oleg Drokin (3): staging/lustre/ldlm: Drop unused blocking_refs flock field staging/lustre/ldlm: Reinstate ldlm_enqueue_pack() staging/lustre: Get rid of LIBLUSTRE_CLIENT and its users Patrick Farrell (1): staging/lustre/ptlrpc: Suppress error for flock requests drivers/staging/lustre/lustre/include/cl_object.h | 22 -- drivers/staging/lustre/lustre/include/lustre_dlm.h | 2 - drivers/staging/lustre/lustre/include/lustre_lib.h | 2 - .../staging/lustre/lustre/include/obd_support.h | 3 + drivers/staging/lustre/lustre/ldlm/ldlm_extent.c | 20 ++ drivers/staging/lustre/lustre/ldlm/ldlm_lib.c | 3 + drivers/staging/lustre/lustre/ldlm/ldlm_pool.c | 9 - drivers/staging/lustre/lustre/ldlm/ldlm_request.c | 61 ++-- drivers/staging/lustre/lustre/llite/file.c | 26 +- drivers/staging/lustre/lustre/llite/lcommon_cl.c | 11 +- drivers/staging/lustre/lustre/llite/lcommon_misc.c | 8 +- drivers/staging/lustre/lustre/llite/llite_mmap.c | 59 ++-- drivers/staging/lustre/lustre/llite/namei.c | 10 + drivers/staging/lustre/lustre/llite/rw.c | 21 +- drivers/staging/lustre/lustre/llite/rw26.c | 52 +-- drivers/staging/lustre/lustre/llite/vvp_page.c | 1 - drivers/staging/lustre/lustre/lov/lov_io.c | 5 - drivers/staging/lustre/lustre/lov/lov_object.c | 7 +- .../staging/lustre/lustre/obdclass/cl_internal.h | 23 -- drivers/staging/lustre/lustre/obdclass/cl_io.c | 1 - drivers/staging/lustre/lustre/obdclass/cl_object.c | 389 ++++----------------- drivers/staging/lustre/lustre/obdclass/lu_object.c | 4 - .../staging/lustre/lustre/obdecho/echo_client.c | 14 +- drivers/staging/lustre/lustre/osc/osc_cache.c | 6 +- drivers/staging/lustre/lustre/osc/osc_lock.c | 116 +++--- drivers/staging/lustre/lustre/osc/osc_page.c | 6 +- drivers/staging/lustre/lustre/osc/osc_request.c | 1 + drivers/staging/lustre/lustre/ptlrpc/client.c | 4 +- drivers/staging/lustre/lustre/ptlrpc/import.c | 19 +- drivers/staging/lustre/lustre/ptlrpc/service.c | 3 +- 30 files changed, 298 insertions(+), 610 deletions(-) -- 2.7.4 From green at linuxhacker.ru Thu Nov 3 01:24:57 2016 From: green at linuxhacker.ru (Oleg Drokin) Date: Wed, 2 Nov 2016 21:24:57 -0400 Subject: [lustre-devel] [PATCH 07/14] staging/lustre/ldlm: Reinstate ldlm_enqueue_pack() In-Reply-To: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> References: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> Message-ID: <1478136304-867780-8-git-send-email-green@linuxhacker.ru> The function becomes used again with the next patch, so bring it back from dead, only this time make it static. Reverts: bf2a033360f7 ("staging/lustre/ldlm: Remove unused ldlm_enqueue_pack()") Signed-off-by: Oleg Drokin --- drivers/staging/lustre/lustre/ldlm/ldlm_request.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c index 6e704c7..1b9ae77 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c @@ -657,6 +657,27 @@ int ldlm_prep_enqueue_req(struct obd_export *exp, struct ptlrpc_request *req, } EXPORT_SYMBOL(ldlm_prep_enqueue_req); +static struct ptlrpc_request *ldlm_enqueue_pack(struct obd_export *exp, + int lvb_len) +{ + struct ptlrpc_request *req; + int rc; + + req = ptlrpc_request_alloc(class_exp2cliimp(exp), &RQF_LDLM_ENQUEUE); + if (!req) + return ERR_PTR(-ENOMEM); + + rc = ldlm_prep_enqueue_req(exp, req, NULL, 0); + if (rc) { + ptlrpc_request_free(req); + return ERR_PTR(rc); + } + + req_capsule_set_size(&req->rq_pill, &RMF_DLM_LVB, RCL_SERVER, lvb_len); + ptlrpc_request_set_replen(req); + return req; +} + /** * Client-side lock enqueue. * -- 2.7.4 From green at linuxhacker.ru Thu Nov 3 01:25:00 2016 From: green at linuxhacker.ru (Oleg Drokin) Date: Wed, 2 Nov 2016 21:25:00 -0400 Subject: [lustre-devel] [PATCH 10/14] staging/lustre/llite: protect from accessing NULL lli_clob In-Reply-To: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> References: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> Message-ID: <1478136304-867780-11-git-send-email-green@linuxhacker.ru> From: Bobi Jam Need to check file's lli_clob object before calling lov_read_and_clear_async_rc(). Signed-off-by: Bobi Jam Reviewed-by: Jinshan Xiong Reviewed-by: Oleg Drokin Reviewed-on: http://review.whamcloud.com/23031 Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-8682 Signed-off-by: Oleg Drokin --- drivers/staging/lustre/lustre/llite/file.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/drivers/staging/lustre/lustre/llite/file.c b/drivers/staging/lustre/lustre/llite/file.c index c1c7551..7886840 100644 --- a/drivers/staging/lustre/lustre/llite/file.c +++ b/drivers/staging/lustre/lustre/llite/file.c @@ -2328,9 +2328,11 @@ int ll_fsync(struct file *file, loff_t start, loff_t end, int datasync) lli->lli_async_rc = 0; if (rc == 0) rc = err; - err = lov_read_and_clear_async_rc(lli->lli_clob); - if (rc == 0) - rc = err; + if (lli->lli_clob) { + err = lov_read_and_clear_async_rc(lli->lli_clob); + if (rc == 0) + rc = err; + } } err = md_sync(ll_i2sbi(inode)->ll_md_exp, ll_inode2fid(inode), &req); -- 2.7.4 From green at linuxhacker.ru Thu Nov 3 01:24:56 2016 From: green at linuxhacker.ru (Oleg Drokin) Date: Wed, 2 Nov 2016 21:24:56 -0400 Subject: [lustre-devel] [PATCH 06/14] staging/lustre/llite: drop_caches hangs in cl_inode_fini() In-Reply-To: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> References: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> Message-ID: <1478136304-867780-7-git-send-email-green@linuxhacker.ru> From: Andrew Perepechko This patch releases cl_pages on error in ll_write_begin() to avoid memory and object reference leaks. Also, it reuses per-cpu lu_env in ll_invalidatepage() in the same way as done in ll_releasepage(). Signed-off-by: Andrew Perepechko Seagate-bug-id: MRP-3504 Reviewed-on: http://review.whamcloud.com/22745 Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-8509 Reviewed-by: Jinshan Xiong Reviewed-by: Bobi Jam Signed-off-by: Oleg Drokin --- drivers/staging/lustre/lustre/llite/rw26.c | 36 ++++++++++++++++-------------- 1 file changed, 19 insertions(+), 17 deletions(-) diff --git a/drivers/staging/lustre/lustre/llite/rw26.c b/drivers/staging/lustre/lustre/llite/rw26.c index 1a08a9d..ca45b44 100644 --- a/drivers/staging/lustre/lustre/llite/rw26.c +++ b/drivers/staging/lustre/lustre/llite/rw26.c @@ -71,8 +71,6 @@ static void ll_invalidatepage(struct page *vmpage, unsigned int offset, struct cl_page *page; struct cl_object *obj; - int refcheck; - LASSERT(PageLocked(vmpage)); LASSERT(!PageWriteback(vmpage)); @@ -82,21 +80,21 @@ static void ll_invalidatepage(struct page *vmpage, unsigned int offset, * happening with locked page too */ if (offset == 0 && length == PAGE_SIZE) { - env = cl_env_get(&refcheck); - if (!IS_ERR(env)) { - inode = vmpage->mapping->host; - obj = ll_i2info(inode)->lli_clob; - if (obj) { - page = cl_vmpage_page(vmpage, obj); - if (page) { - cl_page_delete(env, page); - cl_page_put(env, page); - } - } else { - LASSERT(vmpage->private == 0); + /* See the comment in ll_releasepage() */ + env = cl_env_percpu_get(); + LASSERT(!IS_ERR(env)); + inode = vmpage->mapping->host; + obj = ll_i2info(inode)->lli_clob; + if (obj) { + page = cl_vmpage_page(vmpage, obj); + if (page) { + cl_page_delete(env, page); + cl_page_put(env, page); } - cl_env_put(env, &refcheck); + } else { + LASSERT(vmpage->private == 0); } + cl_env_percpu_put(env); } } @@ -466,9 +464,9 @@ static int ll_write_begin(struct file *file, struct address_space *mapping, struct page **pagep, void **fsdata) { struct ll_cl_context *lcc; - const struct lu_env *env; + const struct lu_env *env = NULL; struct cl_io *io; - struct cl_page *page; + struct cl_page *page = NULL; struct cl_object *clob = ll_i2info(mapping->host)->lli_clob; pgoff_t index = pos >> PAGE_SHIFT; struct page *vmpage = NULL; @@ -556,6 +554,10 @@ static int ll_write_begin(struct file *file, struct address_space *mapping, unlock_page(vmpage); put_page(vmpage); } + if (!IS_ERR_OR_NULL(page)) { + lu_ref_del(&page->cp_reference, "cl_io", io); + cl_page_put(env, page); + } } else { *pagep = vmpage; *fsdata = lcc; -- 2.7.4 From green at linuxhacker.ru Thu Nov 3 01:24:51 2016 From: green at linuxhacker.ru (Oleg Drokin) Date: Wed, 2 Nov 2016 21:24:51 -0400 Subject: [lustre-devel] [PATCH 01/14] staging/lustre/ldlm: Drop unused blocking_refs flock field In-Reply-To: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> References: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> Message-ID: <1478136304-867780-2-git-send-email-green@linuxhacker.ru> blocking_refs is only used on the server, so drop it on the client. Signed-off-by: Oleg Drokin --- drivers/staging/lustre/lustre/include/lustre_dlm.h | 2 -- 1 file changed, 2 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre_dlm.h b/drivers/staging/lustre/lustre/include/lustre_dlm.h index 1c6b7b8..f770b86 100644 --- a/drivers/staging/lustre/lustre/include/lustre_dlm.h +++ b/drivers/staging/lustre/lustre/include/lustre_dlm.h @@ -550,8 +550,6 @@ struct ldlm_flock { __u64 owner; __u64 blocking_owner; struct obd_export *blocking_export; - /* Protected by the hash lock */ - __u32 blocking_refs; __u32 pid; }; -- 2.7.4 From green at linuxhacker.ru Thu Nov 3 01:24:58 2016 From: green at linuxhacker.ru (Oleg Drokin) Date: Wed, 2 Nov 2016 21:24:58 -0400 Subject: [lustre-devel] [PATCH 08/14] staging/lustre/ldlm: engage ELC for all ldlm enqueue req In-Reply-To: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> References: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> Message-ID: <1478136304-867780-9-git-send-email-green@linuxhacker.ru> From: Hongchao Zhang If there is no request passed into ldlm_cli_enqueue, the enqueue request will not engage ELC to drop unneeded locks. currently, this kind of request is mainly related to EXTENT locks enqueue requests (except for glimpse EXTENT lock for it has an intent). Signed-off-by: Hongchao Zhang Reviewed-on: http://review.whamcloud.com/21739 Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-8209 Reviewed-by: Andreas Dilger Reviewed-by: Vitaly Fertman Signed-off-by: Oleg Drokin --- drivers/staging/lustre/lustre/ldlm/ldlm_request.c | 21 ++++----------------- 1 file changed, 4 insertions(+), 17 deletions(-) diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c index 1b9ae77..c5d00d1 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c @@ -748,17 +748,14 @@ int ldlm_cli_enqueue(struct obd_export *exp, struct ptlrpc_request **reqp, lock->l_last_activity = ktime_get_real_seconds(); /* lock not sent to server yet */ - if (!reqp || !*reqp) { - req = ptlrpc_request_alloc_pack(class_exp2cliimp(exp), - &RQF_LDLM_ENQUEUE, - LUSTRE_DLM_VERSION, - LDLM_ENQUEUE); - if (!req) { + req = ldlm_enqueue_pack(exp, lvb_len); + if (IS_ERR(req)) { failed_lock_cleanup(ns, lock, einfo->ei_mode); LDLM_LOCK_RELEASE(lock); - return -ENOMEM; + return PTR_ERR(req); } + req_passed_in = 0; if (reqp) *reqp = req; @@ -778,16 +775,6 @@ int ldlm_cli_enqueue(struct obd_export *exp, struct ptlrpc_request **reqp, body->lock_flags = ldlm_flags_to_wire(*flags); body->lock_handle[0] = *lockh; - /* Continue as normal. */ - if (!req_passed_in) { - if (lvb_len > 0) - req_capsule_extend(&req->rq_pill, - &RQF_LDLM_ENQUEUE_LVB); - req_capsule_set_size(&req->rq_pill, &RMF_DLM_LVB, RCL_SERVER, - lvb_len); - ptlrpc_request_set_replen(req); - } - /* * Liblustre client doesn't get extent locks, except for O_APPEND case * where [0, OBD_OBJECT_EOF] lock is taken, or truncate, where -- 2.7.4 From green at linuxhacker.ru Thu Nov 3 01:25:03 2016 From: green at linuxhacker.ru (Oleg Drokin) Date: Wed, 2 Nov 2016 21:25:03 -0400 Subject: [lustre-devel] [PATCH 13/14] staging/lustre/llite: do not clear uptodate bit in page delete In-Reply-To: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> References: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> Message-ID: <1478136304-867780-14-git-send-email-green@linuxhacker.ru> From: Jinshan Xiong Otherwise, if the race between page fault and truncate occurs, it will cause the page fault routine to return an EIO error. In filemap_fault() { page_not_uptodate: ... ClearPageError(page); error = mapping->a_ops->readpage(file, page); if (!error) { wait_on_page_locked(page); if (!PageUptodate(page)) error = -EIO; } ... } However, I tend to think this is a defect in kernel implementation, because it assumes PageUptodate shouldn't be cleared but file read routine doesn't make the same assumption. Signed-off-by: Jinshan Xiong Reviewed-on: http://review.whamcloud.com/22827 Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-8633 Reviewed-by: Li Dongyang Reviewed-by: Bobi Jam Signed-off-by: Oleg Drokin --- drivers/staging/lustre/lustre/llite/vvp_page.c | 1 - 1 file changed, 1 deletion(-) diff --git a/drivers/staging/lustre/lustre/llite/vvp_page.c b/drivers/staging/lustre/lustre/llite/vvp_page.c index 25490a5..23d6630 100644 --- a/drivers/staging/lustre/lustre/llite/vvp_page.c +++ b/drivers/staging/lustre/lustre/llite/vvp_page.c @@ -166,7 +166,6 @@ static void vvp_page_delete(const struct lu_env *env, refc = atomic_dec_return(&page->cp_ref); LASSERTF(refc >= 1, "page = %p, refc = %d\n", page, refc); - ClearPageUptodate(vmpage); ClearPagePrivate(vmpage); vmpage->private = 0; /* -- 2.7.4 From green at linuxhacker.ru Thu Nov 3 01:25:02 2016 From: green at linuxhacker.ru (Oleg Drokin) Date: Wed, 2 Nov 2016 21:25:02 -0400 Subject: [lustre-devel] [PATCH 12/14] staging/lustre/llite: update ras window correctly In-Reply-To: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> References: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> Message-ID: <1478136304-867780-13-git-send-email-green@linuxhacker.ru> From: Bobi Jam When stride-RA hit case miss, we only reset normal sequential read-ahead window, but not reset the stride IO to avoid the overhead of re-detecting stride IO. While when the normal RA window is set to not insect with the stride-RA window, when we try to increase the stride-RA window length later, the presumption does not hold. This patch resets the stride IO as well in this case. Signed-off-by: Bobi Jam Reviewed-on: http://review.whamcloud.com/23032 Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-8683 Reviewed-by: wangdi Reviewed-by: Jinshan Xiong Signed-off-by: Oleg Drokin --- drivers/staging/lustre/lustre/llite/rw.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/drivers/staging/lustre/lustre/llite/rw.c b/drivers/staging/lustre/lustre/llite/rw.c index d2515a8..e34017d 100644 --- a/drivers/staging/lustre/lustre/llite/rw.c +++ b/drivers/staging/lustre/lustre/llite/rw.c @@ -809,13 +809,20 @@ static void ras_update(struct ll_sb_info *sbi, struct inode *inode, if (ra_miss) { if (index_in_stride_window(ras, index) && stride_io_mode(ras)) { - /*If stride-RA hit cache miss, the stride dector - *will not be reset to avoid the overhead of - *redetecting read-ahead mode - */ if (index != ras->ras_last_readpage + 1) ras->ras_consecutive_pages = 0; ras_reset(inode, ras, index); + + /* If stride-RA hit cache miss, the stride + * detector will not be reset to avoid the + * overhead of redetecting read-ahead mode, + * but on the condition that the stride window + * is still intersect with normal sequential + * read-ahead window. + */ + if (ras->ras_window_start < + ras->ras_stride_offset) + ras_stride_reset(ras); RAS_CDEBUG(ras); } else { /* Reset both stride window and normal RA -- 2.7.4 From green at linuxhacker.ru Thu Nov 3 01:24:59 2016 From: green at linuxhacker.ru (Oleg Drokin) Date: Wed, 2 Nov 2016 21:24:59 -0400 Subject: [lustre-devel] [PATCH 09/14] staging/lustre/ptlrpc: Suppress error for flock requests In-Reply-To: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> References: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> Message-ID: <1478136304-867780-10-git-send-email-green@linuxhacker.ru> From: Patrick Farrell -EAGAIN is a normal return when requesting POSIX flocks. We can't recognize exactly that case here, but it's the only case that should result in -EAGAIN on LDLM_ENQUEUE, so don't print to console in that case. Signed-off-by: Patrick Farrell Reviewed-on: http://review.whamcloud.com/22856 Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-8658 Reviewed-by: Andreas Dilger Reviewed-by: Bob Glossman Signed-off-by: Oleg Drokin --- drivers/staging/lustre/lustre/ptlrpc/client.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c index 7cbfb4c..bb7ae4e 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/client.c +++ b/drivers/staging/lustre/lustre/ptlrpc/client.c @@ -1191,7 +1191,9 @@ static int ptlrpc_check_status(struct ptlrpc_request *req) lnet_nid_t nid = imp->imp_connection->c_peer.nid; __u32 opc = lustre_msg_get_opc(req->rq_reqmsg); - if (ptlrpc_console_allow(req)) + /* -EAGAIN is normal when using POSIX flocks */ + if (ptlrpc_console_allow(req) && + !(opc == LDLM_ENQUEUE && err == -EAGAIN)) LCONSOLE_ERROR_MSG(0x011, "%s: operation %s to node %s failed: rc = %d\n", imp->imp_obd->obd_name, ll_opcode2str(opc), -- 2.7.4 From green at linuxhacker.ru Thu Nov 3 01:25:04 2016 From: green at linuxhacker.ru (Oleg Drokin) Date: Wed, 2 Nov 2016 21:25:04 -0400 Subject: [lustre-devel] [PATCH 14/14] staging/lustre: Get rid of LIBLUSTRE_CLIENT and its users In-Reply-To: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> References: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> Message-ID: <1478136304-867780-15-git-send-email-green@linuxhacker.ru> This define only made sense in a userspace library client, not in the kernel. Signed-off-by: Oleg Drokin --- drivers/staging/lustre/lustre/include/lustre_lib.h | 2 -- drivers/staging/lustre/lustre/ldlm/ldlm_request.c | 15 +-------------- 2 files changed, 1 insertion(+), 16 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre_lib.h b/drivers/staging/lustre/lustre/include/lustre_lib.h index 6b23191..27f3148 100644 --- a/drivers/staging/lustre/lustre/include/lustre_lib.h +++ b/drivers/staging/lustre/lustre/include/lustre_lib.h @@ -350,8 +350,6 @@ do { \ l_wait_event_exclusive_head(wq, condition, &lwi); \ }) -#define LIBLUSTRE_CLIENT (0) - /** @} lib */ #endif /* _LUSTRE_LIB_H */ diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c index c5d00d1..6a96f2c 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c @@ -475,12 +475,7 @@ int ldlm_cli_enqueue_fini(struct obd_export *exp, struct ptlrpc_request *req, "client-side enqueue, new policy data"); } - if ((*flags) & LDLM_FL_AST_SENT || - /* Cancel extent locks as soon as possible on a liblustre client, - * because it cannot handle asynchronous ASTs robustly (see - * bug 7311). - */ - (LIBLUSTRE_CLIENT && type == LDLM_EXTENT)) { + if ((*flags) & LDLM_FL_AST_SENT) { lock_res_and_lock(lock); lock->l_flags |= LDLM_FL_CBPENDING | LDLM_FL_BL_AST; unlock_res_and_lock(lock); @@ -775,14 +770,6 @@ int ldlm_cli_enqueue(struct obd_export *exp, struct ptlrpc_request **reqp, body->lock_flags = ldlm_flags_to_wire(*flags); body->lock_handle[0] = *lockh; - /* - * Liblustre client doesn't get extent locks, except for O_APPEND case - * where [0, OBD_OBJECT_EOF] lock is taken, or truncate, where - * [i_size, OBD_OBJECT_EOF] lock is taken. - */ - LASSERT(ergo(LIBLUSTRE_CLIENT, einfo->ei_type != LDLM_EXTENT || - policy->l_extent.end == OBD_OBJECT_EOF)); - if (async) { LASSERT(reqp); return 0; -- 2.7.4 From green at linuxhacker.ru Thu Nov 3 01:25:01 2016 From: green at linuxhacker.ru (Oleg Drokin) Date: Wed, 2 Nov 2016 21:25:01 -0400 Subject: [lustre-devel] [PATCH 11/14] staging/lustre/ptlrpc: Correctly calculate hrp->hrp_nthrs In-Reply-To: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> References: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> Message-ID: <1478136304-867780-12-git-send-email-green@linuxhacker.ru> From: Amir Shehata cpu_pattern can specify exactly 1 cpu in a partition: "0[0]". That means CPT0 will have CPU 0. CPU 0 can have hyperthreading enabled. This combination would result in weight = cfs_cpu_ht_nsiblings(0); hrp->hrp_nthrs = cfs_cpt_weight(ptlrpc_hr.hr_cpt_table, i); hrp->hrp_nthrs /= weight; evaluating to 0. Where cfs_cpt_weight(ptlrpc_hr.hr_cpt_table, i) == 1 weight == 2 Therefore, if hrp_nthrs becomes zero, just set it to 1. Signed-off-by: Amir Shehata Reviewed-on: http://review.whamcloud.com/19106 Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-8492 Reviewed-by: Liang Zhen Reviewed-by: Doug Oucharek Reviewed-by: James Simmons Signed-off-by: Oleg Drokin --- drivers/staging/lustre/lustre/ptlrpc/service.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/staging/lustre/lustre/ptlrpc/service.c b/drivers/staging/lustre/lustre/ptlrpc/service.c index 72f3930..fc754e7 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/service.c +++ b/drivers/staging/lustre/lustre/ptlrpc/service.c @@ -2541,8 +2541,9 @@ int ptlrpc_hr_init(void) hrp->hrp_nthrs = cfs_cpt_weight(ptlrpc_hr.hr_cpt_table, i); hrp->hrp_nthrs /= weight; + if (hrp->hrp_nthrs == 0) + hrp->hrp_nthrs = 1; - LASSERT(hrp->hrp_nthrs > 0); hrp->hrp_thrs = kzalloc_node(hrp->hrp_nthrs * sizeof(*hrt), GFP_NOFS, cfs_cpt_spread_node(ptlrpc_hr.hr_cpt_table, -- 2.7.4 From oleg.drokin at intel.com Thu Nov 3 04:24:01 2016 From: oleg.drokin at intel.com (Oleg Drokin) Date: Thu, 3 Nov 2016 00:24:01 -0400 Subject: [lustre-devel] New tag 2.8.60 Message-ID: <65318749-5517-4F61-BF7A-A8CAA31E7637@intel.com> Hello! I just tagged 2.8.60 in the master development branch. Here's the changelog: Alex Zhuravlev (4): LU-8527 osd: ot_credits must be 32bit LU-8695 target: use -1 as an offset to declare write LU-8775 osd: do not report special writes in brw stats LU-8573 lnet: Revert LU-7650 patches Amir Shehata (1): LU-8492 ptlrpc: Correctly calculate hrp->hrp_nthrs Andreas Dilger (4): LU-8668 tests: print more information about dd failure LU-3289 ssk: fix SK_IV_REV_START on 32-bit systems LU-8124 osd-zfs: fix statfs small blocksize inode estimate LU-930 doc: move DLC doc to Documentation dir Andriy Skulysh (1): LU-8347 ldlm: granting conflicting locks Artem Blagodarenko (2): LU-8239 utils: llanalyze logs parser fix LU-8239 utils: llanalyze drops useful logs Bob Glossman (4): LU-8669 kernel: kernel update RHEL6.8 [2.6.32-642.6.1.el6] LU-8692 kernel: kernel update RHEL7.2 [3.10.0-327.36.2.el7] LU-8751 kernel: kernel update RHEL7.2 [3.10.0-327.36.3.el7] LU-8755 kernel: kernel update [SLES12 SP1 3.12.62-60.64.8] Bobi Jam (4): LU-8682 llite: protect from accessing NULL lli_clob LU-8413 test: limit # of processes for sanity test_101f LU-8683 readahead: update ras window correctly LU-8680 osc: soft lock - osc_makes_rpc() Bruno Faccini (1): LU-8685 kernel: jbd2: fix incorrect unlock on j_list_lock Chris Horn (1): LU-8733 gnilnd: Remove read capability of cksum_test procfile Chuck Fossen (1): LU-8429 gnilnd: Option to not reconnect after conn timeout Di Wang (3): LU-8667 osp: validate FID before initializing precreate seq LU-8250 mdd: move linkea prepare out of transaction. LU-7206 mdd: stop orphan cleanup before finish FLD Elena Gryaznova (1): LU-8705 tests: do not skip lnet-selftest for DNE Fan Yong (5): LU-8592 mdt: hold mdt_device::mdt_md_root until service stop LU-8569 lfsck: cleanup lfsck requests list before exit LU-6635 lfsck: block replacing the OST-object for test LU-8574 osd-ldiskfs: fix FID-in-dirent properly LU-7593 target: take ted_lcd_lock after transaction started Frank Zago (1): LU-7988 hsm: make mdt_hsm_cdt_start a static function Gu Zheng (1): LU-4931 ladvise: add code for ladvise_hdr into wirecheck.c Hongchao Zhang (1): LU-8209 ldlm: engage ELC for all ldlm enqueue req Jadhav Vikram (1): LU-8575 lod: clear ost usable flag to avoid striping. James Simmons (7): LU-6245 libcfs: remove last of types abstractions LU-6245 libcfs: remove cfs_fs_time handling LU-8648 all: remove all Sun license and URL references LU-6387 tests: fix lp_utils build issues on Power8 LU-8660 mgs: handle return code of server_make_name() LU-8697 llite: remove IS_ERR(master_inode) check LU-6303 osc: remove handling cl_avail_grant less than zero Jeremy Filizetti (3): LU-3289 gss: Change the handling of keys for SK LU-3289 gss: Fix issues with SK privacy and integrity mode LU-8590 gss: Move DH parameter generation out of upcall Jian Yu (1): LU-8311 doc: add NIDs examples to mkfs.lustre and mount.lustre Jinshan Xiong (1): LU-8633 llite: do not clear uptodate bit in page delete John L. Hammond (3): LU-8654 obd: access ocd_connect_flags2 only when present LU-8723 llapi: correct open() handling in llapi_obd_statfs() LU-8731 utils: propagate errors in lfs df Kit Westneat (1): LU-8450 nodemap: modify ldlm_revoke_export_locks Lai Siyao (1): LU-8749 osd-ldiskfs: inherit S_ISGID correctly Li Xi (2): LU-4931 ofd: use thread buffer for ladvise LU-8289 utils: add ll_decode_linkea tool Lokesh Nagappa Jaliminche (1): LU-7919 mount: Buffer overflow issue while parsing mount Mikhail Pershin (1): LU-8089 lwp: change lwp export only at first connect Minh Diep (3): LU-8519 build: make SLES use lbuild kernel-devel LU-8636 build: prevent src download at the same time LU-8707 build: fix lbuild-sles for kernel_module_package Nathaniel Clark (4): LU-8630 build: Fix definition of KMP_MODDIR LU-7111 revert: b=22386 disallow wrong conf_param options LU-8713 utils: Try loading zfs.ko during zfs_init LU-8694 docs: ZFS hostid mkfs.lustre(8) man page update Niu Yawei (6): LU-7903 ptlrpc: leaked rs on difficult reply LU-8645 ptlrpc: update imp_known_replied_xid on resend-replay LU-8681 osd: ingore ENODATA during unlink agent parent LU-6808 ptlrpc: no need to reassign mbits for replay LU-8631 quota: better error message for 'lfs quota' LU-8748 osd-zfs: set block size of echo object Oleg Drokin (2): LU-8246 ldlm: Do not grant a lock twice in a race New tag 2.8.60 Parinay Kondekar (1): LU-8226 tests: Change check_catastrophe() to check_node_health() Patrick Farrell (1): LU-8658 ptlrpc: Suppress error for flock requests Rahul Deshmukh (1): LU-1882 llite: Adding timed wait in ll_umount_begin Steve Guminski (2): LU-8189 osc: osc_match_base prototype differs from declaration LU-8186 llite: Typo in ll_rw_extents_stats_pp_seq_show Yang Sheng (2): LU-7501 utils: keep lfs arguments consistent LU-8378 all: remove set but unused variables From andreas.dilger at intel.com Thu Nov 3 22:12:26 2016 From: andreas.dilger at intel.com (Dilger, Andreas) Date: Thu, 3 Nov 2016 22:12:26 +0000 Subject: [lustre-devel] [PATCH] staging: lustre: fixed shadowed variable in socklnd_cb.c In-Reply-To: <1478210059-6091-1-git-send-email-andrew.kanner@gmail.com> References: <1478210059-6091-1-git-send-email-andrew.kanner@gmail.com> Message-ID: On Nov 3, 2016, at 15:54, Andrew Kanner wrote: > > Changed variable 'tx' name in local scope > Fixed: sparse warning: > socklnd_cb.c:2476:41: warning: symbol 'tx' shadows an earlier one > socklnd_cb.c:2435:25: originally declared here Looking at this more closely (or from a greater distance, hard to say), the outer-scope "tx" is used only after this inner-scope "tx", so in fact there is no benefit to having the inner-scope declaration at all. Removing it may save a stack variable (depending on how the compiler optimizes), and shouldn't affect functionality. Cheers, Andreas > Signed-off-by: Andrew Kanner > --- > drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c > index c1c6f60..03fe4e5 100644 > --- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c > +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c > @@ -2473,11 +2473,11 @@ ksocknal_check_peer_timeouts(int idx) > * holding only shared lock > */ > if (!list_empty(&peer->ksnp_tx_queue)) { > - struct ksock_tx *tx = list_entry(peer->ksnp_tx_queue.next, > + struct ksock_tx *_tx = list_entry(peer->ksnp_tx_queue.next, > struct ksock_tx, tx_list); > > if (cfs_time_aftereq(cfs_time_current(), > - tx->tx_deadline)) { > + _tx->tx_deadline)) { > ksocknal_peer_addref(peer); > read_unlock(&ksocknal_data.ksnd_global_lock); > > -- > 2.1.4 > From andreas.dilger at intel.com Thu Nov 3 23:31:18 2016 From: andreas.dilger at intel.com (Dilger, Andreas) Date: Thu, 3 Nov 2016 23:31:18 +0000 Subject: [lustre-devel] [PATCH v2] staging: lustre: fixed shadowed variable in socklnd_cb.c In-Reply-To: <1478213386-13853-1-git-send-email-andrew.kanner@gmail.com> References: <1478213386-13853-1-git-send-email-andrew.kanner@gmail.com> Message-ID: <8314C33E-2E5A-4E25-8AE9-FDC5140E08FD@intel.com> On Nov 3, 2016, at 16:49, Andrew Kanner wrote: > > Removed redundant declaration of variable 'tx' in local scope > Fixed: sparse warning: > socklnd_cb.c:2476:41: warning: symbol 'tx' shadows an earlier one > socklnd_cb.c:2435:25: originally declared here > > Signed-off-by: Andrew Kanner Reviewed-by: Andreas Dilger > --- > drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c > index c1c6f60..f31f4a1 100644 > --- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c > +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c > @@ -2473,8 +2473,8 @@ ksocknal_check_peer_timeouts(int idx) > * holding only shared lock > */ > if (!list_empty(&peer->ksnp_tx_queue)) { > - struct ksock_tx *tx = list_entry(peer->ksnp_tx_queue.next, > - struct ksock_tx, tx_list); > + tx = list_entry(peer->ksnp_tx_queue.next, > + struct ksock_tx, tx_list); > > if (cfs_time_aftereq(cfs_time_current(), > tx->tx_deadline)) { > -- > 2.1.4 > From mahfouz.saif.elyazal at gmail.com Fri Nov 4 08:37:36 2016 From: mahfouz.saif.elyazal at gmail.com (Aya Mahfouz) Date: Fri, 4 Nov 2016 10:37:36 +0200 Subject: [lustre-devel] [PATCH 1/2] staging: lustre: replace uses of class_devno_max by MAX_OBD_DEVICES In-Reply-To: <8FDBA22B-F6A6-4CA7-B997-41327603E8A9@intel.com> References: <4ba436cdccbea6c6dbc631f8cf75cd80f4f9ddb4.1476739569.git.mahfouz.saif.elyazal@gmail.com> <20161025164730.GA27553@waves> <8FDBA22B-F6A6-4CA7-B997-41327603E8A9@intel.com> Message-ID: On Thu, Nov 3, 2016 at 1:05 AM, Dilger, Andreas wrote: > On Oct 25, 2016, at 10:47, Aya Mahfouz > wrote: > > > > On Mon, Oct 17, 2016 at 10:38:31PM +0000, Dilger, Andreas wrote: > >> On Oct 17, 2016, at 15:46, Aya Mahfouz > wrote: > >>> > >>> class_devno_max is an inline function that returns > >>> MAX_OBD_DEVICES. Replace all calls to the function > >>> by MAX_OBD_DEVICES. > >> > >> Thanks for your patch, but unfortunately it can't be accepted. > >> > >> This function was added in preparation of being able to tune the maximum > >> number of storage devices dynamically, rather than having to hard code > it > >> to the maximum possible number of servers that a client can possibly > >> connect to. > >> > >> While the current maximum of 8192 servers has been enough for current > >> filesystems, I'd rather move in the direction of dynamically handling > this > >> limit rather than re-introducing a hard-coded constant throughout the > code. > >> > > Hello, > > > > I would like to proceed with implementing the function if possible. > > Kindly direct me to some starting pointers. > > Hi Aya, > thanks for offering to look into this. > > There are several ways to approach this problem to make the allocation > of the obd_devs[] array dynamic. In most cases, there isn't any value > to dynamically shrink this array, since the filesystem(s) will typically > be mounted until the node is rebooted, and it is only in the tens of KB > size range, so this will not affect ongoing operations, and that simplifies > the implementation. > > The easiest way would be to have a dynamically-sized obd_devs[] array that > is reallocated in class_newdev() in PAGE_SIZE chunks whenever the current > array has no more free slots and copied to the new array, using > obd_dev_lock > to protect the array while it is being reallocated and copied. In most > cases, this would save memory over the static array (not many filesystems > have so many servers), but for the few sites that have 10000+ servers they > don't need to change the source to handle this. Using libcfs_kvzalloc() > would avoid issues with allocating large chunks of memory. > > There are a few places where obd_devs[] is accessed outside obd_dev_lock > that would need to be fixed now that this array may be changed at runtime. > > A second approach that may scale better is to change obd_devs from an array > to a doubly linked list (using standard list_head helpers). In many cases > the whole list is seached linearly, and most of the uses of class_num2obd() > are just used to walk that list in order, which could be replaced with > list_for_each_entry() list traversal. The class_name2dev() function should > be changed to return the pointer to the obd_device structure, and a new > helper class_dev2num() would just return the obd_minor number from the > obd_device struct for the one use in class_resolve_dev_name(). Using a > linked list has the advantage that there is no need to search for free > slots > in the array, since devices would be removed from the list when it is > freed. > > Cheers, Andreas > > Thanks Andreas! Will start looking into it. -- Kind Regards, Aya Saif El-yazal Mahfouz > >> One comment inline below, if you still want to submit a patch. > >> > >>> Signed-off-by: Aya Mahfouz > >>> --- > >>> drivers/staging/lustre/lustre/obdclass/class_obd.c | 6 +++--- > >>> drivers/staging/lustre/lustre/obdclass/genops.c | 22 > +++++++++++----------- > >>> .../lustre/lustre/obdclass/linux/linux-module.c | 6 +++--- > >>> 3 files changed, 17 insertions(+), 17 deletions(-) > >>> > >>> diff --git a/drivers/staging/lustre/lustre/obdclass/class_obd.c > b/drivers/staging/lustre/lustre/obdclass/class_obd.c > >>> index 2b21675..b775c74 100644 > >>> --- a/drivers/staging/lustre/lustre/obdclass/class_obd.c > >>> +++ b/drivers/staging/lustre/lustre/obdclass/class_obd.c > >>> @@ -345,7 +345,7 @@ int class_handle_ioctl(unsigned int cmd, unsigned > long arg) > >>> goto out; > >>> } > >>> obd = class_name2obd(data->ioc_inlbuf4); > >>> - } else if (data->ioc_dev < class_devno_max()) { > >>> + } else if (data->ioc_dev < MAX_OBD_DEVICES) { > >>> obd = class_num2obd(data->ioc_dev); > >>> } else { > >>> CERROR("OBD ioctl: No device\n"); > >>> @@ -498,7 +498,7 @@ static int __init obdclass_init(void) > >>> } > >>> > >>> /* This struct is already zeroed for us (static global) */ > >>> - for (i = 0; i < class_devno_max(); i++) > >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) > >>> obd_devs[i] = NULL; > >> > >> This block can just be removed entirely. It used to do something > useful, > >> but through a series of changes it has become useless. > >> > >> Cheers, Andreas > >> > >>> /* Default the dirty page cache cap to 1/2 of system memory. > >>> @@ -548,7 +548,7 @@ static void obdclass_exit(void) > >>> lustre_unregister_fs(); > >>> > >>> misc_deregister(&obd_psdev); > >>> - for (i = 0; i < class_devno_max(); i++) { > >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) { > >>> struct obd_device *obd = class_num2obd(i); > >>> > >>> if (obd && obd->obd_set_up && > >>> diff --git a/drivers/staging/lustre/lustre/obdclass/genops.c > b/drivers/staging/lustre/lustre/obdclass/genops.c > >>> index 99c2da6..af4fc58 100644 > >>> --- a/drivers/staging/lustre/lustre/obdclass/genops.c > >>> +++ b/drivers/staging/lustre/lustre/obdclass/genops.c > >>> @@ -290,7 +290,7 @@ struct obd_device *class_newdev(const char > *type_name, const char *name) > >>> LASSERT(newdev->obd_magic == OBD_DEVICE_MAGIC); > >>> > >>> write_lock(&obd_dev_lock); > >>> - for (i = 0; i < class_devno_max(); i++) { > >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) { > >>> struct obd_device *obd = class_num2obd(i); > >>> > >>> if (obd && (strcmp(name, obd->obd_name) == 0)) { > >>> @@ -322,9 +322,9 @@ struct obd_device *class_newdev(const char > *type_name, const char *name) > >>> } > >>> write_unlock(&obd_dev_lock); > >>> > >>> - if (!result && i >= class_devno_max()) { > >>> + if (!result && i >= MAX_OBD_DEVICES) { > >>> CERROR("all %u OBD devices used, increase > MAX_OBD_DEVICES\n", > >>> - class_devno_max()); > >>> + MAX_OBD_DEVICES); > >>> result = ERR_PTR(-EOVERFLOW); > >>> goto out; > >>> } > >>> @@ -372,7 +372,7 @@ int class_name2dev(const char *name) > >>> return -1; > >>> > >>> read_lock(&obd_dev_lock); > >>> - for (i = 0; i < class_devno_max(); i++) { > >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) { > >>> struct obd_device *obd = class_num2obd(i); > >>> > >>> if (obd && strcmp(name, obd->obd_name) == 0) { > >>> @@ -397,7 +397,7 @@ struct obd_device *class_name2obd(const char *name) > >>> { > >>> int dev = class_name2dev(name); > >>> > >>> - if (dev < 0 || dev > class_devno_max()) > >>> + if (dev < 0 || dev > MAX_OBD_DEVICES) > >>> return NULL; > >>> return class_num2obd(dev); > >>> } > >>> @@ -408,7 +408,7 @@ int class_uuid2dev(struct obd_uuid *uuid) > >>> int i; > >>> > >>> read_lock(&obd_dev_lock); > >>> - for (i = 0; i < class_devno_max(); i++) { > >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) { > >>> struct obd_device *obd = class_num2obd(i); > >>> > >>> if (obd && obd_uuid_equals(uuid, &obd->obd_uuid)) { > >>> @@ -435,7 +435,7 @@ struct obd_device *class_num2obd(int num) > >>> { > >>> struct obd_device *obd = NULL; > >>> > >>> - if (num < class_devno_max()) { > >>> + if (num < MAX_OBD_DEVICES) { > >>> obd = obd_devs[num]; > >>> if (!obd) > >>> return NULL; > >>> @@ -463,7 +463,7 @@ struct obd_device *class_find_client_obd(struct > obd_uuid *tgt_uuid, > >>> int i; > >>> > >>> read_lock(&obd_dev_lock); > >>> - for (i = 0; i < class_devno_max(); i++) { > >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) { > >>> struct obd_device *obd = class_num2obd(i); > >>> > >>> if (!obd) > >>> @@ -496,13 +496,13 @@ struct obd_device *class_devices_in_group(struct > obd_uuid *grp_uuid, int *next) > >>> > >>> if (!next) > >>> i = 0; > >>> - else if (*next >= 0 && *next < class_devno_max()) > >>> + else if (*next >= 0 && *next < MAX_OBD_DEVICES) > >>> i = *next; > >>> else > >>> return NULL; > >>> > >>> read_lock(&obd_dev_lock); > >>> - for (; i < class_devno_max(); i++) { > >>> + for (; i < MAX_OBD_DEVICES; i++) { > >>> struct obd_device *obd = class_num2obd(i); > >>> > >>> if (!obd) > >>> @@ -533,7 +533,7 @@ int class_notify_sptlrpc_conf(const char *fsname, > int namelen) > >>> LASSERT(namelen > 0); > >>> > >>> read_lock(&obd_dev_lock); > >>> - for (i = 0; i < class_devno_max(); i++) { > >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) { > >>> obd = class_num2obd(i); > >>> > >>> if (!obd || obd->obd_set_up == 0 || obd->obd_stopping) > >>> diff --git a/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c > b/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c > >>> index 33342bf..ca5b466 100644 > >>> --- a/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c > >>> +++ b/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c > >>> @@ -228,7 +228,7 @@ static ssize_t health_show(struct kobject *kobj, > struct attribute *attr, > >>> return sprintf(buf, "LBUG\n"); > >>> > >>> read_lock(&obd_dev_lock); > >>> - for (i = 0; i < class_devno_max(); i++) { > >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) { > >>> struct obd_device *obd; > >>> > >>> obd = class_num2obd(i); > >>> @@ -326,7 +326,7 @@ static struct attribute *lustre_attrs[] = { > >>> > >>> static void *obd_device_list_seq_start(struct seq_file *p, loff_t > *pos) > >>> { > >>> - if (*pos >= class_devno_max()) > >>> + if (*pos >= MAX_OBD_DEVICES) > >>> return NULL; > >>> > >>> return pos; > >>> @@ -339,7 +339,7 @@ static void obd_device_list_seq_stop(struct > seq_file *p, void *v) > >>> static void *obd_device_list_seq_next(struct seq_file *p, void *v, > loff_t *pos) > >>> { > >>> ++*pos; > >>> - if (*pos >= class_devno_max()) > >>> + if (*pos >= MAX_OBD_DEVICES) > >>> return NULL; > >>> > >>> return pos; > >>> -- > >>> 2.5.0 > >>> > >>> > >>> -- > >>> Kind Regards, > >>> Aya Saif El-yazal Mahfouz > >>> _______________________________________________ > >>> lustre-devel mailing list > >>> lustre-devel at lists.lustre.org > >>> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org > >> > > > > -- > > Kind Regards, > > Aya Saif El-yazal Mahfouz > > _______________________________________________ > > lustre-devel mailing list > > lustre-devel at lists.lustre.org > > http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From oleg.drokin at intel.com Sun Nov 6 17:26:32 2016 From: oleg.drokin at intel.com (Oleg Drokin) Date: Sun, 6 Nov 2016 12:26:32 -0500 Subject: [lustre-devel] [PATCH 2/2] staging: lustre: obdclass: Add handling of error returned by lustre_cfg_new In-Reply-To: <20161106171123.24929-1-christophe.jaillet@wanadoo.fr> References: <20161106171123.24929-1-christophe.jaillet@wanadoo.fr> Message-ID: <97207CF4-21C1-4351-92BF-6F28E7281CF6@intel.com> Hello! On Nov 6, 2016, at 12:11 PM, Christophe JAILLET wrote: > 'lustre_cfg_new()' can return ERR_PTR(-ENOMEM). > Handle these errors and propagate the error code to the callers. > > Error handling has been rearranged in 'lustre_process_log()' with the > addition of a label in order to free some resources. I wonder if we should just make it return NULL on allocation failure, and then at least the other error handling that is there (i.e. in your other patch) would become correct. This would make handling in mgc_apply_recover_logs incorrect, but it's already geared towards this sort of handling anyway, as it discards the passed error and sets ENOMEM unconditionally (just need to revert 3092c34a in a way). Thanks! > > Signed-off-by: Christophe JAILLET > --- > drivers/staging/lustre/lustre/obdclass/obd_mount.c | 16 ++++++++++++++-- > 1 file changed, 14 insertions(+), 2 deletions(-) > > diff --git a/drivers/staging/lustre/lustre/obdclass/obd_mount.c b/drivers/staging/lustre/lustre/obdclass/obd_mount.c > index 59fbc29aae94..5473615cd338 100644 > --- a/drivers/staging/lustre/lustre/obdclass/obd_mount.c > +++ b/drivers/staging/lustre/lustre/obdclass/obd_mount.c > @@ -89,11 +89,14 @@ int lustre_process_log(struct super_block *sb, char *logname, > lustre_cfg_bufs_set(bufs, 2, cfg, sizeof(*cfg)); > lustre_cfg_bufs_set(bufs, 3, &sb, sizeof(sb)); > lcfg = lustre_cfg_new(LCFG_LOG_START, bufs); > + if (IS_ERR(lcfg)) { > + rc = PTR_ERR(lcfg); > + goto out_free; > + } > + > rc = obd_process_config(mgc, sizeof(*lcfg), lcfg); > lustre_cfg_free(lcfg); > > - kfree(bufs); > - > if (rc == -EINVAL) > LCONSOLE_ERROR_MSG(0x15b, "%s: The configuration from log '%s' failed from the MGS (%d). Make sure this client and the MGS are running compatible versions of Lustre.\n", > mgc->obd_name, logname, rc); > @@ -104,6 +107,9 @@ int lustre_process_log(struct super_block *sb, char *logname, > rc); > > /* class_obd_list(); */ > + > +out_free: > + kfree(bufs); > return rc; > } > EXPORT_SYMBOL(lustre_process_log); > @@ -127,6 +133,9 @@ int lustre_end_log(struct super_block *sb, char *logname, > if (cfg) > lustre_cfg_bufs_set(&bufs, 2, cfg, sizeof(*cfg)); > lcfg = lustre_cfg_new(LCFG_LOG_END, &bufs); > + if (IS_ERR(lcfg)) > + return PTR_ERR(lcfg); > + > rc = obd_process_config(mgc, sizeof(*lcfg), lcfg); > lustre_cfg_free(lcfg); > return rc; > @@ -159,6 +168,9 @@ static int do_lcfg(char *cfgname, lnet_nid_t nid, int cmd, > lustre_cfg_bufs_set_string(&bufs, 4, s4); > > lcfg = lustre_cfg_new(cmd, &bufs); > + if (IS_ERR(lcfg)) > + return PTR_ERR(lcfg); > + > lcfg->lcfg_nid = nid; > rc = class_process_config(lcfg); > lustre_cfg_free(lcfg); > -- > 2.9.3 From jsimmons at infradead.org Mon Nov 7 01:37:31 2016 From: jsimmons at infradead.org (James Simmons) Date: Mon, 7 Nov 2016 01:37:31 +0000 (GMT) Subject: [lustre-devel] [PATCH 02/14] staging/lustre/ldlm: fix export reference problem In-Reply-To: <1478136304-867780-3-git-send-email-green@linuxhacker.ru> References: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> <1478136304-867780-3-git-send-email-green@linuxhacker.ru> Message-ID: > From: Hongchao Zhang > > 1, in client_import_del_conn, the export returned from > class_conn2export is not released after using it. > > 2, in ptlrpc_connect_interpret, the export is not released > if the connect_flags isn't compatible. Reviewed-by: James Simmons > Signed-off-by: Hongchao Zhang > Reviewed-on: http://review.whamcloud.com/22031 > Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-8500 > Reviewed-by: James Simmons > Reviewed-by: Bobi Jam > Signed-off-by: Oleg Drokin > --- > drivers/staging/lustre/lustre/ldlm/ldlm_lib.c | 3 +++ > drivers/staging/lustre/lustre/ptlrpc/import.c | 19 ++++++++++--------- > 2 files changed, 13 insertions(+), 9 deletions(-) > > diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c > index 4f9480e..06d3cc6 100644 > --- a/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c > +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c > @@ -170,6 +170,9 @@ int client_import_del_conn(struct obd_import *imp, struct obd_uuid *uuid) > ptlrpc_connection_put(dlmexp->exp_connection); > dlmexp->exp_connection = NULL; > } > + > + if (dlmexp) > + class_export_put(dlmexp); > } > > list_del(&imp_conn->oic_item); > diff --git a/drivers/staging/lustre/lustre/ptlrpc/import.c b/drivers/staging/lustre/lustre/ptlrpc/import.c > index 46ba5a4..05fd92d 100644 > --- a/drivers/staging/lustre/lustre/ptlrpc/import.c > +++ b/drivers/staging/lustre/lustre/ptlrpc/import.c > @@ -972,6 +972,16 @@ static int ptlrpc_connect_interpret(const struct lu_env *env, > > spin_unlock(&imp->imp_lock); > > + if (!exp) { > + /* This could happen if export is cleaned during the > + * connect attempt > + */ > + CERROR("%s: missing export after connect\n", > + imp->imp_obd->obd_name); > + rc = -ENODEV; > + goto out; > + } > + > /* check that server granted subset of flags we asked for. */ > if ((ocd->ocd_connect_flags & imp->imp_connect_flags_orig) != > ocd->ocd_connect_flags) { > @@ -982,15 +992,6 @@ static int ptlrpc_connect_interpret(const struct lu_env *env, > goto out; > } > > - if (!exp) { > - /* This could happen if export is cleaned during the > - * connect attempt > - */ > - CERROR("%s: missing export after connect\n", > - imp->imp_obd->obd_name); > - rc = -ENODEV; > - goto out; > - } > old_connect_flags = exp_connect_flags(exp); > exp->exp_connect_data = *ocd; > imp->imp_obd->obd_self_export->exp_connect_data = *ocd; > -- > 2.7.4 > > From jsimmons at infradead.org Mon Nov 7 01:38:03 2016 From: jsimmons at infradead.org (James Simmons) Date: Mon, 7 Nov 2016 01:38:03 +0000 (GMT) Subject: [lustre-devel] [PATCH 04/14] staging/lustre/llite: clear inode timestamps after losing UPDATE lock In-Reply-To: <1478136304-867780-5-git-send-email-green@linuxhacker.ru> References: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> <1478136304-867780-5-git-send-email-green@linuxhacker.ru> Message-ID: > From: Niu Yawei > > Otherwise, those leftovers would interfere with new timestamps > especially when the timestamps are set back in time on the other > clients. Reviewed-by: James Simmons > Signed-off-by: Jinshan Xiong > Signed-off-by: Niu Yawei > Reviewed-on: http://review.whamcloud.com/22623 > Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-8446 > Reviewed-by: Bobi Jam > Signed-off-by: Oleg Drokin > --- > drivers/staging/lustre/lustre/llite/namei.c | 10 ++++++++++ > 1 file changed, 10 insertions(+) > > diff --git a/drivers/staging/lustre/lustre/llite/namei.c b/drivers/staging/lustre/lustre/llite/namei.c > index 74d9b73..c268f32 100644 > --- a/drivers/staging/lustre/lustre/llite/namei.c > +++ b/drivers/staging/lustre/lustre/llite/namei.c > @@ -251,6 +251,16 @@ int ll_md_blocking_ast(struct ldlm_lock *lock, struct ldlm_lock_desc *desc, > PFID(ll_inode2fid(inode)), rc); > } > > + if (bits & MDS_INODELOCK_UPDATE) { > + struct ll_inode_info *lli = ll_i2info(inode); > + > + spin_lock(&lli->lli_lock); > + LTIME_S(inode->i_mtime) = 0; > + LTIME_S(inode->i_atime) = 0; > + LTIME_S(inode->i_ctime) = 0; > + spin_unlock(&lli->lli_lock); > + } > + > if ((bits & MDS_INODELOCK_UPDATE) && S_ISDIR(inode->i_mode)) { > struct ll_inode_info *lli = ll_i2info(inode); > > -- > 2.7.4 > > From jsimmons at infradead.org Mon Nov 7 01:38:40 2016 From: jsimmons at infradead.org (James Simmons) Date: Mon, 7 Nov 2016 01:38:40 +0000 (GMT) Subject: [lustre-devel] [PATCH 01/14] staging/lustre/ldlm: Drop unused blocking_refs flock field In-Reply-To: <1478136304-867780-2-git-send-email-green@linuxhacker.ru> References: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> <1478136304-867780-2-git-send-email-green@linuxhacker.ru> Message-ID: > blocking_refs is only used on the server, so drop it on the client. Reviewed-by: James Simmons > Signed-off-by: Oleg Drokin > --- > drivers/staging/lustre/lustre/include/lustre_dlm.h | 2 -- > 1 file changed, 2 deletions(-) > > diff --git a/drivers/staging/lustre/lustre/include/lustre_dlm.h b/drivers/staging/lustre/lustre/include/lustre_dlm.h > index 1c6b7b8..f770b86 100644 > --- a/drivers/staging/lustre/lustre/include/lustre_dlm.h > +++ b/drivers/staging/lustre/lustre/include/lustre_dlm.h > @@ -550,8 +550,6 @@ struct ldlm_flock { > __u64 owner; > __u64 blocking_owner; > struct obd_export *blocking_export; > - /* Protected by the hash lock */ > - __u32 blocking_refs; > __u32 pid; > }; > > -- > 2.7.4 > > From jsimmons at infradead.org Mon Nov 7 01:52:40 2016 From: jsimmons at infradead.org (James Simmons) Date: Mon, 7 Nov 2016 01:52:40 +0000 (GMT) Subject: [lustre-devel] [PATCH 03/14] staging/lustre: conflicting PW & PR extent locks on a client In-Reply-To: <1478136304-867780-4-git-send-email-green@linuxhacker.ru> References: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> <1478136304-867780-4-git-send-email-green@linuxhacker.ru> Message-ID: > From: Andriy Skulysh > > PW lock isn't replayed once a lock is marked > LDLM_FL_CANCELING and glimpse lock doesn't wait for > conflicting locks on the client. So the server will > grant a PR lock in response to the glimpse lock request, > which conflicts with the PW lock in LDLM_FL_CANCELING > state on the client. > > Lock in LDLM_FL_CANCELING state may still have pending IO, > so it should be replayed until LDLM_FL_BL_DONE is set to > avoid granted conflicting lock by a server. Reviewed-by: James Simmons > Seagate-bug-id: MRP-3311 > Signed-off-by: Andriy Skulysh > Reviewed-on: http://review.whamcloud.com/20345 > Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-8175 > Reviewed-by: Jinshan Xiong > Signed-off-by: Oleg Drokin > --- > drivers/staging/lustre/lustre/include/obd_support.h | 3 +++ > drivers/staging/lustre/lustre/ldlm/ldlm_extent.c | 20 ++++++++++++++++++++ > drivers/staging/lustre/lustre/ldlm/ldlm_request.c | 4 ++-- > drivers/staging/lustre/lustre/osc/osc_request.c | 1 + > 4 files changed, 26 insertions(+), 2 deletions(-) > > diff --git a/drivers/staging/lustre/lustre/include/obd_support.h b/drivers/staging/lustre/lustre/include/obd_support.h > index 7f3f8cd..aaedec7 100644 > --- a/drivers/staging/lustre/lustre/include/obd_support.h > +++ b/drivers/staging/lustre/lustre/include/obd_support.h > @@ -321,6 +321,8 @@ extern char obd_jobid_var[]; > #define OBD_FAIL_LDLM_CP_CB_WAIT4 0x322 > #define OBD_FAIL_LDLM_CP_CB_WAIT5 0x323 > > +#define OBD_FAIL_LDLM_GRANT_CHECK 0x32a > + > /* LOCKLESS IO */ > #define OBD_FAIL_LDLM_SET_CONTENTION 0x385 > > @@ -343,6 +345,7 @@ extern char obd_jobid_var[]; > #define OBD_FAIL_OSC_CP_ENQ_RACE 0x410 > #define OBD_FAIL_OSC_NO_GRANT 0x411 > #define OBD_FAIL_OSC_DELAY_SETTIME 0x412 > +#define OBD_FAIL_OSC_DELAY_IO 0x414 > > #define OBD_FAIL_PTLRPC 0x500 > #define OBD_FAIL_PTLRPC_ACK 0x501 > diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c b/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c > index ecf472e..a7b34e4 100644 > --- a/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c > +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c > @@ -193,6 +193,26 @@ void ldlm_extent_add_lock(struct ldlm_resource *res, > * add the locks into grant list, for debug purpose, .. > */ > ldlm_resource_add_lock(res, &res->lr_granted, lock); > + > + if (OBD_FAIL_CHECK(OBD_FAIL_LDLM_GRANT_CHECK)) { > + struct ldlm_lock *lck; > + > + list_for_each_entry_reverse(lck, &res->lr_granted, > + l_res_link) { > + if (lck == lock) > + continue; > + if (lockmode_compat(lck->l_granted_mode, > + lock->l_granted_mode)) > + continue; > + if (ldlm_extent_overlap(&lck->l_req_extent, > + &lock->l_req_extent)) { > + CDEBUG(D_ERROR, "granting conflicting lock %p %p\n", > + lck, lock); > + ldlm_resource_dump(D_ERROR, res); > + LBUG(); > + } > + } > + } > } > > /** Remove cancelled lock from resource interval tree. */ > diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c > index 43856ff..6e704c7 100644 > --- a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c > +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c > @@ -1846,7 +1846,7 @@ static int ldlm_chain_lock_for_replay(struct ldlm_lock *lock, void *closure) > * bug 17614: locks being actively cancelled. Get a reference > * on a lock so that it does not disappear under us (e.g. due to cancel) > */ > - if (!(lock->l_flags & (LDLM_FL_FAILED | LDLM_FL_CANCELING))) { > + if (!(lock->l_flags & (LDLM_FL_FAILED | LDLM_FL_BL_DONE))) { > list_add(&lock->l_pending_chain, list); > LDLM_LOCK_GET(lock); > } > @@ -1915,7 +1915,7 @@ static int replay_one_lock(struct obd_import *imp, struct ldlm_lock *lock) > int flags; > > /* Bug 11974: Do not replay a lock which is actively being canceled */ > - if (ldlm_is_canceling(lock)) { > + if (ldlm_is_bl_done(lock)) { > LDLM_DEBUG(lock, "Not replaying canceled lock:"); > return 0; > } > diff --git a/drivers/staging/lustre/lustre/osc/osc_request.c b/drivers/staging/lustre/lustre/osc/osc_request.c > index 091558e..8023561 100644 > --- a/drivers/staging/lustre/lustre/osc/osc_request.c > +++ b/drivers/staging/lustre/lustre/osc/osc_request.c > @@ -1823,6 +1823,7 @@ int osc_build_rpc(const struct lu_env *env, struct client_obd *cli, > DEBUG_REQ(D_INODE, req, "%d pages, aa %p. now %ur/%dw in flight", > page_count, aa, cli->cl_r_in_flight, > cli->cl_w_in_flight); > + OBD_FAIL_TIMEOUT(OBD_FAIL_OSC_DELAY_IO, cfs_fail_val); > > ptlrpcd_add_req(req); > rc = 0; > -- > 2.7.4 > > From jsimmons at infradead.org Mon Nov 7 01:53:06 2016 From: jsimmons at infradead.org (James Simmons) Date: Mon, 7 Nov 2016 01:53:06 +0000 (GMT) Subject: [lustre-devel] [PATCH 05/14] staging/lustre: Get rid of cl_env hash table In-Reply-To: <1478136304-867780-6-git-send-email-green@linuxhacker.ru> References: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> <1478136304-867780-6-git-send-email-green@linuxhacker.ru> Message-ID: > From: Jinshan Xiong > > cl_env hash table is under heavy contention when there are lots of > processes doing IO at the same time; > reduce lock contention by replacing cl_env cache with percpu array; > remove cl_env_nested_get() and cl_env_nested_put(); > remove cl_env_reenter() and cl_env_reexit(); Reviewed-by: James Simmons > Signed-off-by: Jinshan Xiong > Reviewed-on: http://review.whamcloud.com/20254 > Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-4257 > Reviewed-by: Andreas Dilger > Reviewed-by: Bobi Jam > Signed-off-by: Oleg Drokin > --- > drivers/staging/lustre/lustre/include/cl_object.h | 22 -- > drivers/staging/lustre/lustre/ldlm/ldlm_pool.c | 9 - > drivers/staging/lustre/lustre/llite/file.c | 18 +- > drivers/staging/lustre/lustre/llite/lcommon_cl.c | 11 +- > drivers/staging/lustre/lustre/llite/lcommon_misc.c | 8 +- > drivers/staging/lustre/lustre/llite/llite_mmap.c | 59 ++-- > drivers/staging/lustre/lustre/llite/rw.c | 6 +- > drivers/staging/lustre/lustre/llite/rw26.c | 16 +- > drivers/staging/lustre/lustre/lov/lov_io.c | 5 - > drivers/staging/lustre/lustre/lov/lov_object.c | 7 +- > .../staging/lustre/lustre/obdclass/cl_internal.h | 23 -- > drivers/staging/lustre/lustre/obdclass/cl_io.c | 1 - > drivers/staging/lustre/lustre/obdclass/cl_object.c | 389 ++++----------------- > drivers/staging/lustre/lustre/obdclass/lu_object.c | 4 - > .../staging/lustre/lustre/obdecho/echo_client.c | 14 +- > drivers/staging/lustre/lustre/osc/osc_cache.c | 6 +- > drivers/staging/lustre/lustre/osc/osc_lock.c | 116 +++--- > drivers/staging/lustre/lustre/osc/osc_page.c | 6 +- > 18 files changed, 183 insertions(+), 537 deletions(-) > > diff --git a/drivers/staging/lustre/lustre/include/cl_object.h b/drivers/staging/lustre/lustre/include/cl_object.h > index 514d650..3fe26e7 100644 > --- a/drivers/staging/lustre/lustre/include/cl_object.h > +++ b/drivers/staging/lustre/lustre/include/cl_object.h > @@ -2640,35 +2640,13 @@ void cl_sync_io_end(const struct lu_env *env, struct cl_sync_io *anchor); > * - allocation and destruction of environment is amortized by caching no > * longer used environments instead of destroying them; > * > - * - there is a notion of "current" environment, attached to the kernel > - * data structure representing current thread Top-level lustre code > - * allocates an environment and makes it current, then calls into > - * non-lustre code, that in turn calls lustre back. Low-level lustre > - * code thus called can fetch environment created by the top-level code > - * and reuse it, avoiding additional environment allocation. > - * Right now, three interfaces can attach the cl_env to running thread: > - * - cl_env_get > - * - cl_env_implant > - * - cl_env_reexit(cl_env_reenter had to be called priorly) > - * > * \see lu_env, lu_context, lu_context_key > * @{ > */ > > -struct cl_env_nest { > - int cen_refcheck; > - void *cen_cookie; > -}; > - > struct lu_env *cl_env_get(int *refcheck); > struct lu_env *cl_env_alloc(int *refcheck, __u32 tags); > -struct lu_env *cl_env_nested_get(struct cl_env_nest *nest); > void cl_env_put(struct lu_env *env, int *refcheck); > -void cl_env_nested_put(struct cl_env_nest *nest, struct lu_env *env); > -void *cl_env_reenter(void); > -void cl_env_reexit(void *cookie); > -void cl_env_implant(struct lu_env *env, int *refcheck); > -void cl_env_unplant(struct lu_env *env, int *refcheck); > unsigned int cl_env_cache_purge(unsigned int nr); > struct lu_env *cl_env_percpu_get(void); > void cl_env_percpu_put(struct lu_env *env); > diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c > index b29c9561..19831c5 100644 > --- a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c > +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c > @@ -794,7 +794,6 @@ static unsigned long ldlm_pools_count(ldlm_side_t client, gfp_t gfp_mask) > int nr_ns; > struct ldlm_namespace *ns; > struct ldlm_namespace *ns_old = NULL; /* loop detection */ > - void *cookie; > > if (client == LDLM_NAMESPACE_CLIENT && !(gfp_mask & __GFP_FS)) > return 0; > @@ -802,8 +801,6 @@ static unsigned long ldlm_pools_count(ldlm_side_t client, gfp_t gfp_mask) > CDEBUG(D_DLMTRACE, "Request to count %s locks from all pools\n", > client == LDLM_NAMESPACE_CLIENT ? "client" : "server"); > > - cookie = cl_env_reenter(); > - > /* > * Find out how many resources we may release. > */ > @@ -812,7 +809,6 @@ static unsigned long ldlm_pools_count(ldlm_side_t client, gfp_t gfp_mask) > mutex_lock(ldlm_namespace_lock(client)); > if (list_empty(ldlm_namespace_list(client))) { > mutex_unlock(ldlm_namespace_lock(client)); > - cl_env_reexit(cookie); > return 0; > } > ns = ldlm_namespace_first_locked(client); > @@ -838,7 +834,6 @@ static unsigned long ldlm_pools_count(ldlm_side_t client, gfp_t gfp_mask) > ldlm_namespace_put(ns); > } > > - cl_env_reexit(cookie); > return total; > } > > @@ -847,13 +842,10 @@ static unsigned long ldlm_pools_scan(ldlm_side_t client, int nr, gfp_t gfp_mask) > unsigned long freed = 0; > int tmp, nr_ns; > struct ldlm_namespace *ns; > - void *cookie; > > if (client == LDLM_NAMESPACE_CLIENT && !(gfp_mask & __GFP_FS)) > return -1; > > - cookie = cl_env_reenter(); > - > /* > * Shrink at least ldlm_namespace_nr_read(client) namespaces. > */ > @@ -883,7 +875,6 @@ static unsigned long ldlm_pools_scan(ldlm_side_t client, int nr, gfp_t gfp_mask) > freed += ldlm_pool_shrink(&ns->ns_pool, cancel, gfp_mask); > ldlm_namespace_put(ns); > } > - cl_env_reexit(cookie); > /* > * we only decrease the SLV in server pools shrinker, return > * SHRINK_STOP to kernel to avoid needless loop. LU-1128 > diff --git a/drivers/staging/lustre/lustre/llite/file.c b/drivers/staging/lustre/lustre/llite/file.c > index 0ae78c7..c1c7551 100644 > --- a/drivers/staging/lustre/lustre/llite/file.c > +++ b/drivers/staging/lustre/lustre/llite/file.c > @@ -1584,11 +1584,11 @@ int ll_data_version(struct inode *inode, __u64 *data_version, int flags) > */ > int ll_hsm_release(struct inode *inode) > { > - struct cl_env_nest nest; > struct lu_env *env; > struct obd_client_handle *och = NULL; > __u64 data_version = 0; > int rc; > + int refcheck; > > CDEBUG(D_INODE, "%s: Releasing file "DFID".\n", > ll_get_fsname(inode->i_sb, NULL, 0), > @@ -1605,14 +1605,14 @@ int ll_hsm_release(struct inode *inode) > if (rc != 0) > goto out; > > - env = cl_env_nested_get(&nest); > + env = cl_env_get(&refcheck); > if (IS_ERR(env)) { > rc = PTR_ERR(env); > goto out; > } > > ll_merge_attr(env, inode); > - cl_env_nested_put(&nest, env); > + cl_env_put(env, &refcheck); > > /* Release the file. > * NB: lease lock handle is released in mdc_hsm_release_pack() because > @@ -2268,17 +2268,17 @@ static int ll_flush(struct file *file, fl_owner_t id) > int cl_sync_file_range(struct inode *inode, loff_t start, loff_t end, > enum cl_fsync_mode mode, int ignore_layout) > { > - struct cl_env_nest nest; > struct lu_env *env; > struct cl_io *io; > struct cl_fsync_io *fio; > int result; > + int refcheck; > > if (mode != CL_FSYNC_NONE && mode != CL_FSYNC_LOCAL && > mode != CL_FSYNC_DISCARD && mode != CL_FSYNC_ALL) > return -EINVAL; > > - env = cl_env_nested_get(&nest); > + env = cl_env_get(&refcheck); > if (IS_ERR(env)) > return PTR_ERR(env); > > @@ -2301,7 +2301,7 @@ int cl_sync_file_range(struct inode *inode, loff_t start, loff_t end, > if (result == 0) > result = fio->fi_nr_written; > cl_io_fini(env, io); > - cl_env_nested_put(&nest, env); > + cl_env_put(env, &refcheck); > > return result; > } > @@ -3149,14 +3149,14 @@ int ll_layout_conf(struct inode *inode, const struct cl_object_conf *conf) > { > struct ll_inode_info *lli = ll_i2info(inode); > struct cl_object *obj = lli->lli_clob; > - struct cl_env_nest nest; > struct lu_env *env; > int rc; > + int refcheck; > > if (!obj) > return 0; > > - env = cl_env_nested_get(&nest); > + env = cl_env_get(&refcheck); > if (IS_ERR(env)) > return PTR_ERR(env); > > @@ -3190,7 +3190,7 @@ int ll_layout_conf(struct inode *inode, const struct cl_object_conf *conf) > ll_layout_version_set(lli, cl.cl_layout_gen); > } > out: > - cl_env_nested_put(&nest, env); > + cl_env_put(env, &refcheck); > return rc; > } > > diff --git a/drivers/staging/lustre/lustre/llite/lcommon_cl.c b/drivers/staging/lustre/lustre/llite/lcommon_cl.c > index 56e414b..dd1cfd8 100644 > --- a/drivers/staging/lustre/lustre/llite/lcommon_cl.c > +++ b/drivers/staging/lustre/lustre/llite/lcommon_cl.c > @@ -246,15 +246,11 @@ void cl_inode_fini(struct inode *inode) > int emergency; > > if (clob) { > - void *cookie; > - > - cookie = cl_env_reenter(); > env = cl_env_get(&refcheck); > emergency = IS_ERR(env); > if (emergency) { > mutex_lock(&cl_inode_fini_guard); > LASSERT(cl_inode_fini_env); > - cl_env_implant(cl_inode_fini_env, &refcheck); > env = cl_inode_fini_env; > } > /* > @@ -266,13 +262,10 @@ void cl_inode_fini(struct inode *inode) > lu_object_ref_del(&clob->co_lu, "inode", inode); > cl_object_put_last(env, clob); > lli->lli_clob = NULL; > - if (emergency) { > - cl_env_unplant(cl_inode_fini_env, &refcheck); > + if (emergency) > mutex_unlock(&cl_inode_fini_guard); > - } else { > + else > cl_env_put(env, &refcheck); > - } > - cl_env_reexit(cookie); > } > } > > diff --git a/drivers/staging/lustre/lustre/llite/lcommon_misc.c b/drivers/staging/lustre/lustre/llite/lcommon_misc.c > index 1558b55..f48660e 100644 > --- a/drivers/staging/lustre/lustre/llite/lcommon_misc.c > +++ b/drivers/staging/lustre/lustre/llite/lcommon_misc.c > @@ -162,13 +162,11 @@ int cl_get_grouplock(struct cl_object *obj, unsigned long gid, int nonblock, > return rc; > } > > - cg->lg_env = cl_env_get(&refcheck); > + cg->lg_env = env; > cg->lg_io = io; > cg->lg_lock = lock; > cg->lg_gid = gid; > - LASSERT(cg->lg_env == env); > > - cl_env_unplant(env, &refcheck); > return 0; > } > > @@ -177,14 +175,10 @@ void cl_put_grouplock(struct ll_grouplock *cg) > struct lu_env *env = cg->lg_env; > struct cl_io *io = cg->lg_io; > struct cl_lock *lock = cg->lg_lock; > - int refcheck; > > LASSERT(cg->lg_env); > LASSERT(cg->lg_gid); > > - cl_env_implant(env, &refcheck); > - cl_env_put(env, &refcheck); > - > cl_lock_release(env, lock); > cl_io_fini(env, io); > cl_env_put(env, NULL); > diff --git a/drivers/staging/lustre/lustre/llite/llite_mmap.c b/drivers/staging/lustre/lustre/llite/llite_mmap.c > index 4366918..ad2a699 100644 > --- a/drivers/staging/lustre/lustre/llite/llite_mmap.c > +++ b/drivers/staging/lustre/lustre/llite/llite_mmap.c > @@ -80,43 +80,24 @@ struct vm_area_struct *our_vma(struct mm_struct *mm, unsigned long addr, > * API independent part for page fault initialization. > * \param vma - virtual memory area addressed to page fault > * \param env - corespondent lu_env to processing > - * \param nest - nested level > * \param index - page index corespondent to fault. > * \parm ra_flags - vma readahead flags. > * > - * \return allocated and initialized env for fault operation. > - * \retval EINVAL if env can't allocated > - * \return other error codes from cl_io_init. > + * \return error codes from cl_io_init. > */ > static struct cl_io * > -ll_fault_io_init(struct vm_area_struct *vma, struct lu_env **env_ret, > - struct cl_env_nest *nest, pgoff_t index, > - unsigned long *ra_flags) > +ll_fault_io_init(struct lu_env *env, struct vm_area_struct *vma, > + pgoff_t index, unsigned long *ra_flags) > { > struct file *file = vma->vm_file; > struct inode *inode = file_inode(file); > struct cl_io *io; > struct cl_fault_io *fio; > - struct lu_env *env; > int rc; > > - *env_ret = NULL; > if (ll_file_nolock(file)) > return ERR_PTR(-EOPNOTSUPP); > > - /* > - * page fault can be called when lustre IO is > - * already active for the current thread, e.g., when doing read/write > - * against user level buffer mapped from Lustre buffer. To avoid > - * stomping on existing context, optionally force an allocation of a new > - * one. > - */ > - env = cl_env_nested_get(nest); > - if (IS_ERR(env)) > - return ERR_PTR(-EINVAL); > - > - *env_ret = env; > - > restart: > io = vvp_env_thread_io(env); > io->ci_obj = ll_i2info(inode)->lli_clob; > @@ -155,7 +136,6 @@ ll_fault_io_init(struct vm_area_struct *vma, struct lu_env **env_ret, > if (io->ci_need_restart) > goto restart; > > - cl_env_nested_put(nest, env); > io = ERR_PTR(rc); > } > > @@ -169,13 +149,17 @@ static int ll_page_mkwrite0(struct vm_area_struct *vma, struct page *vmpage, > struct lu_env *env; > struct cl_io *io; > struct vvp_io *vio; > - struct cl_env_nest nest; > int result; > + int refcheck; > sigset_t set; > struct inode *inode; > struct ll_inode_info *lli; > > - io = ll_fault_io_init(vma, &env, &nest, vmpage->index, NULL); > + env = cl_env_get(&refcheck); > + if (IS_ERR(env)) > + return PTR_ERR(env); > + > + io = ll_fault_io_init(env, vma, vmpage->index, NULL); > if (IS_ERR(io)) { > result = PTR_ERR(io); > goto out; > @@ -240,8 +224,8 @@ static int ll_page_mkwrite0(struct vm_area_struct *vma, struct page *vmpage, > > out_io: > cl_io_fini(env, io); > - cl_env_nested_put(&nest, env); > out: > + cl_env_put(env, &refcheck); > CDEBUG(D_MMAP, "%s mkwrite with %d\n", current->comm, result); > LASSERT(ergo(result == 0, PageLocked(vmpage))); > > @@ -285,13 +269,19 @@ static int ll_fault0(struct vm_area_struct *vma, struct vm_fault *vmf) > struct vvp_io *vio = NULL; > struct page *vmpage; > unsigned long ra_flags; > - struct cl_env_nest nest; > - int result; > + int result = 0; > int fault_ret = 0; > + int refcheck; > > - io = ll_fault_io_init(vma, &env, &nest, vmf->pgoff, &ra_flags); > - if (IS_ERR(io)) > - return to_fault_error(PTR_ERR(io)); > + env = cl_env_get(&refcheck); > + if (IS_ERR(env)) > + return PTR_ERR(env); > + > + io = ll_fault_io_init(env, vma, vmf->pgoff, &ra_flags); > + if (IS_ERR(io)) { > + result = to_fault_error(PTR_ERR(io)); > + goto out; > + } > > result = io->ci_result; > if (result == 0) { > @@ -322,14 +312,15 @@ static int ll_fault0(struct vm_area_struct *vma, struct vm_fault *vmf) > } > } > cl_io_fini(env, io); > - cl_env_nested_put(&nest, env); > > vma->vm_flags |= ra_flags; > + > +out: > + cl_env_put(env, &refcheck); > if (result != 0 && !(fault_ret & VM_FAULT_RETRY)) > fault_ret |= to_fault_error(result); > > - CDEBUG(D_MMAP, "%s fault %d/%d\n", > - current->comm, fault_ret, result); > + CDEBUG(D_MMAP, "%s fault %d/%d\n", current->comm, fault_ret, result); > return fault_ret; > } > > diff --git a/drivers/staging/lustre/lustre/llite/rw.c b/drivers/staging/lustre/lustre/llite/rw.c > index 80cb8e0..d2515a8 100644 > --- a/drivers/staging/lustre/lustre/llite/rw.c > +++ b/drivers/staging/lustre/lustre/llite/rw.c > @@ -896,17 +896,17 @@ int ll_writepage(struct page *vmpage, struct writeback_control *wbc) > struct cl_io *io; > struct cl_page *page; > struct cl_object *clob; > - struct cl_env_nest nest; > bool redirtied = false; > bool unlocked = false; > int result; > + int refcheck; > > LASSERT(PageLocked(vmpage)); > LASSERT(!PageWriteback(vmpage)); > > LASSERT(ll_i2dtexp(inode)); > > - env = cl_env_nested_get(&nest); > + env = cl_env_get(&refcheck); > if (IS_ERR(env)) { > result = PTR_ERR(env); > goto out; > @@ -971,7 +971,7 @@ int ll_writepage(struct page *vmpage, struct writeback_control *wbc) > } > } > > - cl_env_nested_put(&nest, env); > + cl_env_put(env, &refcheck); > goto out; > > out: > diff --git a/drivers/staging/lustre/lustre/llite/rw26.c b/drivers/staging/lustre/lustre/llite/rw26.c > index 67010be..1a08a9d 100644 > --- a/drivers/staging/lustre/lustre/llite/rw26.c > +++ b/drivers/staging/lustre/lustre/llite/rw26.c > @@ -103,7 +103,6 @@ static void ll_invalidatepage(struct page *vmpage, unsigned int offset, > static int ll_releasepage(struct page *vmpage, gfp_t gfp_mask) > { > struct lu_env *env; > - void *cookie; > struct cl_object *obj; > struct cl_page *page; > struct address_space *mapping; > @@ -129,7 +128,6 @@ static int ll_releasepage(struct page *vmpage, gfp_t gfp_mask) > if (!page) > return 1; > > - cookie = cl_env_reenter(); > env = cl_env_percpu_get(); > LASSERT(!IS_ERR(env)); > > @@ -155,7 +153,6 @@ static int ll_releasepage(struct page *vmpage, gfp_t gfp_mask) > cl_page_put(env, page); > > cl_env_percpu_put(env); > - cl_env_reexit(cookie); > return result; > } > > @@ -340,7 +337,8 @@ static ssize_t ll_direct_IO_26_seg(const struct lu_env *env, struct cl_io *io, > PAGE_SIZE) & ~(DT_MAX_BRW_SIZE - 1)) > static ssize_t ll_direct_IO_26(struct kiocb *iocb, struct iov_iter *iter) > { > - struct lu_env *env; > + struct ll_cl_context *lcc; > + const struct lu_env *env; > struct cl_io *io; > struct file *file = iocb->ki_filp; > struct inode *inode = file->f_mapping->host; > @@ -348,7 +346,6 @@ static ssize_t ll_direct_IO_26(struct kiocb *iocb, struct iov_iter *iter) > ssize_t count = iov_iter_count(iter); > ssize_t tot_bytes = 0, result = 0; > long size = MAX_DIO_SIZE; > - int refcheck; > > /* FIXME: io smaller than PAGE_SIZE is broken on ia64 ??? */ > if ((file_offset & ~PAGE_MASK) || (count & ~PAGE_MASK)) > @@ -363,9 +360,13 @@ static ssize_t ll_direct_IO_26(struct kiocb *iocb, struct iov_iter *iter) > if (iov_iter_alignment(iter) & ~PAGE_MASK) > return -EINVAL; > > - env = cl_env_get(&refcheck); > + lcc = ll_cl_find(file); > + if (!lcc) > + return -EIO; > + > + env = lcc->lcc_env; > LASSERT(!IS_ERR(env)); > - io = vvp_env_io(env)->vui_cl.cis_io; > + io = lcc->lcc_io; > LASSERT(io); > > while (iov_iter_count(iter)) { > @@ -422,7 +423,6 @@ static ssize_t ll_direct_IO_26(struct kiocb *iocb, struct iov_iter *iter) > vio->u.write.vui_written += tot_bytes; > } > > - cl_env_put(env, &refcheck); > return tot_bytes ? tot_bytes : result; > } > > diff --git a/drivers/staging/lustre/lustre/lov/lov_io.c b/drivers/staging/lustre/lustre/lov/lov_io.c > index 011ab0f..002326c 100644 > --- a/drivers/staging/lustre/lustre/lov/lov_io.c > +++ b/drivers/staging/lustre/lustre/lov/lov_io.c > @@ -167,12 +167,7 @@ static int lov_io_sub_init(const struct lu_env *env, struct lov_io *lio, > sub->sub_env = ld->ld_emrg[stripe]->emrg_env; > sub->sub_borrowed = 1; > } else { > - void *cookie; > - > - /* obtain new environment */ > - cookie = cl_env_reenter(); > sub->sub_env = cl_env_get(&sub->sub_refcheck); > - cl_env_reexit(cookie); > if (IS_ERR(sub->sub_env)) > result = PTR_ERR(sub->sub_env); > > diff --git a/drivers/staging/lustre/lustre/lov/lov_object.c b/drivers/staging/lustre/lustre/lov/lov_object.c > index 711a3b9..317311c 100644 > --- a/drivers/staging/lustre/lustre/lov/lov_object.c > +++ b/drivers/staging/lustre/lustre/lov/lov_object.c > @@ -729,19 +729,15 @@ static int lov_layout_change(const struct lu_env *unused, > union lov_layout_state *state = &lov->u; > const struct lov_layout_operations *old_ops; > const struct lov_layout_operations *new_ops; > - void *cookie; > struct lu_env *env; > int refcheck; > int rc; > > LASSERT(0 <= lov->lo_type && lov->lo_type < ARRAY_SIZE(lov_dispatch)); > > - cookie = cl_env_reenter(); > env = cl_env_get(&refcheck); > - if (IS_ERR(env)) { > - cl_env_reexit(cookie); > + if (IS_ERR(env)) > return PTR_ERR(env); > - } > > LASSERT(0 <= llt && llt < ARRAY_SIZE(lov_dispatch)); > > @@ -782,7 +778,6 @@ static int lov_layout_change(const struct lu_env *unused, > lov->lo_type = llt; > out: > cl_env_put(env, &refcheck); > - cl_env_reexit(cookie); > return rc; > } > > diff --git a/drivers/staging/lustre/lustre/obdclass/cl_internal.h b/drivers/staging/lustre/lustre/obdclass/cl_internal.h > index e866754..7b403fb 100644 > --- a/drivers/staging/lustre/lustre/obdclass/cl_internal.h > +++ b/drivers/staging/lustre/lustre/obdclass/cl_internal.h > @@ -50,25 +50,6 @@ enum clt_nesting_level { > }; > > /** > - * Counters used to check correctness of cl_lock interface usage. > - */ > -struct cl_thread_counters { > - /** > - * Number of outstanding calls to cl_lock_mutex_get() made by the > - * current thread. For debugging. > - */ > - int ctc_nr_locks_locked; > - /** List of locked locks. */ > - struct lu_ref ctc_locks_locked; > - /** Number of outstanding holds on locks. */ > - int ctc_nr_held; > - /** Number of outstanding uses on locks. */ > - int ctc_nr_used; > - /** Number of held extent locks. */ > - int ctc_nr_locks_acquired; > -}; > - > -/** > * Thread local state internal for generic cl-code. > */ > struct cl_thread_info { > @@ -83,10 +64,6 @@ struct cl_thread_info { > */ > struct cl_lock_descr clt_descr; > struct cl_page_list clt_list; > - /** > - * Counters for every level of lock nesting. > - */ > - struct cl_thread_counters clt_counters[CNL_NR]; > /** @} debugging */ > > /* > diff --git a/drivers/staging/lustre/lustre/obdclass/cl_io.c b/drivers/staging/lustre/lustre/obdclass/cl_io.c > index c5621ad..af8e4d6 100644 > --- a/drivers/staging/lustre/lustre/obdclass/cl_io.c > +++ b/drivers/staging/lustre/lustre/obdclass/cl_io.c > @@ -412,7 +412,6 @@ void cl_io_unlock(const struct lu_env *env, struct cl_io *io) > scan->cis_iop->op[io->ci_type].cio_unlock(env, scan); > } > io->ci_state = CIS_UNLOCKED; > - LASSERT(!cl_env_info(env)->clt_counters[CNL_TOP].ctc_nr_locks_acquired); > } > EXPORT_SYMBOL(cl_io_unlock); > > diff --git a/drivers/staging/lustre/lustre/obdclass/cl_object.c b/drivers/staging/lustre/lustre/obdclass/cl_object.c > index b4600c4..f5d4e23 100644 > --- a/drivers/staging/lustre/lustre/obdclass/cl_object.c > +++ b/drivers/staging/lustre/lustre/obdclass/cl_object.c > @@ -544,36 +544,20 @@ EXPORT_SYMBOL(cl_site_stats_print); > * bz20044, bz22683. > */ > > -static LIST_HEAD(cl_envs); > -static unsigned int cl_envs_cached_nr; > -static unsigned int cl_envs_cached_max = 128; /* XXX: prototype: arbitrary limit > - * for now. > - */ > -static DEFINE_SPINLOCK(cl_envs_guard); > +static unsigned int cl_envs_cached_max = 32; /* XXX: prototype: arbitrary limit > + * for now. > + */ > +static struct cl_env_cache { > + rwlock_t cec_guard; > + unsigned int cec_count; > + struct list_head cec_envs; > +} *cl_envs = NULL; > > struct cl_env { > void *ce_magic; > struct lu_env ce_lu; > struct lu_context ce_ses; > > - /** > - * This allows cl_env to be entered into cl_env_hash which implements > - * the current thread -> client environment lookup. > - */ > - struct hlist_node ce_node; > - /** > - * Owner for the current cl_env. > - * > - * If LL_TASK_CL_ENV is defined, this point to the owning current, > - * only for debugging purpose ; > - * Otherwise hash is used, and this is the key for cfs_hash. > - * Now current thread pid is stored. Note using thread pointer would > - * lead to unbalanced hash because of its specific allocation locality > - * and could be varied for different platforms and OSes, even different > - * OS versions. > - */ > - void *ce_owner; > - > /* > * Linkage into global list of all client environments. Used for > * garbage collection. > @@ -597,122 +581,13 @@ static void cl_env_init0(struct cl_env *cle, void *debug) > { > LASSERT(cle->ce_ref == 0); > LASSERT(cle->ce_magic == &cl_env_init0); > - LASSERT(!cle->ce_debug && !cle->ce_owner); > + LASSERT(!cle->ce_debug); > > cle->ce_ref = 1; > cle->ce_debug = debug; > CL_ENV_INC(busy); > } > > -/* > - * The implementation of using hash table to connect cl_env and thread > - */ > - > -static struct cfs_hash *cl_env_hash; > - > -static unsigned cl_env_hops_hash(struct cfs_hash *lh, > - const void *key, unsigned mask) > -{ > -#if BITS_PER_LONG == 64 > - return cfs_hash_u64_hash((__u64)key, mask); > -#else > - return cfs_hash_u32_hash((__u32)key, mask); > -#endif > -} > - > -static void *cl_env_hops_obj(struct hlist_node *hn) > -{ > - struct cl_env *cle = hlist_entry(hn, struct cl_env, ce_node); > - > - LASSERT(cle->ce_magic == &cl_env_init0); > - return (void *)cle; > -} > - > -static int cl_env_hops_keycmp(const void *key, struct hlist_node *hn) > -{ > - struct cl_env *cle = cl_env_hops_obj(hn); > - > - LASSERT(cle->ce_owner); > - return (key == cle->ce_owner); > -} > - > -static void cl_env_hops_noop(struct cfs_hash *hs, struct hlist_node *hn) > -{ > - struct cl_env *cle = hlist_entry(hn, struct cl_env, ce_node); > - > - LASSERT(cle->ce_magic == &cl_env_init0); > -} > - > -static struct cfs_hash_ops cl_env_hops = { > - .hs_hash = cl_env_hops_hash, > - .hs_key = cl_env_hops_obj, > - .hs_keycmp = cl_env_hops_keycmp, > - .hs_object = cl_env_hops_obj, > - .hs_get = cl_env_hops_noop, > - .hs_put_locked = cl_env_hops_noop, > -}; > - > -static inline struct cl_env *cl_env_fetch(void) > -{ > - struct cl_env *cle; > - > - cle = cfs_hash_lookup(cl_env_hash, (void *)(long)current->pid); > - LASSERT(ergo(cle, cle->ce_magic == &cl_env_init0)); > - return cle; > -} > - > -static inline void cl_env_attach(struct cl_env *cle) > -{ > - if (cle) { > - int rc; > - > - LASSERT(!cle->ce_owner); > - cle->ce_owner = (void *)(long)current->pid; > - rc = cfs_hash_add_unique(cl_env_hash, cle->ce_owner, > - &cle->ce_node); > - LASSERT(rc == 0); > - } > -} > - > -static inline void cl_env_do_detach(struct cl_env *cle) > -{ > - void *cookie; > - > - LASSERT(cle->ce_owner == (void *)(long)current->pid); > - cookie = cfs_hash_del(cl_env_hash, cle->ce_owner, > - &cle->ce_node); > - LASSERT(cookie == cle); > - cle->ce_owner = NULL; > -} > - > -static int cl_env_store_init(void) > -{ > - cl_env_hash = cfs_hash_create("cl_env", > - HASH_CL_ENV_BITS, HASH_CL_ENV_BITS, > - HASH_CL_ENV_BKT_BITS, 0, > - CFS_HASH_MIN_THETA, > - CFS_HASH_MAX_THETA, > - &cl_env_hops, > - CFS_HASH_RW_BKTLOCK); > - return cl_env_hash ? 0 : -ENOMEM; > -} > - > -static void cl_env_store_fini(void) > -{ > - cfs_hash_putref(cl_env_hash); > -} > - > -static inline struct cl_env *cl_env_detach(struct cl_env *cle) > -{ > - if (!cle) > - cle = cl_env_fetch(); > - > - if (cle && cle->ce_owner) > - cl_env_do_detach(cle); > - > - return cle; > -} > - > static struct lu_env *cl_env_new(__u32 ctx_tags, __u32 ses_tags, void *debug) > { > struct lu_env *env; > @@ -762,16 +637,20 @@ static struct lu_env *cl_env_obtain(void *debug) > { > struct cl_env *cle; > struct lu_env *env; > + int cpu = get_cpu(); > > - spin_lock(&cl_envs_guard); > - LASSERT(equi(cl_envs_cached_nr == 0, list_empty(&cl_envs))); > - if (cl_envs_cached_nr > 0) { > + read_lock(&cl_envs[cpu].cec_guard); > + LASSERT(equi(cl_envs[cpu].cec_count == 0, > + list_empty(&cl_envs[cpu].cec_envs))); > + if (cl_envs[cpu].cec_count > 0) { > int rc; > > - cle = container_of(cl_envs.next, struct cl_env, ce_linkage); > + cle = container_of(cl_envs[cpu].cec_envs.next, struct cl_env, > + ce_linkage); > list_del_init(&cle->ce_linkage); > - cl_envs_cached_nr--; > - spin_unlock(&cl_envs_guard); > + cl_envs[cpu].cec_count--; > + read_unlock(&cl_envs[cpu].cec_guard); > + put_cpu(); > > env = &cle->ce_lu; > rc = lu_env_refill(env); > @@ -784,7 +663,8 @@ static struct lu_env *cl_env_obtain(void *debug) > env = ERR_PTR(rc); > } > } else { > - spin_unlock(&cl_envs_guard); > + read_unlock(&cl_envs[cpu].cec_guard); > + put_cpu(); > env = cl_env_new(lu_context_tags_default, > lu_session_tags_default, debug); > } > @@ -796,27 +676,6 @@ static inline struct cl_env *cl_env_container(struct lu_env *env) > return container_of(env, struct cl_env, ce_lu); > } > > -static struct lu_env *cl_env_peek(int *refcheck) > -{ > - struct lu_env *env; > - struct cl_env *cle; > - > - CL_ENV_INC(lookup); > - > - /* check that we don't go far from untrusted pointer */ > - CLASSERT(offsetof(struct cl_env, ce_magic) == 0); > - > - env = NULL; > - cle = cl_env_fetch(); > - if (cle) { > - CL_ENV_INC(hit); > - env = &cle->ce_lu; > - *refcheck = ++cle->ce_ref; > - } > - CDEBUG(D_OTHER, "%d@%p\n", cle ? cle->ce_ref : 0, cle); > - return env; > -} > - > /** > * Returns lu_env: if there already is an environment associated with the > * current thread, it is returned, otherwise, new environment is allocated. > @@ -834,17 +693,13 @@ struct lu_env *cl_env_get(int *refcheck) > { > struct lu_env *env; > > - env = cl_env_peek(refcheck); > - if (!env) { > - env = cl_env_obtain(__builtin_return_address(0)); > - if (!IS_ERR(env)) { > - struct cl_env *cle; > + env = cl_env_obtain(__builtin_return_address(0)); > + if (!IS_ERR(env)) { > + struct cl_env *cle; > > - cle = cl_env_container(env); > - cl_env_attach(cle); > - *refcheck = cle->ce_ref; > - CDEBUG(D_OTHER, "%d@%p\n", cle->ce_ref, cle); > - } > + cle = cl_env_container(env); > + *refcheck = cle->ce_ref; > + CDEBUG(D_OTHER, "%d@%p\n", cle->ce_ref, cle); > } > return env; > } > @@ -859,7 +714,6 @@ struct lu_env *cl_env_alloc(int *refcheck, __u32 tags) > { > struct lu_env *env; > > - LASSERT(!cl_env_peek(refcheck)); > env = cl_env_new(tags, tags, __builtin_return_address(0)); > if (!IS_ERR(env)) { > struct cl_env *cle; > @@ -874,7 +728,6 @@ EXPORT_SYMBOL(cl_env_alloc); > > static void cl_env_exit(struct cl_env *cle) > { > - LASSERT(!cle->ce_owner); > lu_context_exit(&cle->ce_lu.le_ctx); > lu_context_exit(&cle->ce_ses); > } > @@ -887,20 +740,25 @@ static void cl_env_exit(struct cl_env *cle) > unsigned int cl_env_cache_purge(unsigned int nr) > { > struct cl_env *cle; > + unsigned int i; > > - spin_lock(&cl_envs_guard); > - for (; !list_empty(&cl_envs) && nr > 0; --nr) { > - cle = container_of(cl_envs.next, struct cl_env, ce_linkage); > - list_del_init(&cle->ce_linkage); > - LASSERT(cl_envs_cached_nr > 0); > - cl_envs_cached_nr--; > - spin_unlock(&cl_envs_guard); > + for_each_possible_cpu(i) { > + write_lock(&cl_envs[i].cec_guard); > + for (; !list_empty(&cl_envs[i].cec_envs) && nr > 0; --nr) { > + cle = container_of(cl_envs[i].cec_envs.next, > + struct cl_env, ce_linkage); > + list_del_init(&cle->ce_linkage); > + LASSERT(cl_envs[i].cec_count > 0); > + cl_envs[i].cec_count--; > + write_unlock(&cl_envs[i].cec_guard); > > - cl_env_fini(cle); > - spin_lock(&cl_envs_guard); > + cl_env_fini(cle); > + write_lock(&cl_envs[i].cec_guard); > + } > + LASSERT(equi(cl_envs[i].cec_count == 0, > + list_empty(&cl_envs[i].cec_envs))); > + write_unlock(&cl_envs[i].cec_guard); > } > - LASSERT(equi(cl_envs_cached_nr == 0, list_empty(&cl_envs))); > - spin_unlock(&cl_envs_guard); > return nr; > } > EXPORT_SYMBOL(cl_env_cache_purge); > @@ -923,8 +781,9 @@ void cl_env_put(struct lu_env *env, int *refcheck) > > CDEBUG(D_OTHER, "%d@%p\n", cle->ce_ref, cle); > if (--cle->ce_ref == 0) { > + int cpu = get_cpu(); > + > CL_ENV_DEC(busy); > - cl_env_detach(cle); > cle->ce_debug = NULL; > cl_env_exit(cle); > /* > @@ -933,107 +792,22 @@ void cl_env_put(struct lu_env *env, int *refcheck) > * Return environment to the cache only when it was allocated > * with the standard tags. > */ > - if (cl_envs_cached_nr < cl_envs_cached_max && > + if (cl_envs[cpu].cec_count < cl_envs_cached_max && > (env->le_ctx.lc_tags & ~LCT_HAS_EXIT) == LCT_CL_THREAD && > (env->le_ses->lc_tags & ~LCT_HAS_EXIT) == LCT_SESSION) { > - spin_lock(&cl_envs_guard); > - list_add(&cle->ce_linkage, &cl_envs); > - cl_envs_cached_nr++; > - spin_unlock(&cl_envs_guard); > + read_lock(&cl_envs[cpu].cec_guard); > + list_add(&cle->ce_linkage, &cl_envs[cpu].cec_envs); > + cl_envs[cpu].cec_count++; > + read_unlock(&cl_envs[cpu].cec_guard); > } else { > cl_env_fini(cle); > } > + put_cpu(); > } > } > EXPORT_SYMBOL(cl_env_put); > > /** > - * Declares a point of re-entrancy. > - * > - * \see cl_env_reexit() > - */ > -void *cl_env_reenter(void) > -{ > - return cl_env_detach(NULL); > -} > -EXPORT_SYMBOL(cl_env_reenter); > - > -/** > - * Exits re-entrancy. > - */ > -void cl_env_reexit(void *cookie) > -{ > - cl_env_detach(NULL); > - cl_env_attach(cookie); > -} > -EXPORT_SYMBOL(cl_env_reexit); > - > -/** > - * Setup user-supplied \a env as a current environment. This is to be used to > - * guaranteed that environment exists even when cl_env_get() fails. It is up > - * to user to ensure proper concurrency control. > - * > - * \see cl_env_unplant() > - */ > -void cl_env_implant(struct lu_env *env, int *refcheck) > -{ > - struct cl_env *cle = cl_env_container(env); > - > - LASSERT(cle->ce_ref > 0); > - > - cl_env_attach(cle); > - cl_env_get(refcheck); > - CDEBUG(D_OTHER, "%d@%p\n", cle->ce_ref, cle); > -} > -EXPORT_SYMBOL(cl_env_implant); > - > -/** > - * Detach environment installed earlier by cl_env_implant(). > - */ > -void cl_env_unplant(struct lu_env *env, int *refcheck) > -{ > - struct cl_env *cle = cl_env_container(env); > - > - LASSERT(cle->ce_ref > 1); > - > - CDEBUG(D_OTHER, "%d@%p\n", cle->ce_ref, cle); > - > - cl_env_detach(cle); > - cl_env_put(env, refcheck); > -} > -EXPORT_SYMBOL(cl_env_unplant); > - > -struct lu_env *cl_env_nested_get(struct cl_env_nest *nest) > -{ > - struct lu_env *env; > - > - nest->cen_cookie = NULL; > - env = cl_env_peek(&nest->cen_refcheck); > - if (env) { > - if (!cl_io_is_going(env)) > - return env; > - cl_env_put(env, &nest->cen_refcheck); > - nest->cen_cookie = cl_env_reenter(); > - } > - env = cl_env_get(&nest->cen_refcheck); > - if (IS_ERR(env)) { > - cl_env_reexit(nest->cen_cookie); > - return env; > - } > - > - LASSERT(!cl_io_is_going(env)); > - return env; > -} > -EXPORT_SYMBOL(cl_env_nested_get); > - > -void cl_env_nested_put(struct cl_env_nest *nest, struct lu_env *env) > -{ > - cl_env_put(env, &nest->cen_refcheck); > - cl_env_reexit(nest->cen_cookie); > -} > -EXPORT_SYMBOL(cl_env_nested_put); > - > -/** > * Converts struct ost_lvb to struct cl_attr. > * > * \see cl_attr2lvb > @@ -1060,6 +834,10 @@ static int cl_env_percpu_init(void) > for_each_possible_cpu(i) { > struct lu_env *env; > > + rwlock_init(&cl_envs[i].cec_guard); > + INIT_LIST_HEAD(&cl_envs[i].cec_envs); > + cl_envs[i].cec_count = 0; > + > cle = &cl_env_percpu[i]; > env = &cle->ce_lu; > > @@ -1127,7 +905,6 @@ void cl_env_percpu_put(struct lu_env *env) > LASSERT(cle->ce_ref == 0); > > CL_ENV_DEC(busy); > - cl_env_detach(cle); > cle->ce_debug = NULL; > > put_cpu(); > @@ -1141,7 +918,6 @@ struct lu_env *cl_env_percpu_get(void) > cle = &cl_env_percpu[get_cpu()]; > cl_env_init0(cle, __builtin_return_address(0)); > > - cl_env_attach(cle); > return &cle->ce_lu; > } > EXPORT_SYMBOL(cl_env_percpu_get); > @@ -1205,51 +981,19 @@ LU_KEY_INIT_FINI(cl0, struct cl_thread_info); > static void *cl_key_init(const struct lu_context *ctx, > struct lu_context_key *key) > { > - struct cl_thread_info *info; > - > - info = cl0_key_init(ctx, key); > - if (!IS_ERR(info)) { > - size_t i; > - > - for (i = 0; i < ARRAY_SIZE(info->clt_counters); ++i) > - lu_ref_init(&info->clt_counters[i].ctc_locks_locked); > - } > - return info; > + return cl0_key_init(ctx, key); > } > > static void cl_key_fini(const struct lu_context *ctx, > struct lu_context_key *key, void *data) > { > - struct cl_thread_info *info; > - size_t i; > - > - info = data; > - for (i = 0; i < ARRAY_SIZE(info->clt_counters); ++i) > - lu_ref_fini(&info->clt_counters[i].ctc_locks_locked); > cl0_key_fini(ctx, key, data); > } > > -static void cl_key_exit(const struct lu_context *ctx, > - struct lu_context_key *key, void *data) > -{ > - struct cl_thread_info *info = data; > - size_t i; > - > - for (i = 0; i < ARRAY_SIZE(info->clt_counters); ++i) { > - LASSERT(info->clt_counters[i].ctc_nr_held == 0); > - LASSERT(info->clt_counters[i].ctc_nr_used == 0); > - LASSERT(info->clt_counters[i].ctc_nr_locks_acquired == 0); > - LASSERT(info->clt_counters[i].ctc_nr_locks_locked == 0); > - lu_ref_fini(&info->clt_counters[i].ctc_locks_locked); > - lu_ref_init(&info->clt_counters[i].ctc_locks_locked); > - } > -} > - > static struct lu_context_key cl_key = { > .lct_tags = LCT_CL_THREAD, > .lct_init = cl_key_init, > .lct_fini = cl_key_fini, > - .lct_exit = cl_key_exit > }; > > static struct lu_kmem_descr cl_object_caches[] = { > @@ -1273,13 +1017,15 @@ int cl_global_init(void) > { > int result; > > - result = cl_env_store_init(); > - if (result) > - return result; > + cl_envs = kzalloc(sizeof(*cl_envs) * num_possible_cpus(), GFP_KERNEL); > + if (!cl_envs) { > + result = -ENOMEM; > + goto out; > + } > > result = lu_kmem_init(cl_object_caches); > if (result) > - goto out_store; > + goto out_envs; > > LU_CONTEXT_KEY_INIT(&cl_key); > result = lu_context_key_register(&cl_key); > @@ -1289,16 +1035,17 @@ int cl_global_init(void) > result = cl_env_percpu_init(); > if (result) > /* no cl_env_percpu_fini on error */ > - goto out_context; > + goto out_keys; > > return 0; > > -out_context: > +out_keys: > lu_context_key_degister(&cl_key); > out_kmem: > lu_kmem_fini(cl_object_caches); > -out_store: > - cl_env_store_fini(); > +out_envs: > + kfree(cl_envs); > +out: > return result; > } > > @@ -1310,5 +1057,5 @@ void cl_global_fini(void) > cl_env_percpu_fini(); > lu_context_key_degister(&cl_key); > lu_kmem_fini(cl_object_caches); > - cl_env_store_fini(); > + kfree(cl_envs); > } > diff --git a/drivers/staging/lustre/lustre/obdclass/lu_object.c b/drivers/staging/lustre/lustre/obdclass/lu_object.c > index e031fd2..14ca82d 100644 > --- a/drivers/staging/lustre/lustre/obdclass/lu_object.c > +++ b/drivers/staging/lustre/lustre/obdclass/lu_object.c > @@ -1521,10 +1521,6 @@ void lu_context_key_quiesce(struct lu_context_key *key) > > if (!(key->lct_tags & LCT_QUIESCENT)) { > /* > - * XXX layering violation. > - */ > - cl_env_cache_purge(~0); > - /* > * XXX memory barrier has to go here. > */ > spin_lock(&lu_keys_guard); > diff --git a/drivers/staging/lustre/lustre/obdecho/echo_client.c b/drivers/staging/lustre/lustre/obdecho/echo_client.c > index 30caf5f..25ecfb3 100644 > --- a/drivers/staging/lustre/lustre/obdecho/echo_client.c > +++ b/drivers/staging/lustre/lustre/obdecho/echo_client.c > @@ -563,16 +563,10 @@ static void echo_thread_key_fini(const struct lu_context *ctx, > kmem_cache_free(echo_thread_kmem, info); > } > > -static void echo_thread_key_exit(const struct lu_context *ctx, > - struct lu_context_key *key, void *data) > -{ > -} > - > static struct lu_context_key echo_thread_key = { > .lct_tags = LCT_CL_THREAD, > .lct_init = echo_thread_key_init, > .lct_fini = echo_thread_key_fini, > - .lct_exit = echo_thread_key_exit > }; > > static void *echo_session_key_init(const struct lu_context *ctx, > @@ -594,16 +588,10 @@ static void echo_session_key_fini(const struct lu_context *ctx, > kmem_cache_free(echo_session_kmem, session); > } > > -static void echo_session_key_exit(const struct lu_context *ctx, > - struct lu_context_key *key, void *data) > -{ > -} > - > static struct lu_context_key echo_session_key = { > .lct_tags = LCT_SESSION, > .lct_init = echo_session_key_init, > .lct_fini = echo_session_key_fini, > - .lct_exit = echo_session_key_exit > }; > > LU_TYPE_INIT_FINI(echo, &echo_thread_key, &echo_session_key); > @@ -787,6 +775,8 @@ static struct lu_device *echo_device_free(const struct lu_env *env, > cl_device_fini(&ed->ed_cl); > kfree(ed); > > + cl_env_cache_purge(~0); > + > return NULL; > } > > diff --git a/drivers/staging/lustre/lustre/osc/osc_cache.c b/drivers/staging/lustre/lustre/osc/osc_cache.c > index b645957..cbbe85e 100644 > --- a/drivers/staging/lustre/lustre/osc/osc_cache.c > +++ b/drivers/staging/lustre/lustre/osc/osc_cache.c > @@ -977,7 +977,6 @@ static int osc_extent_wait(const struct lu_env *env, struct osc_extent *ext, > static int osc_extent_truncate(struct osc_extent *ext, pgoff_t trunc_index, > bool partial) > { > - struct cl_env_nest nest; > struct lu_env *env; > struct cl_io *io; > struct osc_object *obj = ext->oe_obj; > @@ -990,6 +989,7 @@ static int osc_extent_truncate(struct osc_extent *ext, pgoff_t trunc_index, > int grants = 0; > int nr_pages = 0; > int rc = 0; > + int refcheck; > > LASSERT(sanity_check(ext) == 0); > EASSERT(ext->oe_state == OES_TRUNC, ext); > @@ -999,7 +999,7 @@ static int osc_extent_truncate(struct osc_extent *ext, pgoff_t trunc_index, > * We can't use that env from osc_cache_truncate_start() because > * it's from lov_io_sub and not fully initialized. > */ > - env = cl_env_nested_get(&nest); > + env = cl_env_get(&refcheck); > io = &osc_env_info(env)->oti_io; > io->ci_obj = cl_object_top(osc2cl(obj)); > rc = cl_io_init(env, io, CIT_MISC, io->ci_obj); > @@ -1085,7 +1085,7 @@ static int osc_extent_truncate(struct osc_extent *ext, pgoff_t trunc_index, > > out: > cl_io_fini(env, io); > - cl_env_nested_put(&nest, env); > + cl_env_put(env, &refcheck); > return rc; > } > > diff --git a/drivers/staging/lustre/lustre/osc/osc_lock.c b/drivers/staging/lustre/lustre/osc/osc_lock.c > index 45d5e6d..e337e87 100644 > --- a/drivers/staging/lustre/lustre/osc/osc_lock.c > +++ b/drivers/staging/lustre/lustre/osc/osc_lock.c > @@ -294,10 +294,10 @@ static int osc_lock_upcall(void *cookie, struct lustre_handle *lockh, > struct osc_lock *oscl = cookie; > struct cl_lock_slice *slice = &oscl->ols_cl; > struct lu_env *env; > - struct cl_env_nest nest; > int rc; > + int refcheck; > > - env = cl_env_nested_get(&nest); > + env = cl_env_get(&refcheck); > /* should never happen, similar to osc_ldlm_blocking_ast(). */ > LASSERT(!IS_ERR(env)); > > @@ -336,7 +336,7 @@ static int osc_lock_upcall(void *cookie, struct lustre_handle *lockh, > > if (oscl->ols_owner) > cl_sync_io_note(env, oscl->ols_owner, rc); > - cl_env_nested_put(&nest, env); > + cl_env_put(env, &refcheck); > > return rc; > } > @@ -347,9 +347,9 @@ static int osc_lock_upcall_agl(void *cookie, struct lustre_handle *lockh, > struct osc_object *osc = cookie; > struct ldlm_lock *dlmlock; > struct lu_env *env; > - struct cl_env_nest nest; > + int refcheck; > > - env = cl_env_nested_get(&nest); > + env = cl_env_get(&refcheck); > LASSERT(!IS_ERR(env)); > > if (errcode == ELDLM_LOCK_MATCHED) { > @@ -374,7 +374,7 @@ static int osc_lock_upcall_agl(void *cookie, struct lustre_handle *lockh, > > out: > cl_object_put(env, osc2cl(osc)); > - cl_env_nested_put(&nest, env); > + cl_env_put(env, &refcheck); > return ldlm_error2errno(errcode); > } > > @@ -382,11 +382,11 @@ static int osc_lock_flush(struct osc_object *obj, pgoff_t start, pgoff_t end, > enum cl_lock_mode mode, int discard) > { > struct lu_env *env; > - struct cl_env_nest nest; > + int refcheck; > int rc = 0; > int rc2 = 0; > > - env = cl_env_nested_get(&nest); > + env = cl_env_get(&refcheck); > if (IS_ERR(env)) > return PTR_ERR(env); > > @@ -404,7 +404,7 @@ static int osc_lock_flush(struct osc_object *obj, pgoff_t start, pgoff_t end, > if (rc == 0 && rc2 < 0) > rc = rc2; > > - cl_env_nested_put(&nest, env); > + cl_env_put(env, &refcheck); > return rc; > } > > @@ -536,7 +536,7 @@ static int osc_ldlm_blocking_ast(struct ldlm_lock *dlmlock, > } > case LDLM_CB_CANCELING: { > struct lu_env *env; > - struct cl_env_nest nest; > + int refcheck; > > /* > * This can be called in the context of outer IO, e.g., > @@ -549,14 +549,14 @@ static int osc_ldlm_blocking_ast(struct ldlm_lock *dlmlock, > * new environment has to be created to not corrupt outer > * context. > */ > - env = cl_env_nested_get(&nest); > + env = cl_env_get(&refcheck); > if (IS_ERR(env)) { > result = PTR_ERR(env); > break; > } > > result = osc_dlm_blocking_ast0(env, dlmlock, data, flag); > - cl_env_nested_put(&nest, env); > + cl_env_put(env, &refcheck); > break; > } > default: > @@ -568,61 +568,61 @@ static int osc_ldlm_blocking_ast(struct ldlm_lock *dlmlock, > static int osc_ldlm_glimpse_ast(struct ldlm_lock *dlmlock, void *data) > { > struct ptlrpc_request *req = data; > - struct cl_env_nest nest; > struct lu_env *env; > struct ost_lvb *lvb; > struct req_capsule *cap; > + struct cl_object *obj = NULL; > int result; > + int refcheck; > > LASSERT(lustre_msg_get_opc(req->rq_reqmsg) == LDLM_GL_CALLBACK); > > - env = cl_env_nested_get(&nest); > - if (!IS_ERR(env)) { > - struct cl_object *obj = NULL; > + env = cl_env_get(&refcheck); > + if (IS_ERR(env)) { > + result = PTR_ERR(env); > + goto out; > + } > > - lock_res_and_lock(dlmlock); > - if (dlmlock->l_ast_data) { > - obj = osc2cl(dlmlock->l_ast_data); > - cl_object_get(obj); > - } > - unlock_res_and_lock(dlmlock); > + lock_res_and_lock(dlmlock); > + if (dlmlock->l_ast_data) { > + obj = osc2cl(dlmlock->l_ast_data); > + cl_object_get(obj); > + } > + unlock_res_and_lock(dlmlock); > > - if (obj) { > - /* Do not grab the mutex of cl_lock for glimpse. > - * See LU-1274 for details. > - * BTW, it's okay for cl_lock to be cancelled during > - * this period because server can handle this race. > - * See ldlm_server_glimpse_ast() for details. > - * cl_lock_mutex_get(env, lock); > - */ > - cap = &req->rq_pill; > - req_capsule_extend(cap, &RQF_LDLM_GL_CALLBACK); > - req_capsule_set_size(cap, &RMF_DLM_LVB, RCL_SERVER, > - sizeof(*lvb)); > - result = req_capsule_server_pack(cap); > - if (result == 0) { > - lvb = req_capsule_server_get(cap, &RMF_DLM_LVB); > - result = cl_object_glimpse(env, obj, lvb); > - } > - if (!exp_connect_lvb_type(req->rq_export)) > - req_capsule_shrink(&req->rq_pill, > - &RMF_DLM_LVB, > - sizeof(struct ost_lvb_v1), > - RCL_SERVER); > - cl_object_put(env, obj); > - } else { > - /* > - * These errors are normal races, so we don't want to > - * fill the console with messages by calling > - * ptlrpc_error() > - */ > - lustre_pack_reply(req, 1, NULL, NULL); > - result = -ELDLM_NO_LOCK_DATA; > + if (obj) { > + /* Do not grab the mutex of cl_lock for glimpse. > + * See LU-1274 for details. > + * BTW, it's okay for cl_lock to be cancelled during > + * this period because server can handle this race. > + * See ldlm_server_glimpse_ast() for details. > + * cl_lock_mutex_get(env, lock); > + */ > + cap = &req->rq_pill; > + req_capsule_extend(cap, &RQF_LDLM_GL_CALLBACK); > + req_capsule_set_size(cap, &RMF_DLM_LVB, RCL_SERVER, > + sizeof(*lvb)); > + result = req_capsule_server_pack(cap); > + if (result == 0) { > + lvb = req_capsule_server_get(cap, &RMF_DLM_LVB); > + result = cl_object_glimpse(env, obj, lvb); > } > - cl_env_nested_put(&nest, env); > + if (!exp_connect_lvb_type(req->rq_export)) > + req_capsule_shrink(&req->rq_pill, &RMF_DLM_LVB, > + sizeof(struct ost_lvb_v1), RCL_SERVER); > + cl_object_put(env, obj); > } else { > - result = PTR_ERR(env); > + /* > + * These errors are normal races, so we don't want to > + * fill the console with messages by calling > + * ptlrpc_error() > + */ > + lustre_pack_reply(req, 1, NULL, NULL); > + result = -ELDLM_NO_LOCK_DATA; > } > + cl_env_put(env, &refcheck); > + > +out: > req->rq_status = result; > return result; > } > @@ -677,12 +677,12 @@ static unsigned long osc_lock_weight(const struct lu_env *env, > */ > unsigned long osc_ldlm_weigh_ast(struct ldlm_lock *dlmlock) > { > - struct cl_env_nest nest; > struct lu_env *env; > struct osc_object *obj; > struct osc_lock *oscl; > unsigned long weight; > bool found = false; > + int refcheck; > > might_sleep(); > /* > @@ -692,7 +692,7 @@ unsigned long osc_ldlm_weigh_ast(struct ldlm_lock *dlmlock) > * the upper context because cl_lock_put don't modify environment > * variables. But just in case .. > */ > - env = cl_env_nested_get(&nest); > + env = cl_env_get(&refcheck); > if (IS_ERR(env)) > /* Mostly because lack of memory, do not eliminate this lock */ > return 1; > @@ -722,7 +722,7 @@ unsigned long osc_ldlm_weigh_ast(struct ldlm_lock *dlmlock) > weight = osc_lock_weight(env, obj, &dlmlock->l_policy_data.l_extent); > > out: > - cl_env_nested_put(&nest, env); > + cl_env_put(env, &refcheck); > return weight; > } > > diff --git a/drivers/staging/lustre/lustre/osc/osc_page.c b/drivers/staging/lustre/lustre/osc/osc_page.c > index 2fb0e53..93248d1 100644 > --- a/drivers/staging/lustre/lustre/osc/osc_page.c > +++ b/drivers/staging/lustre/lustre/osc/osc_page.c > @@ -644,15 +644,15 @@ long osc_lru_shrink(const struct lu_env *env, struct client_obd *cli, > > long osc_lru_reclaim(struct client_obd *cli) > { > - struct cl_env_nest nest; > struct lu_env *env; > struct cl_client_cache *cache = cli->cl_cache; > int max_scans; > + int refcheck; > long rc = 0; > > LASSERT(cache); > > - env = cl_env_nested_get(&nest); > + env = cl_env_get(&refcheck); > if (IS_ERR(env)) > return 0; > > @@ -704,7 +704,7 @@ long osc_lru_reclaim(struct client_obd *cli) > spin_unlock(&cache->ccc_lru_lock); > > out: > - cl_env_nested_put(&nest, env); > + cl_env_put(env, &refcheck); > CDEBUG(D_CACHE, "%s: cli %p freed %ld pages.\n", > cli->cl_import->imp_obd->obd_name, cli, rc); > return rc; > -- > 2.7.4 > > From jsimmons at infradead.org Mon Nov 7 01:53:41 2016 From: jsimmons at infradead.org (James Simmons) Date: Mon, 7 Nov 2016 01:53:41 +0000 (GMT) Subject: [lustre-devel] [PATCH 07/14] staging/lustre/ldlm: Reinstate ldlm_enqueue_pack() In-Reply-To: <1478136304-867780-8-git-send-email-green@linuxhacker.ru> References: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> <1478136304-867780-8-git-send-email-green@linuxhacker.ru> Message-ID: > The function becomes used again with the next patch, so bring it back > from dead, only this time make it static. Reviewed-by: James Simmons > Reverts: bf2a033360f7 ("staging/lustre/ldlm: Remove unused ldlm_enqueue_pack()") > Signed-off-by: Oleg Drokin > --- > drivers/staging/lustre/lustre/ldlm/ldlm_request.c | 21 +++++++++++++++++++++ > 1 file changed, 21 insertions(+) > > diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c > index 6e704c7..1b9ae77 100644 > --- a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c > +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c > @@ -657,6 +657,27 @@ int ldlm_prep_enqueue_req(struct obd_export *exp, struct ptlrpc_request *req, > } > EXPORT_SYMBOL(ldlm_prep_enqueue_req); > > +static struct ptlrpc_request *ldlm_enqueue_pack(struct obd_export *exp, > + int lvb_len) > +{ > + struct ptlrpc_request *req; > + int rc; > + > + req = ptlrpc_request_alloc(class_exp2cliimp(exp), &RQF_LDLM_ENQUEUE); > + if (!req) > + return ERR_PTR(-ENOMEM); > + > + rc = ldlm_prep_enqueue_req(exp, req, NULL, 0); > + if (rc) { > + ptlrpc_request_free(req); > + return ERR_PTR(rc); > + } > + > + req_capsule_set_size(&req->rq_pill, &RMF_DLM_LVB, RCL_SERVER, lvb_len); > + ptlrpc_request_set_replen(req); > + return req; > +} > + > /** > * Client-side lock enqueue. > * > -- > 2.7.4 > > From jsimmons at infradead.org Mon Nov 7 01:54:25 2016 From: jsimmons at infradead.org (James Simmons) Date: Mon, 7 Nov 2016 01:54:25 +0000 (GMT) Subject: [lustre-devel] [PATCH 06/14] staging/lustre/llite: drop_caches hangs in cl_inode_fini() In-Reply-To: <1478136304-867780-7-git-send-email-green@linuxhacker.ru> References: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> <1478136304-867780-7-git-send-email-green@linuxhacker.ru> Message-ID: > From: Andrew Perepechko > > This patch releases cl_pages on error in ll_write_begin() > to avoid memory and object reference leaks. Also, it > reuses per-cpu lu_env in ll_invalidatepage() in the same > way as done in ll_releasepage(). Reviewed-by: James Simmons > Signed-off-by: Andrew Perepechko > Seagate-bug-id: MRP-3504 > Reviewed-on: http://review.whamcloud.com/22745 > Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-8509 > Reviewed-by: Jinshan Xiong > Reviewed-by: Bobi Jam > Signed-off-by: Oleg Drokin > --- > drivers/staging/lustre/lustre/llite/rw26.c | 36 ++++++++++++++++-------------- > 1 file changed, 19 insertions(+), 17 deletions(-) > > diff --git a/drivers/staging/lustre/lustre/llite/rw26.c b/drivers/staging/lustre/lustre/llite/rw26.c > index 1a08a9d..ca45b44 100644 > --- a/drivers/staging/lustre/lustre/llite/rw26.c > +++ b/drivers/staging/lustre/lustre/llite/rw26.c > @@ -71,8 +71,6 @@ static void ll_invalidatepage(struct page *vmpage, unsigned int offset, > struct cl_page *page; > struct cl_object *obj; > > - int refcheck; > - > LASSERT(PageLocked(vmpage)); > LASSERT(!PageWriteback(vmpage)); > > @@ -82,21 +80,21 @@ static void ll_invalidatepage(struct page *vmpage, unsigned int offset, > * happening with locked page too > */ > if (offset == 0 && length == PAGE_SIZE) { > - env = cl_env_get(&refcheck); > - if (!IS_ERR(env)) { > - inode = vmpage->mapping->host; > - obj = ll_i2info(inode)->lli_clob; > - if (obj) { > - page = cl_vmpage_page(vmpage, obj); > - if (page) { > - cl_page_delete(env, page); > - cl_page_put(env, page); > - } > - } else { > - LASSERT(vmpage->private == 0); > + /* See the comment in ll_releasepage() */ > + env = cl_env_percpu_get(); > + LASSERT(!IS_ERR(env)); > + inode = vmpage->mapping->host; > + obj = ll_i2info(inode)->lli_clob; > + if (obj) { > + page = cl_vmpage_page(vmpage, obj); > + if (page) { > + cl_page_delete(env, page); > + cl_page_put(env, page); > } > - cl_env_put(env, &refcheck); > + } else { > + LASSERT(vmpage->private == 0); > } > + cl_env_percpu_put(env); > } > } > > @@ -466,9 +464,9 @@ static int ll_write_begin(struct file *file, struct address_space *mapping, > struct page **pagep, void **fsdata) > { > struct ll_cl_context *lcc; > - const struct lu_env *env; > + const struct lu_env *env = NULL; > struct cl_io *io; > - struct cl_page *page; > + struct cl_page *page = NULL; > struct cl_object *clob = ll_i2info(mapping->host)->lli_clob; > pgoff_t index = pos >> PAGE_SHIFT; > struct page *vmpage = NULL; > @@ -556,6 +554,10 @@ static int ll_write_begin(struct file *file, struct address_space *mapping, > unlock_page(vmpage); > put_page(vmpage); > } > + if (!IS_ERR_OR_NULL(page)) { > + lu_ref_del(&page->cp_reference, "cl_io", io); > + cl_page_put(env, page); > + } > } else { > *pagep = vmpage; > *fsdata = lcc; > -- > 2.7.4 > > From jsimmons at infradead.org Mon Nov 7 01:55:21 2016 From: jsimmons at infradead.org (James Simmons) Date: Mon, 7 Nov 2016 01:55:21 +0000 (GMT) Subject: [lustre-devel] [PATCH 08/14] staging/lustre/ldlm: engage ELC for all ldlm enqueue req In-Reply-To: <1478136304-867780-9-git-send-email-green@linuxhacker.ru> References: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> <1478136304-867780-9-git-send-email-green@linuxhacker.ru> Message-ID: > From: Hongchao Zhang > > If there is no request passed into ldlm_cli_enqueue, the enqueue > request will not engage ELC to drop unneeded locks. currently, > this kind of request is mainly related to EXTENT locks enqueue > requests (except for glimpse EXTENT lock for it has an intent). Reviewed-by: James Simmons > Signed-off-by: Hongchao Zhang > Reviewed-on: http://review.whamcloud.com/21739 > Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-8209 > Reviewed-by: Andreas Dilger > Reviewed-by: Vitaly Fertman > Signed-off-by: Oleg Drokin > --- > drivers/staging/lustre/lustre/ldlm/ldlm_request.c | 21 ++++----------------- > 1 file changed, 4 insertions(+), 17 deletions(-) > > diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c > index 1b9ae77..c5d00d1 100644 > --- a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c > +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c > @@ -748,17 +748,14 @@ int ldlm_cli_enqueue(struct obd_export *exp, struct ptlrpc_request **reqp, > lock->l_last_activity = ktime_get_real_seconds(); > > /* lock not sent to server yet */ > - > if (!reqp || !*reqp) { > - req = ptlrpc_request_alloc_pack(class_exp2cliimp(exp), > - &RQF_LDLM_ENQUEUE, > - LUSTRE_DLM_VERSION, > - LDLM_ENQUEUE); > - if (!req) { > + req = ldlm_enqueue_pack(exp, lvb_len); > + if (IS_ERR(req)) { > failed_lock_cleanup(ns, lock, einfo->ei_mode); > LDLM_LOCK_RELEASE(lock); > - return -ENOMEM; > + return PTR_ERR(req); > } > + > req_passed_in = 0; > if (reqp) > *reqp = req; > @@ -778,16 +775,6 @@ int ldlm_cli_enqueue(struct obd_export *exp, struct ptlrpc_request **reqp, > body->lock_flags = ldlm_flags_to_wire(*flags); > body->lock_handle[0] = *lockh; > > - /* Continue as normal. */ > - if (!req_passed_in) { > - if (lvb_len > 0) > - req_capsule_extend(&req->rq_pill, > - &RQF_LDLM_ENQUEUE_LVB); > - req_capsule_set_size(&req->rq_pill, &RMF_DLM_LVB, RCL_SERVER, > - lvb_len); > - ptlrpc_request_set_replen(req); > - } > - > /* > * Liblustre client doesn't get extent locks, except for O_APPEND case > * where [0, OBD_OBJECT_EOF] lock is taken, or truncate, where > -- > 2.7.4 > > From jsimmons at infradead.org Mon Nov 7 01:55:44 2016 From: jsimmons at infradead.org (James Simmons) Date: Mon, 7 Nov 2016 01:55:44 +0000 (GMT) Subject: [lustre-devel] [PATCH 10/14] staging/lustre/llite: protect from accessing NULL lli_clob In-Reply-To: <1478136304-867780-11-git-send-email-green@linuxhacker.ru> References: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> <1478136304-867780-11-git-send-email-green@linuxhacker.ru> Message-ID: > From: Bobi Jam > > Need to check file's lli_clob object before calling > lov_read_and_clear_async_rc(). Reviewed-by: James Simmons > Signed-off-by: Bobi Jam > Reviewed-by: Jinshan Xiong > Reviewed-by: Oleg Drokin > Reviewed-on: http://review.whamcloud.com/23031 > Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-8682 > Signed-off-by: Oleg Drokin > --- > drivers/staging/lustre/lustre/llite/file.c | 8 +++++--- > 1 file changed, 5 insertions(+), 3 deletions(-) > > diff --git a/drivers/staging/lustre/lustre/llite/file.c b/drivers/staging/lustre/lustre/llite/file.c > index c1c7551..7886840 100644 > --- a/drivers/staging/lustre/lustre/llite/file.c > +++ b/drivers/staging/lustre/lustre/llite/file.c > @@ -2328,9 +2328,11 @@ int ll_fsync(struct file *file, loff_t start, loff_t end, int datasync) > lli->lli_async_rc = 0; > if (rc == 0) > rc = err; > - err = lov_read_and_clear_async_rc(lli->lli_clob); > - if (rc == 0) > - rc = err; > + if (lli->lli_clob) { > + err = lov_read_and_clear_async_rc(lli->lli_clob); > + if (rc == 0) > + rc = err; > + } > } > > err = md_sync(ll_i2sbi(inode)->ll_md_exp, ll_inode2fid(inode), &req); > -- > 2.7.4 > > From jsimmons at infradead.org Mon Nov 7 01:56:28 2016 From: jsimmons at infradead.org (James Simmons) Date: Mon, 7 Nov 2016 01:56:28 +0000 (GMT) Subject: [lustre-devel] [PATCH 14/14] staging/lustre: Get rid of LIBLUSTRE_CLIENT and its users In-Reply-To: <1478136304-867780-15-git-send-email-green@linuxhacker.ru> References: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> <1478136304-867780-15-git-send-email-green@linuxhacker.ru> Message-ID: > This define only made sense in a userspace library client, not in the kernel. Reviewed-by: James Simmons > Signed-off-by: Oleg Drokin > --- > drivers/staging/lustre/lustre/include/lustre_lib.h | 2 -- > drivers/staging/lustre/lustre/ldlm/ldlm_request.c | 15 +-------------- > 2 files changed, 1 insertion(+), 16 deletions(-) > > diff --git a/drivers/staging/lustre/lustre/include/lustre_lib.h b/drivers/staging/lustre/lustre/include/lustre_lib.h > index 6b23191..27f3148 100644 > --- a/drivers/staging/lustre/lustre/include/lustre_lib.h > +++ b/drivers/staging/lustre/lustre/include/lustre_lib.h > @@ -350,8 +350,6 @@ do { \ > l_wait_event_exclusive_head(wq, condition, &lwi); \ > }) > > -#define LIBLUSTRE_CLIENT (0) > - > /** @} lib */ > > #endif /* _LUSTRE_LIB_H */ > diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c > index c5d00d1..6a96f2c 100644 > --- a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c > +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c > @@ -475,12 +475,7 @@ int ldlm_cli_enqueue_fini(struct obd_export *exp, struct ptlrpc_request *req, > "client-side enqueue, new policy data"); > } > > - if ((*flags) & LDLM_FL_AST_SENT || > - /* Cancel extent locks as soon as possible on a liblustre client, > - * because it cannot handle asynchronous ASTs robustly (see > - * bug 7311). > - */ > - (LIBLUSTRE_CLIENT && type == LDLM_EXTENT)) { > + if ((*flags) & LDLM_FL_AST_SENT) { > lock_res_and_lock(lock); > lock->l_flags |= LDLM_FL_CBPENDING | LDLM_FL_BL_AST; > unlock_res_and_lock(lock); > @@ -775,14 +770,6 @@ int ldlm_cli_enqueue(struct obd_export *exp, struct ptlrpc_request **reqp, > body->lock_flags = ldlm_flags_to_wire(*flags); > body->lock_handle[0] = *lockh; > > - /* > - * Liblustre client doesn't get extent locks, except for O_APPEND case > - * where [0, OBD_OBJECT_EOF] lock is taken, or truncate, where > - * [i_size, OBD_OBJECT_EOF] lock is taken. > - */ > - LASSERT(ergo(LIBLUSTRE_CLIENT, einfo->ei_type != LDLM_EXTENT || > - policy->l_extent.end == OBD_OBJECT_EOF)); > - > if (async) { > LASSERT(reqp); > return 0; > -- > 2.7.4 > > From jsimmons at infradead.org Mon Nov 7 01:56:56 2016 From: jsimmons at infradead.org (James Simmons) Date: Mon, 7 Nov 2016 01:56:56 +0000 (GMT) Subject: [lustre-devel] [PATCH 13/14] staging/lustre/llite: do not clear uptodate bit in page delete In-Reply-To: <1478136304-867780-14-git-send-email-green@linuxhacker.ru> References: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> <1478136304-867780-14-git-send-email-green@linuxhacker.ru> Message-ID: > From: Jinshan Xiong > > Otherwise, if the race between page fault and truncate occurs, it > will cause the page fault routine to return an EIO error. > > In filemap_fault() { > page_not_uptodate: > ... > ClearPageError(page); > error = mapping->a_ops->readpage(file, page); > if (!error) { > wait_on_page_locked(page); > if (!PageUptodate(page)) > error = -EIO; > } > ... > } > > However, I tend to think this is a defect in kernel implementation, > because it assumes PageUptodate shouldn't be cleared but file read > routine doesn't make the same assumption. Reviewed-by: James Simmons > Signed-off-by: Jinshan Xiong > Reviewed-on: http://review.whamcloud.com/22827 > Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-8633 > Reviewed-by: Li Dongyang > Reviewed-by: Bobi Jam > Signed-off-by: Oleg Drokin > --- > drivers/staging/lustre/lustre/llite/vvp_page.c | 1 - > 1 file changed, 1 deletion(-) > > diff --git a/drivers/staging/lustre/lustre/llite/vvp_page.c b/drivers/staging/lustre/lustre/llite/vvp_page.c > index 25490a5..23d6630 100644 > --- a/drivers/staging/lustre/lustre/llite/vvp_page.c > +++ b/drivers/staging/lustre/lustre/llite/vvp_page.c > @@ -166,7 +166,6 @@ static void vvp_page_delete(const struct lu_env *env, > refc = atomic_dec_return(&page->cp_ref); > LASSERTF(refc >= 1, "page = %p, refc = %d\n", page, refc); > > - ClearPageUptodate(vmpage); > ClearPagePrivate(vmpage); > vmpage->private = 0; > /* > -- > 2.7.4 > > From jsimmons at infradead.org Mon Nov 7 01:57:21 2016 From: jsimmons at infradead.org (James Simmons) Date: Mon, 7 Nov 2016 01:57:21 +0000 (GMT) Subject: [lustre-devel] [PATCH 09/14] staging/lustre/ptlrpc: Suppress error for flock requests In-Reply-To: <1478136304-867780-10-git-send-email-green@linuxhacker.ru> References: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> <1478136304-867780-10-git-send-email-green@linuxhacker.ru> Message-ID: > From: Patrick Farrell > > -EAGAIN is a normal return when requesting POSIX flocks. > We can't recognize exactly that case here, but it's the > only case that should result in -EAGAIN on LDLM_ENQUEUE, so > don't print to console in that case. Reviewed-by: James Simmons > Signed-off-by: Patrick Farrell > Reviewed-on: http://review.whamcloud.com/22856 > Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-8658 > Reviewed-by: Andreas Dilger > Reviewed-by: Bob Glossman > Signed-off-by: Oleg Drokin > --- > drivers/staging/lustre/lustre/ptlrpc/client.c | 4 +++- > 1 file changed, 3 insertions(+), 1 deletion(-) > > diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c > index 7cbfb4c..bb7ae4e 100644 > --- a/drivers/staging/lustre/lustre/ptlrpc/client.c > +++ b/drivers/staging/lustre/lustre/ptlrpc/client.c > @@ -1191,7 +1191,9 @@ static int ptlrpc_check_status(struct ptlrpc_request *req) > lnet_nid_t nid = imp->imp_connection->c_peer.nid; > __u32 opc = lustre_msg_get_opc(req->rq_reqmsg); > > - if (ptlrpc_console_allow(req)) > + /* -EAGAIN is normal when using POSIX flocks */ > + if (ptlrpc_console_allow(req) && > + !(opc == LDLM_ENQUEUE && err == -EAGAIN)) > LCONSOLE_ERROR_MSG(0x011, "%s: operation %s to node %s failed: rc = %d\n", > imp->imp_obd->obd_name, > ll_opcode2str(opc), > -- > 2.7.4 > > From jsimmons at infradead.org Mon Nov 7 01:57:52 2016 From: jsimmons at infradead.org (James Simmons) Date: Mon, 7 Nov 2016 01:57:52 +0000 (GMT) Subject: [lustre-devel] [PATCH 12/14] staging/lustre/llite: update ras window correctly In-Reply-To: <1478136304-867780-13-git-send-email-green@linuxhacker.ru> References: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> <1478136304-867780-13-git-send-email-green@linuxhacker.ru> Message-ID: > From: Bobi Jam > > When stride-RA hit case miss, we only reset normal sequential > read-ahead window, but not reset the stride IO to avoid the overhead > of re-detecting stride IO. While when the normal RA window is set > to not insect with the stride-RA window, when we try to increase > the stride-RA window length later, the presumption does not hold. > > This patch resets the stride IO as well in this case. Reviewed-by: James Simmons > Signed-off-by: Bobi Jam > Reviewed-on: http://review.whamcloud.com/23032 > Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-8683 > Reviewed-by: wangdi > Reviewed-by: Jinshan Xiong > Signed-off-by: Oleg Drokin > --- > drivers/staging/lustre/lustre/llite/rw.c | 15 +++++++++++---- > 1 file changed, 11 insertions(+), 4 deletions(-) > > diff --git a/drivers/staging/lustre/lustre/llite/rw.c b/drivers/staging/lustre/lustre/llite/rw.c > index d2515a8..e34017d 100644 > --- a/drivers/staging/lustre/lustre/llite/rw.c > +++ b/drivers/staging/lustre/lustre/llite/rw.c > @@ -809,13 +809,20 @@ static void ras_update(struct ll_sb_info *sbi, struct inode *inode, > if (ra_miss) { > if (index_in_stride_window(ras, index) && > stride_io_mode(ras)) { > - /*If stride-RA hit cache miss, the stride dector > - *will not be reset to avoid the overhead of > - *redetecting read-ahead mode > - */ > if (index != ras->ras_last_readpage + 1) > ras->ras_consecutive_pages = 0; > ras_reset(inode, ras, index); > + > + /* If stride-RA hit cache miss, the stride > + * detector will not be reset to avoid the > + * overhead of redetecting read-ahead mode, > + * but on the condition that the stride window > + * is still intersect with normal sequential > + * read-ahead window. > + */ > + if (ras->ras_window_start < > + ras->ras_stride_offset) > + ras_stride_reset(ras); > RAS_CDEBUG(ras); > } else { > /* Reset both stride window and normal RA > -- > 2.7.4 > > From jsimmons at infradead.org Mon Nov 7 01:58:17 2016 From: jsimmons at infradead.org (James Simmons) Date: Mon, 7 Nov 2016 01:58:17 +0000 (GMT) Subject: [lustre-devel] [PATCH 11/14] staging/lustre/ptlrpc: Correctly calculate hrp->hrp_nthrs In-Reply-To: <1478136304-867780-12-git-send-email-green@linuxhacker.ru> References: <1478136304-867780-1-git-send-email-green@linuxhacker.ru> <1478136304-867780-12-git-send-email-green@linuxhacker.ru> Message-ID: On Wed, 2 Nov 2016, Oleg Drokin wrote: > From: Amir Shehata > > cpu_pattern can specify exactly 1 cpu in a partition: > "0[0]". That means CPT0 will have CPU 0. CPU 0 can have > hyperthreading enabled. This combination would result in > > weight = cfs_cpu_ht_nsiblings(0); > hrp->hrp_nthrs = cfs_cpt_weight(ptlrpc_hr.hr_cpt_table, i); > hrp->hrp_nthrs /= weight; > > evaluating to 0. Where > cfs_cpt_weight(ptlrpc_hr.hr_cpt_table, i) == 1 > weight == 2 > > Therefore, if hrp_nthrs becomes zero, just set it to 1. Reviewed-by: James Simmons > Signed-off-by: Amir Shehata > Reviewed-on: http://review.whamcloud.com/19106 > Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-8492 > Reviewed-by: Liang Zhen > Reviewed-by: Doug Oucharek > Reviewed-by: James Simmons > Signed-off-by: Oleg Drokin > --- > drivers/staging/lustre/lustre/ptlrpc/service.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/drivers/staging/lustre/lustre/ptlrpc/service.c b/drivers/staging/lustre/lustre/ptlrpc/service.c > index 72f3930..fc754e7 100644 > --- a/drivers/staging/lustre/lustre/ptlrpc/service.c > +++ b/drivers/staging/lustre/lustre/ptlrpc/service.c > @@ -2541,8 +2541,9 @@ int ptlrpc_hr_init(void) > > hrp->hrp_nthrs = cfs_cpt_weight(ptlrpc_hr.hr_cpt_table, i); > hrp->hrp_nthrs /= weight; > + if (hrp->hrp_nthrs == 0) > + hrp->hrp_nthrs = 1; > > - LASSERT(hrp->hrp_nthrs > 0); > hrp->hrp_thrs = > kzalloc_node(hrp->hrp_nthrs * sizeof(*hrt), GFP_NOFS, > cfs_cpt_spread_node(ptlrpc_hr.hr_cpt_table, > -- > 2.7.4 > > From jsimmons at infradead.org Mon Nov 7 02:01:11 2016 From: jsimmons at infradead.org (James Simmons) Date: Mon, 7 Nov 2016 02:01:11 +0000 (GMT) Subject: [lustre-devel] [PATCH v2] staging: lustre: fixed shadowed variable in socklnd_cb.c In-Reply-To: <1478213386-13853-1-git-send-email-andrew.kanner@gmail.com> References: <1478213386-13853-1-git-send-email-andrew.kanner@gmail.com> Message-ID: > Removed redundant declaration of variable 'tx' in local scope > Fixed: sparse warning: > socklnd_cb.c:2476:41: warning: symbol 'tx' shadows an earlier one > socklnd_cb.c:2435:25: originally declared here Reviewed-by: James Simmons > Signed-off-by: Andrew Kanner > --- > drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c > index c1c6f60..f31f4a1 100644 > --- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c > +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c > @@ -2473,8 +2473,8 @@ ksocknal_check_peer_timeouts(int idx) > * holding only shared lock > */ > if (!list_empty(&peer->ksnp_tx_queue)) { > - struct ksock_tx *tx = list_entry(peer->ksnp_tx_queue.next, > - struct ksock_tx, tx_list); > + tx = list_entry(peer->ksnp_tx_queue.next, > + struct ksock_tx, tx_list); > > if (cfs_time_aftereq(cfs_time_current(), > tx->tx_deadline)) { > -- > 2.1.4 > > From jsimmons at infradead.org Mon Nov 7 02:02:20 2016 From: jsimmons at infradead.org (James Simmons) Date: Mon, 7 Nov 2016 02:02:20 +0000 (GMT) Subject: [lustre-devel] [PATCH] staging: lustre: o2iblnd: replace space indentation with tabs In-Reply-To: <20161105000256.11714-1-nicholasjhanley@gmail.com> References: <20161105000256.11714-1-nicholasjhanley@gmail.com> Message-ID: > This patch fixes all CODE_INDENT checkpatch errors in o2iblnd. Reviewed-by: James Simmons > Signed-off-by: Nicholas Hanley > --- > .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c | 2 +- > .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c | 22 +++++++++++----------- > 2 files changed, 12 insertions(+), 12 deletions(-) > > diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c > index 9e88021..13235b0 100644 > --- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c > +++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c > @@ -1637,7 +1637,7 @@ int kiblnd_fmr_pool_map(struct kib_fmr_poolset *fps, struct kib_tx *tx, > { > __u64 *pages = tx->tx_pages; > bool is_rx = (rd != tx->tx_rd); > - bool tx_pages_mapped = 0; > + bool tx_pages_mapped = 0; > struct kib_fmr_pool *fpo; > int npages = 0; > __u64 version; > diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c > index b27de88..92692a2 100644 > --- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c > +++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c > @@ -1912,12 +1912,12 @@ kiblnd_close_conn_locked(struct kib_conn *conn, int error) > libcfs_nid2str(peer->ibp_nid)); > } else { > CNETERR("Closing conn to %s: error %d%s%s%s%s%s\n", > - libcfs_nid2str(peer->ibp_nid), error, > - list_empty(&conn->ibc_tx_queue) ? "" : "(sending)", > - list_empty(&conn->ibc_tx_noops) ? "" : "(sending_noops)", > - list_empty(&conn->ibc_tx_queue_rsrvd) ? "" : "(sending_rsrvd)", > - list_empty(&conn->ibc_tx_queue_nocred) ? "" : "(sending_nocred)", > - list_empty(&conn->ibc_active_txs) ? "" : "(waiting)"); > + libcfs_nid2str(peer->ibp_nid), error, > + list_empty(&conn->ibc_tx_queue) ? "" : "(sending)", > + list_empty(&conn->ibc_tx_noops) ? "" : "(sending_noops)", > + list_empty(&conn->ibc_tx_queue_rsrvd) ? "" : "(sending_rsrvd)", > + list_empty(&conn->ibc_tx_queue_nocred) ? "" : "(sending_nocred)", > + list_empty(&conn->ibc_active_txs) ? "" : "(waiting)"); > } > > dev = ((struct kib_net *)peer->ibp_ni->ni_data)->ibn_dev; > @@ -2643,7 +2643,7 @@ kiblnd_check_reconnect(struct kib_conn *conn, int version, > if (incarnation) > peer->ibp_incarnation = incarnation; > out: > - write_unlock_irqrestore(glock, flags); > + write_unlock_irqrestore(glock, flags); > > CNETERR("%s: %s (%s), %x, %x, msg_size: %d, queue_depth: %d/%d, max_frags: %d/%d\n", > libcfs_nid2str(peer->ibp_nid), > @@ -2651,7 +2651,7 @@ kiblnd_check_reconnect(struct kib_conn *conn, int version, > reason, IBLND_MSG_VERSION, version, msg_size, > conn->ibc_queue_depth, queue_dep, > conn->ibc_max_frags, frag_num); > - /** > + /** > * if conn::ibc_reconnect is TRUE, connd will reconnect to the peer > * while destroying the zombie > */ > @@ -2976,7 +2976,7 @@ kiblnd_cm_callback(struct rdma_cm_id *cmid, struct rdma_cm_event *event) > case RDMA_CM_EVENT_ADDR_ERROR: > peer = (struct kib_peer *)cmid->context; > CNETERR("%s: ADDR ERROR %d\n", > - libcfs_nid2str(peer->ibp_nid), event->status); > + libcfs_nid2str(peer->ibp_nid), event->status); > kiblnd_peer_connect_failed(peer, 1, -EHOSTUNREACH); > kiblnd_peer_decref(peer); > return -EHOSTUNREACH; /* rc destroys cmid */ > @@ -3021,7 +3021,7 @@ kiblnd_cm_callback(struct rdma_cm_id *cmid, struct rdma_cm_event *event) > return kiblnd_active_connect(cmid); > > CNETERR("Can't resolve route for %s: %d\n", > - libcfs_nid2str(peer->ibp_nid), event->status); > + libcfs_nid2str(peer->ibp_nid), event->status); > kiblnd_peer_connect_failed(peer, 1, event->status); > kiblnd_peer_decref(peer); > return event->status; /* rc destroys cmid */ > @@ -3031,7 +3031,7 @@ kiblnd_cm_callback(struct rdma_cm_id *cmid, struct rdma_cm_event *event) > LASSERT(conn->ibc_state == IBLND_CONN_ACTIVE_CONNECT || > conn->ibc_state == IBLND_CONN_PASSIVE_WAIT); > CNETERR("%s: UNREACHABLE %d\n", > - libcfs_nid2str(conn->ibc_peer->ibp_nid), event->status); > + libcfs_nid2str(conn->ibc_peer->ibp_nid), event->status); > kiblnd_connreq_done(conn, -ENETDOWN); > kiblnd_conn_decref(conn); > return 0; > -- > 2.10.1 > > From jsimmons at infradead.org Mon Nov 7 02:03:01 2016 From: jsimmons at infradead.org (James Simmons) Date: Mon, 7 Nov 2016 02:03:01 +0000 (GMT) Subject: [lustre-devel] Failed to build a Lustre Storage Client on OpenVZ Kernel ! Is this a bug? In-Reply-To: References: Message-ID: > Hello, > > I'm trying to build a Lustre client on OpenVZ kernel (version: 042stab120.5, based on RHEL 6.8 kernel > 2.6.32-642.6.1.el6), and some bugs occured:  > > (1) When build version v2_8_59_0 in git tag repo of Lustre, I encountered the following errors: > " > LD [M] /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/fld/ fld.o > CC [M] /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit e/lloop.o > In file included from /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit e/lloop.c:109: > /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit e/llite_internal.h:1469: error: conflicting types for > 'iov_iter_iovec' > include/linux/fs.h:765: note: previous definition of 'iov_iter_iovec' was here > /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit e/llite_internal.h: In function 'iov_iter_iovec': > /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit e/llite_internal.h:1472: error: 'const struct > iov_iter' has no member named 'iov' > /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit e/llite_internal.h:1473: error: 'const struct > iov_iter' has no member named 'iov' > cc1: warnings being treated as errors > /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit e/llite_internal.h:1473: error: type defaults to 'int' > in declaration of '_min2' > ... > " > > (2) When build old version v2_8_50_0 in git tag repo of Lustre, the following errors display:  > " > CC [M] /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit e/vvp_lock.o > CC [M] /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit e/vvp_io.o > /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit e/vvp_io.c: In function 'vvp_mmap_locks': > /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit e/vvp_io.c:423: error: 'struct iov_iter' has no member > named 'iov' > /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit e/vvp_io.c: In function 'vvp_io_advance': > /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit e/vvp_io.c:514: error: 'struct iov_iter' has no member > named 'iov' > ... > " > > Files with full error context are attached. > > So, How can I fix the problem? Is this a bug which need to be fixed? Can you post your config.log. > Thanks! Any suggestion is appreciated! > > Best Wishes! > > From joe at perches.com Mon Nov 7 02:47:41 2016 From: joe at perches.com (Joe Perches) Date: Sun, 06 Nov 2016 18:47:41 -0800 Subject: [lustre-devel] [PATCH] staging: lustre: o2iblnd: replace space indentation with tabs In-Reply-To: References: <20161105000256.11714-1-nicholasjhanley@gmail.com> Message-ID: <1478486861.13362.2.camel@perches.com> On Mon, 2016-11-07 at 02:02 +0000, James Simmons wrote: > > This patch fixes all CODE_INDENT checkpatch errors in o2iblnd. > Reviewed-by: James Simmons [] > > diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c [] > > @@ -1637,7 +1637,7 @@ int kiblnd_fmr_pool_map(struct kib_fmr_poolset *fps, struct kib_tx *tx, > > { > > __u64 *pages = tx->tx_pages; > > bool is_rx = (rd != tx->tx_rd); > > - bool tx_pages_mapped = 0; > > + bool tx_pages_mapped = 0; It's generally better to use bool assignments to true/false not 1/0. From jsimmons at infradead.org Mon Nov 7 03:47:03 2016 From: jsimmons at infradead.org (James Simmons) Date: Mon, 7 Nov 2016 03:47:03 +0000 (GMT) Subject: [lustre-devel] [PATCH 22/29] staging: lustre: llite: add LL_IOC_FUTIMES_3 In-Reply-To: <20161030145959.GA31097@kroah.com> References: <1477606323-30325-1-git-send-email-jsimmons@infradead.org> <1477606323-30325-23-git-send-email-jsimmons@infradead.org> <20161030145959.GA31097@kroah.com> Message-ID: > On Thu, Oct 27, 2016 at 06:11:56PM -0400, James Simmons wrote: > > From: John L. Hammond > > > > Add a new regular file ioctl LL_IOC_FUTIMES_3 similar to futimes() but > > which allows setting of all three inode timestamps. Use this ioctl > > during HSM restore to ensure that the volatile file has the same > > timestamps as the file to be restored. Strengthen sanity-hsm test_24a > > to check that archive, release, and restore do not change a file's > > ctime. Add sanity-hsm test_24e to check that tar will succeed when it > > encounters a HSM released file. > > This sounds odd, why is this filesystem the only one that needs a > "special" futimes? Don't make up new syscalls by making an ioctl > please, make a new syscall if that's what you really need! This is a special case dealing with file being retrieve from tape backup. The file being restored ended up with the wrong timestamps. This ioctl was used to retrieve the correct timestamps from our meta data servers. Another difference is that our futimes changes all the timestamps. I see the main issue is wrapping such functionality into ioctls. We have the have developed something, ladvise, along the lines of fadvise. We will have to discuss doing it using syscall instead. From jsimmons at infradead.org Mon Nov 7 03:55:36 2016 From: jsimmons at infradead.org (James Simmons) Date: Mon, 7 Nov 2016 03:55:36 +0000 (GMT) Subject: [lustre-devel] [PATCH] staging: lustre: o2iblnd: replace space indentation with tabs In-Reply-To: <1478486861.13362.2.camel@perches.com> References: <20161105000256.11714-1-nicholasjhanley@gmail.com> <1478486861.13362.2.camel@perches.com> Message-ID: > On Mon, 2016-11-07 at 02:02 +0000, James Simmons wrote: > > > This patch fixes all CODE_INDENT checkpatch errors in o2iblnd. > > Reviewed-by: James Simmons > [] > > > diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c > [] > > > @@ -1637,7 +1637,7 @@ int kiblnd_fmr_pool_map(struct kib_fmr_poolset *fps, struct kib_tx *tx, > > > { > > > __u64 *pages = tx->tx_pages; > > > bool is_rx = (rd != tx->tx_rd); > > > - bool tx_pages_mapped = 0; > > > + bool tx_pages_mapped = 0; > > It's generally better to use bool assignments to true/false not 1/0. Nicholas can you resubmit with tx_pages_mapped = false. Looking at the lustre code its the only place like this. Thanks for pointing this out. From jsimmons at infradead.org Mon Nov 7 04:10:16 2016 From: jsimmons at infradead.org (James Simmons) Date: Mon, 7 Nov 2016 04:10:16 +0000 (GMT) Subject: [lustre-devel] [PATCH 2/2] staging: lustre: obdclass: Add handling of error returned by lustre_cfg_new In-Reply-To: <97207CF4-21C1-4351-92BF-6F28E7281CF6@intel.com> References: <20161106171123.24929-1-christophe.jaillet@wanadoo.fr> <97207CF4-21C1-4351-92BF-6F28E7281CF6@intel.com> Message-ID: > On Nov 6, 2016, at 12:11 PM, Christophe JAILLET wrote: > > > 'lustre_cfg_new()' can return ERR_PTR(-ENOMEM). > > Handle these errors and propagate the error code to the callers. > > > > Error handling has been rearranged in 'lustre_process_log()' with the > > addition of a label in order to free some resources. > > I wonder if we should just make it return NULL on allocation failure, > and then at least the other error handling that is there (i.e. in your other patch) > would become correct. > This would make handling in mgc_apply_recover_logs incorrect, but it's already > geared towards this sort of handling anyway, as it discards the passed error > and sets ENOMEM unconditionally (just need to revert 3092c34a in a way). The header lustre_cfg.h is meant to be a UAPI header file. It is used for our userland tools but with the current shape of lustre_cfg.h upstream our tools will not build with it. So having kzalloc and kfree in this header is incorrect. To do this right I need to update our user land tools as well so we should hold off on these patches. From jsimmons at infradead.org Mon Nov 7 04:19:19 2016 From: jsimmons at infradead.org (James Simmons) Date: Mon, 7 Nov 2016 04:19:19 +0000 (GMT) Subject: [lustre-devel] [PATCH 1/2] staging: lustre: replace uses of class_devno_max by MAX_OBD_DEVICES In-Reply-To: References: <4ba436cdccbea6c6dbc631f8cf75cd80f4f9ddb4.1476739569.git.mahfouz.saif.elyazal@gmail.com> <20161025164730.GA27553@waves> <8FDBA22B-F6A6-4CA7-B997-41327603E8A9@intel.com> Message-ID: > On Thu, Nov 3, 2016 at 1:05 AM, Dilger, Andreas wrote: > On Oct 25, 2016, at 10:47, Aya Mahfouz wrote: > > > > On Mon, Oct 17, 2016 at 10:38:31PM +0000, Dilger, Andreas wrote: > >> On Oct 17, 2016, at 15:46, Aya Mahfouz wrote: > >>> > >>> class_devno_max is an inline function that returns > >>> MAX_OBD_DEVICES. Replace all calls to the function > >>> by MAX_OBD_DEVICES. > >> > >> Thanks for your patch, but unfortunately it can't be accepted. > >> > >> This function was added in preparation of being able to tune the maximum > >> number of storage devices dynamically, rather than having to hard code it > >> to the maximum possible number of servers that a client can possibly > >> connect to. > >> > >> While the current maximum of 8192 servers has been enough for current > >> filesystems, I'd rather move in the direction of dynamically handling this > >> limit rather than re-introducing a hard-coded constant throughout the code. > >> > > Hello, > > > > I would like to proceed with implementing the function if possible. > > Kindly direct me to some starting pointers. > > Hi Aya, > thanks for offering to look into this. > > There are several ways to approach this problem  to make the allocation > of the obd_devs[] array dynamic.  In most cases, there isn't any value > to dynamically shrink this array, since the filesystem(s) will typically > be mounted until the node is rebooted, and it is only in the tens of KB > size range, so this will not affect ongoing operations, and that simplifies > the implementation. > > The easiest way would be to have a dynamically-sized obd_devs[] array that > is reallocated in class_newdev() in PAGE_SIZE chunks whenever the current > array has no more free slots and copied to the new array, using obd_dev_lock > to protect the array while it is being reallocated and copied.  In most > cases, this would save memory over the static array (not many filesystems > have so many servers), but for the few sites that have 10000+ servers they > don't need to change the source to handle this.  Using libcfs_kvzalloc() > would avoid issues with allocating large chunks of memory. > > There are a few places where obd_devs[] is accessed outside obd_dev_lock > that would need to be fixed now that this array may be changed at runtime. > > A second approach that may scale better is to change obd_devs from an array > to a doubly linked list (using standard list_head helpers).  In many cases > the whole list is seached linearly, and most of the uses of class_num2obd() > are just used to walk that list in order, which could be replaced with > list_for_each_entry() list traversal.  The class_name2dev() function should > be changed to return the pointer to the obd_device structure, and a new > helper class_dev2num() would just return the obd_minor number from the > obd_device struct for the one use in class_resolve_dev_name().  Using a > linked list has the advantage that there is no need to search for free slots > in the array, since devices would be removed from the list when it is freed. > > Cheers, Andreas > > Thanks Andreas! Will start looking into it. Just to let you know I opened a ticket for you. https://jira.hpdd.intel.com/browse/LU-8802 This way wwe can track the progress and have Lustre developers assigened to look at your work. Thanks for stepping forward. > -- > Kind Regards, > Aya Saif El-yazal Mahfouz >   > >> One comment inline below, if you still want to submit a patch. > >> > >>> Signed-off-by: Aya Mahfouz > >>> --- > >>> drivers/staging/lustre/lustre/obdclass/class_obd.c |  6 +++--- > >>> drivers/staging/lustre/lustre/obdclass/genops.c    | 22 +++++++++++----------- > >>> .../lustre/lustre/obdclass/linux/linux-module.c    |  6 +++--- > >>> 3 files changed, 17 insertions(+), 17 deletions(-) > >>> > >>> diff --git a/drivers/staging/lustre/lustre/obdclass/class_obd.c > b/drivers/staging/lustre/lustre/obdclass/class_obd.c > >>> index 2b21675..b775c74 100644 > >>> --- a/drivers/staging/lustre/lustre/obdclass/class_obd.c > >>> +++ b/drivers/staging/lustre/lustre/obdclass/class_obd.c > >>> @@ -345,7 +345,7 @@ int class_handle_ioctl(unsigned int cmd, unsigned long arg) > >>>                     goto out; > >>>             } > >>>             obd = class_name2obd(data->ioc_inlbuf4); > >>> -   } else if (data->ioc_dev < class_devno_max()) { > >>> +   } else if (data->ioc_dev < MAX_OBD_DEVICES) { > >>>             obd = class_num2obd(data->ioc_dev); > >>>     } else { > >>>             CERROR("OBD ioctl: No device\n"); > >>> @@ -498,7 +498,7 @@ static int __init obdclass_init(void) > >>>     } > >>> > >>>     /* This struct is already zeroed for us (static global) */ > >>> -   for (i = 0; i < class_devno_max(); i++) > >>> +   for (i = 0; i < MAX_OBD_DEVICES; i++) > >>>             obd_devs[i] = NULL; > >> > >> This block can just be removed entirely.  It used to do something useful, > >> but through a series of changes it has become useless. > >> > >> Cheers, Andreas > >> > >>>     /* Default the dirty page cache cap to 1/2 of system memory. > >>> @@ -548,7 +548,7 @@ static void obdclass_exit(void) > >>>     lustre_unregister_fs(); > >>> > >>>     misc_deregister(&obd_psdev); > >>> -   for (i = 0; i < class_devno_max(); i++) { > >>> +   for (i = 0; i < MAX_OBD_DEVICES; i++) { > >>>             struct obd_device *obd = class_num2obd(i); > >>> > >>>             if (obd && obd->obd_set_up && > >>> diff --git a/drivers/staging/lustre/lustre/obdclass/genops.c > b/drivers/staging/lustre/lustre/obdclass/genops.c > >>> index 99c2da6..af4fc58 100644 > >>> --- a/drivers/staging/lustre/lustre/obdclass/genops.c > >>> +++ b/drivers/staging/lustre/lustre/obdclass/genops.c > >>> @@ -290,7 +290,7 @@ struct obd_device *class_newdev(const char *type_name, const char *name) > >>>     LASSERT(newdev->obd_magic == OBD_DEVICE_MAGIC); > >>> > >>>     write_lock(&obd_dev_lock); > >>> -   for (i = 0; i < class_devno_max(); i++) { > >>> +   for (i = 0; i < MAX_OBD_DEVICES; i++) { > >>>             struct obd_device *obd = class_num2obd(i); > >>> > >>>             if (obd && (strcmp(name, obd->obd_name) == 0)) { > >>> @@ -322,9 +322,9 @@ struct obd_device *class_newdev(const char *type_name, const char *name) > >>>     } > >>>     write_unlock(&obd_dev_lock); > >>> > >>> -   if (!result && i >= class_devno_max()) { > >>> +   if (!result && i >= MAX_OBD_DEVICES) { > >>>             CERROR("all %u OBD devices used, increase MAX_OBD_DEVICES\n", > >>> -                  class_devno_max()); > >>> +                  MAX_OBD_DEVICES); > >>>             result = ERR_PTR(-EOVERFLOW); > >>>             goto out; > >>>     } > >>> @@ -372,7 +372,7 @@ int class_name2dev(const char *name) > >>>             return -1; > >>> > >>>     read_lock(&obd_dev_lock); > >>> -   for (i = 0; i < class_devno_max(); i++) { > >>> +   for (i = 0; i < MAX_OBD_DEVICES; i++) { > >>>             struct obd_device *obd = class_num2obd(i); > >>> > >>>             if (obd && strcmp(name, obd->obd_name) == 0) { > >>> @@ -397,7 +397,7 @@ struct obd_device *class_name2obd(const char *name) > >>> { > >>>     int dev = class_name2dev(name); > >>> > >>> -   if (dev < 0 || dev > class_devno_max()) > >>> +   if (dev < 0 || dev > MAX_OBD_DEVICES) > >>>             return NULL; > >>>     return class_num2obd(dev); > >>> } > >>> @@ -408,7 +408,7 @@ int class_uuid2dev(struct obd_uuid *uuid) > >>>     int i; > >>> > >>>     read_lock(&obd_dev_lock); > >>> -   for (i = 0; i < class_devno_max(); i++) { > >>> +   for (i = 0; i < MAX_OBD_DEVICES; i++) { > >>>             struct obd_device *obd = class_num2obd(i); > >>> > >>>             if (obd && obd_uuid_equals(uuid, &obd->obd_uuid)) { > >>> @@ -435,7 +435,7 @@ struct obd_device *class_num2obd(int num) > >>> { > >>>     struct obd_device *obd = NULL; > >>> > >>> -   if (num < class_devno_max()) { > >>> +   if (num < MAX_OBD_DEVICES) { > >>>             obd = obd_devs[num]; > >>>             if (!obd) > >>>                     return NULL; > >>> @@ -463,7 +463,7 @@ struct obd_device *class_find_client_obd(struct obd_uuid *tgt_uuid, > >>>     int i; > >>> > >>>     read_lock(&obd_dev_lock); > >>> -   for (i = 0; i < class_devno_max(); i++) { > >>> +   for (i = 0; i < MAX_OBD_DEVICES; i++) { > >>>             struct obd_device *obd = class_num2obd(i); > >>> > >>>             if (!obd) > >>> @@ -496,13 +496,13 @@ struct obd_device *class_devices_in_group(struct obd_uuid *grp_uuid, int > *next) > >>> > >>>     if (!next) > >>>             i = 0; > >>> -   else if (*next >= 0 && *next < class_devno_max()) > >>> +   else if (*next >= 0 && *next < MAX_OBD_DEVICES) > >>>             i = *next; > >>>     else > >>>             return NULL; > >>> > >>>     read_lock(&obd_dev_lock); > >>> -   for (; i < class_devno_max(); i++) { > >>> +   for (; i < MAX_OBD_DEVICES; i++) { > >>>             struct obd_device *obd = class_num2obd(i); > >>> > >>>             if (!obd) > >>> @@ -533,7 +533,7 @@ int class_notify_sptlrpc_conf(const char *fsname, int namelen) > >>>     LASSERT(namelen > 0); > >>> > >>>     read_lock(&obd_dev_lock); > >>> -   for (i = 0; i < class_devno_max(); i++) { > >>> +   for (i = 0; i < MAX_OBD_DEVICES; i++) { > >>>             obd = class_num2obd(i); > >>> > >>>             if (!obd || obd->obd_set_up == 0 || obd->obd_stopping) > >>> diff --git a/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c > b/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c > >>> index 33342bf..ca5b466 100644 > >>> --- a/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c > >>> +++ b/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c > >>> @@ -228,7 +228,7 @@ static ssize_t health_show(struct kobject *kobj, struct attribute *attr, > >>>             return sprintf(buf, "LBUG\n"); > >>> > >>>     read_lock(&obd_dev_lock); > >>> -   for (i = 0; i < class_devno_max(); i++) { > >>> +   for (i = 0; i < MAX_OBD_DEVICES; i++) { > >>>             struct obd_device *obd; > >>> > >>>             obd = class_num2obd(i); > >>> @@ -326,7 +326,7 @@ static struct attribute *lustre_attrs[] = { > >>> > >>> static void *obd_device_list_seq_start(struct seq_file *p, loff_t *pos) > >>> { > >>> -   if (*pos >= class_devno_max()) > >>> +   if (*pos >= MAX_OBD_DEVICES) > >>>             return NULL; > >>> > >>>     return pos; > >>> @@ -339,7 +339,7 @@ static void obd_device_list_seq_stop(struct seq_file *p, void *v) > >>> static void *obd_device_list_seq_next(struct seq_file *p, void *v, loff_t *pos) > >>> { > >>>     ++*pos; > >>> -   if (*pos >= class_devno_max()) > >>> +   if (*pos >= MAX_OBD_DEVICES) > >>>             return NULL; > >>> > >>>     return pos; > >>> -- > >>> 2.5.0 > >>> > >>> > >>> -- > >>> Kind Regards, > >>> Aya Saif El-yazal Mahfouz > >>> _______________________________________________ > >>> lustre-devel mailing list > >>> lustre-devel at lists.lustre.org > >>> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org > >> > > > > -- > > Kind Regards, > > Aya Saif El-yazal Mahfouz > > _______________________________________________ > > lustre-devel mailing list > > lustre-devel at lists.lustre.org > > http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org > > > > From oleg.drokin at intel.com Mon Nov 7 04:22:00 2016 From: oleg.drokin at intel.com (Oleg Drokin) Date: Sun, 6 Nov 2016 23:22:00 -0500 Subject: [lustre-devel] [PATCH 1/2] staging: lustre: replace uses of class_devno_max by MAX_OBD_DEVICES In-Reply-To: References: <4ba436cdccbea6c6dbc631f8cf75cd80f4f9ddb4.1476739569.git.mahfouz.saif.elyazal@gmail.com> <20161025164730.GA27553@waves> <8FDBA22B-F6A6-4CA7-B997-41327603E8A9@intel.com> Message-ID: <02D3ADEE-88C0-47EE-9A24-1DF770B17893@intel.com> On Nov 4, 2016, at 4:37 AM, Aya Mahfouz wrote: > > On Thu, Nov 3, 2016 at 1:05 AM, Dilger, Andreas wrote: > On Oct 25, 2016, at 10:47, Aya Mahfouz wrote: > > > > On Mon, Oct 17, 2016 at 10:38:31PM +0000, Dilger, Andreas wrote: > >> On Oct 17, 2016, at 15:46, Aya Mahfouz wrote: > >>> > >>> class_devno_max is an inline function that returns > >>> MAX_OBD_DEVICES. Replace all calls to the function > >>> by MAX_OBD_DEVICES. > >> > >> Thanks for your patch, but unfortunately it can't be accepted. > >> > >> This function was added in preparation of being able to tune the maximum > >> number of storage devices dynamically, rather than having to hard code it > >> to the maximum possible number of servers that a client can possibly > >> connect to. > >> > >> While the current maximum of 8192 servers has been enough for current > >> filesystems, I'd rather move in the direction of dynamically handling this > >> limit rather than re-introducing a hard-coded constant throughout the code. > >> > > Hello, > > > > I would like to proceed with implementing the function if possible. > > Kindly direct me to some starting pointers. > > Hi Aya, > thanks for offering to look into this. > > There are several ways to approach this problem to make the allocation > of the obd_devs[] array dynamic. In most cases, there isn't any value > to dynamically shrink this array, since the filesystem(s) will typically > be mounted until the node is rebooted, and it is only in the tens of KB > size range, so this will not affect ongoing operations, and that simplifies > the implementation. > > The easiest way would be to have a dynamically-sized obd_devs[] array that > is reallocated in class_newdev() in PAGE_SIZE chunks whenever the current > array has no more free slots and copied to the new array, using obd_dev_lock > to protect the array while it is being reallocated and copied. In most > cases, this would save memory over the static array (not many filesystems > have so many servers), but for the few sites that have 10000+ servers they > don't need to change the source to handle this. Using libcfs_kvzalloc() > would avoid issues with allocating large chunks of memory. > > There are a few places where obd_devs[] is accessed outside obd_dev_lock > that would need to be fixed now that this array may be changed at runtime. > > A second approach that may scale better is to change obd_devs from an array > to a doubly linked list (using standard list_head helpers). In many cases > the whole list is seached linearly, and most of the uses of class_num2obd() > are just used to walk that list in order, which could be replaced with > list_for_each_entry() list traversal. The class_name2dev() function should > be changed to return the pointer to the obd_device structure, and a new > helper class_dev2num() would just return the obd_minor number from the > obd_device struct for the one use in class_resolve_dev_name(). Using a > linked list has the advantage that there is no need to search for free slots > in the array, since devices would be removed from the list when it is freed. > > Cheers, Andreas > > Thanks Andreas! Will start looking into it. I also would like to point out that Alexey Lyashkov had an implementation of this in http://review.whamcloud.com/347 but it needed to be reverted as it was way too race-prone in the end. I don't know if Alexey ever improved the patch to actually work (at least there was some talk about it), but even if so, the end result was never contributed back to us. Also please be advised that this is the kind of change that you'll need to have fully functional Lustre setup to verify it works, please let me know if you have any problems setting this up. Thanks! > > -- > Kind Regards, > Aya Saif El-yazal Mahfouz > > >> One comment inline below, if you still want to submit a patch. > >> > >>> Signed-off-by: Aya Mahfouz > >>> --- > >>> drivers/staging/lustre/lustre/obdclass/class_obd.c | 6 +++--- > >>> drivers/staging/lustre/lustre/obdclass/genops.c | 22 +++++++++++----------- > >>> .../lustre/lustre/obdclass/linux/linux-module.c | 6 +++--- > >>> 3 files changed, 17 insertions(+), 17 deletions(-) > >>> > >>> diff --git a/drivers/staging/lustre/lustre/obdclass/class_obd.c b/drivers/staging/lustre/lustre/obdclass/class_obd.c > >>> index 2b21675..b775c74 100644 > >>> --- a/drivers/staging/lustre/lustre/obdclass/class_obd.c > >>> +++ b/drivers/staging/lustre/lustre/obdclass/class_obd.c > >>> @@ -345,7 +345,7 @@ int class_handle_ioctl(unsigned int cmd, unsigned long arg) > >>> goto out; > >>> } > >>> obd = class_name2obd(data->ioc_inlbuf4); > >>> - } else if (data->ioc_dev < class_devno_max()) { > >>> + } else if (data->ioc_dev < MAX_OBD_DEVICES) { > >>> obd = class_num2obd(data->ioc_dev); > >>> } else { > >>> CERROR("OBD ioctl: No device\n"); > >>> @@ -498,7 +498,7 @@ static int __init obdclass_init(void) > >>> } > >>> > >>> /* This struct is already zeroed for us (static global) */ > >>> - for (i = 0; i < class_devno_max(); i++) > >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) > >>> obd_devs[i] = NULL; > >> > >> This block can just be removed entirely. It used to do something useful, > >> but through a series of changes it has become useless. > >> > >> Cheers, Andreas > >> > >>> /* Default the dirty page cache cap to 1/2 of system memory. > >>> @@ -548,7 +548,7 @@ static void obdclass_exit(void) > >>> lustre_unregister_fs(); > >>> > >>> misc_deregister(&obd_psdev); > >>> - for (i = 0; i < class_devno_max(); i++) { > >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) { > >>> struct obd_device *obd = class_num2obd(i); > >>> > >>> if (obd && obd->obd_set_up && > >>> diff --git a/drivers/staging/lustre/lustre/obdclass/genops.c b/drivers/staging/lustre/lustre/obdclass/genops.c > >>> index 99c2da6..af4fc58 100644 > >>> --- a/drivers/staging/lustre/lustre/obdclass/genops.c > >>> +++ b/drivers/staging/lustre/lustre/obdclass/genops.c > >>> @@ -290,7 +290,7 @@ struct obd_device *class_newdev(const char *type_name, const char *name) > >>> LASSERT(newdev->obd_magic == OBD_DEVICE_MAGIC); > >>> > >>> write_lock(&obd_dev_lock); > >>> - for (i = 0; i < class_devno_max(); i++) { > >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) { > >>> struct obd_device *obd = class_num2obd(i); > >>> > >>> if (obd && (strcmp(name, obd->obd_name) == 0)) { > >>> @@ -322,9 +322,9 @@ struct obd_device *class_newdev(const char *type_name, const char *name) > >>> } > >>> write_unlock(&obd_dev_lock); > >>> > >>> - if (!result && i >= class_devno_max()) { > >>> + if (!result && i >= MAX_OBD_DEVICES) { > >>> CERROR("all %u OBD devices used, increase MAX_OBD_DEVICES\n", > >>> - class_devno_max()); > >>> + MAX_OBD_DEVICES); > >>> result = ERR_PTR(-EOVERFLOW); > >>> goto out; > >>> } > >>> @@ -372,7 +372,7 @@ int class_name2dev(const char *name) > >>> return -1; > >>> > >>> read_lock(&obd_dev_lock); > >>> - for (i = 0; i < class_devno_max(); i++) { > >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) { > >>> struct obd_device *obd = class_num2obd(i); > >>> > >>> if (obd && strcmp(name, obd->obd_name) == 0) { > >>> @@ -397,7 +397,7 @@ struct obd_device *class_name2obd(const char *name) > >>> { > >>> int dev = class_name2dev(name); > >>> > >>> - if (dev < 0 || dev > class_devno_max()) > >>> + if (dev < 0 || dev > MAX_OBD_DEVICES) > >>> return NULL; > >>> return class_num2obd(dev); > >>> } > >>> @@ -408,7 +408,7 @@ int class_uuid2dev(struct obd_uuid *uuid) > >>> int i; > >>> > >>> read_lock(&obd_dev_lock); > >>> - for (i = 0; i < class_devno_max(); i++) { > >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) { > >>> struct obd_device *obd = class_num2obd(i); > >>> > >>> if (obd && obd_uuid_equals(uuid, &obd->obd_uuid)) { > >>> @@ -435,7 +435,7 @@ struct obd_device *class_num2obd(int num) > >>> { > >>> struct obd_device *obd = NULL; > >>> > >>> - if (num < class_devno_max()) { > >>> + if (num < MAX_OBD_DEVICES) { > >>> obd = obd_devs[num]; > >>> if (!obd) > >>> return NULL; > >>> @@ -463,7 +463,7 @@ struct obd_device *class_find_client_obd(struct obd_uuid *tgt_uuid, > >>> int i; > >>> > >>> read_lock(&obd_dev_lock); > >>> - for (i = 0; i < class_devno_max(); i++) { > >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) { > >>> struct obd_device *obd = class_num2obd(i); > >>> > >>> if (!obd) > >>> @@ -496,13 +496,13 @@ struct obd_device *class_devices_in_group(struct obd_uuid *grp_uuid, int *next) > >>> > >>> if (!next) > >>> i = 0; > >>> - else if (*next >= 0 && *next < class_devno_max()) > >>> + else if (*next >= 0 && *next < MAX_OBD_DEVICES) > >>> i = *next; > >>> else > >>> return NULL; > >>> > >>> read_lock(&obd_dev_lock); > >>> - for (; i < class_devno_max(); i++) { > >>> + for (; i < MAX_OBD_DEVICES; i++) { > >>> struct obd_device *obd = class_num2obd(i); > >>> > >>> if (!obd) > >>> @@ -533,7 +533,7 @@ int class_notify_sptlrpc_conf(const char *fsname, int namelen) > >>> LASSERT(namelen > 0); > >>> > >>> read_lock(&obd_dev_lock); > >>> - for (i = 0; i < class_devno_max(); i++) { > >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) { > >>> obd = class_num2obd(i); > >>> > >>> if (!obd || obd->obd_set_up == 0 || obd->obd_stopping) > >>> diff --git a/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c b/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c > >>> index 33342bf..ca5b466 100644 > >>> --- a/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c > >>> +++ b/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c > >>> @@ -228,7 +228,7 @@ static ssize_t health_show(struct kobject *kobj, struct attribute *attr, > >>> return sprintf(buf, "LBUG\n"); > >>> > >>> read_lock(&obd_dev_lock); > >>> - for (i = 0; i < class_devno_max(); i++) { > >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) { > >>> struct obd_device *obd; > >>> > >>> obd = class_num2obd(i); > >>> @@ -326,7 +326,7 @@ static struct attribute *lustre_attrs[] = { > >>> > >>> static void *obd_device_list_seq_start(struct seq_file *p, loff_t *pos) > >>> { > >>> - if (*pos >= class_devno_max()) > >>> + if (*pos >= MAX_OBD_DEVICES) > >>> return NULL; > >>> > >>> return pos; > >>> @@ -339,7 +339,7 @@ static void obd_device_list_seq_stop(struct seq_file *p, void *v) > >>> static void *obd_device_list_seq_next(struct seq_file *p, void *v, loff_t *pos) > >>> { > >>> ++*pos; > >>> - if (*pos >= class_devno_max()) > >>> + if (*pos >= MAX_OBD_DEVICES) > >>> return NULL; > >>> > >>> return pos; > >>> -- > >>> 2.5.0 > >>> > >>> > >>> -- > >>> Kind Regards, > >>> Aya Saif El-yazal Mahfouz > >>> _______________________________________________ > >>> lustre-devel mailing list > >>> lustre-devel at lists.lustre.org > >>> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org > >> > > > > -- > > Kind Regards, > > Aya Saif El-yazal Mahfouz > > _______________________________________________ > > lustre-devel mailing list > > lustre-devel at lists.lustre.org > > http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org > > > _______________________________________________ > lustre-devel mailing list > lustre-devel at lists.lustre.org > http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org From gregkh at linuxfoundation.org Mon Nov 7 06:43:33 2016 From: gregkh at linuxfoundation.org (Greg KH) Date: Mon, 7 Nov 2016 07:43:33 +0100 Subject: [lustre-devel] [PATCH] staging: lustre: o2iblnd: replace space indentation with tabs In-Reply-To: References: <20161105000256.11714-1-nicholasjhanley@gmail.com> <1478486861.13362.2.camel@perches.com> Message-ID: <20161107064333.GA6336@kroah.com> On Mon, Nov 07, 2016 at 03:55:36AM +0000, James Simmons wrote: > > > On Mon, 2016-11-07 at 02:02 +0000, James Simmons wrote: > > > > This patch fixes all CODE_INDENT checkpatch errors in o2iblnd. > > > Reviewed-by: James Simmons > > [] > > > > diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c > > [] > > > > @@ -1637,7 +1637,7 @@ int kiblnd_fmr_pool_map(struct kib_fmr_poolset *fps, struct kib_tx *tx, > > > > { > > > > __u64 *pages = tx->tx_pages; > > > > bool is_rx = (rd != tx->tx_rd); > > > > - bool tx_pages_mapped = 0; > > > > + bool tx_pages_mapped = 0; > > > > It's generally better to use bool assignments to true/false not 1/0. > > Nicholas can you resubmit with tx_pages_mapped = false. Looking at the > lustre code its the only place like this. Thanks for pointing this out. No, make it a separate patch, don't change code when you are just changing indentation in the same patch. From mahfouz.saif.elyazal at gmail.com Mon Nov 7 09:01:34 2016 From: mahfouz.saif.elyazal at gmail.com (Aya Mahfouz) Date: Mon, 7 Nov 2016 11:01:34 +0200 Subject: [lustre-devel] [PATCH 1/2] staging: lustre: replace uses of class_devno_max by MAX_OBD_DEVICES In-Reply-To: References: <4ba436cdccbea6c6dbc631f8cf75cd80f4f9ddb4.1476739569.git.mahfouz.saif.elyazal@gmail.com> <20161025164730.GA27553@waves> <8FDBA22B-F6A6-4CA7-B997-41327603E8A9@intel.com> Message-ID: On Mon, Nov 7, 2016 at 6:19 AM, James Simmons wrote: > > > On Thu, Nov 3, 2016 at 1:05 AM, Dilger, Andreas < > andreas.dilger at intel.com> wrote: > > On Oct 25, 2016, at 10:47, Aya Mahfouz < > mahfouz.saif.elyazal at gmail.com> wrote: > > > > > > On Mon, Oct 17, 2016 at 10:38:31PM +0000, Dilger, Andreas wrote: > > >> On Oct 17, 2016, at 15:46, Aya Mahfouz < > mahfouz.saif.elyazal at gmail.com> wrote: > > >>> > > >>> class_devno_max is an inline function that returns > > >>> MAX_OBD_DEVICES. Replace all calls to the function > > >>> by MAX_OBD_DEVICES. > > >> > > >> Thanks for your patch, but unfortunately it can't be accepted. > > >> > > >> This function was added in preparation of being able to tune > the maximum > > >> number of storage devices dynamically, rather than having to > hard code it > > >> to the maximum possible number of servers that a client can > possibly > > >> connect to. > > >> > > >> While the current maximum of 8192 servers has been enough for > current > > >> filesystems, I'd rather move in the direction of dynamically > handling this > > >> limit rather than re-introducing a hard-coded constant > throughout the code. > > >> > > > Hello, > > > > > > I would like to proceed with implementing the function if > possible. > > > Kindly direct me to some starting pointers. > > > > Hi Aya, > > thanks for offering to look into this. > > > > There are several ways to approach this problem to make the > allocation > > of the obd_devs[] array dynamic. In most cases, there isn't any > value > > to dynamically shrink this array, since the filesystem(s) will > typically > > be mounted until the node is rebooted, and it is only in the tens > of KB > > size range, so this will not affect ongoing operations, and that > simplifies > > the implementation. > > > > The easiest way would be to have a dynamically-sized obd_devs[] > array that > > is reallocated in class_newdev() in PAGE_SIZE chunks whenever the > current > > array has no more free slots and copied to the new array, using > obd_dev_lock > > to protect the array while it is being reallocated and copied. In > most > > cases, this would save memory over the static array (not many > filesystems > > have so many servers), but for the few sites that have 10000+ > servers they > > don't need to change the source to handle this. Using > libcfs_kvzalloc() > > would avoid issues with allocating large chunks of memory. > > > > There are a few places where obd_devs[] is accessed outside > obd_dev_lock > > that would need to be fixed now that this array may be changed at > runtime. > > > > A second approach that may scale better is to change obd_devs from > an array > > to a doubly linked list (using standard list_head helpers). In > many cases > > the whole list is seached linearly, and most of the uses of > class_num2obd() > > are just used to walk that list in order, which could be replaced > with > > list_for_each_entry() list traversal. The class_name2dev() > function should > > be changed to return the pointer to the obd_device structure, and > a new > > helper class_dev2num() would just return the obd_minor number from > the > > obd_device struct for the one use in class_resolve_dev_name(). > Using a > > linked list has the advantage that there is no need to search for > free slots > > in the array, since devices would be removed from the list when it > is freed. > > > > Cheers, Andreas > > > > Thanks Andreas! Will start looking into it. > > Just to let you know I opened a ticket for you. > > https://jira.hpdd.intel.com/browse/LU-8802 > > This way wwe can track the progress and have Lustre developers assigened > to look at your work. Thanks for stepping forward. > > Thanks, I've created an account: asaifm > > > -- > > Kind Regards, > > Aya Saif El-yazal Mahfouz > > > > >> One comment inline below, if you still want to submit a patch. > > >> > > >>> Signed-off-by: Aya Mahfouz > > >>> --- > > >>> drivers/staging/lustre/lustre/obdclass/class_obd.c | 6 +++--- > > >>> drivers/staging/lustre/lustre/obdclass/genops.c | 22 > +++++++++++----------- > > >>> .../lustre/lustre/obdclass/linux/linux-module.c | 6 +++--- > > >>> 3 files changed, 17 insertions(+), 17 deletions(-) > > >>> > > >>> diff --git a/drivers/staging/lustre/ > lustre/obdclass/class_obd.c > > b/drivers/staging/lustre/lustre/obdclass/class_obd.c > > >>> index 2b21675..b775c74 100644 > > >>> --- a/drivers/staging/lustre/lustre/obdclass/class_obd.c > > >>> +++ b/drivers/staging/lustre/lustre/obdclass/class_obd.c > > >>> @@ -345,7 +345,7 @@ int class_handle_ioctl(unsigned int cmd, > unsigned long arg) > > >>> goto out; > > >>> } > > >>> obd = class_name2obd(data->ioc_inlbuf4); > > >>> - } else if (data->ioc_dev < class_devno_max()) { > > >>> + } else if (data->ioc_dev < MAX_OBD_DEVICES) { > > >>> obd = class_num2obd(data->ioc_dev); > > >>> } else { > > >>> CERROR("OBD ioctl: No device\n"); > > >>> @@ -498,7 +498,7 @@ static int __init obdclass_init(void) > > >>> } > > >>> > > >>> /* This struct is already zeroed for us (static global) */ > > >>> - for (i = 0; i < class_devno_max(); i++) > > >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) > > >>> obd_devs[i] = NULL; > > >> > > >> This block can just be removed entirely. It used to do > something useful, > > >> but through a series of changes it has become useless. > > >> > > >> Cheers, Andreas > > >> > > >>> /* Default the dirty page cache cap to 1/2 of system > memory. > > >>> @@ -548,7 +548,7 @@ static void obdclass_exit(void) > > >>> lustre_unregister_fs(); > > >>> > > >>> misc_deregister(&obd_psdev); > > >>> - for (i = 0; i < class_devno_max(); i++) { > > >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) { > > >>> struct obd_device *obd = class_num2obd(i); > > >>> > > >>> if (obd && obd->obd_set_up && > > >>> diff --git a/drivers/staging/lustre/lustre/obdclass/genops.c > > b/drivers/staging/lustre/lustre/obdclass/genops.c > > >>> index 99c2da6..af4fc58 100644 > > >>> --- a/drivers/staging/lustre/lustre/obdclass/genops.c > > >>> +++ b/drivers/staging/lustre/lustre/obdclass/genops.c > > >>> @@ -290,7 +290,7 @@ struct obd_device *class_newdev(const char > *type_name, const char *name) > > >>> LASSERT(newdev->obd_magic == OBD_DEVICE_MAGIC); > > >>> > > >>> write_lock(&obd_dev_lock); > > >>> - for (i = 0; i < class_devno_max(); i++) { > > >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) { > > >>> struct obd_device *obd = class_num2obd(i); > > >>> > > >>> if (obd && (strcmp(name, obd->obd_name) == 0)) { > > >>> @@ -322,9 +322,9 @@ struct obd_device *class_newdev(const char > *type_name, const char *name) > > >>> } > > >>> write_unlock(&obd_dev_lock); > > >>> > > >>> - if (!result && i >= class_devno_max()) { > > >>> + if (!result && i >= MAX_OBD_DEVICES) { > > >>> CERROR("all %u OBD devices used, increase > MAX_OBD_DEVICES\n", > > >>> - class_devno_max()); > > >>> + MAX_OBD_DEVICES); > > >>> result = ERR_PTR(-EOVERFLOW); > > >>> goto out; > > >>> } > > >>> @@ -372,7 +372,7 @@ int class_name2dev(const char *name) > > >>> return -1; > > >>> > > >>> read_lock(&obd_dev_lock); > > >>> - for (i = 0; i < class_devno_max(); i++) { > > >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) { > > >>> struct obd_device *obd = class_num2obd(i); > > >>> > > >>> if (obd && strcmp(name, obd->obd_name) == 0) { > > >>> @@ -397,7 +397,7 @@ struct obd_device *class_name2obd(const > char *name) > > >>> { > > >>> int dev = class_name2dev(name); > > >>> > > >>> - if (dev < 0 || dev > class_devno_max()) > > >>> + if (dev < 0 || dev > MAX_OBD_DEVICES) > > >>> return NULL; > > >>> return class_num2obd(dev); > > >>> } > > >>> @@ -408,7 +408,7 @@ int class_uuid2dev(struct obd_uuid *uuid) > > >>> int i; > > >>> > > >>> read_lock(&obd_dev_lock); > > >>> - for (i = 0; i < class_devno_max(); i++) { > > >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) { > > >>> struct obd_device *obd = class_num2obd(i); > > >>> > > >>> if (obd && obd_uuid_equals(uuid, &obd->obd_uuid)) { > > >>> @@ -435,7 +435,7 @@ struct obd_device *class_num2obd(int num) > > >>> { > > >>> struct obd_device *obd = NULL; > > >>> > > >>> - if (num < class_devno_max()) { > > >>> + if (num < MAX_OBD_DEVICES) { > > >>> obd = obd_devs[num]; > > >>> if (!obd) > > >>> return NULL; > > >>> @@ -463,7 +463,7 @@ struct obd_device > *class_find_client_obd(struct obd_uuid *tgt_uuid, > > >>> int i; > > >>> > > >>> read_lock(&obd_dev_lock); > > >>> - for (i = 0; i < class_devno_max(); i++) { > > >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) { > > >>> struct obd_device *obd = class_num2obd(i); > > >>> > > >>> if (!obd) > > >>> @@ -496,13 +496,13 @@ struct obd_device > *class_devices_in_group(struct obd_uuid *grp_uuid, int > > *next) > > >>> > > >>> if (!next) > > >>> i = 0; > > >>> - else if (*next >= 0 && *next < class_devno_max()) > > >>> + else if (*next >= 0 && *next < MAX_OBD_DEVICES) > > >>> i = *next; > > >>> else > > >>> return NULL; > > >>> > > >>> read_lock(&obd_dev_lock); > > >>> - for (; i < class_devno_max(); i++) { > > >>> + for (; i < MAX_OBD_DEVICES; i++) { > > >>> struct obd_device *obd = class_num2obd(i); > > >>> > > >>> if (!obd) > > >>> @@ -533,7 +533,7 @@ int class_notify_sptlrpc_conf(const char > *fsname, int namelen) > > >>> LASSERT(namelen > 0); > > >>> > > >>> read_lock(&obd_dev_lock); > > >>> - for (i = 0; i < class_devno_max(); i++) { > > >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) { > > >>> obd = class_num2obd(i); > > >>> > > >>> if (!obd || obd->obd_set_up == 0 || > obd->obd_stopping) > > >>> diff --git a/drivers/staging/lustre/ > lustre/obdclass/linux/linux-module.c > > b/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c > > >>> index 33342bf..ca5b466 100644 > > >>> --- a/drivers/staging/lustre/lustre/obdclass/linux/linux- > module.c > > >>> +++ b/drivers/staging/lustre/lustre/obdclass/linux/linux- > module.c > > >>> @@ -228,7 +228,7 @@ static ssize_t health_show(struct kobject > *kobj, struct attribute *attr, > > >>> return sprintf(buf, "LBUG\n"); > > >>> > > >>> read_lock(&obd_dev_lock); > > >>> - for (i = 0; i < class_devno_max(); i++) { > > >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) { > > >>> struct obd_device *obd; > > >>> > > >>> obd = class_num2obd(i); > > >>> @@ -326,7 +326,7 @@ static struct attribute *lustre_attrs[] = { > > >>> > > >>> static void *obd_device_list_seq_start(struct seq_file *p, > loff_t *pos) > > >>> { > > >>> - if (*pos >= class_devno_max()) > > >>> + if (*pos >= MAX_OBD_DEVICES) > > >>> return NULL; > > >>> > > >>> return pos; > > >>> @@ -339,7 +339,7 @@ static void obd_device_list_seq_stop(struct > seq_file *p, void *v) > > >>> static void *obd_device_list_seq_next(struct seq_file *p, > void *v, loff_t *pos) > > >>> { > > >>> ++*pos; > > >>> - if (*pos >= class_devno_max()) > > >>> + if (*pos >= MAX_OBD_DEVICES) > > >>> return NULL; > > >>> > > >>> return pos; > > >>> -- > > >>> 2.5.0 > > >>> > > >>> > > >>> -- > > >>> Kind Regards, > > >>> Aya Saif El-yazal Mahfouz > > >>> _______________________________________________ > > >>> lustre-devel mailing list > > >>> lustre-devel at lists.lustre.org > > >>> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org > > >> > > > > > > -- > > > Kind Regards, > > > Aya Saif El-yazal Mahfouz > > > _______________________________________________ > > > lustre-devel mailing list > > > lustre-devel at lists.lustre.org > > > http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mahfouz.saif.elyazal at gmail.com Mon Nov 7 09:07:58 2016 From: mahfouz.saif.elyazal at gmail.com (Aya Mahfouz) Date: Mon, 7 Nov 2016 11:07:58 +0200 Subject: [lustre-devel] [PATCH 1/2] staging: lustre: replace uses of class_devno_max by MAX_OBD_DEVICES In-Reply-To: <02D3ADEE-88C0-47EE-9A24-1DF770B17893@intel.com> References: <4ba436cdccbea6c6dbc631f8cf75cd80f4f9ddb4.1476739569.git.mahfouz.saif.elyazal@gmail.com> <20161025164730.GA27553@waves> <8FDBA22B-F6A6-4CA7-B997-41327603E8A9@intel.com> <02D3ADEE-88C0-47EE-9A24-1DF770B17893@intel.com> Message-ID: On Mon, Nov 7, 2016 at 6:22 AM, Oleg Drokin wrote: > > On Nov 4, 2016, at 4:37 AM, Aya Mahfouz wrote: > > > > > On Thu, Nov 3, 2016 at 1:05 AM, Dilger, Andreas < > andreas.dilger at intel.com> wrote: > > On Oct 25, 2016, at 10:47, Aya Mahfouz > wrote: > > > > > > On Mon, Oct 17, 2016 at 10:38:31PM +0000, Dilger, Andreas wrote: > > >> On Oct 17, 2016, at 15:46, Aya Mahfouz com> wrote: > > >>> > > >>> class_devno_max is an inline function that returns > > >>> MAX_OBD_DEVICES. Replace all calls to the function > > >>> by MAX_OBD_DEVICES. > > >> > > >> Thanks for your patch, but unfortunately it can't be accepted. > > >> > > >> This function was added in preparation of being able to tune the > maximum > > >> number of storage devices dynamically, rather than having to hard > code it > > >> to the maximum possible number of servers that a client can possibly > > >> connect to. > > >> > > >> While the current maximum of 8192 servers has been enough for current > > >> filesystems, I'd rather move in the direction of dynamically handling > this > > >> limit rather than re-introducing a hard-coded constant throughout the > code. > > >> > > > Hello, > > > > > > I would like to proceed with implementing the function if possible. > > > Kindly direct me to some starting pointers. > > > > Hi Aya, > > thanks for offering to look into this. > > > > There are several ways to approach this problem to make the allocation > > of the obd_devs[] array dynamic. In most cases, there isn't any value > > to dynamically shrink this array, since the filesystem(s) will typically > > be mounted until the node is rebooted, and it is only in the tens of KB > > size range, so this will not affect ongoing operations, and that > simplifies > > the implementation. > > > > The easiest way would be to have a dynamically-sized obd_devs[] array > that > > is reallocated in class_newdev() in PAGE_SIZE chunks whenever the current > > array has no more free slots and copied to the new array, using > obd_dev_lock > > to protect the array while it is being reallocated and copied. In most > > cases, this would save memory over the static array (not many filesystems > > have so many servers), but for the few sites that have 10000+ servers > they > > don't need to change the source to handle this. Using libcfs_kvzalloc() > > would avoid issues with allocating large chunks of memory. > > > > There are a few places where obd_devs[] is accessed outside obd_dev_lock > > that would need to be fixed now that this array may be changed at > runtime. > > > > A second approach that may scale better is to change obd_devs from an > array > > to a doubly linked list (using standard list_head helpers). In many > cases > > the whole list is seached linearly, and most of the uses of > class_num2obd() > > are just used to walk that list in order, which could be replaced with > > list_for_each_entry() list traversal. The class_name2dev() function > should > > be changed to return the pointer to the obd_device structure, and a new > > helper class_dev2num() would just return the obd_minor number from the > > obd_device struct for the one use in class_resolve_dev_name(). Using a > > linked list has the advantage that there is no need to search for free > slots > > in the array, since devices would be removed from the list when it is > freed. > > > > Cheers, Andreas > > > > Thanks Andreas! Will start looking into it. > > I also would like to point out that Alexey Lyashkov had an implementation > of this in http://review.whamcloud.com/347 but it needed to be reverted > as it was way too race-prone in the end. > I don't know if Alexey ever improved the patch to actually work (at least > there was some talk about it), but even if so, the end result was never > contributed back to us. > > Also please be advised that this is the kind of change that you'll need to > have fully functional Lustre setup to verify it works, > please let me know if you have any problems setting this up. > > Thanks! > I'm currently trying to understand the concerned parts of the lustre code. I will try to setup the lustre file system and take a look at Alexey's implementation this week. Thank you, Aya Saif El-yazal Mahfouz > > > > > -- > > Kind Regards, > > Aya Saif El-yazal Mahfouz > > > > >> One comment inline below, if you still want to submit a patch. > > >> > > >>> Signed-off-by: Aya Mahfouz > > >>> --- > > >>> drivers/staging/lustre/lustre/obdclass/class_obd.c | 6 +++--- > > >>> drivers/staging/lustre/lustre/obdclass/genops.c | 22 > +++++++++++----------- > > >>> .../lustre/lustre/obdclass/linux/linux-module.c | 6 +++--- > > >>> 3 files changed, 17 insertions(+), 17 deletions(-) > > >>> > > >>> diff --git a/drivers/staging/lustre/lustre/obdclass/class_obd.c > b/drivers/staging/lustre/lustre/obdclass/class_obd.c > > >>> index 2b21675..b775c74 100644 > > >>> --- a/drivers/staging/lustre/lustre/obdclass/class_obd.c > > >>> +++ b/drivers/staging/lustre/lustre/obdclass/class_obd.c > > >>> @@ -345,7 +345,7 @@ int class_handle_ioctl(unsigned int cmd, > unsigned long arg) > > >>> goto out; > > >>> } > > >>> obd = class_name2obd(data->ioc_inlbuf4); > > >>> - } else if (data->ioc_dev < class_devno_max()) { > > >>> + } else if (data->ioc_dev < MAX_OBD_DEVICES) { > > >>> obd = class_num2obd(data->ioc_dev); > > >>> } else { > > >>> CERROR("OBD ioctl: No device\n"); > > >>> @@ -498,7 +498,7 @@ static int __init obdclass_init(void) > > >>> } > > >>> > > >>> /* This struct is already zeroed for us (static global) */ > > >>> - for (i = 0; i < class_devno_max(); i++) > > >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) > > >>> obd_devs[i] = NULL; > > >> > > >> This block can just be removed entirely. It used to do something > useful, > > >> but through a series of changes it has become useless. > > >> > > >> Cheers, Andreas > > >> > > >>> /* Default the dirty page cache cap to 1/2 of system memory. > > >>> @@ -548,7 +548,7 @@ static void obdclass_exit(void) > > >>> lustre_unregister_fs(); > > >>> > > >>> misc_deregister(&obd_psdev); > > >>> - for (i = 0; i < class_devno_max(); i++) { > > >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) { > > >>> struct obd_device *obd = class_num2obd(i); > > >>> > > >>> if (obd && obd->obd_set_up && > > >>> diff --git a/drivers/staging/lustre/lustre/obdclass/genops.c > b/drivers/staging/lustre/lustre/obdclass/genops.c > > >>> index 99c2da6..af4fc58 100644 > > >>> --- a/drivers/staging/lustre/lustre/obdclass/genops.c > > >>> +++ b/drivers/staging/lustre/lustre/obdclass/genops.c > > >>> @@ -290,7 +290,7 @@ struct obd_device *class_newdev(const char > *type_name, const char *name) > > >>> LASSERT(newdev->obd_magic == OBD_DEVICE_MAGIC); > > >>> > > >>> write_lock(&obd_dev_lock); > > >>> - for (i = 0; i < class_devno_max(); i++) { > > >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) { > > >>> struct obd_device *obd = class_num2obd(i); > > >>> > > >>> if (obd && (strcmp(name, obd->obd_name) == 0)) { > > >>> @@ -322,9 +322,9 @@ struct obd_device *class_newdev(const char > *type_name, const char *name) > > >>> } > > >>> write_unlock(&obd_dev_lock); > > >>> > > >>> - if (!result && i >= class_devno_max()) { > > >>> + if (!result && i >= MAX_OBD_DEVICES) { > > >>> CERROR("all %u OBD devices used, increase > MAX_OBD_DEVICES\n", > > >>> - class_devno_max()); > > >>> + MAX_OBD_DEVICES); > > >>> result = ERR_PTR(-EOVERFLOW); > > >>> goto out; > > >>> } > > >>> @@ -372,7 +372,7 @@ int class_name2dev(const char *name) > > >>> return -1; > > >>> > > >>> read_lock(&obd_dev_lock); > > >>> - for (i = 0; i < class_devno_max(); i++) { > > >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) { > > >>> struct obd_device *obd = class_num2obd(i); > > >>> > > >>> if (obd && strcmp(name, obd->obd_name) == 0) { > > >>> @@ -397,7 +397,7 @@ struct obd_device *class_name2obd(const char > *name) > > >>> { > > >>> int dev = class_name2dev(name); > > >>> > > >>> - if (dev < 0 || dev > class_devno_max()) > > >>> + if (dev < 0 || dev > MAX_OBD_DEVICES) > > >>> return NULL; > > >>> return class_num2obd(dev); > > >>> } > > >>> @@ -408,7 +408,7 @@ int class_uuid2dev(struct obd_uuid *uuid) > > >>> int i; > > >>> > > >>> read_lock(&obd_dev_lock); > > >>> - for (i = 0; i < class_devno_max(); i++) { > > >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) { > > >>> struct obd_device *obd = class_num2obd(i); > > >>> > > >>> if (obd && obd_uuid_equals(uuid, &obd->obd_uuid)) { > > >>> @@ -435,7 +435,7 @@ struct obd_device *class_num2obd(int num) > > >>> { > > >>> struct obd_device *obd = NULL; > > >>> > > >>> - if (num < class_devno_max()) { > > >>> + if (num < MAX_OBD_DEVICES) { > > >>> obd = obd_devs[num]; > > >>> if (!obd) > > >>> return NULL; > > >>> @@ -463,7 +463,7 @@ struct obd_device *class_find_client_obd(struct > obd_uuid *tgt_uuid, > > >>> int i; > > >>> > > >>> read_lock(&obd_dev_lock); > > >>> - for (i = 0; i < class_devno_max(); i++) { > > >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) { > > >>> struct obd_device *obd = class_num2obd(i); > > >>> > > >>> if (!obd) > > >>> @@ -496,13 +496,13 @@ struct obd_device > *class_devices_in_group(struct obd_uuid *grp_uuid, int *next) > > >>> > > >>> if (!next) > > >>> i = 0; > > >>> - else if (*next >= 0 && *next < class_devno_max()) > > >>> + else if (*next >= 0 && *next < MAX_OBD_DEVICES) > > >>> i = *next; > > >>> else > > >>> return NULL; > > >>> > > >>> read_lock(&obd_dev_lock); > > >>> - for (; i < class_devno_max(); i++) { > > >>> + for (; i < MAX_OBD_DEVICES; i++) { > > >>> struct obd_device *obd = class_num2obd(i); > > >>> > > >>> if (!obd) > > >>> @@ -533,7 +533,7 @@ int class_notify_sptlrpc_conf(const char > *fsname, int namelen) > > >>> LASSERT(namelen > 0); > > >>> > > >>> read_lock(&obd_dev_lock); > > >>> - for (i = 0; i < class_devno_max(); i++) { > > >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) { > > >>> obd = class_num2obd(i); > > >>> > > >>> if (!obd || obd->obd_set_up == 0 || obd->obd_stopping) > > >>> diff --git a/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c > b/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c > > >>> index 33342bf..ca5b466 100644 > > >>> --- a/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c > > >>> +++ b/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c > > >>> @@ -228,7 +228,7 @@ static ssize_t health_show(struct kobject *kobj, > struct attribute *attr, > > >>> return sprintf(buf, "LBUG\n"); > > >>> > > >>> read_lock(&obd_dev_lock); > > >>> - for (i = 0; i < class_devno_max(); i++) { > > >>> + for (i = 0; i < MAX_OBD_DEVICES; i++) { > > >>> struct obd_device *obd; > > >>> > > >>> obd = class_num2obd(i); > > >>> @@ -326,7 +326,7 @@ static struct attribute *lustre_attrs[] = { > > >>> > > >>> static void *obd_device_list_seq_start(struct seq_file *p, loff_t > *pos) > > >>> { > > >>> - if (*pos >= class_devno_max()) > > >>> + if (*pos >= MAX_OBD_DEVICES) > > >>> return NULL; > > >>> > > >>> return pos; > > >>> @@ -339,7 +339,7 @@ static void obd_device_list_seq_stop(struct > seq_file *p, void *v) > > >>> static void *obd_device_list_seq_next(struct seq_file *p, void *v, > loff_t *pos) > > >>> { > > >>> ++*pos; > > >>> - if (*pos >= class_devno_max()) > > >>> + if (*pos >= MAX_OBD_DEVICES) > > >>> return NULL; > > >>> > > >>> return pos; > > >>> -- > > >>> 2.5.0 > > >>> > > >>> > > >>> -- > > >>> Kind Regards, > > >>> Aya Saif El-yazal Mahfouz > > >>> _______________________________________________ > > >>> lustre-devel mailing list > > >>> lustre-devel at lists.lustre.org > > >>> http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org > > >> > > > > > > -- > > > Kind Regards, > > > Aya Saif El-yazal Mahfouz > > > _______________________________________________ > > > lustre-devel mailing list > > > lustre-devel at lists.lustre.org > > > http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org > > > > > > _______________________________________________ > > lustre-devel mailing list > > lustre-devel at lists.lustre.org > > http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gregkh at linuxfoundation.org Mon Nov 7 10:38:32 2016 From: gregkh at linuxfoundation.org (Greg Kroah-Hartman) Date: Mon, 7 Nov 2016 11:38:32 +0100 Subject: [lustre-devel] [PATCH v2] staging: lustre: mdc: manage number of modify RPCs in flight In-Reply-To: <1478112412-8200-1-git-send-email-jsimmons@infradead.org> References: <1478112412-8200-1-git-send-email-jsimmons@infradead.org> Message-ID: <20161107103832.GA19871@kroah.com> On Wed, Nov 02, 2016 at 02:46:52PM -0400, James Simmons wrote: > From: Gregoire Pichon > > This patch is the main client part of a new feature that supports > multiple modify metadata RPCs in parallel. Its goal is to improve > metadata operations performance of a single client, while maintening > the consistency of MDT reply reconstruction and MDT recovery > mechanisms. > > It allows to manage the number of modify RPCs in flight within > the client obd structure and to assign a virtual index (the tag) to > each modify RPC to help server side cleaning of reply data. > > The mdc component uses this feature to send multiple modify RPCs > in parallel. > > Signed-off-by: Gregoire Pichon > Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-5319 > Reviewed-on: http://review.whamcloud.com/14374 > Reviewed-by: Andreas Dilger > Reviewed-by: Alex Zhuravlev > Reviewed-by: Oleg Drokin > Signed-off-by: James Simmons > --- > drivers/staging/lustre/lustre/fid/fid_request.c | 4 - > drivers/staging/lustre/lustre/include/lustre_mdc.h | 24 +++ > drivers/staging/lustre/lustre/include/obd.h | 7 +- > drivers/staging/lustre/lustre/include/obd_class.h | 6 + > drivers/staging/lustre/lustre/ldlm/ldlm_lib.c | 37 +++++ > drivers/staging/lustre/lustre/mdc/lproc_mdc.c | 23 +++ > drivers/staging/lustre/lustre/mdc/mdc_locks.c | 13 +- > drivers/staging/lustre/lustre/mdc/mdc_reint.c | 23 +-- > drivers/staging/lustre/lustre/mdc/mdc_request.c | 50 ++----- > drivers/staging/lustre/lustre/obdclass/genops.c | 171 +++++++++++++++++++- > 10 files changed, 293 insertions(+), 65 deletions(-) What changed from v1 of this patch? You always need to put that below the --- line so we can remember what is going on here... Can you make a v3 with that information please? thanks, greg k-h From gregkh at linuxfoundation.org Mon Nov 7 10:40:30 2016 From: gregkh at linuxfoundation.org (Greg KH) Date: Mon, 7 Nov 2016 11:40:30 +0100 Subject: [lustre-devel] [PATCH 2/2] staging: lustre: obdclass: Add handling of error returned by lustre_cfg_new In-Reply-To: References: <20161106171123.24929-1-christophe.jaillet@wanadoo.fr> <97207CF4-21C1-4351-92BF-6F28E7281CF6@intel.com> Message-ID: <20161107104030.GB19871@kroah.com> On Mon, Nov 07, 2016 at 04:10:16AM +0000, James Simmons wrote: > > > On Nov 6, 2016, at 12:11 PM, Christophe JAILLET wrote: > > > > > 'lustre_cfg_new()' can return ERR_PTR(-ENOMEM). > > > Handle these errors and propagate the error code to the callers. > > > > > > Error handling has been rearranged in 'lustre_process_log()' with the > > > addition of a label in order to free some resources. > > > > I wonder if we should just make it return NULL on allocation failure, > > and then at least the other error handling that is there (i.e. in your other patch) > > would become correct. > > This would make handling in mgc_apply_recover_logs incorrect, but it's already > > geared towards this sort of handling anyway, as it discards the passed error > > and sets ENOMEM unconditionally (just need to revert 3092c34a in a way). > > The header lustre_cfg.h is meant to be a UAPI header file. It is used for > our userland tools but with the current shape of lustre_cfg.h upstream our > tools will not build with it. So having kzalloc and kfree in this header > is incorrect. To do this right I need to update our user land tools as > well so we should hold off on these patches. Ok, but the code as-is today is incorrect, so that should get fixed somehow, soon... thanks, greg k-h From gregkh at linuxfoundation.org Mon Nov 7 11:38:41 2016 From: gregkh at linuxfoundation.org (Greg KH) Date: Mon, 7 Nov 2016 12:38:41 +0100 Subject: [lustre-devel] [PATCH] Staging: lustre: llite: refactor lov_object_fiemap() In-Reply-To: References: Message-ID: <20161107113841.GA12277@kroah.com> On Mon, Nov 07, 2016 at 10:56:19AM +0000, Xu, Bobijam wrote: > * Change loff_t to u64 in lov_object_fiemap() since loff_t is a > signed value type. > * Add fiemap_for_stripe() to get file map extent from each stripe > device. > > Reported-by: Dan Carpenter > Signed-off-by: Bobi Jam Why are you doing two different things in the same patch and also not cc:ing the maintainers of the driver you are modifying? Please break this up and fix the cc: issues. thanks, greg k-h From andreas.dilger at intel.com Mon Nov 7 21:20:30 2016 From: andreas.dilger at intel.com (Dilger, Andreas) Date: Mon, 7 Nov 2016 21:20:30 +0000 Subject: [lustre-devel] [PATCH] staging: lustre: o2iblnd: use bool assignment to true/false In-Reply-To: <20161107190138.5730-1-nicholasjhanley@gmail.com> References: <20161107190138.5730-1-nicholasjhanley@gmail.com> Message-ID: <96662E15-7BBE-41BB-9CEB-07BBA34AA90A@intel.com> On Nov 7, 2016, at 12:01, Nicholas Hanley wrote: > > Replace 0 with false in tx_pages_mapped = 0 to be consistent with > the rest of the lustre code. > > Signed-off-by: Nicholas Hanley Reviewed-by: Andreas Dilger > --- > drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c > index 13235b0..e2fc65f 100644 > --- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c > +++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c > @@ -1637,7 +1637,7 @@ int kiblnd_fmr_pool_map(struct kib_fmr_poolset *fps, struct kib_tx *tx, > { > __u64 *pages = tx->tx_pages; > bool is_rx = (rd != tx->tx_rd); > - bool tx_pages_mapped = 0; > + bool tx_pages_mapped = false; > struct kib_fmr_pool *fpo; > int npages = 0; > __u64 version; > -- > 2.10.2 > From andreas.dilger at intel.com Mon Nov 7 21:33:26 2016 From: andreas.dilger at intel.com (Dilger, Andreas) Date: Mon, 7 Nov 2016 21:33:26 +0000 Subject: [lustre-devel] [PATCH 2/2] staging: lustre: obdclass: Add handling of error returned by lustre_cfg_new In-Reply-To: <97207CF4-21C1-4351-92BF-6F28E7281CF6@intel.com> References: <20161106171123.24929-1-christophe.jaillet@wanadoo.fr> <97207CF4-21C1-4351-92BF-6F28E7281CF6@intel.com> Message-ID: On Nov 6, 2016, at 10:26, Drokin, Oleg wrote: > > Hello! > > On Nov 6, 2016, at 12:11 PM, Christophe JAILLET wrote: > >> 'lustre_cfg_new()' can return ERR_PTR(-ENOMEM). >> Handle these errors and propagate the error code to the callers. >> >> Error handling has been rearranged in 'lustre_process_log()' with the >> addition of a label in order to free some resources. > > I wonder if we should just make it return NULL on allocation failure, > and then at least the other error handling that is there (i.e. in your other patch) > would become correct. > This would make handling in mgc_apply_recover_logs incorrect, but it's already > geared towards this sort of handling anyway, as it discards the passed error > and sets ENOMEM unconditionally (just need to revert 3092c34a in a way). I'd agree with Oleg that returning NULL is the preferable solution here. There are also callers of lustre_cfg_new() in class_config_llog_handler(), do_lcfg(), and lustre_end_log() that do not check error returns at all that should be fixed at the same time. Cheers, Andreas >> >> Signed-off-by: Christophe JAILLET >> --- >> drivers/staging/lustre/lustre/obdclass/obd_mount.c | 16 ++++++++++++++-- >> 1 file changed, 14 insertions(+), 2 deletions(-) >> >> diff --git a/drivers/staging/lustre/lustre/obdclass/obd_mount.c b/drivers/staging/lustre/lustre/obdclass/obd_mount.c >> index 59fbc29aae94..5473615cd338 100644 >> --- a/drivers/staging/lustre/lustre/obdclass/obd_mount.c >> +++ b/drivers/staging/lustre/lustre/obdclass/obd_mount.c >> @@ -89,11 +89,14 @@ int lustre_process_log(struct super_block *sb, char *logname, >> lustre_cfg_bufs_set(bufs, 2, cfg, sizeof(*cfg)); >> lustre_cfg_bufs_set(bufs, 3, &sb, sizeof(sb)); >> lcfg = lustre_cfg_new(LCFG_LOG_START, bufs); >> + if (IS_ERR(lcfg)) { >> + rc = PTR_ERR(lcfg); >> + goto out_free; >> + } >> + >> rc = obd_process_config(mgc, sizeof(*lcfg), lcfg); >> lustre_cfg_free(lcfg); >> >> - kfree(bufs); >> - >> if (rc == -EINVAL) >> LCONSOLE_ERROR_MSG(0x15b, "%s: The configuration from log '%s' failed from the MGS (%d). Make sure this client and the MGS are running compatible versions of Lustre.\n", >> mgc->obd_name, logname, rc); >> @@ -104,6 +107,9 @@ int lustre_process_log(struct super_block *sb, char *logname, >> rc); >> >> /* class_obd_list(); */ >> + >> +out_free: >> + kfree(bufs); >> return rc; >> } >> EXPORT_SYMBOL(lustre_process_log); >> @@ -127,6 +133,9 @@ int lustre_end_log(struct super_block *sb, char *logname, >> if (cfg) >> lustre_cfg_bufs_set(&bufs, 2, cfg, sizeof(*cfg)); >> lcfg = lustre_cfg_new(LCFG_LOG_END, &bufs); >> + if (IS_ERR(lcfg)) >> + return PTR_ERR(lcfg); >> + >> rc = obd_process_config(mgc, sizeof(*lcfg), lcfg); >> lustre_cfg_free(lcfg); >> return rc; >> @@ -159,6 +168,9 @@ static int do_lcfg(char *cfgname, lnet_nid_t nid, int cmd, >> lustre_cfg_bufs_set_string(&bufs, 4, s4); >> >> lcfg = lustre_cfg_new(cmd, &bufs); >> + if (IS_ERR(lcfg)) >> + return PTR_ERR(lcfg); >> + >> lcfg->lcfg_nid = nid; >> rc = class_process_config(lcfg); >> lustre_cfg_free(lcfg); >> -- >> 2.9.3 > From andreas.dilger at intel.com Mon Nov 7 21:50:32 2016 From: andreas.dilger at intel.com (Dilger, Andreas) Date: Mon, 7 Nov 2016 21:50:32 +0000 Subject: [lustre-devel] [PATCH 1/2] staging: lustre: replace uses of class_devno_max by MAX_OBD_DEVICES In-Reply-To: References: <4ba436cdccbea6c6dbc631f8cf75cd80f4f9ddb4.1476739569.git.mahfouz.saif.elyazal@gmail.com> <20161025164730.GA27553@waves> <8FDBA22B-F6A6-4CA7-B997-41327603E8A9@intel.com> <02D3ADEE-88C0-47EE-9A24-1DF770B17893@intel.com> Message-ID: On Nov 7, 2016, at 02:07, Aya Mahfouz wrote: > > On Mon, Nov 7, 2016 at 6:22 AM, Oleg Drokin wrote: > >> On Nov 4, 2016, at 4:37 AM, Aya Mahfouz wrote: >> >> > >> > On Thu, Nov 3, 2016 at 1:05 AM, Dilger, Andreas wrote: >> > On Oct 25, 2016, at 10:47, Aya Mahfouz wrote: >> > > >> > > On Mon, Oct 17, 2016 at 10:38:31PM +0000, Dilger, Andreas wrote: >> > >> On Oct 17, 2016, at 15:46, Aya Mahfouz wrote: >> > >>> >> > >>> class_devno_max is an inline function that returns >> > >>> MAX_OBD_DEVICES. Replace all calls to the function >> > >>> by MAX_OBD_DEVICES. >> > >> >> > >> Thanks for your patch, but unfortunately it can't be accepted. >> > >> >> > >> This function was added in preparation of being able to tune the maximum >> > >> number of storage devices dynamically, rather than having to hard code it >> > >> to the maximum possible number of servers that a client can possibly >> > >> connect to. >> > >> >> > >> While the current maximum of 8192 servers has been enough for current >> > >> filesystems, I'd rather move in the direction of dynamically handling this >> > >> limit rather than re-introducing a hard-coded constant throughout the code. >> > >> >> > > Hello, >> > > >> > > I would like to proceed with implementing the function if possible. >> > > Kindly direct me to some starting pointers. >> > >> > Hi Aya, >> > thanks for offering to look into this. >> > >> > There are several ways to approach this problem to make the allocation >> > of the obd_devs[] array dynamic. In most cases, there isn't any value >> > to dynamically shrink this array, since the filesystem(s) will typically >> > be mounted until the node is rebooted, and it is only in the tens of KB >> > size range, so this will not affect ongoing operations, and that simplifies >> > the implementation. >> > >> > The easiest way would be to have a dynamically-sized obd_devs[] array that >> > is reallocated in class_newdev() in PAGE_SIZE chunks whenever the current >> > array has no more free slots and copied to the new array, using obd_dev_lock >> > to protect the array while it is being reallocated and copied. In most >> > cases, this would save memory over the static array (not many filesystems >> > have so many servers), but for the few sites that have 10000+ servers they >> > don't need to change the source to handle this. Using libcfs_kvzalloc() >> > would avoid issues with allocating large chunks of memory. >> > >> > There are a few places where obd_devs[] is accessed outside obd_dev_lock >> > that would need to be fixed now that this array may be changed at runtime. >> > >> > A second approach that may scale better is to change obd_devs from an array >> > to a doubly linked list (using standard list_head helpers). In many cases >> > the whole list is seached linearly, and most of the uses of class_num2obd() >> > are just used to walk that list in order, which could be replaced with >> > list_for_each_entry() list traversal. The class_name2dev() function should >> > be changed to return the pointer to the obd_device structure, and a new >> > helper class_dev2num() would just return the obd_minor number from the >> > obd_device struct for the one use in class_resolve_dev_name(). Using a >> > linked list has the advantage that there is no need to search for free slots >> > in the array, since devices would be removed from the list when it is freed. >> > >> > Cheers, Andreas >> > >> > Thanks Andreas! Will start looking into it. >> >> I also would like to point out that Alexey Lyashkov had an implementation >> of this in http://review.whamcloud.com/347 but it needed to be reverted >> as it was way too race-prone in the end. >> I don't know if Alexey ever improved the patch to actually work (at least >> there was some talk about it), but even if so, the end result was never >> contributed back to us. >> >> Also please be advised that this is the kind of change that you'll need to >> have fully functional Lustre setup to verify it works, please let me know >> if you have any problems setting this up. >> >> Thanks! > > I'm currently trying to understand the concerned parts of the lustre > code. I will try to setup the lustre file system and take a look at > Alexey's implementation this week. I think it makes sense to split this change into a couple of smaller chunks than what was in the original patch, if the original patch is used at all. The proposals I made above are relatively simple to implement and change the existing code to use the new dynamic array (or list) handling without introducing a lot of complexity. Then, separately, it would be possible to move the code over from linear list walking to hash-based device name/UUID lookups, which is where things got more complex and error-prone. Having the changes done in a series of smaller and easily understood patches makes the code much easier to review, easier to test in case a bug is introduced, and often provides value at each step (i.e. dynamic device count) even if the later patches are held up or rejected for one reason or another. Cheers, Andreas From jsimmons at infradead.org Mon Nov 7 22:23:23 2016 From: jsimmons at infradead.org (James Simmons) Date: Mon, 7 Nov 2016 17:23:23 -0500 Subject: [lustre-devel] [PATCH v3] staging: lustre: mdc: manage number of modify RPCs in flight Message-ID: <1478557403-31072-1-git-send-email-jsimmons@infradead.org> From: Gregoire Pichon This patch is the main client part of a new feature that supports multiple modify metadata RPCs in parallel. Its goal is to improve metadata operations performance of a single client, while maintening the consistency of MDT reply reconstruction and MDT recovery mechanisms. It allows to manage the number of modify RPCs in flight within the client obd structure and to assign a virtual index (the tag) to each modify RPC to help server side cleaning of reply data. The mdc component uses this feature to send multiple modify RPCs in parallel. Changelog: v1) Initial patch with incorrect print out of timestamp in obd_mod_rpc_stats_seq_show. v2) Fixed up obd_mod_rpc_stats_seq_show to print out in nanoseconds v3) Add this changelog Signed-off-by: Gregoire Pichon Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-5319 Reviewed-on: http://review.whamcloud.com/14374 Reviewed-by: Andreas Dilger Reviewed-by: Alex Zhuravlev Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- drivers/staging/lustre/lustre/fid/fid_request.c | 4 - drivers/staging/lustre/lustre/include/lustre_mdc.h | 24 +++ drivers/staging/lustre/lustre/include/obd.h | 7 +- drivers/staging/lustre/lustre/include/obd_class.h | 6 + drivers/staging/lustre/lustre/ldlm/ldlm_lib.c | 37 +++++ drivers/staging/lustre/lustre/mdc/lproc_mdc.c | 23 +++ drivers/staging/lustre/lustre/mdc/mdc_locks.c | 13 +- drivers/staging/lustre/lustre/mdc/mdc_reint.c | 23 +-- drivers/staging/lustre/lustre/mdc/mdc_request.c | 50 ++----- drivers/staging/lustre/lustre/obdclass/genops.c | 171 +++++++++++++++++++- 10 files changed, 293 insertions(+), 65 deletions(-) diff --git a/drivers/staging/lustre/lustre/fid/fid_request.c b/drivers/staging/lustre/lustre/fid/fid_request.c index 1148b9a..999f250 100644 --- a/drivers/staging/lustre/lustre/fid/fid_request.c +++ b/drivers/staging/lustre/lustre/fid/fid_request.c @@ -112,11 +112,7 @@ static int seq_client_rpc(struct lu_client_seq *seq, ptlrpc_at_set_req_timeout(req); - if (opc != SEQ_ALLOC_SUPER && seq->lcs_type == LUSTRE_SEQ_METADATA) - mdc_get_rpc_lock(exp->exp_obd->u.cli.cl_rpc_lock, NULL); rc = ptlrpc_queue_wait(req); - if (opc != SEQ_ALLOC_SUPER && seq->lcs_type == LUSTRE_SEQ_METADATA) - mdc_put_rpc_lock(exp->exp_obd->u.cli.cl_rpc_lock, NULL); if (rc) goto out_req; diff --git a/drivers/staging/lustre/lustre/include/lustre_mdc.h b/drivers/staging/lustre/lustre/include/lustre_mdc.h index 92a5c0f..198ceb0 100644 --- a/drivers/staging/lustre/lustre/include/lustre_mdc.h +++ b/drivers/staging/lustre/lustre/include/lustre_mdc.h @@ -156,6 +156,30 @@ static inline void mdc_put_rpc_lock(struct mdc_rpc_lock *lck, mutex_unlock(&lck->rpcl_mutex); } +static inline void mdc_get_mod_rpc_slot(struct ptlrpc_request *req, + struct lookup_intent *it) +{ + struct client_obd *cli = &req->rq_import->imp_obd->u.cli; + u32 opc; + u16 tag; + + opc = lustre_msg_get_opc(req->rq_reqmsg); + tag = obd_get_mod_rpc_slot(cli, opc, it); + lustre_msg_set_tag(req->rq_reqmsg, tag); +} + +static inline void mdc_put_mod_rpc_slot(struct ptlrpc_request *req, + struct lookup_intent *it) +{ + struct client_obd *cli = &req->rq_import->imp_obd->u.cli; + u32 opc; + u16 tag; + + opc = lustre_msg_get_opc(req->rq_reqmsg); + tag = lustre_msg_get_tag(req->rq_reqmsg); + obd_put_mod_rpc_slot(cli, opc, it, tag); +} + /** * Update the maximum possible easize. * diff --git a/drivers/staging/lustre/lustre/include/obd.h b/drivers/staging/lustre/lustre/include/obd.h index c8a6e23..09e3e71 100644 --- a/drivers/staging/lustre/lustre/include/obd.h +++ b/drivers/staging/lustre/lustre/include/obd.h @@ -263,14 +263,17 @@ struct client_obd { wait_queue_head_t cl_destroy_waitq; struct mdc_rpc_lock *cl_rpc_lock; - struct mdc_rpc_lock *cl_close_lock; /* modify rpcs in flight * currently used for metadata only */ spinlock_t cl_mod_rpcs_lock; u16 cl_max_mod_rpcs_in_flight; - + u16 cl_mod_rpcs_in_flight; + u16 cl_close_rpcs_in_flight; + wait_queue_head_t cl_mod_rpcs_waitq; + unsigned long *cl_mod_tag_bitmap; + struct obd_histogram cl_mod_rpcs_hist; /* mgc datastruct */ atomic_t cl_mgc_refcount; diff --git a/drivers/staging/lustre/lustre/include/obd_class.h b/drivers/staging/lustre/lustre/include/obd_class.h index 70b355e..f79133c 100644 --- a/drivers/staging/lustre/lustre/include/obd_class.h +++ b/drivers/staging/lustre/lustre/include/obd_class.h @@ -101,6 +101,12 @@ struct obd_device *class_devices_in_group(struct obd_uuid *grp_uuid, __u32 obd_get_max_rpcs_in_flight(struct client_obd *cli); int obd_set_max_rpcs_in_flight(struct client_obd *cli, __u32 max); int obd_set_max_mod_rpcs_in_flight(struct client_obd *cli, u16 max); +int obd_mod_rpc_stats_seq_show(struct client_obd *cli, struct seq_file *seq); + +u16 obd_get_mod_rpc_slot(struct client_obd *cli, u32 opc, + struct lookup_intent *it); +void obd_put_mod_rpc_slot(struct client_obd *cli, u32 opc, + struct lookup_intent *it, u16 tag); struct llog_handle; struct llog_rec_hdr; diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c index 4f9480e..ee4bb56 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c @@ -372,6 +372,25 @@ int client_obd_setup(struct obd_device *obddev, struct lustre_cfg *lcfg) } else { cli->cl_max_rpcs_in_flight = OBD_MAX_RIF_DEFAULT; } + + spin_lock_init(&cli->cl_mod_rpcs_lock); + spin_lock_init(&cli->cl_mod_rpcs_hist.oh_lock); + cli->cl_max_mod_rpcs_in_flight = 0; + cli->cl_mod_rpcs_in_flight = 0; + cli->cl_close_rpcs_in_flight = 0; + init_waitqueue_head(&cli->cl_mod_rpcs_waitq); + cli->cl_mod_tag_bitmap = NULL; + + if (connect_op == MDS_CONNECT) { + cli->cl_max_mod_rpcs_in_flight = cli->cl_max_rpcs_in_flight - 1; + cli->cl_mod_tag_bitmap = kcalloc(BITS_TO_LONGS(OBD_MAX_RIF_MAX), + sizeof(long), GFP_NOFS); + if (!cli->cl_mod_tag_bitmap) { + rc = -ENOMEM; + goto err; + } + } + rc = ldlm_get_ref(); if (rc) { CERROR("ldlm_get_ref failed: %d\n", rc); @@ -431,12 +450,16 @@ int client_obd_setup(struct obd_device *obddev, struct lustre_cfg *lcfg) err_ldlm: ldlm_put_ref(); err: + kfree(cli->cl_mod_tag_bitmap); + cli->cl_mod_tag_bitmap = NULL; return rc; } EXPORT_SYMBOL(client_obd_setup); int client_obd_cleanup(struct obd_device *obddev) { + struct client_obd *cli = &obddev->u.cli; + ldlm_namespace_free_post(obddev->obd_namespace); obddev->obd_namespace = NULL; @@ -444,6 +467,10 @@ int client_obd_cleanup(struct obd_device *obddev) LASSERT(!obddev->u.cli.cl_import); ldlm_put_ref(); + + kfree(cli->cl_mod_tag_bitmap); + cli->cl_mod_tag_bitmap = NULL; + return 0; } EXPORT_SYMBOL(client_obd_cleanup); @@ -458,6 +485,7 @@ int client_connect_import(const struct lu_env *env, struct obd_import *imp = cli->cl_import; struct obd_connect_data *ocd; struct lustre_handle conn = { 0 }; + bool is_mdc = false; int rc; *exp = NULL; @@ -484,6 +512,10 @@ int client_connect_import(const struct lu_env *env, ocd = &imp->imp_connect_data; if (data) { *ocd = *data; + is_mdc = !strncmp(imp->imp_obd->obd_type->typ_name, + LUSTRE_MDC_NAME, 3); + if (is_mdc) + data->ocd_connect_flags |= OBD_CONNECT_MULTIMODRPCS; imp->imp_connect_flags_orig = data->ocd_connect_flags; } @@ -499,6 +531,11 @@ int client_connect_import(const struct lu_env *env, ocd->ocd_connect_flags, "old %#llx, new %#llx\n", data->ocd_connect_flags, ocd->ocd_connect_flags); data->ocd_connect_flags = ocd->ocd_connect_flags; + /* clear the flag as it was not set and is not known + * by upper layers + */ + if (is_mdc) + data->ocd_connect_flags &= ~OBD_CONNECT_MULTIMODRPCS; } ptlrpc_pinger_add_import(imp); diff --git a/drivers/staging/lustre/lustre/mdc/lproc_mdc.c b/drivers/staging/lustre/lustre/mdc/lproc_mdc.c index 76b9afc..9021c46 100644 --- a/drivers/staging/lustre/lustre/mdc/lproc_mdc.c +++ b/drivers/staging/lustre/lustre/mdc/lproc_mdc.c @@ -146,6 +146,27 @@ static ssize_t max_mod_rpcs_in_flight_store(struct kobject *kobj, } LUSTRE_RW_ATTR(max_mod_rpcs_in_flight); +static int mdc_rpc_stats_seq_show(struct seq_file *seq, void *v) +{ + struct obd_device *dev = seq->private; + + return obd_mod_rpc_stats_seq_show(&dev->u.cli, seq); +} + +static ssize_t mdc_rpc_stats_seq_write(struct file *file, + const char __user *buf, + size_t len, loff_t *off) +{ + struct seq_file *seq = file->private_data; + struct obd_device *dev = seq->private; + struct client_obd *cli = &dev->u.cli; + + lprocfs_oh_clear(&cli->cl_mod_rpcs_hist); + + return len; +} +LPROC_SEQ_FOPS(mdc_rpc_stats); + LPROC_SEQ_FOPS_WR_ONLY(mdc, ping); LPROC_SEQ_FOPS_RO_TYPE(mdc, connect_flags); @@ -185,6 +206,8 @@ static ssize_t max_pages_per_rpc_show(struct kobject *kobj, { "import", &mdc_import_fops, NULL, 0 }, { "state", &mdc_state_fops, NULL, 0 }, { "pinger_recov", &mdc_pinger_recov_fops, NULL, 0 }, + { .name = "rpc_stats", + .fops = &mdc_rpc_stats_fops }, { NULL } }; diff --git a/drivers/staging/lustre/lustre/mdc/mdc_locks.c b/drivers/staging/lustre/lustre/mdc/mdc_locks.c index b9ca140..42a128f 100644 --- a/drivers/staging/lustre/lustre/mdc/mdc_locks.c +++ b/drivers/staging/lustre/lustre/mdc/mdc_locks.c @@ -766,15 +766,16 @@ int mdc_enqueue(struct obd_export *exp, struct ldlm_enqueue_info *einfo, req->rq_sent = ktime_get_real_seconds() + resends; } - /* It is important to obtain rpc_lock first (if applicable), so that - * threads that are serialised with rpc_lock are not polluting our - * rpcs in flight counter. We do not do flock request limiting, though + /* It is important to obtain modify RPC slot first (if applicable), so + * that threads that are waiting for a modify RPC slot are not polluting + * our rpcs in flight counter. + * We do not do flock request limiting, though */ if (it) { - mdc_get_rpc_lock(obddev->u.cli.cl_rpc_lock, it); + mdc_get_mod_rpc_slot(req, it); rc = obd_get_request_slot(&obddev->u.cli); if (rc != 0) { - mdc_put_rpc_lock(obddev->u.cli.cl_rpc_lock, it); + mdc_put_mod_rpc_slot(req, it); mdc_clear_replay_flag(req, 0); ptlrpc_req_finished(req); return rc; @@ -801,7 +802,7 @@ int mdc_enqueue(struct obd_export *exp, struct ldlm_enqueue_info *einfo, } obd_put_request_slot(&obddev->u.cli); - mdc_put_rpc_lock(obddev->u.cli.cl_rpc_lock, it); + mdc_put_mod_rpc_slot(req, it); if (rc < 0) { CDEBUG(D_INFO, "%s: ldlm_cli_enqueue failed: rc = %d\n", diff --git a/drivers/staging/lustre/lustre/mdc/mdc_reint.c b/drivers/staging/lustre/lustre/mdc/mdc_reint.c index 1847e5a..b551c57 100644 --- a/drivers/staging/lustre/lustre/mdc/mdc_reint.c +++ b/drivers/staging/lustre/lustre/mdc/mdc_reint.c @@ -40,17 +40,15 @@ #include "../include/lustre_fid.h" /* mdc_setattr does its own semaphore handling */ -static int mdc_reint(struct ptlrpc_request *request, - struct mdc_rpc_lock *rpc_lock, - int level) +static int mdc_reint(struct ptlrpc_request *request, int level) { int rc; request->rq_send_state = level; - mdc_get_rpc_lock(rpc_lock, NULL); + mdc_get_mod_rpc_slot(request, NULL); rc = ptlrpc_queue_wait(request); - mdc_put_rpc_lock(rpc_lock, NULL); + mdc_put_mod_rpc_slot(request, NULL); if (rc) CDEBUG(D_INFO, "error in handling %d\n", rc); else if (!req_capsule_server_get(&request->rq_pill, &RMF_MDT_BODY)) @@ -103,8 +101,6 @@ int mdc_setattr(struct obd_export *exp, struct md_op_data *op_data, { LIST_HEAD(cancels); struct ptlrpc_request *req; - struct mdc_rpc_lock *rpc_lock; - struct obd_device *obd = exp->exp_obd; int count = 0, rc; __u64 bits; @@ -131,8 +127,6 @@ int mdc_setattr(struct obd_export *exp, struct md_op_data *op_data, return rc; } - rpc_lock = obd->u.cli.cl_rpc_lock; - if (op_data->op_attr.ia_valid & (ATTR_MTIME | ATTR_CTIME)) CDEBUG(D_INODE, "setting mtime %ld, ctime %ld\n", LTIME_S(op_data->op_attr.ia_mtime), @@ -141,7 +135,7 @@ int mdc_setattr(struct obd_export *exp, struct md_op_data *op_data, ptlrpc_request_set_replen(req); - rc = mdc_reint(req, rpc_lock, LUSTRE_IMP_FULL); + rc = mdc_reint(req, LUSTRE_IMP_FULL); if (rc == -ERESTARTSYS) rc = 0; @@ -220,7 +214,7 @@ int mdc_create(struct obd_export *exp, struct md_op_data *op_data, } level = LUSTRE_IMP_FULL; resend: - rc = mdc_reint(req, exp->exp_obd->u.cli.cl_rpc_lock, level); + rc = mdc_reint(req, level); /* Resend if we were told to. */ if (rc == -ERESTARTSYS) { @@ -292,7 +286,7 @@ int mdc_unlink(struct obd_export *exp, struct md_op_data *op_data, *request = req; - rc = mdc_reint(req, obd->u.cli.cl_rpc_lock, LUSTRE_IMP_FULL); + rc = mdc_reint(req, LUSTRE_IMP_FULL); if (rc == -ERESTARTSYS) rc = 0; return rc; @@ -302,7 +296,6 @@ int mdc_link(struct obd_export *exp, struct md_op_data *op_data, struct ptlrpc_request **request) { LIST_HEAD(cancels); - struct obd_device *obd = exp->exp_obd; struct ptlrpc_request *req; int count = 0, rc; @@ -334,7 +327,7 @@ int mdc_link(struct obd_export *exp, struct md_op_data *op_data, mdc_link_pack(req, op_data); ptlrpc_request_set_replen(req); - rc = mdc_reint(req, obd->u.cli.cl_rpc_lock, LUSTRE_IMP_FULL); + rc = mdc_reint(req, LUSTRE_IMP_FULL); *request = req; if (rc == -ERESTARTSYS) rc = 0; @@ -398,7 +391,7 @@ int mdc_rename(struct obd_export *exp, struct md_op_data *op_data, obd->u.cli.cl_default_mds_easize); ptlrpc_request_set_replen(req); - rc = mdc_reint(req, obd->u.cli.cl_rpc_lock, LUSTRE_IMP_FULL); + rc = mdc_reint(req, LUSTRE_IMP_FULL); *request = req; if (rc == -ERESTARTSYS) rc = 0; diff --git a/drivers/staging/lustre/lustre/mdc/mdc_request.c b/drivers/staging/lustre/lustre/mdc/mdc_request.c index c620f5c..e3a167a 100644 --- a/drivers/staging/lustre/lustre/mdc/mdc_request.c +++ b/drivers/staging/lustre/lustre/mdc/mdc_request.c @@ -327,12 +327,12 @@ static int mdc_xattr_common(struct obd_export *exp, /* make rpc */ if (opcode == MDS_REINT) - mdc_get_rpc_lock(exp->exp_obd->u.cli.cl_rpc_lock, NULL); + mdc_get_mod_rpc_slot(req, NULL); rc = ptlrpc_queue_wait(req); if (opcode == MDS_REINT) - mdc_put_rpc_lock(exp->exp_obd->u.cli.cl_rpc_lock, NULL); + mdc_put_mod_rpc_slot(req, NULL); if (rc) ptlrpc_req_finished(req); @@ -775,9 +775,9 @@ static int mdc_close(struct obd_export *exp, struct md_op_data *op_data, ptlrpc_request_set_replen(req); - mdc_get_rpc_lock(obd->u.cli.cl_close_lock, NULL); + mdc_get_mod_rpc_slot(req, NULL); rc = ptlrpc_queue_wait(req); - mdc_put_rpc_lock(obd->u.cli.cl_close_lock, NULL); + mdc_put_mod_rpc_slot(req, NULL); if (!req->rq_repmsg) { CDEBUG(D_RPCTRACE, "request failed to send: %p, %d\n", req, @@ -1490,9 +1490,9 @@ static int mdc_ioc_hsm_progress(struct obd_export *exp, ptlrpc_request_set_replen(req); - mdc_get_rpc_lock(exp->exp_obd->u.cli.cl_rpc_lock, NULL); + mdc_get_mod_rpc_slot(req, NULL); rc = ptlrpc_queue_wait(req); - mdc_put_rpc_lock(exp->exp_obd->u.cli.cl_rpc_lock, NULL); + mdc_put_mod_rpc_slot(req, NULL); out: ptlrpc_req_finished(req); return rc; @@ -1670,9 +1670,9 @@ static int mdc_ioc_hsm_state_set(struct obd_export *exp, ptlrpc_request_set_replen(req); - mdc_get_rpc_lock(exp->exp_obd->u.cli.cl_rpc_lock, NULL); + mdc_get_mod_rpc_slot(req, NULL); rc = ptlrpc_queue_wait(req); - mdc_put_rpc_lock(exp->exp_obd->u.cli.cl_rpc_lock, NULL); + mdc_put_mod_rpc_slot(req, NULL); out: ptlrpc_req_finished(req); return rc; @@ -1735,9 +1735,9 @@ static int mdc_ioc_hsm_request(struct obd_export *exp, ptlrpc_request_set_replen(req); - mdc_get_rpc_lock(exp->exp_obd->u.cli.cl_rpc_lock, NULL); + mdc_get_mod_rpc_slot(req, NULL); rc = ptlrpc_queue_wait(req); - mdc_put_rpc_lock(exp->exp_obd->u.cli.cl_rpc_lock, NULL); + mdc_put_mod_rpc_slot(req, NULL); out: ptlrpc_req_finished(req); return rc; @@ -2582,29 +2582,17 @@ static void mdc_llog_finish(struct obd_device *obd) static int mdc_setup(struct obd_device *obd, struct lustre_cfg *cfg) { - struct client_obd *cli = &obd->u.cli; struct lprocfs_static_vars lvars = { NULL }; int rc; - cli->cl_rpc_lock = kzalloc(sizeof(*cli->cl_rpc_lock), GFP_NOFS); - if (!cli->cl_rpc_lock) - return -ENOMEM; - mdc_init_rpc_lock(cli->cl_rpc_lock); - rc = ptlrpcd_addref(); if (rc < 0) - goto err_rpc_lock; - - cli->cl_close_lock = kzalloc(sizeof(*cli->cl_close_lock), GFP_NOFS); - if (!cli->cl_close_lock) { - rc = -ENOMEM; - goto err_ptlrpcd_decref; - } - mdc_init_rpc_lock(cli->cl_close_lock); + return rc; rc = client_obd_setup(obd, cfg); if (rc) - goto err_close_lock; + goto err_ptlrpcd_decref; + lprocfs_mdc_init_vars(&lvars); lprocfs_obd_setup(obd, lvars.obd_vars, lvars.sysfs_vars); sptlrpc_lprocfs_cliobd_attach(obd); @@ -2621,17 +2609,10 @@ static int mdc_setup(struct obd_device *obd, struct lustre_cfg *cfg) return rc; } - spin_lock_init(&cli->cl_mod_rpcs_lock); - cli->cl_max_mod_rpcs_in_flight = OBD_MAX_RIF_DEFAULT - 1; - return rc; -err_close_lock: - kfree(cli->cl_close_lock); err_ptlrpcd_decref: ptlrpcd_decref(); -err_rpc_lock: - kfree(cli->cl_rpc_lock); return rc; } @@ -2679,11 +2660,6 @@ static int mdc_precleanup(struct obd_device *obd, enum obd_cleanup_stage stage) static int mdc_cleanup(struct obd_device *obd) { - struct client_obd *cli = &obd->u.cli; - - kfree(cli->cl_rpc_lock); - kfree(cli->cl_close_lock); - ptlrpcd_decref(); return client_obd_cleanup(obd); diff --git a/drivers/staging/lustre/lustre/obdclass/genops.c b/drivers/staging/lustre/lustre/obdclass/genops.c index 62e6636..438d619 100644 --- a/drivers/staging/lustre/lustre/obdclass/genops.c +++ b/drivers/staging/lustre/lustre/obdclass/genops.c @@ -1461,6 +1461,7 @@ int obd_set_max_mod_rpcs_in_flight(struct client_obd *cli, __u16 max) { struct obd_connect_data *ocd; u16 maxmodrpcs; + u16 prev; if (max > OBD_MAX_RIF_MAX || max < 1) return -ERANGE; @@ -1486,10 +1487,178 @@ int obd_set_max_mod_rpcs_in_flight(struct client_obd *cli, __u16 max) return -ERANGE; } + spin_lock(&cli->cl_mod_rpcs_lock); + + prev = cli->cl_max_mod_rpcs_in_flight; cli->cl_max_mod_rpcs_in_flight = max; - /* will have to wakeup waiters if max has been increased */ + /* wakeup waiters if limit has been increased */ + if (cli->cl_max_mod_rpcs_in_flight > prev) + wake_up(&cli->cl_mod_rpcs_waitq); + + spin_unlock(&cli->cl_mod_rpcs_lock); return 0; } EXPORT_SYMBOL(obd_set_max_mod_rpcs_in_flight); + +#define pct(a, b) (b ? (a * 100) / b : 0) + +int obd_mod_rpc_stats_seq_show(struct client_obd *cli, struct seq_file *seq) +{ + unsigned long mod_tot = 0, mod_cum; + struct timespec64 now; + int i; + + ktime_get_real_ts64(&now); + + spin_lock(&cli->cl_mod_rpcs_lock); + + seq_printf(seq, "snapshot_time: %llu.%9lu (secs.nsecs)\n", + (s64)now.tv_sec, (unsigned long)now.tv_nsec); + seq_printf(seq, "modify_RPCs_in_flight: %hu\n", + cli->cl_mod_rpcs_in_flight); + + seq_puts(seq, "\n\t\t\tmodify\n"); + seq_puts(seq, "rpcs in flight rpcs %% cum %%\n"); + + mod_tot = lprocfs_oh_sum(&cli->cl_mod_rpcs_hist); + + mod_cum = 0; + for (i = 0; i < OBD_HIST_MAX; i++) { + unsigned long mod = cli->cl_mod_rpcs_hist.oh_buckets[i]; + + mod_cum += mod; + seq_printf(seq, "%d:\t\t%10lu %3lu %3lu\n", + i, mod, pct(mod, mod_tot), + pct(mod_cum, mod_tot)); + if (mod_cum == mod_tot) + break; + } + + spin_unlock(&cli->cl_mod_rpcs_lock); + + return 0; +} +EXPORT_SYMBOL(obd_mod_rpc_stats_seq_show); +#undef pct + +/* + * The number of modify RPCs sent in parallel is limited + * because the server has a finite number of slots per client to + * store request result and ensure reply reconstruction when needed. + * On the client, this limit is stored in cl_max_mod_rpcs_in_flight + * that takes into account server limit and cl_max_rpcs_in_flight + * value. + * On the MDC client, to avoid a potential deadlock (see Bugzilla 3462), + * one close request is allowed above the maximum. + */ +static inline bool obd_mod_rpc_slot_avail_locked(struct client_obd *cli, + bool close_req) +{ + bool avail; + + /* A slot is available if + * - number of modify RPCs in flight is less than the max + * - it's a close RPC and no other close request is in flight + */ + avail = cli->cl_mod_rpcs_in_flight < cli->cl_max_mod_rpcs_in_flight || + (close_req && !cli->cl_close_rpcs_in_flight); + + return avail; +} + +static inline bool obd_mod_rpc_slot_avail(struct client_obd *cli, + bool close_req) +{ + bool avail; + + spin_lock(&cli->cl_mod_rpcs_lock); + avail = obd_mod_rpc_slot_avail_locked(cli, close_req); + spin_unlock(&cli->cl_mod_rpcs_lock); + return avail; +} + +/* Get a modify RPC slot from the obd client @cli according + * to the kind of operation @opc that is going to be sent + * and the intent @it of the operation if it applies. + * If the maximum number of modify RPCs in flight is reached + * the thread is put to sleep. + * Returns the tag to be set in the request message. Tag 0 + * is reserved for non-modifying requests. + */ +u16 obd_get_mod_rpc_slot(struct client_obd *cli, __u32 opc, + struct lookup_intent *it) +{ + struct l_wait_info lwi = LWI_INTR(NULL, NULL); + bool close_req = false; + u16 i, max; + + /* read-only metadata RPCs don't consume a slot on MDT + * for reply reconstruction + */ + if (it && (it->it_op == IT_GETATTR || it->it_op == IT_LOOKUP || + it->it_op == IT_LAYOUT || it->it_op == IT_READDIR)) + return 0; + + if (opc == MDS_CLOSE) + close_req = true; + + do { + spin_lock(&cli->cl_mod_rpcs_lock); + max = cli->cl_max_mod_rpcs_in_flight; + if (obd_mod_rpc_slot_avail_locked(cli, close_req)) { + /* there is a slot available */ + cli->cl_mod_rpcs_in_flight++; + if (close_req) + cli->cl_close_rpcs_in_flight++; + lprocfs_oh_tally(&cli->cl_mod_rpcs_hist, + cli->cl_mod_rpcs_in_flight); + /* find a free tag */ + i = find_first_zero_bit(cli->cl_mod_tag_bitmap, + max + 1); + LASSERT(i < OBD_MAX_RIF_MAX); + LASSERT(!test_and_set_bit(i, cli->cl_mod_tag_bitmap)); + spin_unlock(&cli->cl_mod_rpcs_lock); + /* tag 0 is reserved for non-modify RPCs */ + return i + 1; + } + spin_unlock(&cli->cl_mod_rpcs_lock); + + CDEBUG(D_RPCTRACE, "%s: sleeping for a modify RPC slot opc %u, max %hu\n", + cli->cl_import->imp_obd->obd_name, opc, max); + + l_wait_event(cli->cl_mod_rpcs_waitq, + obd_mod_rpc_slot_avail(cli, close_req), &lwi); + } while (true); +} +EXPORT_SYMBOL(obd_get_mod_rpc_slot); + +/* + * Put a modify RPC slot from the obd client @cli according + * to the kind of operation @opc that has been sent and the + * intent @it of the operation if it applies. + */ +void obd_put_mod_rpc_slot(struct client_obd *cli, u32 opc, + struct lookup_intent *it, u16 tag) +{ + bool close_req = false; + + if (it && (it->it_op == IT_GETATTR || it->it_op == IT_LOOKUP || + it->it_op == IT_LAYOUT || it->it_op == IT_READDIR)) + return; + + if (opc == MDS_CLOSE) + close_req = true; + + spin_lock(&cli->cl_mod_rpcs_lock); + cli->cl_mod_rpcs_in_flight--; + if (close_req) + cli->cl_close_rpcs_in_flight--; + /* release the tag in the bitmap */ + LASSERT(tag - 1 < OBD_MAX_RIF_MAX); + LASSERT(test_and_clear_bit(tag - 1, cli->cl_mod_tag_bitmap) != 0); + spin_unlock(&cli->cl_mod_rpcs_lock); + wake_up(&cli->cl_mod_rpcs_waitq); +} +EXPORT_SYMBOL(obd_put_mod_rpc_slot); -- 1.7.1 From oleg.drokin at intel.com Mon Nov 7 22:46:00 2016 From: oleg.drokin at intel.com (Oleg Drokin) Date: Mon, 7 Nov 2016 17:46:00 -0500 Subject: [lustre-devel] [PATCH 2/2] staging: lustre: obdclass: Add handling of error returned by lustre_cfg_new In-Reply-To: References: <20161106171123.24929-1-christophe.jaillet@wanadoo.fr> <97207CF4-21C1-4351-92BF-6F28E7281CF6@intel.com> Message-ID: On Nov 7, 2016, at 4:33 PM, Dilger, Andreas wrote: > On Nov 6, 2016, at 10:26, Drokin, Oleg wrote: >> >> Hello! >> >> On Nov 6, 2016, at 12:11 PM, Christophe JAILLET wrote: >> >>> 'lustre_cfg_new()' can return ERR_PTR(-ENOMEM). >>> Handle these errors and propagate the error code to the callers. >>> >>> Error handling has been rearranged in 'lustre_process_log()' with the >>> addition of a label in order to free some resources. >> >> I wonder if we should just make it return NULL on allocation failure, >> and then at least the other error handling that is there (i.e. in your other patch) >> would become correct. >> This would make handling in mgc_apply_recover_logs incorrect, but it's already >> geared towards this sort of handling anyway, as it discards the passed error >> and sets ENOMEM unconditionally (just need to revert 3092c34a in a way). > > I'd agree with Oleg that returning NULL is the preferable solution here. > > There are also callers of lustre_cfg_new() in class_config_llog_handler(), > do_lcfg(), and lustre_end_log() that do not check error returns at all that > should be fixed at the same time. This patch was actually doing it. > > Cheers, Andreas > >>> >>> Signed-off-by: Christophe JAILLET >>> --- >>> drivers/staging/lustre/lustre/obdclass/obd_mount.c | 16 ++++++++++++++-- >>> 1 file changed, 14 insertions(+), 2 deletions(-) >>> >>> diff --git a/drivers/staging/lustre/lustre/obdclass/obd_mount.c b/drivers/staging/lustre/lustre/obdclass/obd_mount.c >>> index 59fbc29aae94..5473615cd338 100644 >>> --- a/drivers/staging/lustre/lustre/obdclass/obd_mount.c >>> +++ b/drivers/staging/lustre/lustre/obdclass/obd_mount.c >>> @@ -89,11 +89,14 @@ int lustre_process_log(struct super_block *sb, char *logname, >>> lustre_cfg_bufs_set(bufs, 2, cfg, sizeof(*cfg)); >>> lustre_cfg_bufs_set(bufs, 3, &sb, sizeof(sb)); >>> lcfg = lustre_cfg_new(LCFG_LOG_START, bufs); >>> + if (IS_ERR(lcfg)) { >>> + rc = PTR_ERR(lcfg); >>> + goto out_free; >>> + } >>> + >>> rc = obd_process_config(mgc, sizeof(*lcfg), lcfg); >>> lustre_cfg_free(lcfg); >>> >>> - kfree(bufs); >>> - >>> if (rc == -EINVAL) >>> LCONSOLE_ERROR_MSG(0x15b, "%s: The configuration from log '%s' failed from the MGS (%d). Make sure this client and the MGS are running compatible versions of Lustre.\n", >>> mgc->obd_name, logname, rc); >>> @@ -104,6 +107,9 @@ int lustre_process_log(struct super_block *sb, char *logname, >>> rc); >>> >>> /* class_obd_list(); */ >>> + >>> +out_free: >>> + kfree(bufs); >>> return rc; >>> } >>> EXPORT_SYMBOL(lustre_process_log); >>> @@ -127,6 +133,9 @@ int lustre_end_log(struct super_block *sb, char *logname, >>> if (cfg) >>> lustre_cfg_bufs_set(&bufs, 2, cfg, sizeof(*cfg)); >>> lcfg = lustre_cfg_new(LCFG_LOG_END, &bufs); >>> + if (IS_ERR(lcfg)) >>> + return PTR_ERR(lcfg); >>> + >>> rc = obd_process_config(mgc, sizeof(*lcfg), lcfg); >>> lustre_cfg_free(lcfg); >>> return rc; >>> @@ -159,6 +168,9 @@ static int do_lcfg(char *cfgname, lnet_nid_t nid, int cmd, >>> lustre_cfg_bufs_set_string(&bufs, 4, s4); >>> >>> lcfg = lustre_cfg_new(cmd, &bufs); >>> + if (IS_ERR(lcfg)) >>> + return PTR_ERR(lcfg); >>> + >>> lcfg->lcfg_nid = nid; >>> rc = class_process_config(lcfg); >>> lustre_cfg_free(lcfg); >>> -- >>> 2.9.3 >> From jsimmons at infradead.org Mon Nov 7 22:52:32 2016 From: jsimmons at infradead.org (James Simmons) Date: Mon, 7 Nov 2016 17:52:32 -0500 Subject: [lustre-devel] [PATCH] staging: lustre: obd: rename health sysfs file to health_check Message-ID: <1478559152-3624-1-git-send-email-jsimmons@infradead.org> Testing revealed that the sysfs file health is actually the wrong name. Rename to the proper name health_check. Signed-off-by: James Simmons --- .../lustre/lustre/obdclass/linux/linux-module.c | 8 ++++---- drivers/staging/lustre/sysfs-fs-lustre | 2 +- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c b/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c index be09e04..9f5e829 100644 --- a/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c +++ b/drivers/staging/lustre/lustre/obdclass/linux/linux-module.c @@ -217,8 +217,8 @@ static ssize_t pinger_show(struct kobject *kobj, struct attribute *attr, return sprintf(buf, "%s\n", "on"); } -static ssize_t health_show(struct kobject *kobj, struct attribute *attr, - char *buf) +static ssize_t +health_check_show(struct kobject *kobj, struct attribute *attr, char *buf) { bool healthy = true; int i; @@ -311,14 +311,14 @@ static ssize_t jobid_name_store(struct kobject *kobj, struct attribute *attr, LUSTRE_RO_ATTR(version); LUSTRE_RO_ATTR(pinger); -LUSTRE_RO_ATTR(health); +LUSTRE_RO_ATTR(health_check); LUSTRE_RW_ATTR(jobid_var); LUSTRE_RW_ATTR(jobid_name); static struct attribute *lustre_attrs[] = { &lustre_attr_version.attr, &lustre_attr_pinger.attr, - &lustre_attr_health.attr, + &lustre_attr_health_check.attr, &lustre_attr_jobid_name.attr, &lustre_attr_jobid_var.attr, NULL, diff --git a/drivers/staging/lustre/sysfs-fs-lustre b/drivers/staging/lustre/sysfs-fs-lustre index 20206ba..8691c65 100644 --- a/drivers/staging/lustre/sysfs-fs-lustre +++ b/drivers/staging/lustre/sysfs-fs-lustre @@ -11,7 +11,7 @@ Description: Shows if the lustre module has pinger support. "on" means yes and "off" means no. -What: /sys/fs/lustre/health +What: /sys/fs/lustre/health_check Date: May 2015 Contact: "Oleg Drokin" Description: -- 1.7.1 From jsimmons at infradead.org Tue Nov 8 02:47:20 2016 From: jsimmons at infradead.org (James Simmons) Date: Mon, 7 Nov 2016 21:47:20 -0500 Subject: [lustre-devel] [PATCH] staging: lustre: ldlm: pl_recalc time handling is wrong Message-ID: <1478573240-12850-1-git-send-email-jsimmons@infradead.org> The ldlm_pool field pl_recalc_time is set to the current monotonic clock value but the interval period is calculated with the wall clock. This means the interval period will always be far larger than the pl_recalc_period, which is just a small interval time period. The correct thing to do is to use monotomic clock current value instead of the wall clocks value when calculating recalc_interval_sec. Signed-off-by: James Simmons --- drivers/staging/lustre/lustre/ldlm/ldlm_pool.c | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c index 19831c5..30d4f80 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c @@ -256,7 +256,7 @@ static int ldlm_cli_pool_recalc(struct ldlm_pool *pl) time64_t recalc_interval_sec; int ret; - recalc_interval_sec = ktime_get_real_seconds() - pl->pl_recalc_time; + recalc_interval_sec = ktime_get_seconds() - pl->pl_recalc_time; if (recalc_interval_sec < pl->pl_recalc_period) return 0; @@ -264,7 +264,7 @@ static int ldlm_cli_pool_recalc(struct ldlm_pool *pl) /* * Check if we need to recalc lists now. */ - recalc_interval_sec = ktime_get_real_seconds() - pl->pl_recalc_time; + recalc_interval_sec = ktime_get_seconds() - pl->pl_recalc_time; if (recalc_interval_sec < pl->pl_recalc_period) { spin_unlock(&pl->pl_lock); return 0; @@ -301,7 +301,7 @@ static int ldlm_cli_pool_recalc(struct ldlm_pool *pl) * Time of LRU resizing might be longer than period, * so update after LRU resizing rather than before it. */ - pl->pl_recalc_time = ktime_get_real_seconds(); + pl->pl_recalc_time = ktime_get_seconds(); lprocfs_counter_add(pl->pl_stats, LDLM_POOL_TIMING_STAT, recalc_interval_sec); spin_unlock(&pl->pl_lock); -- 1.7.1 From andreas.dilger at intel.com Wed Nov 9 03:50:29 2016 From: andreas.dilger at intel.com (Dilger, Andreas) Date: Wed, 9 Nov 2016 03:50:29 +0000 Subject: [lustre-devel] [PATCH] staging: lustre: ldlm: pl_recalc time handling is wrong In-Reply-To: <1478573240-12850-1-git-send-email-jsimmons@infradead.org> References: <1478573240-12850-1-git-send-email-jsimmons@infradead.org> Message-ID: <4C1355CC-5CF8-4AB3-95F1-B356EC357D00@intel.com> On Nov 7, 2016, at 19:47, James Simmons wrote: > > The ldlm_pool field pl_recalc_time is set to the current > monotonic clock value but the interval period is calculated > with the wall clock. This means the interval period will > always be far larger than the pl_recalc_period, which is > just a small interval time period. The correct thing to > do is to use monotomic clock current value instead of the > wall clocks value when calculating recalc_interval_sec. It looks like this was introduced by commit 8f83409cf "staging/lustre: use 64-bit time for pl_recalc" but that patch changed get_seconds() to a mix of ktime_get_seconds() and ktime_get_real_seconds() for an unknown reason. It doesn't appear that there is any difference in overhead between the two (on 64-bit at least). Since the ldlm pool recalculation interval is actually driven in response to load on the server, it makes sense to use the "real" time instead of the monotonic time (if I understand correctly) if the client is in a VM that may periodically be blocked and "miss time" compared to the outside world. Using the "real" clock, the recalc_interval_sec will correctly reflect the actual elapsed time rather than just the number of ticks inside the VM. Is my understanding of these different clocks correct? Cheers, Andreas > > Signed-off-by: James Simmons > --- > drivers/staging/lustre/lustre/ldlm/ldlm_pool.c | 6 +++--- > 1 files changed, 3 insertions(+), 3 deletions(-) > > diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c > index 19831c5..30d4f80 100644 > --- a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c > +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c > @@ -256,7 +256,7 @@ static int ldlm_cli_pool_recalc(struct ldlm_pool *pl) > time64_t recalc_interval_sec; > int ret; > > - recalc_interval_sec = ktime_get_real_seconds() - pl->pl_recalc_time; > + recalc_interval_sec = ktime_get_seconds() - pl->pl_recalc_time; > if (recalc_interval_sec < pl->pl_recalc_period) > return 0; > > @@ -264,7 +264,7 @@ static int ldlm_cli_pool_recalc(struct ldlm_pool *pl) > /* > * Check if we need to recalc lists now. > */ > - recalc_interval_sec = ktime_get_real_seconds() - pl->pl_recalc_time; > + recalc_interval_sec = ktime_get_seconds() - pl->pl_recalc_time; > if (recalc_interval_sec < pl->pl_recalc_period) { > spin_unlock(&pl->pl_lock); > return 0; > @@ -301,7 +301,7 @@ static int ldlm_cli_pool_recalc(struct ldlm_pool *pl) > * Time of LRU resizing might be longer than period, > * so update after LRU resizing rather than before it. > */ > - pl->pl_recalc_time = ktime_get_real_seconds(); > + pl->pl_recalc_time = ktime_get_seconds(); > lprocfs_counter_add(pl->pl_stats, LDLM_POOL_TIMING_STAT, > recalc_interval_sec); > spin_unlock(&pl->pl_lock); > -- > 1.7.1 > > _______________________________________________ > lustre-devel mailing list > lustre-devel at lists.lustre.org > http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org From arnd at arndb.de Wed Nov 9 16:00:42 2016 From: arnd at arndb.de (Arnd Bergmann) Date: Wed, 09 Nov 2016 17:00:42 +0100 Subject: [lustre-devel] [PATCH] staging: lustre: ldlm: pl_recalc time handling is wrong In-Reply-To: <4C1355CC-5CF8-4AB3-95F1-B356EC357D00@intel.com> References: <1478573240-12850-1-git-send-email-jsimmons@infradead.org> <4C1355CC-5CF8-4AB3-95F1-B356EC357D00@intel.com> Message-ID: <1594836.LAbS3nYoXQ@wuerfel> On Wednesday, November 9, 2016 3:50:29 AM CET Dilger, Andreas wrote: > On Nov 7, 2016, at 19:47, James Simmons wrote: > > > > The ldlm_pool field pl_recalc_time is set to the current > > monotonic clock value but the interval period is calculated > > with the wall clock. This means the interval period will > > always be far larger than the pl_recalc_period, which is > > just a small interval time period. The correct thing to > > do is to use monotomic clock current value instead of the > > wall clocks value when calculating recalc_interval_sec. > > It looks like this was introduced by commit 8f83409cf > "staging/lustre: use 64-bit time for pl_recalc" but that patch changed > get_seconds() to a mix of ktime_get_seconds() and ktime_get_real_seconds() > for an unknown reason. It doesn't appear that there is any difference > in overhead between the two (on 64-bit at least). It was meant to use ktime_get_real_seconds() consistently, very sorry about the mistake. I don't remember exactly how we got there, I assume I had started out using ktime_get_seconds() and then moved to ktime_get_real_seconds() later but missed the last three instances. > Since the ldlm pool recalculation interval is actually driven in response to > load on the server, it makes sense to use the "real" time instead of the > monotonic time (if I understand correctly) if the client is in a VM that > may periodically be blocked and "miss time" compared to the outside world. > Using the "real" clock, the recalc_interval_sec will correctly reflect the > actual elapsed time rather than just the number of ticks inside the VM. > > Is my understanding of these different clocks correct? No, this is not the difference: monotonic and real time always tick at exactly the same rate, the only difference is the starting point. monotonic time starts at boot and can not be adjusted, while real time is set to reflect the UTC time base and gets initialized from the real time clock at boot, or using settimeofday(2) or NTP later on. In my changelog text, I wrote keeping the 'real' instead of 'monotonic' time because of the debug prints. the intention here is simply to have the console log keep the same numbers as "date +%s" for absolute values. The patch that James suggested converting everything to ktime_get_seconds() would result in the same intervals that have always been there (until I broke it by using time domains inconsistently), but would mean we could use a u32 type for pl_recalc_time and pl_recalc_period because that doesn't overflow until 136 years after booting. (signed time_t overflows 68 years after 1970, i.e 2038). Arnd From adilger at dilger.ca Wed Nov 9 22:57:53 2016 From: adilger at dilger.ca (Andreas Dilger) Date: Wed, 9 Nov 2016 15:57:53 -0700 Subject: [lustre-devel] [PATCH] f2fs: support multiple devices In-Reply-To: <20161109205653.70061-1-jaegeuk@kernel.org> References: <20161109205653.70061-1-jaegeuk@kernel.org> Message-ID: <0D1876A8-BB77-4C1A-BE4F-B4A0E81DD4EA@dilger.ca> On Nov 9, 2016, at 1:56 PM, Jaegeuk Kim wrote: > > This patch implements multiple devices support for f2fs. > Given multiple devices by mkfs.f2fs, f2fs shows them entirely as one big > volume under one f2fs instance. > > Internal block management is very simple, but we will modify block > allocation and background GC policy to boost IO speed by exploiting them > accoording to each device speed. How will you integrate this into FIEMAP, since it is now possible if a file is split across multiple devices then it will return ambiguous block numbers for a file. I've been meaning to merge the FIEMAP handling in Lustre to support multiple devices in a single filesystem, so that this can be detected in userspace. struct ll_fiemap_extent { __u64 fe_logical; /* logical offset in bytes for the start of * the extent from the beginning of the file */ __u64 fe_physical; /* physical offset in bytes for the start * of the extent from the beginning of the disk */ __u64 fe_length; /* length in bytes for this extent */ __u64 fe_reserved64[2]; __u32 fe_flags; /* FIEMAP_EXTENT_* flags for this extent */ __u32 fe_device; /* device number for this extent */ __u32 fe_reserved[2]; }; This adds the 32-bit "fe_device" field, which would optionally be filled in by the filesystem (zero otherwise). It would return the kernel device number (i.e. st_dev), or for network filesystem (with FIEMAP_EXTENT_NET set) this could just return an integer device number since the device number is meaningless (and may conflict) on a remote system. Since AFAIK Btrfs also has multiple device support there are an increasing number of places where this would be useful. Cheers, Andreas > > Signed-off-by: Jaegeuk Kim > --- > fs/f2fs/data.c | 55 ++++++++++++++++--- > fs/f2fs/f2fs.h | 29 ++++++++-- > fs/f2fs/segment.c | 119 +++++++++++++++++++++++++++++------------ > fs/f2fs/super.c | 138 ++++++++++++++++++++++++++++++++++++++---------- > include/linux/f2fs_fs.h | 10 +++- > 5 files changed, 277 insertions(+), 74 deletions(-) > > diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c > index 47ded0c..e2be24e 100644 > --- a/fs/f2fs/data.c > +++ b/fs/f2fs/data.c > @@ -88,6 +88,46 @@ static void f2fs_write_end_io(struct bio *bio) > } > > /* > + * Return true, if pre_bio's bdev is same as its target device. > + */ > +struct block_device *f2fs_target_device(struct f2fs_sb_info *sbi, > + block_t blk_addr, struct bio *bio) > +{ > + struct block_device *bdev = sbi->sb->s_bdev; > + int i; > + > + for (i = 0; i < sbi->s_ndevs; i++) { > + if (FDEV(i).start_blk <= blk_addr && > + FDEV(i).end_blk >= blk_addr) { > + blk_addr -= FDEV(i).start_blk; > + bdev = FDEV(i).bdev; > + break; > + } > + } > + if (bio) { > + bio->bi_bdev = bdev; > + bio->bi_iter.bi_sector = SECTOR_FROM_BLOCK(blk_addr); > + } > + return bdev; > +} > + > +int f2fs_target_device_index(struct f2fs_sb_info *sbi, block_t blkaddr) > +{ > + int i; > + > + for (i = 0; i < sbi->s_ndevs; i++) > + if (FDEV(i).start_blk <= blkaddr && FDEV(i).end_blk >= blkaddr) > + return i; > + return 0; > +} > + > +static bool __same_bdev(struct f2fs_sb_info *sbi, > + block_t blk_addr, struct bio *bio) > +{ > + return f2fs_target_device(sbi, blk_addr, NULL) == bio->bi_bdev; > +} > + > +/* > * Low-level block read/write IO operations. > */ > static struct bio *__bio_alloc(struct f2fs_sb_info *sbi, block_t blk_addr, > @@ -97,8 +137,7 @@ static struct bio *__bio_alloc(struct f2fs_sb_info *sbi, block_t blk_addr, > > bio = f2fs_bio_alloc(npages); > > - bio->bi_bdev = sbi->sb->s_bdev; > - bio->bi_iter.bi_sector = SECTOR_FROM_BLOCK(blk_addr); > + f2fs_target_device(sbi, blk_addr, bio); > bio->bi_end_io = is_read ? f2fs_read_end_io : f2fs_write_end_io; > bio->bi_private = is_read ? NULL : sbi; > > @@ -273,7 +312,8 @@ void f2fs_submit_page_mbio(struct f2fs_io_info *fio) > down_write(&io->io_rwsem); > > if (io->bio && (io->last_block_in_bio != fio->new_blkaddr - 1 || > - (io->fio.op != fio->op || io->fio.op_flags != fio->op_flags))) > + (io->fio.op != fio->op || io->fio.op_flags != fio->op_flags) || > + !__same_bdev(sbi, fio->new_blkaddr, io->bio))) > __submit_merged_bio(io); > alloc_new: > if (io->bio == NULL) { > @@ -965,7 +1005,6 @@ static struct bio *f2fs_grab_bio(struct inode *inode, block_t blkaddr, > { > struct f2fs_sb_info *sbi = F2FS_I_SB(inode); > struct fscrypt_ctx *ctx = NULL; > - struct block_device *bdev = sbi->sb->s_bdev; > struct bio *bio; > > if (f2fs_encrypted_inode(inode) && S_ISREG(inode->i_mode)) { > @@ -983,8 +1022,7 @@ static struct bio *f2fs_grab_bio(struct inode *inode, block_t blkaddr, > fscrypt_release_ctx(ctx); > return ERR_PTR(-ENOMEM); > } > - bio->bi_bdev = bdev; > - bio->bi_iter.bi_sector = SECTOR_FROM_BLOCK(blkaddr); > + f2fs_target_device(sbi, blkaddr, bio); > bio->bi_end_io = f2fs_read_end_io; > bio->bi_private = ctx; > > @@ -1079,7 +1117,8 @@ static int f2fs_mpage_readpages(struct address_space *mapping, > * This page will go to BIO. Do we need to send this > * BIO off first? > */ > - if (bio && (last_block_in_bio != block_nr - 1)) { > + if (bio && (last_block_in_bio != block_nr - 1 || > + !__same_bdev(F2FS_I_SB(inode), block_nr, bio))) { > submit_and_realloc: > __submit_bio(F2FS_I_SB(inode), bio, DATA); > bio = NULL; > @@ -1738,6 +1777,8 @@ static ssize_t f2fs_direct_IO(struct kiocb *iocb, struct iov_iter *iter) > return 0; > if (test_opt(F2FS_I_SB(inode), LFS)) > return 0; > + if (F2FS_I_SB(inode)->s_ndevs) > + return 0; > > trace_f2fs_direct_IO_enter(inode, offset, count, rw); > > diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h > index 9650514..1737c45 100644 > --- a/fs/f2fs/f2fs.h > +++ b/fs/f2fs/f2fs.h > @@ -730,6 +730,20 @@ struct f2fs_bio_info { > struct rw_semaphore io_rwsem; /* blocking op for bio */ > }; > > +#define FDEV(i) (sbi->devs[i]) > +#define RDEV(i) (raw_super->devs[i]) > +struct f2fs_dev_info { > + struct block_device *bdev; > + char path[MAX_PATH_LEN]; > + unsigned int total_segments; > + block_t start_blk; > + block_t end_blk; > +#ifdef CONFIG_BLK_DEV_ZONED > + unsigned int nr_blkz; /* Total number of zones */ > + u8 *blkz_type; /* Array of zones type */ > +#endif > +}; > + > enum inode_type { > DIR_INODE, /* for dirty dir inode */ > FILE_INODE, /* for dirty regular/symlink inode */ > @@ -778,10 +792,8 @@ struct f2fs_sb_info { > #endif > > #ifdef CONFIG_BLK_DEV_ZONED > - unsigned int nr_blkz; /* Total number of zones */ > unsigned int blocks_per_blkz; /* F2FS blocks per zone */ > unsigned int log_blocks_per_blkz; /* log2 F2FS blocks per zone */ > - u8 *blkz_type; /* Array of zones type */ > #endif > > /* for node-related operations */ > @@ -897,6 +909,8 @@ struct f2fs_sb_info { > > /* For shrinker support */ > struct list_head s_list; > + int s_ndevs; /* number of devices */ > + struct f2fs_dev_info *devs; /* for device list */ > struct mutex umount_mutex; > unsigned int shrinker_run_no; > > @@ -2159,6 +2173,9 @@ void f2fs_submit_merged_bio_cond(struct f2fs_sb_info *, struct inode *, > void f2fs_flush_merged_bios(struct f2fs_sb_info *); > int f2fs_submit_page_bio(struct f2fs_io_info *); > void f2fs_submit_page_mbio(struct f2fs_io_info *); > +struct block_device *f2fs_target_device(struct f2fs_sb_info *, > + block_t, struct bio *); > +int f2fs_target_device_index(struct f2fs_sb_info *, block_t); > void set_data_blkaddr(struct dnode_of_data *); > void f2fs_update_data_blkaddr(struct dnode_of_data *, block_t); > int reserve_new_blocks(struct dnode_of_data *, blkcnt_t); > @@ -2446,11 +2463,15 @@ static inline int f2fs_sb_mounted_blkzoned(struct super_block *sb) > > #ifdef CONFIG_BLK_DEV_ZONED > static inline int get_blkz_type(struct f2fs_sb_info *sbi, > - block_t blkaddr) > + struct block_device *bdev, block_t blkaddr) > { > unsigned int zno = blkaddr >> sbi->log_blocks_per_blkz; > + int i; > > - return sbi->blkz_type[zno]; > + for (i = 0; i < sbi->s_ndevs; i++) > + if (FDEV(i).bdev == bdev) > + return FDEV(i).blkz_type[zno]; > + return -EINVAL; > } > #endif > > diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c > index 7fb7dd3..ef727d1 100644 > --- a/fs/f2fs/segment.c > +++ b/fs/f2fs/segment.c > @@ -403,6 +403,33 @@ void f2fs_balance_fs_bg(struct f2fs_sb_info *sbi) > } > } > > +static int __submit_flush_wait(struct block_device *bdev) > +{ > + struct bio *bio = f2fs_bio_alloc(0); > + int ret; > + > + bio_set_op_attrs(bio, REQ_OP_WRITE, WRITE_FLUSH); > + bio->bi_bdev = bdev; > + ret = submit_bio_wait(bio); > + bio_put(bio); > + return ret; > +} > + > +static int submit_flush_wait(struct f2fs_sb_info *sbi) > +{ > + int ret = __submit_flush_wait(sbi->sb->s_bdev); > + int i; > + > + if (sbi->s_ndevs && !ret) { > + for (i = 1; i < sbi->s_ndevs; i++) { > + ret = __submit_flush_wait(FDEV(i).bdev); > + if (ret) > + break; > + } > + } > + return ret; > +} > + > static int issue_flush_thread(void *data) > { > struct f2fs_sb_info *sbi = data; > @@ -413,25 +440,18 @@ static int issue_flush_thread(void *data) > return 0; > > if (!llist_empty(&fcc->issue_list)) { > - struct bio *bio; > struct flush_cmd *cmd, *next; > int ret; > > - bio = f2fs_bio_alloc(0); > - > fcc->dispatch_list = llist_del_all(&fcc->issue_list); > fcc->dispatch_list = llist_reverse_order(fcc->dispatch_list); > > - bio->bi_bdev = sbi->sb->s_bdev; > - bio_set_op_attrs(bio, REQ_OP_WRITE, WRITE_FLUSH); > - ret = submit_bio_wait(bio); > - > + ret = submit_flush_wait(sbi); > llist_for_each_entry_safe(cmd, next, > fcc->dispatch_list, llnode) { > cmd->ret = ret; > complete(&cmd->wait); > } > - bio_put(bio); > fcc->dispatch_list = NULL; > } > > @@ -452,15 +472,11 @@ int f2fs_issue_flush(struct f2fs_sb_info *sbi) > return 0; > > if (!test_opt(sbi, FLUSH_MERGE) || !atomic_read(&fcc->submit_flush)) { > - struct bio *bio = f2fs_bio_alloc(0); > int ret; > > atomic_inc(&fcc->submit_flush); > - bio->bi_bdev = sbi->sb->s_bdev; > - bio_set_op_attrs(bio, REQ_OP_WRITE, WRITE_FLUSH); > - ret = submit_bio_wait(bio); > + ret = submit_flush_wait(sbi); > atomic_dec(&fcc->submit_flush); > - bio_put(bio); > return ret; > } > > @@ -637,14 +653,18 @@ static void f2fs_submit_bio_wait_endio(struct bio *bio) > > /* this function is copied from blkdev_issue_discard from block/blk-lib.c */ > static int __f2fs_issue_discard_async(struct f2fs_sb_info *sbi, > - block_t blkstart, block_t blklen) > + struct block_device *bdev, block_t blkstart, block_t blklen) > { > - struct block_device *bdev = sbi->sb->s_bdev; > struct bio *bio = NULL; > int err; > > trace_f2fs_issue_discard(sbi->sb, blkstart, blklen); > > + if (sbi->s_ndevs) { > + int devi = f2fs_target_device_index(sbi, blkstart); > + > + blkstart -= FDEV(devi).start_blk; > + } > err = __blkdev_issue_discard(bdev, > SECTOR_FROM_BLOCK(blkstart), > SECTOR_FROM_BLOCK(blklen), > @@ -662,18 +682,24 @@ static int __f2fs_issue_discard_async(struct f2fs_sb_info *sbi, > } > > #ifdef CONFIG_BLK_DEV_ZONED > -static int f2fs_issue_discard_zone(struct f2fs_sb_info *sbi, > - block_t blkstart, block_t blklen) > +static int __f2fs_issue_discard_zone(struct f2fs_sb_info *sbi, > + struct block_device *bdev, block_t blkstart, block_t blklen) > { > - sector_t sector = SECTOR_FROM_BLOCK(blkstart); > sector_t nr_sects = SECTOR_FROM_BLOCK(blklen); > - struct block_device *bdev = sbi->sb->s_bdev; > + sector_t sector; > + int devi = 0; > > - if (nr_sects != bdev_zone_size(bdev)) { > + if (sbi->s_ndevs) { > + devi = f2fs_target_device_index(sbi, blkstart); > + blkstart -= FDEV(devi).start_blk; > + } > + sector = SECTOR_FROM_BLOCK(blkstart); > + > + if (sector % bdev_zone_size(bdev) || nr_sects != bdev_zone_size(bdev)) { > f2fs_msg(sbi->sb, KERN_INFO, > - "Unaligned discard attempted (sector %llu + %llu)", > - (unsigned long long)sector, > - (unsigned long long)nr_sects); > + "(%d) %s: Unaligned discard attempted (block %x + %x)", > + devi, sbi->s_ndevs ? FDEV(devi).path: "", > + blkstart, blklen); > return -EIO; > } > > @@ -682,14 +708,12 @@ static int f2fs_issue_discard_zone(struct f2fs_sb_info *sbi, > * use regular discard if the drive supports it. For sequential > * zones, reset the zone write pointer. > */ > - switch (get_blkz_type(sbi, blkstart)) { > + switch (get_blkz_type(sbi, bdev, blkstart)) { > > case BLK_ZONE_TYPE_CONVENTIONAL: > if (!blk_queue_discard(bdev_get_queue(bdev))) > return 0; > - return __f2fs_issue_discard_async(sbi, blkstart, > - blklen); > - > + return __f2fs_issue_discard_async(sbi, bdev, blkstart, blklen); > case BLK_ZONE_TYPE_SEQWRITE_REQ: > case BLK_ZONE_TYPE_SEQWRITE_PREF: > trace_f2fs_issue_reset_zone(sbi->sb, blkstart); > @@ -702,14 +726,45 @@ static int f2fs_issue_discard_zone(struct f2fs_sb_info *sbi, > } > #endif > > +static int __issue_discard_async(struct f2fs_sb_info *sbi, > + struct block_device *bdev, block_t blkstart, block_t blklen) > +{ > +#ifdef CONFIG_BLK_DEV_ZONED > + if (f2fs_sb_mounted_blkzoned(sbi->sb) && > + bdev_zoned_model(bdev) != BLK_ZONED_NONE) > + return __f2fs_issue_discard_zone(sbi, bdev, blkstart, blklen); > +#endif > + return __f2fs_issue_discard_async(sbi, bdev, blkstart, blklen); > +} > + > static int f2fs_issue_discard(struct f2fs_sb_info *sbi, > block_t blkstart, block_t blklen) > { > + sector_t start = blkstart, len = 0; > + struct block_device *bdev; > struct seg_entry *se; > unsigned int offset; > block_t i; > + int err = 0; > + > + bdev = f2fs_target_device(sbi, blkstart, NULL); > + > + for (i = blkstart; i < blkstart + blklen; i++, len++) { > + if (i != start) { > + struct block_device *bdev2 = > + f2fs_target_device(sbi, i, NULL); > + > + if (bdev2 != bdev) { > + err = __issue_discard_async(sbi, bdev, > + start, len); > + if (err) > + return err; > + bdev = bdev2; > + start = i; > + len = 0; > + } > + } > > - for (i = blkstart; i < blkstart + blklen; i++) { > se = get_seg_entry(sbi, GET_SEGNO(sbi, i)); > offset = GET_BLKOFF_FROM_SEG0(sbi, i); > > @@ -717,11 +772,9 @@ static int f2fs_issue_discard(struct f2fs_sb_info *sbi, > sbi->discard_blks--; > } > > -#ifdef CONFIG_BLK_DEV_ZONED > - if (f2fs_sb_mounted_blkzoned(sbi->sb)) > - return f2fs_issue_discard_zone(sbi, blkstart, blklen); > -#endif > - return __f2fs_issue_discard_async(sbi, blkstart, blklen); > + if (len) > + err = __issue_discard_async(sbi, bdev, start, len); > + return err; > } > > static void __add_discard_entry(struct f2fs_sb_info *sbi, > diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c > index eca9aea..4ccbb86 100644 > --- a/fs/f2fs/super.c > +++ b/fs/f2fs/super.c > @@ -713,6 +713,19 @@ static void destroy_percpu_info(struct f2fs_sb_info *sbi) > percpu_counter_destroy(&sbi->total_valid_inode_count); > } > > +static void destroy_device_list(struct f2fs_sb_info *sbi) > +{ > + int i; > + > + for (i = 0; i < sbi->s_ndevs; i++) { > + blkdev_put(FDEV(i).bdev, FMODE_EXCL); > +#ifdef CONFIG_BLK_DEV_ZONED > + kfree(FDEV(i).blkz_type); > +#endif > + } > + kfree(sbi->devs); > +} > + > static void f2fs_put_super(struct super_block *sb) > { > struct f2fs_sb_info *sbi = F2FS_SB(sb); > @@ -773,6 +786,8 @@ static void f2fs_put_super(struct super_block *sb) > crypto_free_shash(sbi->s_chksum_driver); > kfree(sbi->raw_super); > > + destroy_device_list(sbi); > + > destroy_percpu_info(sbi); > kfree(sbi); > } > @@ -1516,9 +1531,9 @@ static int init_percpu_info(struct f2fs_sb_info *sbi) > } > > #ifdef CONFIG_BLK_DEV_ZONED > -static int init_blkz_info(struct f2fs_sb_info *sbi) > +static int init_blkz_info(struct f2fs_sb_info *sbi, int devi) > { > - struct block_device *bdev = sbi->sb->s_bdev; > + struct block_device *bdev = FDEV(devi).bdev; > sector_t nr_sectors = bdev->bd_part->nr_sects; > sector_t sector = 0; > struct blk_zone *zones; > @@ -1529,15 +1544,21 @@ static int init_blkz_info(struct f2fs_sb_info *sbi) > if (!f2fs_sb_mounted_blkzoned(sbi->sb)) > return 0; > > + if (sbi->blocks_per_blkz && sbi->blocks_per_blkz != > + SECTOR_TO_BLOCK(bdev_zone_size(bdev))) > + return -EINVAL; > sbi->blocks_per_blkz = SECTOR_TO_BLOCK(bdev_zone_size(bdev)); > + if (sbi->log_blocks_per_blkz && sbi->log_blocks_per_blkz != > + __ilog2_u32(sbi->blocks_per_blkz)) > + return -EINVAL; > sbi->log_blocks_per_blkz = __ilog2_u32(sbi->blocks_per_blkz); > - sbi->nr_blkz = SECTOR_TO_BLOCK(nr_sectors) >> > - sbi->log_blocks_per_blkz; > + FDEV(devi).nr_blkz = SECTOR_TO_BLOCK(nr_sectors) >> > + sbi->log_blocks_per_blkz; > if (nr_sectors & (bdev_zone_size(bdev) - 1)) > - sbi->nr_blkz++; > + FDEV(devi).nr_blkz++; > > - sbi->blkz_type = kmalloc(sbi->nr_blkz, GFP_KERNEL); > - if (!sbi->blkz_type) > + FDEV(devi).blkz_type = kmalloc(FDEV(devi).nr_blkz, GFP_KERNEL); > + if (!FDEV(devi).blkz_type) > return -ENOMEM; > > #define F2FS_REPORT_NR_ZONES 4096 > @@ -1562,7 +1583,7 @@ static int init_blkz_info(struct f2fs_sb_info *sbi) > } > > for (i = 0; i < nr_zones; i++) { > - sbi->blkz_type[n] = zones[i].type; > + FDEV(devi).blkz_type[n] = zones[i].type; > sector += zones[i].len; > n++; > } > @@ -1666,6 +1687,77 @@ int f2fs_commit_super(struct f2fs_sb_info *sbi, bool recover) > return err; > } > > +static int f2fs_scan_devices(struct f2fs_sb_info *sbi) > +{ > + struct f2fs_super_block *raw_super = F2FS_RAW_SUPER(sbi); > + int i; > + > + for (i = 0; i < MAX_DEVICES; i++) { > + if (!RDEV(i).path[0]) > + return 0; > + > + if (i == 0) { > + sbi->devs = kzalloc(sizeof(struct f2fs_dev_info) * > + MAX_DEVICES, GFP_KERNEL); > + if (!sbi->devs) > + return -ENOMEM; > + } > + > + memcpy(FDEV(i).path, RDEV(i).path, MAX_PATH_LEN); > + FDEV(i).total_segments = le32_to_cpu(RDEV(i).total_segments); > + if (i == 0) { > + FDEV(i).start_blk = 0; > + FDEV(i).end_blk = FDEV(i).start_blk + > + (FDEV(i).total_segments << > + sbi->log_blocks_per_seg) - 1 + > + le32_to_cpu(raw_super->segment0_blkaddr); > + } else { > + FDEV(i).start_blk = FDEV(i - 1).end_blk + 1; > + FDEV(i).end_blk = FDEV(i).start_blk + > + (FDEV(i).total_segments << > + sbi->log_blocks_per_seg) - 1; > + } > + > + FDEV(i).bdev = blkdev_get_by_path(FDEV(i).path, > + sbi->sb->s_mode, sbi->sb->s_type); > + if (IS_ERR(FDEV(i).bdev)) > + return PTR_ERR(FDEV(i).bdev); > + > + /* to release errored devices */ > + sbi->s_ndevs = i + 1; > + > +#ifdef CONFIG_BLK_DEV_ZONED > + if (bdev_zoned_model(FDEV(i).bdev) == BLK_ZONED_HM && > + !f2fs_sb_mounted_blkzoned(sbi->sb)) { > + f2fs_msg(sbi->sb, KERN_ERR, > + "Zoned block device feature not enabled\n"); > + return -EINVAL; > + } > + if (bdev_zoned_model(FDEV(i).bdev) != BLK_ZONED_NONE) { > + if (init_blkz_info(sbi, i)) { > + f2fs_msg(sbi->sb, KERN_ERR, > + "Failed to initialize F2FS blkzone information"); > + return -EINVAL; > + } > + f2fs_msg(sbi->sb, KERN_INFO, > + "Mount Device [%2d]: %20s, %8u, %8x - %8x (zone: %s)", > + i, FDEV(i).path, > + FDEV(i).total_segments, > + FDEV(i).start_blk, FDEV(i).end_blk, > + bdev_zoned_model(FDEV(i).bdev) == BLK_ZONED_HA ? > + "Host-aware" : "Host-managed"); > + continue; > + } > +#endif > + f2fs_msg(sbi->sb, KERN_INFO, > + "Mount Device [%2d]: %20s, %8u, %8x - %8x", > + i, FDEV(i).path, > + FDEV(i).total_segments, > + FDEV(i).start_blk, FDEV(i).end_blk); > + } > + return 0; > +} > + > static int f2fs_fill_super(struct super_block *sb, void *data, int silent) > { > struct f2fs_sb_info *sbi; > @@ -1724,15 +1816,7 @@ static int f2fs_fill_super(struct super_block *sb, void *data, int silent) > "Zoned block device support is not enabled\n"); > goto free_sb_buf; > } > -#else > - if (bdev_zoned_model(sb->s_bdev) == BLK_ZONED_HM && > - !f2fs_sb_mounted_blkzoned(sb)) { > - f2fs_msg(sb, KERN_ERR, > - "Zoned block device feature not enabled\n"); > - goto free_sb_buf; > - } > #endif > - > default_options(sbi); > /* parse mount options */ > options = kstrdup((const char *)data, GFP_KERNEL); > @@ -1802,6 +1886,13 @@ static int f2fs_fill_super(struct super_block *sb, void *data, int silent) > goto free_meta_inode; > } > > + /* Initialize device list */ > + err = f2fs_scan_devices(sbi); > + if (err) { > + f2fs_msg(sb, KERN_ERR, "Failed to find devices"); > + goto free_devices; > + } > + > sbi->total_valid_node_count = > le32_to_cpu(sbi->ckpt->valid_node_count); > percpu_counter_set(&sbi->total_valid_inode_count, > @@ -1820,15 +1911,6 @@ static int f2fs_fill_super(struct super_block *sb, void *data, int silent) > > init_ino_entry_info(sbi); > > -#ifdef CONFIG_BLK_DEV_ZONED > - err = init_blkz_info(sbi); > - if (err) { > - f2fs_msg(sb, KERN_ERR, > - "Failed to initialize F2FS blkzone information"); > - goto free_blkz; > - } > -#endif > - > /* setup f2fs internal modules */ > err = build_segment_manager(sbi); > if (err) { > @@ -2007,10 +2089,8 @@ static int f2fs_fill_super(struct super_block *sb, void *data, int silent) > destroy_node_manager(sbi); > free_sm: > destroy_segment_manager(sbi); > -#ifdef CONFIG_BLK_DEV_ZONED > -free_blkz: > - kfree(sbi->blkz_type); > -#endif > +free_devices: > + destroy_device_list(sbi); > kfree(sbi->ckpt); > free_meta_inode: > make_bad_inode(sbi->meta_inode); > diff --git a/include/linux/f2fs_fs.h b/include/linux/f2fs_fs.h > index 422630b..cea41a1 100644 > --- a/include/linux/f2fs_fs.h > +++ b/include/linux/f2fs_fs.h > @@ -52,10 +52,17 @@ > > #define VERSION_LEN 256 > #define MAX_VOLUME_NAME 512 > +#define MAX_PATH_LEN 64 > +#define MAX_DEVICES 8 > > /* > * For superblock > */ > +struct f2fs_device { > + __u8 path[MAX_PATH_LEN]; > + __le32 total_segments; > +} __packed; > + > struct f2fs_super_block { > __le32 magic; /* Magic Number */ > __le16 major_ver; /* Major Version */ > @@ -94,7 +101,8 @@ struct f2fs_super_block { > __le32 feature; /* defined features */ > __u8 encryption_level; /* versioning level for encryption */ > __u8 encrypt_pw_salt[16]; /* Salt used for string2key algorithm */ > - __u8 reserved[871]; /* valid reserved region */ > + struct f2fs_device devs[MAX_DEVICES]; /* device list */ > + __u8 reserved[327]; /* valid reserved region */ > } __packed; > > /* > -- > 2.8.3 Cheers, Andreas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP using GPGMail URL: From gregkh at linuxfoundation.org Wed Nov 9 16:03:54 2016 From: gregkh at linuxfoundation.org (Greg Kroah-Hartman) Date: Wed, 9 Nov 2016 17:03:54 +0100 Subject: [lustre-devel] [PATCH v3] staging: lustre: mdc: manage number of modify RPCs in flight In-Reply-To: <1478557403-31072-1-git-send-email-jsimmons@infradead.org> References: <1478557403-31072-1-git-send-email-jsimmons@infradead.org> Message-ID: <20161109160354.GC9111@kroah.com> On Mon, Nov 07, 2016 at 05:23:23PM -0500, James Simmons wrote: > From: Gregoire Pichon > > This patch is the main client part of a new feature that supports > multiple modify metadata RPCs in parallel. Its goal is to improve > metadata operations performance of a single client, while maintening > the consistency of MDT reply reconstruction and MDT recovery > mechanisms. > > It allows to manage the number of modify RPCs in flight within > the client obd structure and to assign a virtual index (the tag) to > each modify RPC to help server side cleaning of reply data. > > The mdc component uses this feature to send multiple modify RPCs > in parallel. > > Changelog: > > v1) Initial patch with incorrect print out of timestamp in > obd_mod_rpc_stats_seq_show. > > v2) Fixed up obd_mod_rpc_stats_seq_show to print out in nanoseconds > > v3) Add this changelog changelog goes below the --- line, you don't want that in the final commit, right? Please fix up and resend. thanks, greg k-h From gregkh at linuxfoundation.org Wed Nov 9 16:05:07 2016 From: gregkh at linuxfoundation.org (Greg KH) Date: Wed, 9 Nov 2016 17:05:07 +0100 Subject: [lustre-devel] [PATCH 1/2] staging: lustre: llite: use u64 instead of loff_t in lov_object_fiemap() In-Reply-To: References: Message-ID: <20161109160507.GD9111@kroah.com> On Tue, Nov 08, 2016 at 12:13:59PM +0000, Xu, Bobijam wrote: > Change loff_t to u64 in lov_object_fiemap() since loff_t is a signed > value type. > > Otherwise there could be an overflow in > drivers/staging/lustre/lustre/lov/lov_object.c:1241 lov_object_fiemap() > warn: signed overflow undefined. 'fm_start + fm_length < fm_start' > > Reported-by: Dan Carpenter > Signed-off-by: Bobi Jam > --- > drivers/staging/lustre/lustre/lov/lov_object.c | 38 +++++++++++++------------- > 1 file changed, 19 insertions(+), 19 deletions(-) Why did you not cc: the maintainers of this filesystem with these patches? Please resend them both and do so. thanks, greg k-h From gregkh at linuxfoundation.org Thu Nov 10 12:21:08 2016 From: gregkh at linuxfoundation.org (Greg Kroah-Hartman) Date: Thu, 10 Nov 2016 13:21:08 +0100 Subject: [lustre-devel] [PATCH] staging: lustre: ldlm: pl_recalc time handling is wrong In-Reply-To: <1594836.LAbS3nYoXQ@wuerfel> References: <1478573240-12850-1-git-send-email-jsimmons@infradead.org> <4C1355CC-5CF8-4AB3-95F1-B356EC357D00@intel.com> <1594836.LAbS3nYoXQ@wuerfel> Message-ID: <20161110122108.GA7580@kroah.com> On Wed, Nov 09, 2016 at 05:00:42PM +0100, Arnd Bergmann wrote: > On Wednesday, November 9, 2016 3:50:29 AM CET Dilger, Andreas wrote: > > On Nov 7, 2016, at 19:47, James Simmons wrote: > > > > > > The ldlm_pool field pl_recalc_time is set to the current > > > monotonic clock value but the interval period is calculated > > > with the wall clock. This means the interval period will > > > always be far larger than the pl_recalc_period, which is > > > just a small interval time period. The correct thing to > > > do is to use monotomic clock current value instead of the > > > wall clocks value when calculating recalc_interval_sec. > > > > It looks like this was introduced by commit 8f83409cf > > "staging/lustre: use 64-bit time for pl_recalc" but that patch changed > > get_seconds() to a mix of ktime_get_seconds() and ktime_get_real_seconds() > > for an unknown reason. It doesn't appear that there is any difference > > in overhead between the two (on 64-bit at least). > > It was meant to use ktime_get_real_seconds() consistently, very sorry > about the mistake. I don't remember exactly how we got there, I assume > I had started out using ktime_get_seconds() and then moved to > ktime_get_real_seconds() later but missed the last three instances. > > > Since the ldlm pool recalculation interval is actually driven in response to > > load on the server, it makes sense to use the "real" time instead of the > > monotonic time (if I understand correctly) if the client is in a VM that > > may periodically be blocked and "miss time" compared to the outside world. > > Using the "real" clock, the recalc_interval_sec will correctly reflect the > > actual elapsed time rather than just the number of ticks inside the VM. > > > > Is my understanding of these different clocks correct? > > No, this is not the difference: monotonic and real time always tick > at exactly the same rate, the only difference is the starting point. > monotonic time starts at boot and can not be adjusted, while real > time is set to reflect the UTC time base and gets initialized > from the real time clock at boot, or using settimeofday(2) or > NTP later on. > > In my changelog text, I wrote > > keeping the 'real' instead of 'monotonic' time because of the > debug prints. > > the intention here is simply to have the console log keep the > same numbers as "date +%s" for absolute values. The patch that > James suggested converting everything to ktime_get_seconds() > would result in the same intervals that have always been there > (until I broke it by using time domains inconsistently), but > would mean we could use a u32 type for pl_recalc_time and > pl_recalc_period because that doesn't overflow until 136 years > after booting. (signed time_t overflows 68 years after 1970, > i.e 2038). So, is this patch correct and should be merged to the tree, or not? confused, greg k-h From dan.carpenter at oracle.com Thu Nov 10 12:50:51 2016 From: dan.carpenter at oracle.com (Dan Carpenter) Date: Thu, 10 Nov 2016 15:50:51 +0300 Subject: [lustre-devel] [patch] staging/lustre/osc: indent an if statement Message-ID: <20161110125050.GA18577@mwanda> We accidentally removed a tab here. Let's add it back, and some curly braces as well since this is a muti-line indent. Signed-off-by: Dan Carpenter diff --git a/drivers/staging/lustre/lustre/osc/osc_lock.c b/drivers/staging/lustre/lustre/osc/osc_lock.c index e337e87..bfc8d38 100644 --- a/drivers/staging/lustre/lustre/osc/osc_lock.c +++ b/drivers/staging/lustre/lustre/osc/osc_lock.c @@ -607,9 +607,11 @@ static int osc_ldlm_glimpse_ast(struct ldlm_lock *dlmlock, void *data) lvb = req_capsule_server_get(cap, &RMF_DLM_LVB); result = cl_object_glimpse(env, obj, lvb); } - if (!exp_connect_lvb_type(req->rq_export)) - req_capsule_shrink(&req->rq_pill, &RMF_DLM_LVB, - sizeof(struct ost_lvb_v1), RCL_SERVER); + if (!exp_connect_lvb_type(req->rq_export)) { + req_capsule_shrink(&req->rq_pill, &RMF_DLM_LVB, + sizeof(struct ost_lvb_v1), + RCL_SERVER); + } cl_object_put(env, obj); } else { /* From arnd at arndb.de Thu Nov 10 15:01:18 2016 From: arnd at arndb.de (Arnd Bergmann) Date: Thu, 10 Nov 2016 16:01:18 +0100 Subject: [lustre-devel] [PATCH] staging: lustre: ldlm: pl_recalc time handling is wrong In-Reply-To: <20161110122108.GA7580@kroah.com> References: <1478573240-12850-1-git-send-email-jsimmons@infradead.org> <1594836.LAbS3nYoXQ@wuerfel> <20161110122108.GA7580@kroah.com> Message-ID: <5149342.oboLeYkUcs@wuerfel> On Thursday, November 10, 2016 1:21:08 PM CET Greg Kroah-Hartman wrote: > > > > the intention here is simply to have the console log keep the > > same numbers as "date +%s" for absolute values. The patch that > > James suggested converting everything to ktime_get_seconds() > > would result in the same intervals that have always been there > > (until I broke it by using time domains inconsistently), but > > would mean we could use a u32 type for pl_recalc_time and > > pl_recalc_period because that doesn't overflow until 136 years > > after booting. (signed time_t overflows 68 years after 1970, > > i.e 2038). > > So, is this patch correct and should be merged to the tree, or not? > No, I think it's wrong in a different way as before. After my patch, there were six instances of ktime_get_real_seconds() and three instances of ktime_get_seconds(), and that was wrong. James's patch converts three of the six instances to ktime_get_seconds(), which is equally wrong, just different. We can either convert the six ktime_get_real_seconds() to ktime_get_seconds(), or convert the four (one was added in the meantime) ktime_get_seconds() to ktime_get_real_seconds(). I'll follow up with a patch for the latter as it is what I had originally intended. We can also do what James intended instead. Arnd From arnd at arndb.de Thu Nov 10 15:21:20 2016 From: arnd at arndb.de (Arnd Bergmann) Date: Thu, 10 Nov 2016 16:21:20 +0100 Subject: [lustre-devel] [PATCH v2] staging: lustre: ldlm: pl_recalc time handling is wrong In-Reply-To: <20161110122108.GA7580@kroah.com> References: <1478573240-12850-1-git-send-email-jsimmons@infradead.org> <1594836.LAbS3nYoXQ@wuerfel> <20161110122108.GA7580@kroah.com> Message-ID: <4839295.T6k2aHkh3K@wuerfel> James Simmons reports: > The ldlm_pool field pl_recalc_time is set to the current > monotonic clock value but the interval period is calculated > with the wall clock. This means the interval period will > always be far larger than the pl_recalc_period, which is > just a small interval time period. The correct thing to > do is to use monotomic clock current value instead of the > wall clocks value when calculating recalc_interval_sec. This broke when I converted the 32-bit get_seconds() into ktime_get_{real_,}seconds() inconsistently. Either one of those two would have worked, but mixing them does not. Staying with the original intention of the patch, this changes the ktime_get_seconds() calls into ktime_get_real_seconds(), using real time instead of mononic time. Cc: stable at vger.kernel.org # v4.4+ Fixes: 8f83409cf238 ("staging/lustre: use 64-bit time for pl_recalc") Reported-by: James Simmons Signed-off-by: Arnd Bergmann --- v2: James' patch was similarly incomplete to mine, as it only addressed some of the calls. With this new version, all ktime accessors use the same time domain. diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c index 19831c555c49..b820309d70e3 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c @@ -356,10 +356,10 @@ static int ldlm_pool_recalc(struct ldlm_pool *pl) u32 recalc_interval_sec; int count; - recalc_interval_sec = ktime_get_seconds() - pl->pl_recalc_time; + recalc_interval_sec = ktime_get_real_seconds() - pl->pl_recalc_time; if (recalc_interval_sec > 0) { spin_lock(&pl->pl_lock); - recalc_interval_sec = ktime_get_seconds() - pl->pl_recalc_time; + recalc_interval_sec = ktime_get_real_seconds() - pl->pl_recalc_time; if (recalc_interval_sec > 0) { /* @@ -382,7 +382,7 @@ static int ldlm_pool_recalc(struct ldlm_pool *pl) count); } - recalc_interval_sec = pl->pl_recalc_time - ktime_get_seconds() + + recalc_interval_sec = pl->pl_recalc_time - ktime_get_real_seconds() + pl->pl_recalc_period; if (recalc_interval_sec <= 0) { /* DEBUG: should be re-removed after LU-4536 is fixed */ @@ -657,7 +657,7 @@ int ldlm_pool_init(struct ldlm_pool *pl, struct ldlm_namespace *ns, spin_lock_init(&pl->pl_lock); atomic_set(&pl->pl_granted, 0); - pl->pl_recalc_time = ktime_get_seconds(); + pl->pl_recalc_time = ktime_get_real_seconds(); atomic_set(&pl->pl_lock_volume_factor, 1); atomic_set(&pl->pl_grant_rate, 0); From jsimmons at infradead.org Thu Nov 10 15:51:13 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 10:51:13 -0500 Subject: [lustre-devel] [PATCH v4] staging: lustre: mdc: manage number of modify RPCs in flight Message-ID: <1478793073-21814-1-git-send-email-jsimmons@infradead.org> From: Gregoire Pichon This patch is the main client part of a new feature that supports multiple modify metadata RPCs in parallel. Its goal is to improve metadata operations performance of a single client, while maintening the consistency of MDT reply reconstruction and MDT recovery mechanisms. It allows to manage the number of modify RPCs in flight within the client obd structure and to assign a virtual index (the tag) to each modify RPC to help server side cleaning of reply data. The mdc component uses this feature to send multiple modify RPCs in parallel. Signed-off-by: Gregoire Pichon Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-5319 Reviewed-on: http://review.whamcloud.com/14374 Reviewed-by: Andreas Dilger Reviewed-by: Alex Zhuravlev Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- Changelog: v1) Initial patch with incorrect print out of timestamp in obd_mod_rpc_stats_seq_show. v2) Fixed up obd_mod_rpc_stats_seq_show to print out in nanoseconds v3) Add this changelog v4) and place in right spot drivers/staging/lustre/lustre/fid/fid_request.c | 4 - drivers/staging/lustre/lustre/include/lustre_mdc.h | 24 +++ drivers/staging/lustre/lustre/include/obd.h | 7 +- drivers/staging/lustre/lustre/include/obd_class.h | 6 + drivers/staging/lustre/lustre/ldlm/ldlm_lib.c | 37 +++++ drivers/staging/lustre/lustre/mdc/lproc_mdc.c | 23 +++ drivers/staging/lustre/lustre/mdc/mdc_locks.c | 13 +- drivers/staging/lustre/lustre/mdc/mdc_reint.c | 23 +-- drivers/staging/lustre/lustre/mdc/mdc_request.c | 50 ++----- drivers/staging/lustre/lustre/obdclass/genops.c | 171 +++++++++++++++++++- 10 files changed, 293 insertions(+), 65 deletions(-) diff --git a/drivers/staging/lustre/lustre/fid/fid_request.c b/drivers/staging/lustre/lustre/fid/fid_request.c index 1148b9a..999f250 100644 --- a/drivers/staging/lustre/lustre/fid/fid_request.c +++ b/drivers/staging/lustre/lustre/fid/fid_request.c @@ -112,11 +112,7 @@ static int seq_client_rpc(struct lu_client_seq *seq, ptlrpc_at_set_req_timeout(req); - if (opc != SEQ_ALLOC_SUPER && seq->lcs_type == LUSTRE_SEQ_METADATA) - mdc_get_rpc_lock(exp->exp_obd->u.cli.cl_rpc_lock, NULL); rc = ptlrpc_queue_wait(req); - if (opc != SEQ_ALLOC_SUPER && seq->lcs_type == LUSTRE_SEQ_METADATA) - mdc_put_rpc_lock(exp->exp_obd->u.cli.cl_rpc_lock, NULL); if (rc) goto out_req; diff --git a/drivers/staging/lustre/lustre/include/lustre_mdc.h b/drivers/staging/lustre/lustre/include/lustre_mdc.h index 92a5c0f..198ceb0 100644 --- a/drivers/staging/lustre/lustre/include/lustre_mdc.h +++ b/drivers/staging/lustre/lustre/include/lustre_mdc.h @@ -156,6 +156,30 @@ static inline void mdc_put_rpc_lock(struct mdc_rpc_lock *lck, mutex_unlock(&lck->rpcl_mutex); } +static inline void mdc_get_mod_rpc_slot(struct ptlrpc_request *req, + struct lookup_intent *it) +{ + struct client_obd *cli = &req->rq_import->imp_obd->u.cli; + u32 opc; + u16 tag; + + opc = lustre_msg_get_opc(req->rq_reqmsg); + tag = obd_get_mod_rpc_slot(cli, opc, it); + lustre_msg_set_tag(req->rq_reqmsg, tag); +} + +static inline void mdc_put_mod_rpc_slot(struct ptlrpc_request *req, + struct lookup_intent *it) +{ + struct client_obd *cli = &req->rq_import->imp_obd->u.cli; + u32 opc; + u16 tag; + + opc = lustre_msg_get_opc(req->rq_reqmsg); + tag = lustre_msg_get_tag(req->rq_reqmsg); + obd_put_mod_rpc_slot(cli, opc, it, tag); +} + /** * Update the maximum possible easize. * diff --git a/drivers/staging/lustre/lustre/include/obd.h b/drivers/staging/lustre/lustre/include/obd.h index c8a6e23..09e3e71 100644 --- a/drivers/staging/lustre/lustre/include/obd.h +++ b/drivers/staging/lustre/lustre/include/obd.h @@ -263,14 +263,17 @@ struct client_obd { wait_queue_head_t cl_destroy_waitq; struct mdc_rpc_lock *cl_rpc_lock; - struct mdc_rpc_lock *cl_close_lock; /* modify rpcs in flight * currently used for metadata only */ spinlock_t cl_mod_rpcs_lock; u16 cl_max_mod_rpcs_in_flight; - + u16 cl_mod_rpcs_in_flight; + u16 cl_close_rpcs_in_flight; + wait_queue_head_t cl_mod_rpcs_waitq; + unsigned long *cl_mod_tag_bitmap; + struct obd_histogram cl_mod_rpcs_hist; /* mgc datastruct */ atomic_t cl_mgc_refcount; diff --git a/drivers/staging/lustre/lustre/include/obd_class.h b/drivers/staging/lustre/lustre/include/obd_class.h index 70b355e..f79133c 100644 --- a/drivers/staging/lustre/lustre/include/obd_class.h +++ b/drivers/staging/lustre/lustre/include/obd_class.h @@ -101,6 +101,12 @@ struct obd_device *class_devices_in_group(struct obd_uuid *grp_uuid, __u32 obd_get_max_rpcs_in_flight(struct client_obd *cli); int obd_set_max_rpcs_in_flight(struct client_obd *cli, __u32 max); int obd_set_max_mod_rpcs_in_flight(struct client_obd *cli, u16 max); +int obd_mod_rpc_stats_seq_show(struct client_obd *cli, struct seq_file *seq); + +u16 obd_get_mod_rpc_slot(struct client_obd *cli, u32 opc, + struct lookup_intent *it); +void obd_put_mod_rpc_slot(struct client_obd *cli, u32 opc, + struct lookup_intent *it, u16 tag); struct llog_handle; struct llog_rec_hdr; diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c index 4f9480e..ee4bb56 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c @@ -372,6 +372,25 @@ int client_obd_setup(struct obd_device *obddev, struct lustre_cfg *lcfg) } else { cli->cl_max_rpcs_in_flight = OBD_MAX_RIF_DEFAULT; } + + spin_lock_init(&cli->cl_mod_rpcs_lock); + spin_lock_init(&cli->cl_mod_rpcs_hist.oh_lock); + cli->cl_max_mod_rpcs_in_flight = 0; + cli->cl_mod_rpcs_in_flight = 0; + cli->cl_close_rpcs_in_flight = 0; + init_waitqueue_head(&cli->cl_mod_rpcs_waitq); + cli->cl_mod_tag_bitmap = NULL; + + if (connect_op == MDS_CONNECT) { + cli->cl_max_mod_rpcs_in_flight = cli->cl_max_rpcs_in_flight - 1; + cli->cl_mod_tag_bitmap = kcalloc(BITS_TO_LONGS(OBD_MAX_RIF_MAX), + sizeof(long), GFP_NOFS); + if (!cli->cl_mod_tag_bitmap) { + rc = -ENOMEM; + goto err; + } + } + rc = ldlm_get_ref(); if (rc) { CERROR("ldlm_get_ref failed: %d\n", rc); @@ -431,12 +450,16 @@ int client_obd_setup(struct obd_device *obddev, struct lustre_cfg *lcfg) err_ldlm: ldlm_put_ref(); err: + kfree(cli->cl_mod_tag_bitmap); + cli->cl_mod_tag_bitmap = NULL; return rc; } EXPORT_SYMBOL(client_obd_setup); int client_obd_cleanup(struct obd_device *obddev) { + struct client_obd *cli = &obddev->u.cli; + ldlm_namespace_free_post(obddev->obd_namespace); obddev->obd_namespace = NULL; @@ -444,6 +467,10 @@ int client_obd_cleanup(struct obd_device *obddev) LASSERT(!obddev->u.cli.cl_import); ldlm_put_ref(); + + kfree(cli->cl_mod_tag_bitmap); + cli->cl_mod_tag_bitmap = NULL; + return 0; } EXPORT_SYMBOL(client_obd_cleanup); @@ -458,6 +485,7 @@ int client_connect_import(const struct lu_env *env, struct obd_import *imp = cli->cl_import; struct obd_connect_data *ocd; struct lustre_handle conn = { 0 }; + bool is_mdc = false; int rc; *exp = NULL; @@ -484,6 +512,10 @@ int client_connect_import(const struct lu_env *env, ocd = &imp->imp_connect_data; if (data) { *ocd = *data; + is_mdc = !strncmp(imp->imp_obd->obd_type->typ_name, + LUSTRE_MDC_NAME, 3); + if (is_mdc) + data->ocd_connect_flags |= OBD_CONNECT_MULTIMODRPCS; imp->imp_connect_flags_orig = data->ocd_connect_flags; } @@ -499,6 +531,11 @@ int client_connect_import(const struct lu_env *env, ocd->ocd_connect_flags, "old %#llx, new %#llx\n", data->ocd_connect_flags, ocd->ocd_connect_flags); data->ocd_connect_flags = ocd->ocd_connect_flags; + /* clear the flag as it was not set and is not known + * by upper layers + */ + if (is_mdc) + data->ocd_connect_flags &= ~OBD_CONNECT_MULTIMODRPCS; } ptlrpc_pinger_add_import(imp); diff --git a/drivers/staging/lustre/lustre/mdc/lproc_mdc.c b/drivers/staging/lustre/lustre/mdc/lproc_mdc.c index 76b9afc..9021c46 100644 --- a/drivers/staging/lustre/lustre/mdc/lproc_mdc.c +++ b/drivers/staging/lustre/lustre/mdc/lproc_mdc.c @@ -146,6 +146,27 @@ static ssize_t max_mod_rpcs_in_flight_store(struct kobject *kobj, } LUSTRE_RW_ATTR(max_mod_rpcs_in_flight); +static int mdc_rpc_stats_seq_show(struct seq_file *seq, void *v) +{ + struct obd_device *dev = seq->private; + + return obd_mod_rpc_stats_seq_show(&dev->u.cli, seq); +} + +static ssize_t mdc_rpc_stats_seq_write(struct file *file, + const char __user *buf, + size_t len, loff_t *off) +{ + struct seq_file *seq = file->private_data; + struct obd_device *dev = seq->private; + struct client_obd *cli = &dev->u.cli; + + lprocfs_oh_clear(&cli->cl_mod_rpcs_hist); + + return len; +} +LPROC_SEQ_FOPS(mdc_rpc_stats); + LPROC_SEQ_FOPS_WR_ONLY(mdc, ping); LPROC_SEQ_FOPS_RO_TYPE(mdc, connect_flags); @@ -185,6 +206,8 @@ static ssize_t max_pages_per_rpc_show(struct kobject *kobj, { "import", &mdc_import_fops, NULL, 0 }, { "state", &mdc_state_fops, NULL, 0 }, { "pinger_recov", &mdc_pinger_recov_fops, NULL, 0 }, + { .name = "rpc_stats", + .fops = &mdc_rpc_stats_fops }, { NULL } }; diff --git a/drivers/staging/lustre/lustre/mdc/mdc_locks.c b/drivers/staging/lustre/lustre/mdc/mdc_locks.c index b9ca140..42a128f 100644 --- a/drivers/staging/lustre/lustre/mdc/mdc_locks.c +++ b/drivers/staging/lustre/lustre/mdc/mdc_locks.c @@ -766,15 +766,16 @@ int mdc_enqueue(struct obd_export *exp, struct ldlm_enqueue_info *einfo, req->rq_sent = ktime_get_real_seconds() + resends; } - /* It is important to obtain rpc_lock first (if applicable), so that - * threads that are serialised with rpc_lock are not polluting our - * rpcs in flight counter. We do not do flock request limiting, though + /* It is important to obtain modify RPC slot first (if applicable), so + * that threads that are waiting for a modify RPC slot are not polluting + * our rpcs in flight counter. + * We do not do flock request limiting, though */ if (it) { - mdc_get_rpc_lock(obddev->u.cli.cl_rpc_lock, it); + mdc_get_mod_rpc_slot(req, it); rc = obd_get_request_slot(&obddev->u.cli); if (rc != 0) { - mdc_put_rpc_lock(obddev->u.cli.cl_rpc_lock, it); + mdc_put_mod_rpc_slot(req, it); mdc_clear_replay_flag(req, 0); ptlrpc_req_finished(req); return rc; @@ -801,7 +802,7 @@ int mdc_enqueue(struct obd_export *exp, struct ldlm_enqueue_info *einfo, } obd_put_request_slot(&obddev->u.cli); - mdc_put_rpc_lock(obddev->u.cli.cl_rpc_lock, it); + mdc_put_mod_rpc_slot(req, it); if (rc < 0) { CDEBUG(D_INFO, "%s: ldlm_cli_enqueue failed: rc = %d\n", diff --git a/drivers/staging/lustre/lustre/mdc/mdc_reint.c b/drivers/staging/lustre/lustre/mdc/mdc_reint.c index 1847e5a..b551c57 100644 --- a/drivers/staging/lustre/lustre/mdc/mdc_reint.c +++ b/drivers/staging/lustre/lustre/mdc/mdc_reint.c @@ -40,17 +40,15 @@ #include "../include/lustre_fid.h" /* mdc_setattr does its own semaphore handling */ -static int mdc_reint(struct ptlrpc_request *request, - struct mdc_rpc_lock *rpc_lock, - int level) +static int mdc_reint(struct ptlrpc_request *request, int level) { int rc; request->rq_send_state = level; - mdc_get_rpc_lock(rpc_lock, NULL); + mdc_get_mod_rpc_slot(request, NULL); rc = ptlrpc_queue_wait(request); - mdc_put_rpc_lock(rpc_lock, NULL); + mdc_put_mod_rpc_slot(request, NULL); if (rc) CDEBUG(D_INFO, "error in handling %d\n", rc); else if (!req_capsule_server_get(&request->rq_pill, &RMF_MDT_BODY)) @@ -103,8 +101,6 @@ int mdc_setattr(struct obd_export *exp, struct md_op_data *op_data, { LIST_HEAD(cancels); struct ptlrpc_request *req; - struct mdc_rpc_lock *rpc_lock; - struct obd_device *obd = exp->exp_obd; int count = 0, rc; __u64 bits; @@ -131,8 +127,6 @@ int mdc_setattr(struct obd_export *exp, struct md_op_data *op_data, return rc; } - rpc_lock = obd->u.cli.cl_rpc_lock; - if (op_data->op_attr.ia_valid & (ATTR_MTIME | ATTR_CTIME)) CDEBUG(D_INODE, "setting mtime %ld, ctime %ld\n", LTIME_S(op_data->op_attr.ia_mtime), @@ -141,7 +135,7 @@ int mdc_setattr(struct obd_export *exp, struct md_op_data *op_data, ptlrpc_request_set_replen(req); - rc = mdc_reint(req, rpc_lock, LUSTRE_IMP_FULL); + rc = mdc_reint(req, LUSTRE_IMP_FULL); if (rc == -ERESTARTSYS) rc = 0; @@ -220,7 +214,7 @@ int mdc_create(struct obd_export *exp, struct md_op_data *op_data, } level = LUSTRE_IMP_FULL; resend: - rc = mdc_reint(req, exp->exp_obd->u.cli.cl_rpc_lock, level); + rc = mdc_reint(req, level); /* Resend if we were told to. */ if (rc == -ERESTARTSYS) { @@ -292,7 +286,7 @@ int mdc_unlink(struct obd_export *exp, struct md_op_data *op_data, *request = req; - rc = mdc_reint(req, obd->u.cli.cl_rpc_lock, LUSTRE_IMP_FULL); + rc = mdc_reint(req, LUSTRE_IMP_FULL); if (rc == -ERESTARTSYS) rc = 0; return rc; @@ -302,7 +296,6 @@ int mdc_link(struct obd_export *exp, struct md_op_data *op_data, struct ptlrpc_request **request) { LIST_HEAD(cancels); - struct obd_device *obd = exp->exp_obd; struct ptlrpc_request *req; int count = 0, rc; @@ -334,7 +327,7 @@ int mdc_link(struct obd_export *exp, struct md_op_data *op_data, mdc_link_pack(req, op_data); ptlrpc_request_set_replen(req); - rc = mdc_reint(req, obd->u.cli.cl_rpc_lock, LUSTRE_IMP_FULL); + rc = mdc_reint(req, LUSTRE_IMP_FULL); *request = req; if (rc == -ERESTARTSYS) rc = 0; @@ -398,7 +391,7 @@ int mdc_rename(struct obd_export *exp, struct md_op_data *op_data, obd->u.cli.cl_default_mds_easize); ptlrpc_request_set_replen(req); - rc = mdc_reint(req, obd->u.cli.cl_rpc_lock, LUSTRE_IMP_FULL); + rc = mdc_reint(req, LUSTRE_IMP_FULL); *request = req; if (rc == -ERESTARTSYS) rc = 0; diff --git a/drivers/staging/lustre/lustre/mdc/mdc_request.c b/drivers/staging/lustre/lustre/mdc/mdc_request.c index c620f5c..e3a167a 100644 --- a/drivers/staging/lustre/lustre/mdc/mdc_request.c +++ b/drivers/staging/lustre/lustre/mdc/mdc_request.c @@ -327,12 +327,12 @@ static int mdc_xattr_common(struct obd_export *exp, /* make rpc */ if (opcode == MDS_REINT) - mdc_get_rpc_lock(exp->exp_obd->u.cli.cl_rpc_lock, NULL); + mdc_get_mod_rpc_slot(req, NULL); rc = ptlrpc_queue_wait(req); if (opcode == MDS_REINT) - mdc_put_rpc_lock(exp->exp_obd->u.cli.cl_rpc_lock, NULL); + mdc_put_mod_rpc_slot(req, NULL); if (rc) ptlrpc_req_finished(req); @@ -775,9 +775,9 @@ static int mdc_close(struct obd_export *exp, struct md_op_data *op_data, ptlrpc_request_set_replen(req); - mdc_get_rpc_lock(obd->u.cli.cl_close_lock, NULL); + mdc_get_mod_rpc_slot(req, NULL); rc = ptlrpc_queue_wait(req); - mdc_put_rpc_lock(obd->u.cli.cl_close_lock, NULL); + mdc_put_mod_rpc_slot(req, NULL); if (!req->rq_repmsg) { CDEBUG(D_RPCTRACE, "request failed to send: %p, %d\n", req, @@ -1490,9 +1490,9 @@ static int mdc_ioc_hsm_progress(struct obd_export *exp, ptlrpc_request_set_replen(req); - mdc_get_rpc_lock(exp->exp_obd->u.cli.cl_rpc_lock, NULL); + mdc_get_mod_rpc_slot(req, NULL); rc = ptlrpc_queue_wait(req); - mdc_put_rpc_lock(exp->exp_obd->u.cli.cl_rpc_lock, NULL); + mdc_put_mod_rpc_slot(req, NULL); out: ptlrpc_req_finished(req); return rc; @@ -1670,9 +1670,9 @@ static int mdc_ioc_hsm_state_set(struct obd_export *exp, ptlrpc_request_set_replen(req); - mdc_get_rpc_lock(exp->exp_obd->u.cli.cl_rpc_lock, NULL); + mdc_get_mod_rpc_slot(req, NULL); rc = ptlrpc_queue_wait(req); - mdc_put_rpc_lock(exp->exp_obd->u.cli.cl_rpc_lock, NULL); + mdc_put_mod_rpc_slot(req, NULL); out: ptlrpc_req_finished(req); return rc; @@ -1735,9 +1735,9 @@ static int mdc_ioc_hsm_request(struct obd_export *exp, ptlrpc_request_set_replen(req); - mdc_get_rpc_lock(exp->exp_obd->u.cli.cl_rpc_lock, NULL); + mdc_get_mod_rpc_slot(req, NULL); rc = ptlrpc_queue_wait(req); - mdc_put_rpc_lock(exp->exp_obd->u.cli.cl_rpc_lock, NULL); + mdc_put_mod_rpc_slot(req, NULL); out: ptlrpc_req_finished(req); return rc; @@ -2582,29 +2582,17 @@ static void mdc_llog_finish(struct obd_device *obd) static int mdc_setup(struct obd_device *obd, struct lustre_cfg *cfg) { - struct client_obd *cli = &obd->u.cli; struct lprocfs_static_vars lvars = { NULL }; int rc; - cli->cl_rpc_lock = kzalloc(sizeof(*cli->cl_rpc_lock), GFP_NOFS); - if (!cli->cl_rpc_lock) - return -ENOMEM; - mdc_init_rpc_lock(cli->cl_rpc_lock); - rc = ptlrpcd_addref(); if (rc < 0) - goto err_rpc_lock; - - cli->cl_close_lock = kzalloc(sizeof(*cli->cl_close_lock), GFP_NOFS); - if (!cli->cl_close_lock) { - rc = -ENOMEM; - goto err_ptlrpcd_decref; - } - mdc_init_rpc_lock(cli->cl_close_lock); + return rc; rc = client_obd_setup(obd, cfg); if (rc) - goto err_close_lock; + goto err_ptlrpcd_decref; + lprocfs_mdc_init_vars(&lvars); lprocfs_obd_setup(obd, lvars.obd_vars, lvars.sysfs_vars); sptlrpc_lprocfs_cliobd_attach(obd); @@ -2621,17 +2609,10 @@ static int mdc_setup(struct obd_device *obd, struct lustre_cfg *cfg) return rc; } - spin_lock_init(&cli->cl_mod_rpcs_lock); - cli->cl_max_mod_rpcs_in_flight = OBD_MAX_RIF_DEFAULT - 1; - return rc; -err_close_lock: - kfree(cli->cl_close_lock); err_ptlrpcd_decref: ptlrpcd_decref(); -err_rpc_lock: - kfree(cli->cl_rpc_lock); return rc; } @@ -2679,11 +2660,6 @@ static int mdc_precleanup(struct obd_device *obd, enum obd_cleanup_stage stage) static int mdc_cleanup(struct obd_device *obd) { - struct client_obd *cli = &obd->u.cli; - - kfree(cli->cl_rpc_lock); - kfree(cli->cl_close_lock); - ptlrpcd_decref(); return client_obd_cleanup(obd); diff --git a/drivers/staging/lustre/lustre/obdclass/genops.c b/drivers/staging/lustre/lustre/obdclass/genops.c index 62e6636..438d619 100644 --- a/drivers/staging/lustre/lustre/obdclass/genops.c +++ b/drivers/staging/lustre/lustre/obdclass/genops.c @@ -1461,6 +1461,7 @@ int obd_set_max_mod_rpcs_in_flight(struct client_obd *cli, __u16 max) { struct obd_connect_data *ocd; u16 maxmodrpcs; + u16 prev; if (max > OBD_MAX_RIF_MAX || max < 1) return -ERANGE; @@ -1486,10 +1487,178 @@ int obd_set_max_mod_rpcs_in_flight(struct client_obd *cli, __u16 max) return -ERANGE; } + spin_lock(&cli->cl_mod_rpcs_lock); + + prev = cli->cl_max_mod_rpcs_in_flight; cli->cl_max_mod_rpcs_in_flight = max; - /* will have to wakeup waiters if max has been increased */ + /* wakeup waiters if limit has been increased */ + if (cli->cl_max_mod_rpcs_in_flight > prev) + wake_up(&cli->cl_mod_rpcs_waitq); + + spin_unlock(&cli->cl_mod_rpcs_lock); return 0; } EXPORT_SYMBOL(obd_set_max_mod_rpcs_in_flight); + +#define pct(a, b) (b ? (a * 100) / b : 0) + +int obd_mod_rpc_stats_seq_show(struct client_obd *cli, struct seq_file *seq) +{ + unsigned long mod_tot = 0, mod_cum; + struct timespec64 now; + int i; + + ktime_get_real_ts64(&now); + + spin_lock(&cli->cl_mod_rpcs_lock); + + seq_printf(seq, "snapshot_time: %llu.%9lu (secs.nsecs)\n", + (s64)now.tv_sec, (unsigned long)now.tv_nsec); + seq_printf(seq, "modify_RPCs_in_flight: %hu\n", + cli->cl_mod_rpcs_in_flight); + + seq_puts(seq, "\n\t\t\tmodify\n"); + seq_puts(seq, "rpcs in flight rpcs %% cum %%\n"); + + mod_tot = lprocfs_oh_sum(&cli->cl_mod_rpcs_hist); + + mod_cum = 0; + for (i = 0; i < OBD_HIST_MAX; i++) { + unsigned long mod = cli->cl_mod_rpcs_hist.oh_buckets[i]; + + mod_cum += mod; + seq_printf(seq, "%d:\t\t%10lu %3lu %3lu\n", + i, mod, pct(mod, mod_tot), + pct(mod_cum, mod_tot)); + if (mod_cum == mod_tot) + break; + } + + spin_unlock(&cli->cl_mod_rpcs_lock); + + return 0; +} +EXPORT_SYMBOL(obd_mod_rpc_stats_seq_show); +#undef pct + +/* + * The number of modify RPCs sent in parallel is limited + * because the server has a finite number of slots per client to + * store request result and ensure reply reconstruction when needed. + * On the client, this limit is stored in cl_max_mod_rpcs_in_flight + * that takes into account server limit and cl_max_rpcs_in_flight + * value. + * On the MDC client, to avoid a potential deadlock (see Bugzilla 3462), + * one close request is allowed above the maximum. + */ +static inline bool obd_mod_rpc_slot_avail_locked(struct client_obd *cli, + bool close_req) +{ + bool avail; + + /* A slot is available if + * - number of modify RPCs in flight is less than the max + * - it's a close RPC and no other close request is in flight + */ + avail = cli->cl_mod_rpcs_in_flight < cli->cl_max_mod_rpcs_in_flight || + (close_req && !cli->cl_close_rpcs_in_flight); + + return avail; +} + +static inline bool obd_mod_rpc_slot_avail(struct client_obd *cli, + bool close_req) +{ + bool avail; + + spin_lock(&cli->cl_mod_rpcs_lock); + avail = obd_mod_rpc_slot_avail_locked(cli, close_req); + spin_unlock(&cli->cl_mod_rpcs_lock); + return avail; +} + +/* Get a modify RPC slot from the obd client @cli according + * to the kind of operation @opc that is going to be sent + * and the intent @it of the operation if it applies. + * If the maximum number of modify RPCs in flight is reached + * the thread is put to sleep. + * Returns the tag to be set in the request message. Tag 0 + * is reserved for non-modifying requests. + */ +u16 obd_get_mod_rpc_slot(struct client_obd *cli, __u32 opc, + struct lookup_intent *it) +{ + struct l_wait_info lwi = LWI_INTR(NULL, NULL); + bool close_req = false; + u16 i, max; + + /* read-only metadata RPCs don't consume a slot on MDT + * for reply reconstruction + */ + if (it && (it->it_op == IT_GETATTR || it->it_op == IT_LOOKUP || + it->it_op == IT_LAYOUT || it->it_op == IT_READDIR)) + return 0; + + if (opc == MDS_CLOSE) + close_req = true; + + do { + spin_lock(&cli->cl_mod_rpcs_lock); + max = cli->cl_max_mod_rpcs_in_flight; + if (obd_mod_rpc_slot_avail_locked(cli, close_req)) { + /* there is a slot available */ + cli->cl_mod_rpcs_in_flight++; + if (close_req) + cli->cl_close_rpcs_in_flight++; + lprocfs_oh_tally(&cli->cl_mod_rpcs_hist, + cli->cl_mod_rpcs_in_flight); + /* find a free tag */ + i = find_first_zero_bit(cli->cl_mod_tag_bitmap, + max + 1); + LASSERT(i < OBD_MAX_RIF_MAX); + LASSERT(!test_and_set_bit(i, cli->cl_mod_tag_bitmap)); + spin_unlock(&cli->cl_mod_rpcs_lock); + /* tag 0 is reserved for non-modify RPCs */ + return i + 1; + } + spin_unlock(&cli->cl_mod_rpcs_lock); + + CDEBUG(D_RPCTRACE, "%s: sleeping for a modify RPC slot opc %u, max %hu\n", + cli->cl_import->imp_obd->obd_name, opc, max); + + l_wait_event(cli->cl_mod_rpcs_waitq, + obd_mod_rpc_slot_avail(cli, close_req), &lwi); + } while (true); +} +EXPORT_SYMBOL(obd_get_mod_rpc_slot); + +/* + * Put a modify RPC slot from the obd client @cli according + * to the kind of operation @opc that has been sent and the + * intent @it of the operation if it applies. + */ +void obd_put_mod_rpc_slot(struct client_obd *cli, u32 opc, + struct lookup_intent *it, u16 tag) +{ + bool close_req = false; + + if (it && (it->it_op == IT_GETATTR || it->it_op == IT_LOOKUP || + it->it_op == IT_LAYOUT || it->it_op == IT_READDIR)) + return; + + if (opc == MDS_CLOSE) + close_req = true; + + spin_lock(&cli->cl_mod_rpcs_lock); + cli->cl_mod_rpcs_in_flight--; + if (close_req) + cli->cl_close_rpcs_in_flight--; + /* release the tag in the bitmap */ + LASSERT(tag - 1 < OBD_MAX_RIF_MAX); + LASSERT(test_and_clear_bit(tag - 1, cli->cl_mod_tag_bitmap) != 0); + spin_unlock(&cli->cl_mod_rpcs_lock); + wake_up(&cli->cl_mod_rpcs_waitq); +} +EXPORT_SYMBOL(obd_put_mod_rpc_slot); -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:30:30 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:30:30 -0500 Subject: [lustre-devel] [PATCH 00/35] second batch of missing lustre 2.8 patches Message-ID: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> More fixes missing from the upstream client. Also a nice cleanup with the removal of cl_req which is no longer needed. More cleanup for lustre_idl.h which is a uapi header. Like the last batch these patches are independent of order. Aditya Pandit (1): staging: lustre: llite: tar restore fails for HSM released files. Alexander Boyko (1): staging: lustre: ptlrpc: race at req processing Andreas Dilger (4): staging: lustre: obdclass: remove structure holes to reduce memory staging: lustre: mdc: remove console spew from mdc_ioc_fid2path staging: lustre: misc: clean up DFID related error messages staging: lustre: idl: clean up file attribute flags Andrew Wellington (1): staging: lustre: llite: support SELinux context labelling Ben Evans (1): staging: lustre: ptlrpc: Move IT_* definitions to lustre_idl.h Bobi Jam (1): staging: lustre: lov: init LOV stripe type beforehand Chennaiah Palla (1): staging: lustre: obdclass: add export for lprocfs_stats_alloc_one() Gregoire Pichon (1): staging: lustre: osc: fix max_dirty_mb tunable setting limit Henri Doreau (3): staging: lustre: hsm: Use file lease to implement migration staging: lustre: nrs: serialize executions of nrs_policy_stop staging: lustre: obd: Remove dead code in precleanup Hiroya Nozaki (1): staging: lustre: llite: ll_write_begin/end not passing on errors Hongchao Zhang (1): staging: lustre: ptlrpc: reset imp_replay_cursor Jian Yu (1): staging: lustre: mount: fix lmd_parse() to handle commas in expr_list Jinshan Xiong (4): staging: lustre: osc: Performance tune for LRU staging: lustre: clio: get rid of cl_req staging: lustre: osc: osc_extent should hold refcount to osc_object staging: lustre: osc: Do not merge extents with partial pages John L. Hammond (4): staging: lustre: obd: rename obd_unpackmd() to md_unpackmd() staging: lustre: lov: avoid infinite loop in lsm_alloc_plain() staging: lustre: ldlm: improve lock timeout messages staging: lustre: hsm: prevent migration of HSM archived files Lai Siyao (1): staging: lustre: statahead: lock leaks if statahead file recreated Liang Zhen (2): staging: lustre: ptlrpc: mbits is sent within ptlrpc_body staging: lustre: lnet: add offset for selftest brw Mikhail Pershin (1): staging: lustre: llog: fix wrong offset in llog_process_thread() Oleg Drokin (1): staging: lustre: osc: Remove remains of osc_ast_guard wang di (5): staging: lustre: lmv: lock necessary part of lmv_add_target staging: lustre: mgc: IR log failure should not stop mount staging: lustre: lmv: revalidate the dentry for striped dir staging: lustre: llite: lookup master inode by ilookup5_nowait staging: lustre: llite: clear dir stripe md in ll_iget drivers/staging/lustre/include/linux/lnet/lnetst.h | 2 + drivers/staging/lustre/lnet/selftest/brw_test.c | 73 ++++--- drivers/staging/lustre/lnet/selftest/conrpc.c | 8 +- drivers/staging/lustre/lnet/selftest/framework.c | 2 +- drivers/staging/lustre/lnet/selftest/rpc.c | 19 +- drivers/staging/lustre/lnet/selftest/rpc.h | 2 +- drivers/staging/lustre/lnet/selftest/selftest.h | 5 +- drivers/staging/lustre/lustre/fld/fld_internal.h | 5 - drivers/staging/lustre/lustre/fld/fld_request.c | 8 - drivers/staging/lustre/lustre/include/cl_object.h | 242 ++------------------ .../lustre/lustre/include/lustre/lustre_idl.h | 53 ++++- .../lustre/lustre/include/lustre/lustre_user.h | 19 +- drivers/staging/lustre/lustre/include/lustre_dlm.h | 2 +- drivers/staging/lustre/lustre/include/lustre_fld.h | 2 - drivers/staging/lustre/lustre/include/lustre_lmv.h | 13 +- drivers/staging/lustre/lustre/include/lustre_net.h | 12 +- .../lustre/lustre/include/lustre_req_layout.h | 2 +- drivers/staging/lustre/lustre/include/obd.h | 104 +++++---- drivers/staging/lustre/lustre/include/obd_class.h | 68 ++---- drivers/staging/lustre/lustre/ldlm/ldlm_lock.c | 4 +- drivers/staging/lustre/lustre/ldlm/ldlm_request.c | 6 +- drivers/staging/lustre/lustre/llite/Makefile | 2 +- drivers/staging/lustre/lustre/llite/file.c | 237 ++++++++++++------- .../staging/lustre/lustre/llite/llite_internal.h | 12 +- drivers/staging/lustre/lustre/llite/llite_lib.c | 55 ++++- drivers/staging/lustre/lustre/llite/lproc_llite.c | 4 - drivers/staging/lustre/lustre/llite/namei.c | 46 ++++- drivers/staging/lustre/lustre/llite/rw26.c | 5 + drivers/staging/lustre/lustre/llite/statahead.c | 2 + drivers/staging/lustre/lustre/llite/vvp_dev.c | 12 - drivers/staging/lustre/lustre/llite/vvp_internal.h | 8 - drivers/staging/lustre/lustre/llite/vvp_io.c | 78 +++++-- drivers/staging/lustre/lustre/llite/vvp_object.c | 23 ++- drivers/staging/lustre/lustre/llite/vvp_req.c | 115 --------- drivers/staging/lustre/lustre/llite/xattr.c | 61 +++++- drivers/staging/lustre/lustre/lmv/lmv_intent.c | 11 +- drivers/staging/lustre/lustre/lmv/lmv_internal.h | 3 - drivers/staging/lustre/lustre/lmv/lmv_obd.c | 122 ++++------ .../staging/lustre/lustre/lov/lov_cl_internal.h | 26 -- drivers/staging/lustre/lustre/lov/lov_dev.c | 52 ----- drivers/staging/lustre/lustre/lov/lov_ea.c | 14 +- drivers/staging/lustre/lustre/lov/lov_internal.h | 5 + drivers/staging/lustre/lustre/lov/lov_obd.c | 24 -- drivers/staging/lustre/lustre/lov/lov_object.c | 1 + drivers/staging/lustre/lustre/lov/lovsub_dev.c | 61 ----- drivers/staging/lustre/lustre/lov/lovsub_object.c | 22 ++- drivers/staging/lustre/lustre/mdc/mdc_lib.c | 34 ++-- drivers/staging/lustre/lustre/mdc/mdc_request.c | 52 ++--- drivers/staging/lustre/lustre/mgc/mgc_request.c | 55 +++-- drivers/staging/lustre/lustre/obdclass/cl_io.c | 231 +------------------ drivers/staging/lustre/lustre/obdclass/cl_page.c | 19 +-- drivers/staging/lustre/lustre/obdclass/llog.c | 81 +++++-- .../lustre/lustre/obdclass/lprocfs_status.c | 2 + .../staging/lustre/lustre/obdclass/obd_config.c | 2 +- drivers/staging/lustre/lustre/obdclass/obd_mount.c | 96 +++++++- .../staging/lustre/lustre/obdecho/echo_client.c | 4 - drivers/staging/lustre/lustre/osc/lproc_osc.c | 8 +- drivers/staging/lustre/lustre/osc/osc_cache.c | 56 ++--- .../staging/lustre/lustre/osc/osc_cl_internal.h | 44 ++--- drivers/staging/lustre/lustre/osc/osc_dev.c | 15 +-- drivers/staging/lustre/lustre/osc/osc_internal.h | 7 +- drivers/staging/lustre/lustre/osc/osc_io.c | 146 +----------- drivers/staging/lustre/lustre/osc/osc_object.c | 82 ++++++- drivers/staging/lustre/lustre/osc/osc_page.c | 146 +++++++----- drivers/staging/lustre/lustre/osc/osc_quota.c | 4 +- drivers/staging/lustre/lustre/osc/osc_request.c | 207 ++++++----------- drivers/staging/lustre/lustre/ptlrpc/client.c | 76 ++++--- drivers/staging/lustre/lustre/ptlrpc/import.c | 1 + drivers/staging/lustre/lustre/ptlrpc/layout.c | 10 +- drivers/staging/lustre/lustre/ptlrpc/niobuf.c | 63 +++--- drivers/staging/lustre/lustre/ptlrpc/nrs.c | 16 +- .../staging/lustre/lustre/ptlrpc/pack_generic.c | 20 ++- .../staging/lustre/lustre/ptlrpc/ptlrpc_internal.h | 1 + drivers/staging/lustre/lustre/ptlrpc/wiretest.c | 52 ++++- 74 files changed, 1387 insertions(+), 1795 deletions(-) delete mode 100644 drivers/staging/lustre/lustre/llite/vvp_req.c From jsimmons at infradead.org Thu Nov 10 17:30:34 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:30:34 -0500 Subject: [lustre-devel] [PATCH 04/35] staging: lustre: lov: init LOV stripe type beforehand In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <1478799065-24841-5-git-send-email-jsimmons@infradead.org> From: Bobi Jam When lu_object_alloc() reaches to LOV object init, we need initialize its stripe type beforehand, so that if something wrong in the conf buffer, the object chain need to be traversed to free what has been allocated, with LOV object type be set as LLT_EMPTY, and when the LOV part is reached, it won't panic without knowing what stripe type it is. This patch also improves debug messages in lsm_unpackmd_common(), and does not return error if the LOV device is still processing config log while trying to verify a layout buffer. Signed-off-by: Bobi Jam Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6744 Reviewed-on: http://review.whamcloud.com/15362 Reviewed-by: Jinshan Xiong Reviewed-by: Fan Yong Reviewed-by: Andreas Dilger Signed-off-by: James Simmons --- drivers/staging/lustre/lustre/lov/lov_ea.c | 12 ++++++++---- drivers/staging/lustre/lustre/lov/lov_internal.h | 5 +++++ drivers/staging/lustre/lustre/lov/lov_object.c | 1 + 3 files changed, 14 insertions(+), 4 deletions(-) diff --git a/drivers/staging/lustre/lustre/lov/lov_ea.c b/drivers/staging/lustre/lustre/lov/lov_ea.c index 53db170..218f275 100644 --- a/drivers/staging/lustre/lustre/lov/lov_ea.c +++ b/drivers/staging/lustre/lustre/lov/lov_ea.c @@ -177,17 +177,21 @@ static int lsm_unpackmd_common(struct lov_obd *lov, if (lov_oinfo_is_dummy(loi)) continue; - if (loi->loi_ost_idx >= lov->desc.ld_tgt_count) { - CERROR("OST index %d more than OST count %d\n", + if (loi->loi_ost_idx >= lov->desc.ld_tgt_count && + !lov2obd(lov)->obd_process_conf) { + CERROR("%s: OST index %d more than OST count %d\n", + (char *)lov->desc.ld_uuid.uuid, loi->loi_ost_idx, lov->desc.ld_tgt_count); lov_dump_lmm_v1(D_WARNING, lmm); return -EINVAL; } if (!lov->lov_tgts[loi->loi_ost_idx]) { - CERROR("OST index %d missing\n", loi->loi_ost_idx); + CERROR("%s: OST index %d missing\n", + (char *)lov->desc.ld_uuid.uuid, + loi->loi_ost_idx); lov_dump_lmm_v1(D_WARNING, lmm); - return -EINVAL; + continue; } tgt_bytes = lov_tgt_maxbytes(lov->lov_tgts[loi->loi_ost_idx]); diff --git a/drivers/staging/lustre/lustre/lov/lov_internal.h b/drivers/staging/lustre/lustre/lov/lov_internal.h index 5b151bb..774499c 100644 --- a/drivers/staging/lustre/lustre/lov/lov_internal.h +++ b/drivers/staging/lustre/lustre/lov/lov_internal.h @@ -301,4 +301,9 @@ static inline bool lov_oinfo_is_dummy(const struct lov_oinfo *loi) return false; } +static inline struct obd_device *lov2obd(const struct lov_obd *lov) +{ + return container_of0(lov, struct obd_device, u.lov); +} + #endif diff --git a/drivers/staging/lustre/lustre/lov/lov_object.c b/drivers/staging/lustre/lustre/lov/lov_object.c index 317311c..76d4256 100644 --- a/drivers/staging/lustre/lustre/lov/lov_object.c +++ b/drivers/staging/lustre/lustre/lov/lov_object.c @@ -802,6 +802,7 @@ int lov_object_init(const struct lu_env *env, struct lu_object *obj, init_waitqueue_head(&lov->lo_waitq); cl_object_page_init(lu2cl(obj), sizeof(struct lov_page)); + lov->lo_type = LLT_EMPTY; if (cconf->u.coc_layout.lb_buf) { lsm = lov_unpackmd(dev->ld_lov, cconf->u.coc_layout.lb_buf, -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:30:43 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:30:43 -0500 Subject: [lustre-devel] [PATCH 13/35] staging: lustre: llite: lookup master inode by ilookup5_nowait In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <1478799065-24841-14-git-send-email-jsimmons@infradead.org> From: wang di Do not lookup master inode by ilookup5, instead it should use ilookup5_nowait, otherwise it will cause dead lock, 1. Client1 send chmod req to the MDT0, then on MDT0, it enqueues master and all of its slaves lock, (mdt_attr_set() ->mdt_lock_slaves()), after gets master and stripe0 lock, it will send the enqueue request(for stripe1) to MDT1, then MDT1 finds the lock has been granted to client2. Then MDT1 sends blocking ast to client2. 2. At the same time, client2 tries to unlink the striped dir (rm -rf striped_dir), and during lookup, it will hold the master inode of the striped directory, whose inode state is NEW, then tries to revalidate all of its slaves, (ll_prep_inode()->ll_iget()->ll_read_inode2()-> ll_update_inode().). And it will be blocked on the server side because of 1. 3. Then the client get the blocking_ast request, cancel the lock, but being blocked by ilookup5 in ll_md_blocking_ast(), because the inode state is still NEW. Signed-off-by: wang di Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-5344 Reviewed-on: http://review.whamcloud.com/16066 Reviewed-by: John L. Hammond Reviewed-by: Lai Siyao Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- drivers/staging/lustre/lustre/llite/namei.c | 36 ++++++++++++++++++++++++--- 1 files changed, 32 insertions(+), 4 deletions(-) diff --git a/drivers/staging/lustre/lustre/llite/namei.c b/drivers/staging/lustre/lustre/llite/namei.c index c268f32..9ea43e1 100644 --- a/drivers/staging/lustre/lustre/llite/namei.c +++ b/drivers/staging/lustre/lustre/llite/namei.c @@ -286,10 +286,38 @@ int ll_md_blocking_ast(struct ldlm_lock *lock, struct ldlm_lock_desc *desc, hash = cl_fid_build_ino(&lli->lli_pfid, ll_need_32bit_api(ll_i2sbi(inode))); - - master_inode = ilookup5(inode->i_sb, hash, - ll_test_inode_by_fid, - (void *)&lli->lli_pfid); + /* + * Do not lookup the inode with ilookup5, + * otherwise it will cause dead lock, + * + * 1. Client1 send chmod req to the MDT0, then + * on MDT0, it enqueues master and all of its + * slaves lock, (mdt_attr_set() -> + * mdt_lock_slaves()), after gets master and + * stripe0 lock, it will send the enqueue req + * (for stripe1) to MDT1, then MDT1 finds the + * lock has been granted to client2. Then MDT1 + * sends blocking ast to client2. + * + * 2. At the same time, client2 tries to unlink + * the striped dir (rm -rf striped_dir), and + * during lookup, it will hold the master inode + * of the striped directory, whose inode state + * is NEW, then tries to revalidate all of its + * slaves, (ll_prep_inode()->ll_iget()-> + * ll_read_inode2()-> ll_update_inode().). And + * it will be blocked on the server side because + * of 1. + * + * 3. Then the client get the blocking_ast req, + * cancel the lock, but being blocked if using + * ->ilookup5()), because master inode state is + * NEW. + */ + master_inode = ilookup5_nowait(inode->i_sb, + hash, + ll_test_inode_by_fid, + (void *)&lli->lli_pfid); if (master_inode) { ll_invalidate_negative_children(master_inode); iput(master_inode); -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:30:38 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:30:38 -0500 Subject: [lustre-devel] [PATCH 08/35] staging: lustre: lmv: lock necessary part of lmv_add_target In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <1478799065-24841-9-git-send-email-jsimmons@infradead.org> From: wang di Release lmv_init_mutex once the new target is added into lmv_tgt_desc, so lmv_obd_connect will not be serialized. New target should be allowed to added to fld client lists, so FLD can always choose new added target to do the FLD lookup request, and also remove some noise error messages in this process. Signed-off-by: wang di Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6713 Reviewed-on: http://review.whamcloud.com/15269 Reviewed-by: Niu Yawei Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- drivers/staging/lustre/lustre/fld/fld_internal.h | 5 -- drivers/staging/lustre/lustre/fld/fld_request.c | 8 --- drivers/staging/lustre/lustre/include/lustre_fld.h | 2 - drivers/staging/lustre/lustre/lmv/lmv_obd.c | 53 ++++++++++---------- 4 files changed, 27 insertions(+), 41 deletions(-) diff --git a/drivers/staging/lustre/lustre/fld/fld_internal.h b/drivers/staging/lustre/lustre/fld/fld_internal.h index 08eaec7..4a7f0b7 100644 --- a/drivers/staging/lustre/lustre/fld/fld_internal.h +++ b/drivers/staging/lustre/lustre/fld/fld_internal.h @@ -62,11 +62,6 @@ #include "../include/lustre_req_layout.h" #include "../include/lustre_fld.h" -enum { - LUSTRE_FLD_INIT = 1 << 0, - LUSTRE_FLD_RUN = 1 << 1 -}; - struct fld_stats { __u64 fst_count; __u64 fst_cache; diff --git a/drivers/staging/lustre/lustre/fld/fld_request.c b/drivers/staging/lustre/lustre/fld/fld_request.c index 0de72b7..4cade7a 100644 --- a/drivers/staging/lustre/lustre/fld/fld_request.c +++ b/drivers/staging/lustre/lustre/fld/fld_request.c @@ -159,11 +159,6 @@ int fld_client_add_target(struct lu_client_fld *fld, LASSERT(name); LASSERT(tar->ft_srv || tar->ft_exp); - if (fld->lcf_flags != LUSTRE_FLD_INIT) { - CERROR("%s: Attempt to add target %s (idx %llu) on fly - skip it\n", - fld->lcf_name, name, tar->ft_idx); - return 0; - } CDEBUG(D_INFO, "%s: Adding target %s (idx %llu)\n", fld->lcf_name, name, tar->ft_idx); @@ -282,7 +277,6 @@ int fld_client_init(struct lu_client_fld *fld, fld->lcf_count = 0; spin_lock_init(&fld->lcf_lock); fld->lcf_hash = &fld_hash[hash]; - fld->lcf_flags = LUSTRE_FLD_INIT; INIT_LIST_HEAD(&fld->lcf_targets); cache_size = FLD_CLIENT_CACHE_SIZE / @@ -421,8 +415,6 @@ int fld_client_lookup(struct lu_client_fld *fld, u64 seq, u32 *mds, struct lu_fld_target *target; int rc; - fld->lcf_flags |= LUSTRE_FLD_RUN; - rc = fld_cache_lookup(fld->lcf_cache, seq, &res); if (rc == 0) { *mds = res.lsr_index; diff --git a/drivers/staging/lustre/lustre/include/lustre_fld.h b/drivers/staging/lustre/lustre/include/lustre_fld.h index 932410d..6ef1b03 100644 --- a/drivers/staging/lustre/lustre/include/lustre_fld.h +++ b/drivers/staging/lustre/lustre/include/lustre_fld.h @@ -103,8 +103,6 @@ struct lu_client_fld { /** Client fld debugfs entry name. */ char lcf_name[LUSTRE_MDT_MAXNAMELEN]; - - int lcf_flags; }; /* Client methods */ diff --git a/drivers/staging/lustre/lustre/lmv/lmv_obd.c b/drivers/staging/lustre/lustre/lmv/lmv_obd.c index ca78aff..2338556 100644 --- a/drivers/staging/lustre/lustre/lmv/lmv_obd.c +++ b/drivers/staging/lustre/lustre/lmv/lmv_obd.c @@ -387,27 +387,23 @@ static int lmv_add_target(struct obd_device *obd, struct obd_uuid *uuidp, __u32 index, int gen) { struct lmv_obd *lmv = &obd->u.lmv; + struct obd_device *mdc_obd; struct lmv_tgt_desc *tgt; int orig_tgt_count = 0; int rc = 0; CDEBUG(D_CONFIG, "Target uuid: %s. index %d\n", uuidp->uuid, index); - mutex_lock(&lmv->lmv_init_mutex); - - if (lmv->desc.ld_tgt_count == 0) { - struct obd_device *mdc_obd; - - mdc_obd = class_find_client_obd(uuidp, LUSTRE_MDC_NAME, - &obd->obd_uuid); - if (!mdc_obd) { - mutex_unlock(&lmv->lmv_init_mutex); - CERROR("%s: Target %s not attached: rc = %d\n", - obd->obd_name, uuidp->uuid, -EINVAL); - return -EINVAL; - } + mdc_obd = class_find_client_obd(uuidp, LUSTRE_MDC_NAME, + &obd->obd_uuid); + if (!mdc_obd) { + CERROR("%s: Target %s not attached: rc = %d\n", + obd->obd_name, uuidp->uuid, -EINVAL); + return -EINVAL; } + mutex_lock(&lmv->lmv_init_mutex); + if ((index < lmv->tgts_size) && lmv->tgts[index]) { tgt = lmv->tgts[index]; CERROR("%s: UUID %s already assigned at LOV target index %d: rc = %d\n", @@ -463,22 +459,27 @@ static int lmv_add_target(struct obd_device *obd, struct obd_uuid *uuidp, lmv->desc.ld_tgt_count = index + 1; } - if (lmv->connected) { - rc = lmv_connect_mdc(obd, tgt); - if (rc) { - spin_lock(&lmv->lmv_lock); - if (lmv->desc.ld_tgt_count == index + 1) - lmv->desc.ld_tgt_count = orig_tgt_count; - memset(tgt, 0, sizeof(*tgt)); - spin_unlock(&lmv->lmv_lock); - } else { - int easize = sizeof(struct lmv_stripe_md) + - lmv->desc.ld_tgt_count * sizeof(struct lu_fid); - lmv_init_ea_size(obd->obd_self_export, easize, 0); - } + if (!lmv->connected) { + /* lmv_check_connect() will connect this target. */ + mutex_unlock(&lmv->lmv_init_mutex); + return rc; } + /* Otherwise let's connect it ourselves */ mutex_unlock(&lmv->lmv_init_mutex); + rc = lmv_connect_mdc(obd, tgt); + if (rc) { + spin_lock(&lmv->lmv_lock); + if (lmv->desc.ld_tgt_count == index + 1) + lmv->desc.ld_tgt_count = orig_tgt_count; + memset(tgt, 0, sizeof(*tgt)); + spin_unlock(&lmv->lmv_lock); + } else { + int easize = sizeof(struct lmv_stripe_md) + + lmv->desc.ld_tgt_count * sizeof(struct lu_fid); + lmv_init_ea_size(obd->obd_self_export, easize, 0); + } + return rc; } -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:30:36 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:30:36 -0500 Subject: [lustre-devel] [PATCH 06/35] staging: lustre: osc: Performance tune for LRU In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <1478799065-24841-7-git-send-email-jsimmons@infradead.org> From: Jinshan Xiong Early launch page LRU work in osc_io_rw_iter_init(); Change the page LRU shrinking policy by OSC attributes; Delete the contented lock osc_object::oo_seatbelt Signed-off-by: Jinshan Xiong Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-5108 Reviewed-on: http://review.whamcloud.com/10458 Reviewed-by: Bobi Jam Reviewed-by: Fan Yong Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- drivers/staging/lustre/lustre/include/obd.h | 35 ++++- drivers/staging/lustre/lustre/osc/lproc_osc.c | 6 +- drivers/staging/lustre/lustre/osc/osc_cache.c | 34 ++--- .../staging/lustre/lustre/osc/osc_cl_internal.h | 19 --- drivers/staging/lustre/lustre/osc/osc_internal.h | 7 +- drivers/staging/lustre/lustre/osc/osc_io.c | 13 ++- drivers/staging/lustre/lustre/osc/osc_object.c | 9 -- drivers/staging/lustre/lustre/osc/osc_page.c | 146 ++++++++++++-------- drivers/staging/lustre/lustre/osc/osc_quota.c | 4 +- drivers/staging/lustre/lustre/osc/osc_request.c | 17 +-- 10 files changed, 160 insertions(+), 130 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/obd.h b/drivers/staging/lustre/lustre/include/obd.h index 5ebcfb8..2ab7560 100644 --- a/drivers/staging/lustre/lustre/include/obd.h +++ b/drivers/staging/lustre/lustre/include/obd.h @@ -247,15 +247,42 @@ struct client_obd { struct obd_histogram cl_read_offset_hist; struct obd_histogram cl_write_offset_hist; - /* lru for osc caching pages */ + /* LRU for osc caching pages */ struct cl_client_cache *cl_cache; - struct list_head cl_lru_osc; /* member of cl_cache->ccc_lru */ + /** member of cl_cache->ccc_lru */ + struct list_head cl_lru_osc; + /** # of available LRU slots left in the per-OSC cache. + * Available LRU slots are shared by all OSCs of the same file system, + * therefore this is a pointer to cl_client_cache::ccc_lru_left. + */ atomic_long_t *cl_lru_left; + /** # of busy LRU pages. A page is considered busy if it's in writeback + * queue, or in transfer. Busy pages can't be discarded so they are not + * in LRU cache. + */ atomic_long_t cl_lru_busy; + /** # of LRU pages in the cache for this client_obd */ atomic_long_t cl_lru_in_list; + /** # of threads are shrinking LRU cache. To avoid contention, it's not + * allowed to have multiple threads shrinking LRU cache. + */ atomic_t cl_lru_shrinkers; - struct list_head cl_lru_list; /* lru page list */ - spinlock_t cl_lru_list_lock; /* page list protector */ + /** The time when this LRU cache was last used. */ + time64_t cl_lru_last_used; + /** stats: how many reclaims have happened for this client_obd. + * reclaim and shrink - shrink is async, voluntarily rebalancing; + * reclaim is sync, initiated by IO thread when the LRU slots are + * in shortage. + */ + u64 cl_lru_reclaim; + /** List of LRU pages for this client_obd */ + struct list_head cl_lru_list; + /** Lock for LRU page list */ + spinlock_t cl_lru_list_lock; + /** # of unstable pages in this client_obd. + * An unstable page is a page state that WRITE RPC has finished but + * the transaction has NOT yet committed. + */ atomic_long_t cl_unstable_count; /* number of in flight destroy rpcs is limited to max_rpcs_in_flight */ diff --git a/drivers/staging/lustre/lustre/osc/lproc_osc.c b/drivers/staging/lustre/lustre/osc/lproc_osc.c index a837362..4421cfe 100644 --- a/drivers/staging/lustre/lustre/osc/lproc_osc.c +++ b/drivers/staging/lustre/lustre/osc/lproc_osc.c @@ -183,10 +183,12 @@ static int osc_cached_mb_seq_show(struct seq_file *m, void *v) seq_printf(m, "used_mb: %ld\n" - "busy_cnt: %ld\n", + "busy_cnt: %ld\n" + "reclaim: %llu\n", (atomic_long_read(&cli->cl_lru_in_list) + atomic_long_read(&cli->cl_lru_busy)) >> shift, - atomic_long_read(&cli->cl_lru_busy)); + atomic_long_read(&cli->cl_lru_busy), + cli->cl_lru_reclaim); return 0; } diff --git a/drivers/staging/lustre/lustre/osc/osc_cache.c b/drivers/staging/lustre/lustre/osc/osc_cache.c index cbbe85e..b939c9c 100644 --- a/drivers/staging/lustre/lustre/osc/osc_cache.c +++ b/drivers/staging/lustre/lustre/osc/osc_cache.c @@ -959,7 +959,7 @@ static int osc_extent_wait(const struct lu_env *env, struct osc_extent *ext, if (rc == -ETIMEDOUT) { OSC_EXTENT_DUMP(D_ERROR, ext, "%s: wait ext to %u timedout, recovery in progress?\n", - osc_export(obj)->exp_obd->obd_name, state); + cli_name(osc_cli(obj)), state); lwi = LWI_INTR(NULL, NULL); rc = l_wait_event(ext->oe_waitq, extent_wait_cb(ext, state), @@ -1327,7 +1327,6 @@ static int osc_completion(const struct lu_env *env, struct osc_async_page *oap, { struct osc_page *opg = oap2osc_page(oap); struct cl_page *page = oap2cl_page(oap); - struct osc_object *obj = cl2osc(opg->ops_cl.cpl_obj); enum cl_req_type crt; int srvlock; @@ -1350,13 +1349,6 @@ static int osc_completion(const struct lu_env *env, struct osc_async_page *oap, /* Clear opg->ops_transfer_pinned before VM lock is released. */ opg->ops_transfer_pinned = 0; - spin_lock(&obj->oo_seatbelt); - LASSERT(opg->ops_submitter); - LASSERT(!list_empty(&opg->ops_inflight)); - list_del_init(&opg->ops_inflight); - opg->ops_submitter = NULL; - spin_unlock(&obj->oo_seatbelt); - opg->ops_submit_time = 0; srvlock = oap->oap_brw_flags & OBD_BRW_SRVLOCK; @@ -1386,10 +1378,10 @@ static int osc_completion(const struct lu_env *env, struct osc_async_page *oap, #define OSC_DUMP_GRANT(lvl, cli, fmt, args...) do { \ struct client_obd *__tmp = (cli); \ - CDEBUG(lvl, "%s: grant { dirty: %ld/%ld dirty_pages: %ld/%lu " \ + CDEBUG(lvl, "%s: grant { dirty: %lu/%lu dirty_pages: %ld/%lu " \ "dropped: %ld avail: %ld, reserved: %ld, flight: %d }" \ "lru {in list: %ld, left: %ld, waiters: %d }" fmt "\n", \ - __tmp->cl_import->imp_obd->obd_name, \ + cli_name(__tmp), \ __tmp->cl_dirty_pages, __tmp->cl_dirty_max_pages, \ atomic_long_read(&obd_dirty_pages), obd_max_dirty_pages, \ __tmp->cl_lost_grant, __tmp->cl_avail_grant, \ @@ -1627,7 +1619,7 @@ static int osc_enter_cache(const struct lu_env *env, struct client_obd *cli, osc_io_unplug_async(env, cli, NULL); CDEBUG(D_CACHE, "%s: sleeping for cache space @ %p for %p\n", - cli->cl_import->imp_obd->obd_name, &ocw, oap); + cli_name(cli), &ocw, oap); rc = l_wait_event(ocw.ocw_waitq, ocw_granted(cli, &ocw), &lwi); @@ -1671,7 +1663,7 @@ static int osc_enter_cache(const struct lu_env *env, struct client_obd *cli, break; default: CDEBUG(D_CACHE, "%s: event for cache space @ %p never arrived due to %d, fall back to sync i/o\n", - cli->cl_import->imp_obd->obd_name, &ocw, rc); + cli_name(cli), &ocw, rc); break; } out: @@ -2250,14 +2242,9 @@ static int osc_io_unplug0(const struct lu_env *env, struct client_obd *cli, return 0; if (!async) { - /* disable osc_lru_shrink() temporarily to avoid - * potential stack overrun problem. LU-2859 - */ - atomic_inc(&cli->cl_lru_shrinkers); spin_lock(&cli->cl_loi_list_lock); osc_check_rpcs(env, cli); spin_unlock(&cli->cl_loi_list_lock); - atomic_dec(&cli->cl_lru_shrinkers); } else { CDEBUG(D_CACHE, "Queue writeback work for client %p.\n", cli); LASSERT(cli->cl_writeback_work); @@ -2479,7 +2466,6 @@ int osc_teardown_async_page(const struct lu_env *env, struct osc_object *obj, struct osc_page *ops) { struct osc_async_page *oap = &ops->ops_oap; - struct osc_extent *ext = NULL; int rc = 0; LASSERT(oap->oap_magic == OAP_MAGIC); @@ -2487,12 +2473,15 @@ int osc_teardown_async_page(const struct lu_env *env, CDEBUG(D_INFO, "teardown oap %p page %p at index %lu.\n", oap, ops, osc_index(oap2osc(oap))); - osc_object_lock(obj); if (!list_empty(&oap->oap_rpc_item)) { CDEBUG(D_CACHE, "oap %p is not in cache.\n", oap); rc = -EBUSY; } else if (!list_empty(&oap->oap_pending_item)) { + struct osc_extent *ext = NULL; + + osc_object_lock(obj); ext = osc_extent_lookup(obj, osc_index(oap2osc(oap))); + osc_object_unlock(obj); /* only truncated pages are allowed to be taken out. * See osc_extent_truncate() and osc_cache_truncate_start() * for details. @@ -2502,10 +2491,9 @@ int osc_teardown_async_page(const struct lu_env *env, osc_index(oap2osc(oap))); rc = -EBUSY; } + if (ext) + osc_extent_put(env, ext); } - osc_object_unlock(obj); - if (ext) - osc_extent_put(env, ext); return rc; } diff --git a/drivers/staging/lustre/lustre/osc/osc_cl_internal.h b/drivers/staging/lustre/lustre/osc/osc_cl_internal.h index 8a55412..513397a 100644 --- a/drivers/staging/lustre/lustre/osc/osc_cl_internal.h +++ b/drivers/staging/lustre/lustre/osc/osc_cl_internal.h @@ -126,16 +126,6 @@ struct osc_object { int oo_contended; unsigned long oo_contention_time; /** - * List of pages in transfer. - */ - struct list_head oo_inflight[CRT_NR]; - /** - * Lock, protecting osc_page::ops_inflight, because a seat-belt is - * locked during take-off and landing. - */ - spinlock_t oo_seatbelt; - - /** * used by the osc to keep track of what objects to build into rpcs. * Protected by client_obd->cli_loi_list_lock. */ @@ -363,15 +353,6 @@ struct osc_page { */ struct list_head ops_lru; /** - * Linkage into a per-osc_object list of pages in flight. For - * debugging. - */ - struct list_head ops_inflight; - /** - * Thread that submitted this page for transfer. For debugging. - */ - struct task_struct *ops_submitter; - /** * Submit time - the time when the page is starting RPC. For debugging. */ unsigned long ops_submit_time; diff --git a/drivers/staging/lustre/lustre/osc/osc_internal.h b/drivers/staging/lustre/lustre/osc/osc_internal.h index 12d3b58..0e181e2 100644 --- a/drivers/staging/lustre/lustre/osc/osc_internal.h +++ b/drivers/staging/lustre/lustre/osc/osc_internal.h @@ -133,7 +133,7 @@ int osc_build_rpc(const struct lu_env *env, struct client_obd *cli, struct list_head *ext_list, int cmd); long osc_lru_shrink(const struct lu_env *env, struct client_obd *cli, long target, bool force); -long osc_lru_reclaim(struct client_obd *cli); +long osc_lru_reclaim(struct client_obd *cli, unsigned long npages); unsigned long osc_ldlm_weigh_ast(struct ldlm_lock *dlmlock); @@ -155,6 +155,11 @@ static inline unsigned long rpcs_in_flight(struct client_obd *cli) return cli->cl_r_in_flight + cli->cl_w_in_flight; } +static inline char *cli_name(struct client_obd *cli) +{ + return cli->cl_import->imp_obd->obd_name; +} + struct osc_device { struct cl_device od_cl; struct obd_export *od_exp; diff --git a/drivers/staging/lustre/lustre/osc/osc_io.c b/drivers/staging/lustre/lustre/osc/osc_io.c index 3b82d0a..e0f81a4 100644 --- a/drivers/staging/lustre/lustre/osc/osc_io.c +++ b/drivers/staging/lustre/lustre/osc/osc_io.c @@ -372,7 +372,7 @@ static int osc_io_rw_iter_init(const struct lu_env *env, npages = max_pages; c = atomic_long_read(cli->cl_lru_left); - if (c < npages && osc_lru_reclaim(cli) > 0) + if (c < npages && osc_lru_reclaim(cli, npages) > 0) c = atomic_long_read(cli->cl_lru_left); while (c >= npages) { if (c == atomic_long_cmpxchg(cli->cl_lru_left, c, c - npages)) { @@ -381,6 +381,17 @@ static int osc_io_rw_iter_init(const struct lu_env *env, } c = atomic_long_read(cli->cl_lru_left); } + if (atomic_long_read(cli->cl_lru_left) < max_pages) { + /* + * If there aren't enough pages in the per-OSC LRU then + * wake up the LRU thread to try and clear out space, so + * we don't block if pages are being dirtied quickly. + */ + CDEBUG(D_CACHE, "%s: queue LRU, left: %lu/%ld.\n", + cli_name(cli), atomic_long_read(cli->cl_lru_left), + max_pages); + (void)ptlrpcd_queue_work(cli->cl_lru_work); + } return 0; } diff --git a/drivers/staging/lustre/lustre/osc/osc_object.c b/drivers/staging/lustre/lustre/osc/osc_object.c index dc0c173..a6dafbf 100644 --- a/drivers/staging/lustre/lustre/osc/osc_object.c +++ b/drivers/staging/lustre/lustre/osc/osc_object.c @@ -71,13 +71,8 @@ static int osc_object_init(const struct lu_env *env, struct lu_object *obj, { struct osc_object *osc = lu2osc(obj); const struct cl_object_conf *cconf = lu2cl_conf(conf); - int i; osc->oo_oinfo = cconf->u.coc_oinfo; - spin_lock_init(&osc->oo_seatbelt); - for (i = 0; i < CRT_NR; ++i) - INIT_LIST_HEAD(&osc->oo_inflight[i]); - INIT_LIST_HEAD(&osc->oo_ready_item); INIT_LIST_HEAD(&osc->oo_hp_ready_item); INIT_LIST_HEAD(&osc->oo_write_item); @@ -103,10 +98,6 @@ static int osc_object_init(const struct lu_env *env, struct lu_object *obj, static void osc_object_free(const struct lu_env *env, struct lu_object *obj) { struct osc_object *osc = lu2osc(obj); - int i; - - for (i = 0; i < CRT_NR; ++i) - LASSERT(list_empty(&osc->oo_inflight[i])); LASSERT(list_empty(&osc->oo_ready_item)); LASSERT(list_empty(&osc->oo_hp_ready_item)); diff --git a/drivers/staging/lustre/lustre/osc/osc_page.c b/drivers/staging/lustre/lustre/osc/osc_page.c index 93248d1..29dfd5d 100644 --- a/drivers/staging/lustre/lustre/osc/osc_page.c +++ b/drivers/staging/lustre/lustre/osc/osc_page.c @@ -86,11 +86,6 @@ static void osc_page_transfer_add(const struct lu_env *env, struct osc_object *obj = cl2osc(opg->ops_cl.cpl_obj); osc_lru_use(osc_cli(obj), opg); - - spin_lock(&obj->oo_seatbelt); - list_add(&opg->ops_inflight, &obj->oo_inflight[crt]); - opg->ops_submitter = current; - spin_unlock(&obj->oo_seatbelt); } int osc_page_cache_add(const struct lu_env *env, @@ -139,7 +134,7 @@ static int osc_page_print(const struct lu_env *env, struct osc_object *obj = cl2osc(slice->cpl_obj); struct client_obd *cli = &osc_export(obj)->exp_obd->u.cli; - return (*printer)(env, cookie, LUSTRE_OSC_NAME "-page@%p %lu: 1< %#x %d %u %s %s > 2< %llu %u %u %#x %#x | %p %p %p > 3< %s %p %d %lu %d > 4< %d %d %d %lu %s | %s %s %s %s > 5< %s %s %s %s | %d %s | %d %s %s>\n", + return (*printer)(env, cookie, LUSTRE_OSC_NAME "-page@%p %lu: 1< %#x %d %u %s %s > 2< %llu %u %u %#x %#x | %p %p %p > 3< %d %lu %d > 4< %d %d %d %lu %s | %s %s %s %s > 5< %s %s %s %s | %d %s | %d %s %s>\n", opg, osc_index(opg), /* 1 */ oap->oap_magic, oap->oap_cmd, @@ -151,8 +146,7 @@ static int osc_page_print(const struct lu_env *env, oap->oap_async_flags, oap->oap_brw_flags, oap->oap_request, oap->oap_cli, obj, /* 3 */ - osc_list(&opg->ops_inflight), - opg->ops_submitter, opg->ops_transfer_pinned, + opg->ops_transfer_pinned, osc_submit_duration(opg), opg->ops_srvlock, /* 4 */ cli->cl_r_in_flight, cli->cl_w_in_flight, @@ -191,14 +185,6 @@ static void osc_page_delete(const struct lu_env *env, LASSERT(0); } - spin_lock(&obj->oo_seatbelt); - if (opg->ops_submitter) { - LASSERT(!list_empty(&opg->ops_inflight)); - list_del_init(&opg->ops_inflight); - opg->ops_submitter = NULL; - } - spin_unlock(&obj->oo_seatbelt); - osc_lru_del(osc_cli(obj), opg); if (slice->cpl_page->cp_type == CPT_CACHEABLE) { @@ -281,10 +267,6 @@ int osc_page_init(const struct lu_env *env, struct cl_object *obj, cl_page_slice_add(page, &opg->ops_cl, obj, index, &osc_page_ops); } - /* ops_inflight and ops_lru are the same field, but it doesn't - * hurt to initialize it twice :-) - */ - INIT_LIST_HEAD(&opg->ops_inflight); INIT_LIST_HEAD(&opg->ops_lru); /* reserve an LRU space for this page */ @@ -342,16 +324,27 @@ void osc_page_submit(const struct lu_env *env, struct osc_page *opg, * OSC to free slots voluntarily to maintain a reasonable number of free slots * at any time. */ - static DECLARE_WAIT_QUEUE_HEAD(osc_lru_waitq); -/* LRU pages are freed in batch mode. OSC should at least free this - * number of pages to avoid running out of LRU budget, and.. + +/** + * LRU pages are freed in batch mode. OSC should at least free this + * number of pages to avoid running out of LRU slots. */ -static const int lru_shrink_min = 2 << (20 - PAGE_SHIFT); /* 2M */ -/* free this number at most otherwise it will take too long time to finish. */ -static const int lru_shrink_max = 8 << (20 - PAGE_SHIFT); /* 8M */ +static inline int lru_shrink_min(struct client_obd *cli) +{ + return cli->cl_max_pages_per_rpc * 2; +} + +/** + * free this number at most otherwise it will take too long time to finish. + */ +static inline int lru_shrink_max(struct client_obd *cli) +{ + return cli->cl_max_pages_per_rpc * cli->cl_max_rpcs_in_flight; +} -/* Check if we can free LRU slots from this OSC. If there exists LRU waiters, +/** + * Check if we can free LRU slots from this OSC. If there exists LRU waiters, * we should free slots aggressively. In this way, slots are freed in a steady * step to maintain fairness among OSCs. * @@ -368,13 +361,19 @@ static int osc_cache_too_much(struct client_obd *cli) /* if it's going to run out LRU slots, we should free some, but not * too much to maintain fairness among OSCs. */ - if (atomic_long_read(cli->cl_lru_left) < cache->ccc_lru_max >> 4) { + if (atomic_long_read(cli->cl_lru_left) < cache->ccc_lru_max >> 2) { if (pages >= budget) - return lru_shrink_max; + return lru_shrink_max(cli); else if (pages >= budget / 2) - return lru_shrink_min; - } else if (pages >= budget * 2) { - return lru_shrink_min; + return lru_shrink_min(cli); + } else { + time64_t duration = ktime_get_real_seconds(); + + /* knock out pages by duration of no IO activity */ + duration =- cli->cl_lru_last_used; + duration >>= 6; /* approximately 1 minute */ + if (duration > 0 && pages >= budget / duration) + return lru_shrink_min(cli); } return 0; } @@ -382,11 +381,21 @@ static int osc_cache_too_much(struct client_obd *cli) int lru_queue_work(const struct lu_env *env, void *data) { struct client_obd *cli = data; + int count; - CDEBUG(D_CACHE, "Run LRU work for client obd %p.\n", cli); + CDEBUG(D_CACHE, "%s: run LRU work for client obd\n", cli_name(cli)); - if (osc_cache_too_much(cli)) - osc_lru_shrink(env, cli, lru_shrink_max, true); + count = osc_cache_too_much(cli); + if (count > 0) { + int rc = osc_lru_shrink(env, cli, count, false); + + CDEBUG(D_CACHE, "%s: shrank %d/%d pages from client obd\n", + cli_name(cli), rc, count); + if (rc >= count) { + CDEBUG(D_CACHE, "%s: queue again\n", cli_name(cli)); + ptlrpcd_queue_work(cli->cl_lru_work); + } + } return 0; } @@ -413,10 +422,10 @@ void osc_lru_add_batch(struct client_obd *cli, struct list_head *plist) list_splice_tail(&lru, &cli->cl_lru_list); atomic_long_sub(npages, &cli->cl_lru_busy); atomic_long_add(npages, &cli->cl_lru_in_list); + cli->cl_lru_last_used = ktime_get_real_seconds(); spin_unlock(&cli->cl_lru_list_lock); - /* XXX: May set force to be true for better performance */ - if (osc_cache_too_much(cli)) + if (waitqueue_active(&osc_lru_waitq)) (void)ptlrpcd_queue_work(cli->cl_lru_work); } } @@ -449,8 +458,10 @@ static void osc_lru_del(struct client_obd *cli, struct osc_page *opg) * this osc occupies too many LRU pages and kernel is * stealing one of them. */ - if (!memory_pressure_get()) + if (osc_cache_too_much(cli)) { + CDEBUG(D_CACHE, "%s: queue LRU workn", cli_name(cli)); (void)ptlrpcd_queue_work(cli->cl_lru_work); + } wake_up(&osc_lru_waitq); } else { LASSERT(list_empty(&opg->ops_lru)); @@ -482,6 +493,7 @@ static void discard_pagevec(const struct lu_env *env, struct cl_io *io, struct cl_page *page = pvec[i]; LASSERT(cl_page_is_owned(page, io)); + cl_page_delete(env, page); cl_page_discard(env, io, page); cl_page_disown(env, io, page); cl_page_put(env, page); @@ -532,6 +544,8 @@ long osc_lru_shrink(const struct lu_env *env, struct client_obd *cli, if (atomic_long_read(&cli->cl_lru_in_list) == 0 || target <= 0) return 0; + CDEBUG(D_CACHE, "%s: shrinkers: %d, force: %d\n", + cli_name(cli), atomic_read(&cli->cl_lru_shrinkers), force); if (!force) { if (atomic_read(&cli->cl_lru_shrinkers) > 0) return -EBUSY; @@ -548,11 +562,16 @@ long osc_lru_shrink(const struct lu_env *env, struct client_obd *cli, io = &osc_env_info(env)->oti_io; spin_lock(&cli->cl_lru_list_lock); + if (force) + cli->cl_lru_reclaim++; maxscan = min(target << 1, atomic_long_read(&cli->cl_lru_in_list)); list_for_each_entry_safe(opg, temp, &cli->cl_lru_list, ops_lru) { struct cl_page *page; bool will_free = false; + if (!force && atomic_read(&cli->cl_lru_shrinkers) > 1) + break; + if (--maxscan < 0) break; @@ -642,7 +661,13 @@ long osc_lru_shrink(const struct lu_env *env, struct client_obd *cli, return count > 0 ? count : rc; } -long osc_lru_reclaim(struct client_obd *cli) +/** + * Reclaim LRU pages by an IO thread. The caller wants to reclaim at least + * \@npages of LRU slots. For performance consideration, it's better to drop + * LRU pages in batch. Therefore, the actual number is adjusted at least + * max_pages_per_rpc. + */ +long osc_lru_reclaim(struct client_obd *cli, unsigned long npages) { struct lu_env *env; struct cl_client_cache *cache = cli->cl_cache; @@ -656,20 +681,23 @@ long osc_lru_reclaim(struct client_obd *cli) if (IS_ERR(env)) return 0; - rc = osc_lru_shrink(env, cli, osc_cache_too_much(cli), false); - if (rc != 0) { - if (rc == -EBUSY) - rc = 0; - - CDEBUG(D_CACHE, "%s: Free %ld pages from own LRU: %p.\n", - cli->cl_import->imp_obd->obd_name, rc, cli); + npages = max_t(int, npages, cli->cl_max_pages_per_rpc); + CDEBUG(D_CACHE, "%s: start to reclaim %ld pages from LRU\n", + cli_name(cli), npages); + rc = osc_lru_shrink(env, cli, npages, true); + if (rc >= npages) { + CDEBUG(D_CACHE, "%s: reclaimed %ld/%ld pages from LRU\n", + cli_name(cli), rc, npages); + if (osc_cache_too_much(cli) > 0) + ptlrpcd_queue_work(cli->cl_lru_work); goto out; + } else if (rc > 0) { + npages -= rc; } - CDEBUG(D_CACHE, "%s: cli %p no free slots, pages: %ld, busy: %ld.\n", - cli->cl_import->imp_obd->obd_name, cli, - atomic_long_read(&cli->cl_lru_in_list), - atomic_long_read(&cli->cl_lru_busy)); + CDEBUG(D_CACHE, "%s: cli %p no free slots, pages: %ld/%ld, want: %ld\n", + cli_name(cli), cli, atomic_long_read(&cli->cl_lru_in_list), + atomic_long_read(&cli->cl_lru_busy), npages); /* Reclaim LRU slots from other client_obd as it can't free enough * from its own. This should rarely happen. @@ -686,7 +714,7 @@ long osc_lru_reclaim(struct client_obd *cli) cl_lru_osc); CDEBUG(D_CACHE, "%s: cli %p LRU pages: %ld, busy: %ld.\n", - cli->cl_import->imp_obd->obd_name, cli, + cli_name(cli), cli, atomic_long_read(&cli->cl_lru_in_list), atomic_long_read(&cli->cl_lru_busy)); @@ -694,11 +722,12 @@ long osc_lru_reclaim(struct client_obd *cli) if (osc_cache_too_much(cli) > 0) { spin_unlock(&cache->ccc_lru_lock); - rc = osc_lru_shrink(env, cli, osc_cache_too_much(cli), - true); + rc = osc_lru_shrink(env, cli, npages, true); spin_lock(&cache->ccc_lru_lock); - if (rc != 0) + if (rc >= npages) break; + if (rc > 0) + npages -= rc; } } spin_unlock(&cache->ccc_lru_lock); @@ -706,7 +735,7 @@ long osc_lru_reclaim(struct client_obd *cli) out: cl_env_put(env, &refcheck); CDEBUG(D_CACHE, "%s: cli %p freed %ld pages.\n", - cli->cl_import->imp_obd->obd_name, cli, rc); + cli_name(cli), cli, rc); return rc; } @@ -736,7 +765,7 @@ static int osc_lru_reserve(const struct lu_env *env, struct osc_object *obj, LASSERT(atomic_long_read(cli->cl_lru_left) >= 0); while (!atomic_long_add_unless(cli->cl_lru_left, -1, 0)) { /* run out of LRU spaces, try to drop some by itself */ - rc = osc_lru_reclaim(cli); + rc = osc_lru_reclaim(cli, 1); if (rc < 0) break; if (rc > 0) @@ -839,7 +868,7 @@ void osc_dec_unstable_pages(struct ptlrpc_request *req) if (!unstable_count) wake_up_all(&cli->cl_cache->ccc_unstable_waitq); - if (osc_cache_too_much(cli)) + if (waitqueue_active(&osc_lru_waitq)) (void)ptlrpcd_queue_work(cli->cl_lru_work); } @@ -895,8 +924,7 @@ bool osc_over_unstable_soft_limit(struct client_obd *cli) CDEBUG(D_CACHE, "%s: cli: %p unstable pages: %lu, osc unstable pages: %lu\n", - cli->cl_import->imp_obd->obd_name, cli, - unstable_nr, osc_unstable_count); + cli_name(cli), cli, unstable_nr, osc_unstable_count); /* * If the LRU slots are in shortage - 25% remaining AND this OSC diff --git a/drivers/staging/lustre/lustre/osc/osc_quota.c b/drivers/staging/lustre/lustre/osc/osc_quota.c index acdd91a..4bb3b45 100644 --- a/drivers/staging/lustre/lustre/osc/osc_quota.c +++ b/drivers/staging/lustre/lustre/osc/osc_quota.c @@ -106,7 +106,7 @@ int osc_quota_setdq(struct client_obd *cli, const unsigned int qid[], } CDEBUG(D_QUOTA, "%s: setdq to insert for %s %d (%d)\n", - cli->cl_import->imp_obd->obd_name, + cli_name(cli), type == USRQUOTA ? "user" : "group", qid[type], rc); } else { @@ -122,7 +122,7 @@ int osc_quota_setdq(struct client_obd *cli, const unsigned int qid[], kmem_cache_free(osc_quota_kmem, oqi); CDEBUG(D_QUOTA, "%s: setdq to remove for %s %d (%p)\n", - cli->cl_import->imp_obd->obd_name, + cli_name(cli), type == USRQUOTA ? "user" : "group", qid[type], oqi); } diff --git a/drivers/staging/lustre/lustre/osc/osc_request.c b/drivers/staging/lustre/lustre/osc/osc_request.c index 8023561..3daca63 100644 --- a/drivers/staging/lustre/lustre/osc/osc_request.c +++ b/drivers/staging/lustre/lustre/osc/osc_request.c @@ -584,14 +584,13 @@ static void osc_announce_cached(struct client_obd *cli, struct obdo *oa, oa->o_undirty = 0; } else if (unlikely(atomic_long_read(&obd_dirty_pages) - atomic_long_read(&obd_dirty_transit_pages) > - (obd_max_dirty_pages + 1))) { + (long)(obd_max_dirty_pages + 1))) { /* The atomic_read() allowing the atomic_inc() are * not covered by a lock thus they may safely race and trip * this CERROR() unless we add in a small fudge factor (+1). */ - CERROR("%s: dirty %ld + %ld > system dirty_max %lu\n", - cli->cl_import->imp_obd->obd_name, - atomic_long_read(&obd_dirty_pages), + CERROR("%s: dirty %ld + %ld > system dirty_max %ld\n", + cli_name(cli), atomic_long_read(&obd_dirty_pages), atomic_long_read(&obd_dirty_transit_pages), obd_max_dirty_pages); oa->o_undirty = 0; @@ -786,12 +785,10 @@ static int osc_add_shrink_grant(struct client_obd *client) osc_grant_shrink_grant_cb, NULL, &client->cl_grant_shrink_list); if (rc) { - CERROR("add grant client %s error %d\n", - client->cl_import->imp_obd->obd_name, rc); + CERROR("add grant client %s error %d\n", cli_name(client), rc); return rc; } - CDEBUG(D_CACHE, "add grant client %s\n", - client->cl_import->imp_obd->obd_name); + CDEBUG(D_CACHE, "add grant client %s\n", cli_name(client)); osc_update_next_shrink(client); return 0; } @@ -825,8 +822,8 @@ static void osc_init_grant(struct client_obd *cli, struct obd_connect_data *ocd) spin_unlock(&cli->cl_loi_list_lock); CDEBUG(D_CACHE, "%s, setting cl_avail_grant: %ld cl_lost_grant: %ld chunk bits: %d\n", - cli->cl_import->imp_obd->obd_name, - cli->cl_avail_grant, cli->cl_lost_grant, cli->cl_chunkbits); + cli_name(cli), cli->cl_avail_grant, cli->cl_lost_grant, + cli->cl_chunkbits); if (ocd->ocd_connect_flags & OBD_CONNECT_GRANT_SHRINK && list_empty(&cli->cl_grant_shrink_list)) -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:30:45 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:30:45 -0500 Subject: [lustre-devel] [PATCH 15/35] staging: lustre: llite: tar restore fails for HSM released files. In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <1478799065-24841-16-git-send-email-jsimmons@infradead.org> From: Aditya Pandit If you create a file, archive and release it, it keeps only a link and all information in xattr. If you tar the file with --xattr you will store the same striping information and link information in the tar. If you delete the file, the file and archive state does not make sense. Now if you restore the file using tar with xattr having the RELEASED flag turned on, then it is not correct because this is a new file. Hence ignoring the HSM xattr and masking out the "RELEASED" flag for the files, which are not archived. Signed-off-by: Aditya Pandit Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6214 Reviewed-on: http://review.whamcloud.com/16060 Reviewed-by: Andreas Dilger Reviewed-by: frank zago Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- .../lustre/lustre/include/lustre/lustre_user.h | 1 + drivers/staging/lustre/lustre/llite/xattr.c | 61 +++++++++++++++++++- 2 files changed, 60 insertions(+), 2 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre/lustre_user.h b/drivers/staging/lustre/lustre/include/lustre/lustre_user.h index 579ef14..e393ae3 100644 --- a/drivers/staging/lustre/lustre/include/lustre/lustre_user.h +++ b/drivers/staging/lustre/lustre/include/lustre/lustre_user.h @@ -1001,6 +1001,7 @@ struct ioc_data_version { * See HSM_FLAGS below. */ enum hsm_states { + HS_NONE = 0x00000000, HS_EXISTS = 0x00000001, HS_DIRTY = 0x00000002, HS_RELEASED = 0x00000004, diff --git a/drivers/staging/lustre/lustre/llite/xattr.c b/drivers/staging/lustre/lustre/llite/xattr.c index ea3becc..7a848eb 100644 --- a/drivers/staging/lustre/lustre/llite/xattr.c +++ b/drivers/staging/lustre/lustre/llite/xattr.c @@ -112,8 +112,9 @@ static int xattr_type_filter(struct ll_sb_info *sbi, return -EPERM; /* b10667: ignore lustre special xattr for now */ - if ((handler->flags == XATTR_TRUSTED_T && !strcmp(name, "lov")) || - (handler->flags == XATTR_LUSTRE_T && !strcmp(name, "lov"))) + if (!strcmp(name, "hsm") || + ((handler->flags == XATTR_TRUSTED_T && !strcmp(name, "lov")) || + (handler->flags == XATTR_LUSTRE_T && !strcmp(name, "lov")))) return 0; /* b15587: ignore security.capability xattr for now */ @@ -147,6 +148,37 @@ static int xattr_type_filter(struct ll_sb_info *sbi, return 0; } +static int get_hsm_state(struct inode *inode, u32 *hus_states) +{ + struct md_op_data *op_data; + struct hsm_user_state *hus; + int rc; + + hus = kzalloc(sizeof(*hus), GFP_NOFS); + if (!hus) + return -ENOMEM; + + op_data = ll_prep_md_op_data(NULL, inode, NULL, NULL, 0, 0, + LUSTRE_OPC_ANY, hus); + if (!IS_ERR(op_data)) { + rc = obd_iocontrol(LL_IOC_HSM_STATE_GET, ll_i2mdexp(inode), + sizeof(*op_data), op_data, NULL); + if (!rc) + *hus_states = hus->hus_states; + else + CDEBUG(D_VFSTRACE, "obd_iocontrol failed. rc = %d\n", + rc); + + ll_finish_md_op_data(op_data); + } else { + rc = PTR_ERR(op_data); + CDEBUG(D_VFSTRACE, "Could not prepare the opdata. rc = %d\n", + rc); + } + kfree(hus); + return rc; +} + static int ll_xattr_set(const struct xattr_handler *handler, struct dentry *dentry, struct inode *inode, const char *name, const void *value, size_t size, @@ -183,6 +215,31 @@ static int ll_xattr_set(const struct xattr_handler *handler, if (lump && lump->lmm_stripe_offset == 0) lump->lmm_stripe_offset = -1; + /* Avoid anyone directly setting the RELEASED flag. */ + if (lump && (lump->lmm_pattern & LOV_PATTERN_F_RELEASED)) { + /* Only if we have a released flag check if the file + * was indeed archived. + */ + u32 state = HS_NONE; + + rc = get_hsm_state(inode, &state); + if (rc) + return rc; + + if (!(state & HS_ARCHIVED)) { + CDEBUG(D_VFSTRACE, + "hus_states state = %x, pattern = %x\n", + state, lump->lmm_pattern); + /* + * Here the state is: real file is not + * archived but user is requesting to set + * the RELEASED flag so we mask off the + * released flag from the request + */ + lump->lmm_pattern ^= LOV_PATTERN_F_RELEASED; + } + } + if (lump && S_ISREG(inode->i_mode)) { __u64 it_flags = FMODE_WRITE; int lum_size; -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:30:58 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:30:58 -0500 Subject: [lustre-devel] [PATCH 28/35] staging: lustre: osc: Remove remains of osc_ast_guard In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <1478799065-24841-29-git-send-email-jsimmons@infradead.org> From: Oleg Drokin osc_ast_guard has been removed by the clio simplification. Remove the last lock class definition. Signed-off-by: Oleg Drokin Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-7148 Reviewed-on: http://review.whamcloud.com/16392 Reviewed-by: Andreas Dilger Reviewed-by: Bobi Jam Signed-off-by: James Simmons --- drivers/staging/lustre/lustre/osc/osc_dev.c | 2 -- drivers/staging/lustre/lustre/osc/osc_request.c | 1 - 2 files changed, 0 insertions(+), 3 deletions(-) diff --git a/drivers/staging/lustre/lustre/osc/osc_dev.c b/drivers/staging/lustre/lustre/osc/osc_dev.c index 89f288b..c5d62ae 100644 --- a/drivers/staging/lustre/lustre/osc/osc_dev.c +++ b/drivers/staging/lustre/lustre/osc/osc_dev.c @@ -88,8 +88,6 @@ struct lu_kmem_descr osc_caches[] = { } }; -struct lock_class_key osc_ast_guard_class; - /***************************************************************************** * * Type conversions. diff --git a/drivers/staging/lustre/lustre/osc/osc_request.c b/drivers/staging/lustre/lustre/osc/osc_request.c index c1f5e24..3d5cae9 100644 --- a/drivers/staging/lustre/lustre/osc/osc_request.c +++ b/drivers/staging/lustre/lustre/osc/osc_request.c @@ -2789,7 +2789,6 @@ static int osc_process_config(struct obd_device *obd, u32 len, void *buf) }; extern struct lu_kmem_descr osc_caches[]; -extern struct lock_class_key osc_ast_guard_class; static int __init osc_init(void) { -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:31:00 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:31:00 -0500 Subject: [lustre-devel] [PATCH 30/35] staging: lustre: llite: ll_write_begin/end not passing on errors In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <1478799065-24841-31-git-send-email-jsimmons@infradead.org> From: Hiroya Nozaki Because of a implementation of generic_perform_write(), write(2) may return 0 with no errno even if EDQUOT or ENOSPC actually happened in it. This patch fixes the issue with setting a proper errno to ci_result. Signed-off-by: Hiroya Nozaki Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6732 Reviewed-on: http://review.whamcloud.com/15302 Reviewed-by: Bobi Jam Reviewed-by: Andreas Dilger Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- drivers/staging/lustre/lustre/llite/rw26.c | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/drivers/staging/lustre/lustre/llite/rw26.c b/drivers/staging/lustre/lustre/llite/rw26.c index ca45b44..c1b7409 100644 --- a/drivers/staging/lustre/lustre/llite/rw26.c +++ b/drivers/staging/lustre/lustre/llite/rw26.c @@ -478,6 +478,7 @@ static int ll_write_begin(struct file *file, struct address_space *mapping, lcc = ll_cl_find(file); if (!lcc) { + io = NULL; result = -EIO; goto out; } @@ -558,6 +559,8 @@ static int ll_write_begin(struct file *file, struct address_space *mapping, lu_ref_del(&page->cp_reference, "cl_io", io); cl_page_put(env, page); } + if (io) + io->ci_result = result; } else { *pagep = vmpage; *fsdata = lcc; @@ -627,6 +630,8 @@ static int ll_write_end(struct file *file, struct address_space *mapping, file->f_flags & O_SYNC || IS_SYNC(file_inode(file))) result = vvp_io_write_commit(env, io); + if (result < 0) + io->ci_result = result; return result >= 0 ? copied : result; } -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:30:49 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:30:49 -0500 Subject: [lustre-devel] [PATCH 19/35] staging: lustre: obdclass: remove structure holes to reduce memory In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <1478799065-24841-20-git-send-email-jsimmons@infradead.org> From: Andreas Dilger Fix the alignment of fields in commonly-used structures to reduce memory usage on the client and server. Structures fixed: ptlrpc_reply_state: reduced by 8 bytes obd_device: reduced by 16 bytes niobuf_local: reduced by 8 bytes Signed-off-by: Andreas Dilger Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3281 Reviewed-on: http://review.whamcloud.com/16692 Reviewed-by: Dmitry Eremin Reviewed-by: Alex Zhuravlev Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- drivers/staging/lustre/lustre/include/lustre_net.h | 7 ++-- drivers/staging/lustre/lustre/include/obd.h | 38 ++++++++++---------- 2 files changed, 23 insertions(+), 22 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre_net.h b/drivers/staging/lustre/lustre/include/lustre_net.h index 7f549a3..d2cbec3 100644 --- a/drivers/staging/lustre/lustre/include/lustre_net.h +++ b/drivers/staging/lustre/lustre/include/lustre_net.h @@ -438,6 +438,10 @@ struct ptlrpc_reply_state { unsigned long rs_committed:1;/* the transaction was committed * and the rs was dispatched */ + atomic_t rs_refcount; /* number of users */ + /** Number of locks awaiting client ACK */ + int rs_nlocks; + /** Size of the state */ int rs_size; /** opcode */ @@ -450,7 +454,6 @@ struct ptlrpc_reply_state { struct ptlrpc_service_part *rs_svcpt; /** Lnet metadata handle for the reply */ lnet_handle_md_t rs_md_h; - atomic_t rs_refcount; /** Context for the service thread */ struct ptlrpc_svc_ctx *rs_svc_ctx; @@ -467,8 +470,6 @@ struct ptlrpc_reply_state { */ struct lustre_msg *rs_msg; /* reply message */ - /** Number of locks awaiting client ACK */ - int rs_nlocks; /** Handles of locks awaiting client reply ACK */ struct lustre_handle rs_locks[RS_MAX_LOCKS]; /** Lock modes of locks in \a rs_locks */ diff --git a/drivers/staging/lustre/lustre/include/obd.h b/drivers/staging/lustre/lustre/include/obd.h index ffe09a3..a346312 100644 --- a/drivers/staging/lustre/lustre/include/obd.h +++ b/drivers/staging/lustre/lustre/include/obd.h @@ -428,9 +428,9 @@ struct niobuf_local { __u32 lnb_page_offset; __u32 lnb_len; __u32 lnb_flags; + int lnb_rc; struct page *lnb_page; void *lnb_data; - int lnb_rc; }; #define LUSTRE_FLD_NAME "fld" @@ -531,15 +531,14 @@ struct lvfs_run_ctxt { struct obd_device { struct obd_type *obd_type; - __u32 obd_magic; + u32 obd_magic; /* OBD_DEVICE_MAGIC */ + int obd_minor; /* device number: lctl dl */ + struct lu_device *obd_lu_dev; /* common and UUID name of this device */ - char obd_name[MAX_OBD_NAME]; - struct obd_uuid obd_uuid; + struct obd_uuid obd_uuid; + char obd_name[MAX_OBD_NAME]; - struct lu_device *obd_lu_dev; - - int obd_minor; /* bitfield modification is protected by obd_dev_lock */ unsigned long obd_attached:1, /* finished attach */ obd_set_up:1, /* finished setup */ @@ -563,22 +562,22 @@ struct obd_device { unsigned long obd_recovery_expired:1; /* uuid-export hash body */ struct cfs_hash *obd_uuid_hash; - atomic_t obd_refcount; wait_queue_head_t obd_refcount_waitq; struct list_head obd_exports; struct list_head obd_unlinked_exports; struct list_head obd_delayed_exports; + atomic_t obd_refcount; int obd_num_exports; spinlock_t obd_nid_lock; struct ldlm_namespace *obd_namespace; struct ptlrpc_client obd_ldlm_client; /* XXX OST/MDS only */ /* a spinlock is OK for what we do now, may need a semaphore later */ spinlock_t obd_dev_lock; /* protect OBD bitfield above */ - struct mutex obd_dev_mutex; - __u64 obd_last_committed; spinlock_t obd_osfs_lock; struct obd_statfs obd_osfs; /* locked by obd_osfs_lock */ __u64 obd_osfs_age; + u64 obd_last_committed; + struct mutex obd_dev_mutex; struct lvfs_run_ctxt obd_lvfs_ctxt; struct obd_llog_group obd_olg; /* default llog group */ struct obd_device *obd_observer; @@ -592,12 +591,13 @@ struct obd_device { struct lov_obd lov; struct lmv_obd lmv; } u; + /* Fields used by LProcFS */ - unsigned int obd_cntr_base; - struct lprocfs_stats *obd_stats; + struct lprocfs_stats *obd_stats; + unsigned int obd_cntr_base; - unsigned int md_cntr_base; - struct lprocfs_stats *md_stats; + struct lprocfs_stats *md_stats; + unsigned int md_cntr_base; struct dentry *obd_debugfs_entry; struct dentry *obd_svc_debugfs_entry; @@ -609,9 +609,11 @@ struct obd_device { /** * Ldlm pool part. Save last calculated SLV and Limit. */ - rwlock_t obd_pool_lock; - int obd_pool_limit; - __u64 obd_pool_slv; + rwlock_t obd_pool_lock; + u64 obd_pool_slv; + int obd_pool_limit; + + int obd_conn_inprogress; /** * A list of outstanding class_incref()'s against this obd. For @@ -619,8 +621,6 @@ struct obd_device { */ struct lu_ref obd_reference; - int obd_conn_inprogress; - struct kobject obd_kobj; /* sysfs object */ struct completion obd_kobj_unregister; }; -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:30:39 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:30:39 -0500 Subject: [lustre-devel] [PATCH 09/35] staging: lustre: mgc: IR log failure should not stop mount In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <1478799065-24841-10-git-send-email-jsimmons@infradead.org> From: wang di If clients or other targets can not get IR config lock or lock, the mount should continue, instead of failing. Because timeout mechanism will handle the recovery anyway. Signed-off-by: wang di Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6906 Reviewed-on: http://review.whamcloud.com/15728 Reviewed-by: Jinshan Xiong Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- drivers/staging/lustre/lustre/mgc/mgc_request.c | 9 ++++++++- 1 files changed, 8 insertions(+), 1 deletions(-) diff --git a/drivers/staging/lustre/lustre/mgc/mgc_request.c b/drivers/staging/lustre/lustre/mgc/mgc_request.c index 2d6fdd0..e248f33 100644 --- a/drivers/staging/lustre/lustre/mgc/mgc_request.c +++ b/drivers/staging/lustre/lustre/mgc/mgc_request.c @@ -1675,8 +1675,15 @@ int mgc_process_log(struct obd_device *mgc, struct config_llog_data *cld) if (cld_is_recover(cld)) { rc = 0; /* this is not a fatal error for recover log */ - if (rcl == 0) + if (!rcl) { rc = mgc_process_recover_log(mgc, cld); + if (rc) { + CERROR("%s: recover log %s failed: rc = %d not fatal.\n", + mgc->obd_name, cld->cld_logname, rc); + rc = 0; + cld->cld_lostlock = 1; + } + } } else { rc = mgc_process_cfg_log(mgc, cld, rcl != 0); } -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:30:31 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:30:31 -0500 Subject: [lustre-devel] [PATCH 01/35] staging: lustre: hsm: Use file lease to implement migration In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <1478799065-24841-2-git-send-email-jsimmons@infradead.org> From: Henri Doreau Implement non-blocking migration based on exclusive open instead of group lock. Implemented exclusive close operation to atomically put a lease, swap two layouts and close a file. This allows race-free migrations. Make the caller responsible for retrying on failure (EBUSY, EAGAIN) in non-blocking mode. In blocking mode, allow applications to trigger layout swaps using a grouplock they already own, to prevent race conditions between the actual data copy and the layout swap. Updated lfs accordingly. File leases are also taken in blocking mode, so that lfs migrate can issue a warning if an application attempts to open a file that is being migrated and gets blocked. Timestamps (atime/mtime) are set from userland, after the layout swap is performed, to prevent conflicts with the grouplock. lli_trunc_sem is taken/released in the vvp_io layer, under the DLM lock. This re-ordering fixes the original issue between truncate and migrate. Signed-off-by: Henri Doreau Signed-off-by: Jinshan Xiong Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-4840 Reviewed-on: http://review.whamcloud.com/10013 Reviewed-by: John L. Hammond Reviewed-by: frank zago Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- .../lustre/lustre/include/lustre/lustre_idl.h | 5 +- .../lustre/lustre/include/lustre/lustre_user.h | 1 + .../lustre/lustre/include/lustre_req_layout.h | 2 +- drivers/staging/lustre/lustre/llite/file.c | 230 ++++++++++++-------- drivers/staging/lustre/lustre/llite/llite_lib.c | 4 - drivers/staging/lustre/lustre/llite/vvp_io.c | 78 +++++--- drivers/staging/lustre/lustre/mdc/mdc_lib.c | 34 ++-- drivers/staging/lustre/lustre/mdc/mdc_request.c | 7 +- drivers/staging/lustre/lustre/ptlrpc/layout.c | 10 +- 9 files changed, 231 insertions(+), 140 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h index db09f3b..21abab4 100644 --- a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h +++ b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h @@ -1595,7 +1595,9 @@ static inline __u32 lov_mds_md_size(__u16 stripes, __u32 lmm_magic) /* OBD_MD_FLRMTRGETFACL (0x0008000000000000ULL) lfs rgetfacl, obsolete */ #define OBD_MD_FLDATAVERSION (0x0010000000000000ULL) /* iversion sum */ -#define OBD_MD_FLRELEASED (0x0020000000000000ULL) /* file released */ +#define OBD_MD_CLOSE_INTENT_EXECED (0x0020000000000000ULL) /* close intent + * executed + */ #define OBD_MD_DEFAULT_MEA (0x0040000000000000ULL) /* default MEA */ @@ -2127,6 +2129,7 @@ enum mds_op_bias { MDS_OWNEROVERRIDE = 1 << 11, MDS_HSM_RELEASE = 1 << 12, MDS_RENAME_MIGRATE = BIT(13), + MDS_CLOSE_LAYOUT_SWAP = BIT(14), }; /* instance of mdt_reint_rec */ diff --git a/drivers/staging/lustre/lustre/include/lustre/lustre_user.h b/drivers/staging/lustre/lustre/include/lustre/lustre_user.h index 856e2f9..579ef14 100644 --- a/drivers/staging/lustre/lustre/include/lustre/lustre_user.h +++ b/drivers/staging/lustre/lustre/include/lustre/lustre_user.h @@ -650,6 +650,7 @@ struct if_quotactl { #define SWAP_LAYOUTS_CHECK_DV2 (1 << 1) #define SWAP_LAYOUTS_KEEP_MTIME (1 << 2) #define SWAP_LAYOUTS_KEEP_ATIME (1 << 3) +#define SWAP_LAYOUTS_CLOSE BIT(4) /* Swap XATTR_NAME_HSM as well, only on the MDT so far */ #define SWAP_LAYOUTS_MDS_HSM (1 << 31) diff --git a/drivers/staging/lustre/lustre/include/lustre_req_layout.h b/drivers/staging/lustre/lustre/include/lustre_req_layout.h index 78857b3..7657132 100644 --- a/drivers/staging/lustre/lustre/include/lustre_req_layout.h +++ b/drivers/staging/lustre/lustre/include/lustre_req_layout.h @@ -148,7 +148,7 @@ void req_capsule_shrink(struct req_capsule *pill, */ extern struct req_format RQF_MDS_GETATTR_NAME; extern struct req_format RQF_MDS_CLOSE; -extern struct req_format RQF_MDS_RELEASE_CLOSE; +extern struct req_format RQF_MDS_INTENT_CLOSE; extern struct req_format RQF_MDS_CONNECT; extern struct req_format RQF_MDS_DISCONNECT; extern struct req_format RQF_MDS_GET_INFO; diff --git a/drivers/staging/lustre/lustre/llite/file.c b/drivers/staging/lustre/lustre/llite/file.c index 7886840..7adbf31 100644 --- a/drivers/staging/lustre/lustre/llite/file.c +++ b/drivers/staging/lustre/lustre/llite/file.c @@ -113,10 +113,19 @@ static void ll_prepare_close(struct inode *inode, struct md_op_data *op_data, 0, 0, LUSTRE_OPC_ANY, NULL); } +/** + * Perform a close, possibly with a bias. + * The meaning of "data" depends on the value of "bias". + * + * If \a bias is MDS_HSM_RELEASE then \a data is a pointer to the data version. + * If \a bias is MDS_CLOSE_LAYOUT_SWAP then \a data is a pointer to the inode to + * swap layouts with. + */ static int ll_close_inode_openhandle(struct obd_export *md_exp, - struct inode *inode, struct obd_client_handle *och, - const __u64 *data_version) + struct inode *inode, + enum mds_op_bias bias, + void *data) { struct obd_export *exp = ll_i2mdexp(inode); struct md_op_data *op_data; @@ -143,12 +152,26 @@ static int ll_close_inode_openhandle(struct obd_export *md_exp, } ll_prepare_close(inode, op_data, och); - if (data_version) { - /* Pass in data_version implies release. */ + switch (bias) { + case MDS_CLOSE_LAYOUT_SWAP: + LASSERT(data); + op_data->op_bias |= MDS_CLOSE_LAYOUT_SWAP; + op_data->op_data_version = 0; + op_data->op_lease_handle = och->och_lease_handle; + op_data->op_fid2 = *ll_inode2fid(data); + break; + + case MDS_HSM_RELEASE: + LASSERT(data); op_data->op_bias |= MDS_HSM_RELEASE; - op_data->op_data_version = *data_version; + op_data->op_data_version = *(__u64 *)data; op_data->op_lease_handle = och->och_lease_handle; op_data->op_attr.ia_valid |= ATTR_SIZE | ATTR_BLOCKS; + break; + + default: + LASSERT(!data); + break; } rc = md_close(md_exp, op_data, och->och_mod, &req); @@ -169,11 +192,12 @@ static int ll_close_inode_openhandle(struct obd_export *md_exp, spin_unlock(&lli->lli_lock); } - if (rc == 0 && op_data->op_bias & MDS_HSM_RELEASE) { + if (op_data->op_bias & (MDS_HSM_RELEASE | MDS_CLOSE_LAYOUT_SWAP) && + !rc) { struct mdt_body *body; body = req_capsule_server_get(&req->rq_pill, &RMF_MDT_BODY); - if (!(body->mbo_valid & OBD_MD_FLRELEASED)) + if (!(body->mbo_valid & OBD_MD_CLOSE_INTENT_EXECED)) rc = -EBUSY; } @@ -227,7 +251,7 @@ int ll_md_real_close(struct inode *inode, fmode_t fmode) * be closed. */ rc = ll_close_inode_openhandle(ll_i2sbi(inode)->ll_md_exp, - inode, och, NULL); + och, inode, 0, NULL); } return rc; @@ -263,7 +287,8 @@ static int ll_md_close(struct obd_export *md_exp, struct inode *inode, } if (fd->fd_och) { - rc = ll_close_inode_openhandle(md_exp, inode, fd->fd_och, NULL); + rc = ll_close_inode_openhandle(md_exp, fd->fd_och, inode, 0, + NULL); fd->fd_och = NULL; goto out; } @@ -816,7 +841,7 @@ static int ll_md_blocking_lease_ast(struct ldlm_lock *lock, it.it_lock_mode = 0; och->och_lease_handle.cookie = 0ULL; } - rc2 = ll_close_inode_openhandle(sbi->ll_md_exp, inode, och, NULL); + rc2 = ll_close_inode_openhandle(sbi->ll_md_exp, och, inode, 0, NULL); if (rc2 < 0) CERROR("%s: error closing file "DFID": %d\n", ll_get_fsname(inode->i_sb, NULL, 0), @@ -830,6 +855,69 @@ static int ll_md_blocking_lease_ast(struct ldlm_lock *lock, } /** + * Check whether a layout swap can be done between two inodes. + * + * \param[in] inode1 First inode to check + * \param[in] inode2 Second inode to check + * + * \retval 0 on success, layout swap can be performed between both inodes + * \retval negative error code if requirements are not met + */ +static int ll_check_swap_layouts_validity(struct inode *inode1, + struct inode *inode2) +{ + if (!S_ISREG(inode1->i_mode) || !S_ISREG(inode2->i_mode)) + return -EINVAL; + + if (inode_permission(inode1, MAY_WRITE) || + inode_permission(inode2, MAY_WRITE)) + return -EPERM; + + if (inode1->i_sb != inode2->i_sb) + return -EXDEV; + + return 0; +} + +static int ll_swap_layouts_close(struct obd_client_handle *och, + struct inode *inode, struct inode *inode2) +{ + const struct lu_fid *fid1 = ll_inode2fid(inode); + const struct lu_fid *fid2; + int rc; + + CDEBUG(D_INODE, "%s: biased close of file " DFID "\n", + ll_get_fsname(inode->i_sb, NULL, 0), PFID(fid1)); + + rc = ll_check_swap_layouts_validity(inode, inode2); + if (rc < 0) + goto out_free_och; + + /* We now know that inode2 is a lustre inode */ + fid2 = ll_inode2fid(inode2); + + rc = lu_fid_cmp(fid1, fid2); + if (!rc) { + rc = -EINVAL; + goto out_free_och; + } + + /* + * Close the file and swap layouts between inode & inode2. + * NB: lease lock handle is released in mdc_close_layout_swap_pack() + * because we still need it to pack l_remote_handle to MDT. + */ + rc = ll_close_inode_openhandle(ll_i2sbi(inode)->ll_md_exp, och, inode, + MDS_CLOSE_LAYOUT_SWAP, inode2); + + och = NULL; /* freed in ll_close_inode_openhandle() */ + +out_free_och: + kfree(och); + return rc; +} + +/** * Release lease and close the file. * It will check if the lease has ever broken. */ @@ -856,7 +944,7 @@ static int ll_lease_close(struct obd_client_handle *och, struct inode *inode, *lease_broken = cancelled; return ll_close_inode_openhandle(ll_i2sbi(inode)->ll_md_exp, - inode, och, NULL); + och, inode, 0, NULL); } int ll_merge_attr(const struct lu_env *env, struct inode *inode) @@ -1014,11 +1102,9 @@ void ll_io_init(struct cl_io *io, const struct file *file, int write) range_locked = true; } - down_read(&lli->lli_trunc_sem); ll_cl_add(file, env, io); rc = cl_io_loop(env, io); ll_cl_remove(file, env); - up_read(&lli->lli_trunc_sem); if (range_locked) { CDEBUG(D_VFSTRACE, "Range unlock [%llu, %llu]\n", range.rl_node.in_extent.start, @@ -1415,7 +1501,7 @@ int ll_release_openhandle(struct inode *inode, struct lookup_intent *it) ll_och_fill(ll_i2sbi(inode)->ll_md_exp, it, och); rc = ll_close_inode_openhandle(ll_i2sbi(inode)->ll_md_exp, - inode, och, NULL); + och, inode, 0, NULL); out: /* this one is in place of ll_file_open */ if (it_disposition(it, DISP_ENQ_OPEN_REF)) { @@ -1618,8 +1704,8 @@ int ll_hsm_release(struct inode *inode) * NB: lease lock handle is released in mdc_hsm_release_pack() because * we still need it to pack l_remote_handle to MDT. */ - rc = ll_close_inode_openhandle(ll_i2sbi(inode)->ll_md_exp, inode, och, - &data_version); + rc = ll_close_inode_openhandle(ll_i2sbi(inode)->ll_md_exp, och, inode, + MDS_HSM_RELEASE, &data_version); och = NULL; out: @@ -1630,10 +1716,12 @@ int ll_hsm_release(struct inode *inode) } struct ll_swap_stack { - struct iattr ia1, ia2; - __u64 dv1, dv2; - struct inode *inode1, *inode2; - bool check_dv1, check_dv2; + u64 dv1; + u64 dv2; + struct inode *inode1; + struct inode *inode2; + bool check_dv1; + bool check_dv2; }; static int ll_swap_layouts(struct file *file1, struct file *file2, @@ -1653,21 +1741,9 @@ static int ll_swap_layouts(struct file *file1, struct file *file2, llss->inode1 = file_inode(file1); llss->inode2 = file_inode(file2); - if (!S_ISREG(llss->inode2->i_mode)) { - rc = -EINVAL; - goto free; - } - - if (inode_permission(llss->inode1, MAY_WRITE) || - inode_permission(llss->inode2, MAY_WRITE)) { - rc = -EPERM; - goto free; - } - - if (llss->inode2->i_sb != llss->inode1->i_sb) { - rc = -EXDEV; + rc = ll_check_swap_layouts_validity(llss->inode1, llss->inode2); + if (rc < 0) goto free; - } /* we use 2 bool because it is easier to swap than 2 bits */ if (lsl->sl_flags & SWAP_LAYOUTS_CHECK_DV1) @@ -1681,10 +1757,8 @@ static int ll_swap_layouts(struct file *file1, struct file *file2, llss->dv2 = lsl->sl_dv2; rc = lu_fid_cmp(ll_inode2fid(llss->inode1), ll_inode2fid(llss->inode2)); - if (rc == 0) /* same file, done! */ { - rc = 0; + if (!rc) /* same file, done! */ goto free; - } if (rc < 0) { /* sequentialize it */ swap(llss->inode1, llss->inode2); @@ -1706,19 +1780,6 @@ static int ll_swap_layouts(struct file *file1, struct file *file2, } } - /* to be able to restore mtime and atime after swap - * we need to first save them - */ - if (lsl->sl_flags & - (SWAP_LAYOUTS_KEEP_MTIME | SWAP_LAYOUTS_KEEP_ATIME)) { - llss->ia1.ia_mtime = llss->inode1->i_mtime; - llss->ia1.ia_atime = llss->inode1->i_atime; - llss->ia1.ia_valid = ATTR_MTIME | ATTR_ATIME; - llss->ia2.ia_mtime = llss->inode2->i_mtime; - llss->ia2.ia_atime = llss->inode2->i_atime; - llss->ia2.ia_valid = ATTR_MTIME | ATTR_ATIME; - } - /* ultimate check, before swapping the layouts we check if * dataversion has changed (if requested) */ @@ -1768,39 +1829,6 @@ static int ll_swap_layouts(struct file *file1, struct file *file2, ll_put_grouplock(llss->inode1, file1, gid); } - /* rc can be set from obd_iocontrol() or from a GOTO(putgl, ...) */ - if (rc != 0) - goto free; - - /* clear useless flags */ - if (!(lsl->sl_flags & SWAP_LAYOUTS_KEEP_MTIME)) { - llss->ia1.ia_valid &= ~ATTR_MTIME; - llss->ia2.ia_valid &= ~ATTR_MTIME; - } - - if (!(lsl->sl_flags & SWAP_LAYOUTS_KEEP_ATIME)) { - llss->ia1.ia_valid &= ~ATTR_ATIME; - llss->ia2.ia_valid &= ~ATTR_ATIME; - } - - /* update time if requested */ - rc = 0; - if (llss->ia2.ia_valid != 0) { - inode_lock(llss->inode1); - rc = ll_setattr(file1->f_path.dentry, &llss->ia2); - inode_unlock(llss->inode1); - } - - if (llss->ia1.ia_valid != 0) { - int rc1; - - inode_lock(llss->inode2); - rc1 = ll_setattr(file2->f_path.dentry, &llss->ia1); - inode_unlock(llss->inode2); - if (rc == 0) - rc = rc1; - } - free: kfree(llss); @@ -1957,16 +1985,46 @@ static inline long ll_lease_type_from_fmode(fmode_t fmode) sizeof(struct lustre_swap_layouts))) return -EFAULT; - if ((file->f_flags & O_ACCMODE) == 0) /* O_RDONLY */ + if ((file->f_flags & O_ACCMODE) == O_RDONLY) return -EPERM; file2 = fget(lsl.sl_fd); if (!file2) return -EBADF; - rc = -EPERM; - if ((file2->f_flags & O_ACCMODE) != 0) /* O_WRONLY or O_RDWR */ + /* O_WRONLY or O_RDWR */ + if ((file2->f_flags & O_ACCMODE) == O_RDONLY) { + rc = -EPERM; + goto out; + } + + if (lsl.sl_flags & SWAP_LAYOUTS_CLOSE) { + struct obd_client_handle *och = NULL; + struct ll_inode_info *lli; + struct inode *inode2; + + if (lsl.sl_flags != SWAP_LAYOUTS_CLOSE) { + rc = -EINVAL; + goto out; + } + + lli = ll_i2info(inode); + mutex_lock(&lli->lli_och_mutex); + if (fd->fd_lease_och) { + och = fd->fd_lease_och; + fd->fd_lease_och = NULL; + } + mutex_unlock(&lli->lli_och_mutex); + if (!och) { + rc = -ENOLCK; + goto out; + } + inode2 = file_inode(file2); + rc = ll_swap_layouts_close(och, inode, inode2); + } else { rc = ll_swap_layouts(file, file2, &lsl); + } +out: fput(file2); return rc; } diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c index 308da06..c52653e 100644 --- a/drivers/staging/lustre/lustre/llite/llite_lib.c +++ b/drivers/staging/lustre/lustre/llite/llite_lib.c @@ -1524,11 +1524,7 @@ int ll_setattr_raw(struct dentry *dentry, struct iattr *attr, bool hsm_import) * setting times to past, but it is necessary due to possible * time de-synchronization between MDT inode and OST objects */ - if (attr->ia_valid & ATTR_SIZE) - down_write(&lli->lli_trunc_sem); rc = cl_setattr_ost(ll_i2info(inode)->lli_clob, attr, 0); - if (attr->ia_valid & ATTR_SIZE) - up_write(&lli->lli_trunc_sem); } out: if (op_data) diff --git a/drivers/staging/lustre/lustre/llite/vvp_io.c b/drivers/staging/lustre/lustre/llite/vvp_io.c index 3d327b4..33b38bd 100644 --- a/drivers/staging/lustre/lustre/llite/vvp_io.c +++ b/drivers/staging/lustre/lustre/llite/vvp_io.c @@ -589,14 +589,6 @@ static int vvp_do_vmtruncate(struct inode *inode, size_t size) return result; } -static int vvp_io_setattr_trunc(const struct lu_env *env, - const struct cl_io_slice *ios, - struct inode *inode, loff_t size) -{ - inode_dio_wait(inode); - return 0; -} - static int vvp_io_setattr_time(const struct lu_env *env, const struct cl_io_slice *ios) { @@ -627,15 +619,20 @@ static int vvp_io_setattr_start(const struct lu_env *env, { struct cl_io *io = ios->cis_io; struct inode *inode = vvp_object_inode(io->ci_obj); - int result = 0; + struct ll_inode_info *lli = ll_i2info(inode); - inode_lock(inode); - if (cl_io_is_trunc(io)) - result = vvp_io_setattr_trunc(env, ios, inode, - io->u.ci_setattr.sa_attr.lvb_size); - if (!result && io->u.ci_setattr.sa_valid & TIMES_SET_FLAGS) - result = vvp_io_setattr_time(env, ios); - return result; + if (cl_io_is_trunc(io)) { + down_write(&lli->lli_trunc_sem); + inode_lock(inode); + inode_dio_wait(inode); + } else { + inode_lock(inode); + } + + if (io->u.ci_setattr.sa_valid & TIMES_SET_FLAGS) + return vvp_io_setattr_time(env, ios); + + return 0; } static void vvp_io_setattr_end(const struct lu_env *env, @@ -643,14 +640,18 @@ static void vvp_io_setattr_end(const struct lu_env *env, { struct cl_io *io = ios->cis_io; struct inode *inode = vvp_object_inode(io->ci_obj); + struct ll_inode_info *lli = ll_i2info(inode); - if (cl_io_is_trunc(io)) + if (cl_io_is_trunc(io)) { /* Truncate in memory pages - they must be clean pages * because osc has already notified to destroy osc_extents. */ vvp_do_vmtruncate(inode, io->u.ci_setattr.sa_attr.lvb_size); - - inode_unlock(inode); + inode_unlock(inode); + up_write(&lli->lli_trunc_sem); + } else { + inode_unlock(inode); + } } static void vvp_io_setattr_fini(const struct lu_env *env, @@ -666,6 +667,7 @@ static int vvp_io_read_start(const struct lu_env *env, struct cl_io *io = ios->cis_io; struct cl_object *obj = io->ci_obj; struct inode *inode = vvp_object_inode(obj); + struct ll_inode_info *lli = ll_i2info(inode); struct file *file = vio->vui_fd->fd_file; int result; @@ -678,6 +680,8 @@ static int vvp_io_read_start(const struct lu_env *env, CDEBUG(D_VFSTRACE, "read: -> [%lli, %lli)\n", pos, pos + cnt); + down_read(&lli->lli_trunc_sem); + if (!can_populate_pages(env, io, inode)) return 0; @@ -903,10 +907,13 @@ static int vvp_io_write_start(const struct lu_env *env, struct cl_io *io = ios->cis_io; struct cl_object *obj = io->ci_obj; struct inode *inode = vvp_object_inode(obj); + struct ll_inode_info *lli = ll_i2info(inode); ssize_t result = 0; loff_t pos = io->u.ci_wr.wr.crw_pos; size_t cnt = io->u.ci_wr.wr.crw_count; + down_read(&lli->lli_trunc_sem); + if (!can_populate_pages(env, io, inode)) return 0; @@ -990,6 +997,15 @@ static int vvp_io_write_start(const struct lu_env *env, return result; } +static void vvp_io_rw_end(const struct lu_env *env, + const struct cl_io_slice *ios) +{ + struct inode *inode = vvp_object_inode(ios->cis_obj); + struct ll_inode_info *lli = ll_i2info(inode); + + up_read(&lli->lli_trunc_sem); +} + static int vvp_io_kernel_fault(struct vvp_fault_io *cfio) { struct vm_fault *vmf = cfio->ft_vmf; @@ -1042,6 +1058,7 @@ static int vvp_io_fault_start(const struct lu_env *env, struct cl_io *io = ios->cis_io; struct cl_object *obj = io->ci_obj; struct inode *inode = vvp_object_inode(obj); + struct ll_inode_info *lli = ll_i2info(inode); struct cl_fault_io *fio = &io->u.ci_fault; struct vvp_fault_io *cfio = &vio->u.fault; loff_t offset; @@ -1057,6 +1074,8 @@ static int vvp_io_fault_start(const struct lu_env *env, " changed while waiting for the page fault lock\n", PFID(lu_object_fid(&obj->co_lu))); + down_read(&lli->lli_trunc_sem); + /* offset of the last byte on the page */ offset = cl_offset(obj, fio->ft_index + 1) - 1; LASSERT(cl_index(obj, offset) == fio->ft_index); @@ -1204,6 +1223,17 @@ static int vvp_io_fault_start(const struct lu_env *env, return result; } +static void vvp_io_fault_end(const struct lu_env *env, + const struct cl_io_slice *ios) +{ + struct inode *inode = vvp_object_inode(ios->cis_obj); + struct ll_inode_info *lli = ll_i2info(inode); + + CLOBINVRNT(env, ios->cis_io->ci_obj, + vvp_object_invariant(ios->cis_io->ci_obj)); + up_read(&lli->lli_trunc_sem); +} + static int vvp_io_fsync_start(const struct lu_env *env, const struct cl_io_slice *ios) { @@ -1233,18 +1263,13 @@ static int vvp_io_read_ahead(const struct lu_env *env, return result; } -static void vvp_io_end(const struct lu_env *env, const struct cl_io_slice *ios) -{ - CLOBINVRNT(env, ios->cis_io->ci_obj, - vvp_object_invariant(ios->cis_io->ci_obj)); -} - static const struct cl_io_operations vvp_io_ops = { .op = { [CIT_READ] = { .cio_fini = vvp_io_fini, .cio_lock = vvp_io_read_lock, .cio_start = vvp_io_read_start, + .cio_end = vvp_io_rw_end, .cio_advance = vvp_io_advance, }, [CIT_WRITE] = { @@ -1253,6 +1278,7 @@ static void vvp_io_end(const struct lu_env *env, const struct cl_io_slice *ios) .cio_iter_fini = vvp_io_write_iter_fini, .cio_lock = vvp_io_write_lock, .cio_start = vvp_io_write_start, + .cio_end = vvp_io_rw_end, .cio_advance = vvp_io_advance, }, [CIT_SETATTR] = { @@ -1267,7 +1293,7 @@ static void vvp_io_end(const struct lu_env *env, const struct cl_io_slice *ios) .cio_iter_init = vvp_io_fault_iter_init, .cio_lock = vvp_io_fault_lock, .cio_start = vvp_io_fault_start, - .cio_end = vvp_io_end, + .cio_end = vvp_io_fault_end, }, [CIT_FSYNC] = { .cio_start = vvp_io_fsync_start, diff --git a/drivers/staging/lustre/lustre/mdc/mdc_lib.c b/drivers/staging/lustre/lustre/mdc/mdc_lib.c index 1925072..c1990f0 100644 --- a/drivers/staging/lustre/lustre/mdc/mdc_lib.c +++ b/drivers/staging/lustre/lustre/mdc/mdc_lib.c @@ -430,25 +430,29 @@ void mdc_getattr_pack(struct ptlrpc_request *req, __u64 valid, u32 flags, op_data->op_namelen); } -static void mdc_hsm_release_pack(struct ptlrpc_request *req, - struct md_op_data *op_data) +static void mdc_intent_close_pack(struct ptlrpc_request *req, + struct md_op_data *op_data) { - if (op_data->op_bias & MDS_HSM_RELEASE) { - struct close_data *data; - struct ldlm_lock *lock; + enum mds_op_bias bias = op_data->op_bias; + struct close_data *data; + struct ldlm_lock *lock; - data = req_capsule_client_get(&req->rq_pill, &RMF_CLOSE_DATA); + if (!(bias & (MDS_HSM_RELEASE | MDS_CLOSE_LAYOUT_SWAP | + MDS_RENAME_MIGRATE))) + return; - lock = ldlm_handle2lock(&op_data->op_lease_handle); - if (lock) { - data->cd_handle = lock->l_remote_handle; - LDLM_LOCK_PUT(lock); - } - ldlm_cli_cancel(&op_data->op_lease_handle, LCF_LOCAL); + data = req_capsule_client_get(&req->rq_pill, &RMF_CLOSE_DATA); + LASSERT(data); - data->cd_data_version = op_data->op_data_version; - data->cd_fid = op_data->op_fid2; + lock = ldlm_handle2lock(&op_data->op_lease_handle); + if (lock) { + data->cd_handle = lock->l_remote_handle; + LDLM_LOCK_PUT(lock); } + ldlm_cli_cancel(&op_data->op_lease_handle, LCF_LOCAL); + + data->cd_data_version = op_data->op_data_version; + data->cd_fid = op_data->op_fid2; } void mdc_close_pack(struct ptlrpc_request *req, struct md_op_data *op_data) @@ -473,5 +477,5 @@ void mdc_close_pack(struct ptlrpc_request *req, struct md_op_data *op_data) rec->sa_valid &= ~MDS_ATTR_ATIME; mdc_ioepoch_pack(epoch, op_data); - mdc_hsm_release_pack(req, op_data); + mdc_intent_close_pack(req, op_data); } diff --git a/drivers/staging/lustre/lustre/mdc/mdc_request.c b/drivers/staging/lustre/lustre/mdc/mdc_request.c index c620f5c..215b156 100644 --- a/drivers/staging/lustre/lustre/mdc/mdc_request.c +++ b/drivers/staging/lustre/lustre/mdc/mdc_request.c @@ -705,9 +705,8 @@ static int mdc_close(struct obd_export *exp, struct md_op_data *op_data, int rc; int saved_rc = 0; - req_fmt = &RQF_MDS_CLOSE; if (op_data->op_bias & MDS_HSM_RELEASE) { - req_fmt = &RQF_MDS_RELEASE_CLOSE; + req_fmt = &RQF_MDS_INTENT_CLOSE; /* allocate a FID for volatile file */ rc = mdc_fid_alloc(NULL, exp, &op_data->op_fid2, op_data); @@ -717,6 +716,10 @@ static int mdc_close(struct obd_export *exp, struct md_op_data *op_data, /* save the errcode and proceed to close */ saved_rc = rc; } + } else if (op_data->op_bias & MDS_CLOSE_LAYOUT_SWAP) { + req_fmt = &RQF_MDS_INTENT_CLOSE; + } else { + req_fmt = &RQF_MDS_CLOSE; } *request = NULL; diff --git a/drivers/staging/lustre/lustre/ptlrpc/layout.c b/drivers/staging/lustre/lustre/ptlrpc/layout.c index f0e0448..31aa58e 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/layout.c +++ b/drivers/staging/lustre/lustre/ptlrpc/layout.c @@ -121,7 +121,7 @@ &RMF_CAPA1 }; -static const struct req_msg_field *mdt_release_close_client[] = { +static const struct req_msg_field *mdt_intent_close_client[] = { &RMF_PTLRPC_BODY, &RMF_MDT_EPOCH, &RMF_REC_REINT, @@ -666,7 +666,7 @@ &RQF_MDS_GETXATTR, &RQF_MDS_SYNC, &RQF_MDS_CLOSE, - &RQF_MDS_RELEASE_CLOSE, + &RQF_MDS_INTENT_CLOSE, &RQF_MDS_READPAGE, &RQF_MDS_WRITEPAGE, &RQF_MDS_REINT, @@ -1365,10 +1365,10 @@ struct req_format RQF_MDS_CLOSE = mdt_close_client, mds_last_unlink_server); EXPORT_SYMBOL(RQF_MDS_CLOSE); -struct req_format RQF_MDS_RELEASE_CLOSE = +struct req_format RQF_MDS_INTENT_CLOSE = DEFINE_REQ_FMT0("MDS_CLOSE", - mdt_release_close_client, mds_last_unlink_server); -EXPORT_SYMBOL(RQF_MDS_RELEASE_CLOSE); + mdt_intent_close_client, mds_last_unlink_server); +EXPORT_SYMBOL(RQF_MDS_INTENT_CLOSE); struct req_format RQF_MDS_READPAGE = DEFINE_REQ_FMT0("MDS_READPAGE", -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:30:50 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:30:50 -0500 Subject: [lustre-devel] [PATCH 20/35] staging: lustre: ptlrpc: Move IT_* definitions to lustre_idl.h In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <1478799065-24841-21-git-send-email-jsimmons@infradead.org> From: Ben Evans Put IT_* definitions into an enum, as they're sent over the wire, adjust calls, print statements, etc. to use the new enum. Signed-off-by: Ben Evans Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6746 Reviewed-on: http://review.whamcloud.com/16228 Reviewed-by: John L. Hammond Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- .../lustre/lustre/include/lustre/lustre_idl.h | 18 ++++++++++++++++++ drivers/staging/lustre/lustre/include/lustre_dlm.h | 2 +- drivers/staging/lustre/lustre/include/obd.h | 16 ---------------- drivers/staging/lustre/lustre/ldlm/ldlm_lock.c | 4 ++-- 4 files changed, 21 insertions(+), 19 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h index 33deee6..c06909a 100644 --- a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h +++ b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h @@ -2631,6 +2631,24 @@ struct ldlm_flock_wire { void lustre_swab_gl_desc(union ldlm_gl_desc *); +enum ldlm_intent_flags { + IT_OPEN = BIT(0), + IT_CREAT = BIT(1), + IT_OPEN_CREAT = BIT(1) | BIT(0), + IT_READDIR = BIT(2), + IT_GETATTR = BIT(3), + IT_LOOKUP = BIT(4), + IT_UNLINK = BIT(5), + IT_TRUNC = BIT(6), + IT_GETXATTR = BIT(7), + IT_EXEC = BIT(8), + IT_PIN = BIT(9), + IT_LAYOUT = BIT(10), + IT_QUOTA_DQACQ = BIT(11), + IT_QUOTA_CONN = BIT(12), + IT_SETXATTR = BIT(13), +}; + struct ldlm_intent { __u64 opc; }; diff --git a/drivers/staging/lustre/lustre/include/lustre_dlm.h b/drivers/staging/lustre/lustre/include/lustre_dlm.h index f770b86..d94237d 100644 --- a/drivers/staging/lustre/lustre/include/lustre_dlm.h +++ b/drivers/staging/lustre/lustre/include/lustre_dlm.h @@ -977,7 +977,7 @@ struct ldlm_enqueue_info { extern struct obd_ops ldlm_obd_ops; extern char *ldlm_lockname[]; -char *ldlm_it2str(int it); +const char *ldlm_it2str(enum ldlm_intent_flags it); /** * Just a fancy CDEBUG call with log level preset to LDLM_DEBUG. diff --git a/drivers/staging/lustre/lustre/include/obd.h b/drivers/staging/lustre/lustre/include/obd.h index a346312..f74f8bd 100644 --- a/drivers/staging/lustre/lustre/include/obd.h +++ b/drivers/staging/lustre/lustre/include/obd.h @@ -656,22 +656,6 @@ struct obd_device { struct lu_context; -/* /!\ must be coherent with include/linux/namei.h on patched kernel */ -#define IT_OPEN (1 << 0) -#define IT_CREAT (1 << 1) -#define IT_READDIR (1 << 2) -#define IT_GETATTR (1 << 3) -#define IT_LOOKUP (1 << 4) -#define IT_UNLINK (1 << 5) -#define IT_TRUNC (1 << 6) -#define IT_GETXATTR (1 << 7) -#define IT_EXEC (1 << 8) -#define IT_PIN (1 << 9) -#define IT_LAYOUT (1 << 10) -#define IT_QUOTA_DQACQ (1 << 11) -#define IT_QUOTA_CONN (1 << 12) -#define IT_SETXATTR (1 << 13) - static inline int it_to_lock_mode(struct lookup_intent *it) { /* CREAT needs to be tested before open (both could be set) */ diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c index f2044ec..17d532e 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c @@ -105,7 +105,7 @@ void ldlm_convert_policy_to_local(struct obd_export *exp, enum ldlm_type type, convert(wpolicy, lpolicy); } -char *ldlm_it2str(int it) +const char *ldlm_it2str(enum ldlm_intent_flags it) { switch (it) { case IT_OPEN: @@ -127,7 +127,7 @@ void ldlm_convert_policy_to_local(struct obd_export *exp, enum ldlm_type type, case IT_LAYOUT: return "layout"; default: - CERROR("Unknown intent %d\n", it); + CERROR("Unknown intent 0x%08x\n", it); return "UNKNOWN"; } } -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:31:01 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:31:01 -0500 Subject: [lustre-devel] [PATCH 31/35] staging: lustre: obdclass: add export for lprocfs_stats_alloc_one() In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <1478799065-24841-32-git-send-email-jsimmons@infradead.org> From: Chennaiah Palla When compiling the kernel without optimization, when using GCOV, the lprocfs_stats_alloc_one() symbol is not properly exported to other modules and causes the ptlrpc module to fail loading with an unknown symbol. Added EXPORT_SYMBOL(lprocfs_stats_alloc_one) so that this works properly. Signed-off-by: Chennaiah Palla Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-7515 Seagate-bug-id: MRP-3188 Reviewed-on: http://review.whamcloud.com/17443 Reviewed-by: Andreas Dilger Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- .../lustre/lustre/obdclass/lprocfs_status.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c index 8a2f02f..cccc749 100644 --- a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c +++ b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c @@ -1052,6 +1052,7 @@ int lprocfs_stats_alloc_one(struct lprocfs_stats *stats, unsigned int cpuid) } return rc; } +EXPORT_SYMBOL(lprocfs_stats_alloc_one); struct lprocfs_stats *lprocfs_alloc_stats(unsigned int num, enum lprocfs_stats_flags flags) -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:30:55 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:30:55 -0500 Subject: [lustre-devel] [PATCH 25/35] staging: lustre: osc: Do not merge extents with partial pages In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <1478799065-24841-26-git-send-email-jsimmons@infradead.org> From: Jinshan Xiong After range lock is introduced to Lustre, it's possible for multiple threads to submit osc_extents with partial pages, and finally I/O engine may try to merge these extents, which will end up with assert in osc_build_rpc(). In this patch, osc_extent::oe_no_merge is introduced, and this flag is set if osc_extent submitted via osc_io_submit() includes partial pages. This flag is used by I/O engine to stop merging this kind of extents. Signed-off-by: Jinshan Xiong Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6666 Reviewed-on: http://review.whamcloud.com/15468 Reviewed-by: Andreas Dilger Reviewed-by: Patrick Farrell Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- drivers/staging/lustre/lustre/osc/osc_cache.c | 11 +++++++++-- .../staging/lustre/lustre/osc/osc_cl_internal.h | 4 ++++ 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/drivers/staging/lustre/lustre/osc/osc_cache.c b/drivers/staging/lustre/lustre/osc/osc_cache.c index 0ca01b1..b0f030c 100644 --- a/drivers/staging/lustre/lustre/osc/osc_cache.c +++ b/drivers/staging/lustre/lustre/osc/osc_cache.c @@ -1918,7 +1918,8 @@ static int try_to_add_extent_for_io(struct client_obd *cli, } if (tmp->oe_srvlock != ext->oe_srvlock || - !tmp->oe_grants != !ext->oe_grants) + !tmp->oe_grants != !ext->oe_grants || + tmp->oe_no_merge || ext->oe_no_merge) return 0; /* remove break for strict check */ @@ -2649,11 +2650,13 @@ int osc_queue_sync_pages(const struct lu_env *env, struct osc_object *obj, struct osc_async_page *oap, *tmp; int page_count = 0; int mppr = cli->cl_max_pages_per_rpc; + bool can_merge = true; pgoff_t start = CL_PAGE_EOF; pgoff_t end = 0; list_for_each_entry(oap, list, oap_pending_item) { - pgoff_t index = osc_index(oap2osc(oap)); + struct osc_page *opg = oap2osc_page(oap); + pgoff_t index = osc_index(opg); if (index > end) end = index; @@ -2661,6 +2664,9 @@ int osc_queue_sync_pages(const struct lu_env *env, struct osc_object *obj, start = index; ++page_count; mppr <<= (page_count > mppr); + + if (unlikely(opg->ops_from > 0 || opg->ops_to < PAGE_SIZE)) + can_merge = false; } ext = osc_extent_alloc(obj); @@ -2674,6 +2680,7 @@ int osc_queue_sync_pages(const struct lu_env *env, struct osc_object *obj, ext->oe_rw = !!(cmd & OBD_BRW_READ); ext->oe_sync = 1; + ext->oe_no_merge = !can_merge; ext->oe_urgent = 1; ext->oe_start = start; ext->oe_end = end; diff --git a/drivers/staging/lustre/lustre/osc/osc_cl_internal.h b/drivers/staging/lustre/lustre/osc/osc_cl_internal.h index e525110..e4a214d 100644 --- a/drivers/staging/lustre/lustre/osc/osc_cl_internal.h +++ b/drivers/staging/lustre/lustre/osc/osc_cl_internal.h @@ -596,6 +596,10 @@ struct osc_extent { oe_rw:1, /** sync extent, queued by osc_queue_sync_pages() */ oe_sync:1, + /** set if this extent has partial, sync pages. + * Extents with partial page(s) can't merge with others in RPC + */ + oe_no_merge:1, oe_srvlock:1, oe_memalloc:1, /** an ACTIVE extent is going to be truncated, so when this extent -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:30:59 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:30:59 -0500 Subject: [lustre-devel] [PATCH 29/35] staging: lustre: misc: clean up DFID related error messages In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <1478799065-24841-30-git-send-email-jsimmons@infradead.org> From: Andreas Dilger Improve the error messages related to DFID output and parsing for usage in userspace. Signed-off-by: Andreas Dilger Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-1606 Reviewed-on: http://review.whamcloud.com/6156 Reviewed-by: Frank Zago Reviewed-by: Ben Evans Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- .../lustre/lustre/include/lustre/lustre_idl.h | 9 +++++---- .../lustre/lustre/include/lustre/lustre_user.h | 17 ++++++----------- 2 files changed, 11 insertions(+), 15 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h index c06909a..b8ddebd 100644 --- a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h +++ b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h @@ -537,13 +537,14 @@ static inline void ostid_set_id(struct ost_id *oi, __u64 oid) { if (fid_seq_is_mdt0(oi->oi.oi_seq)) { if (oid >= IDIF_MAX_OID) { - CERROR("Bad %llu to set " DOSTID "\n", oid, POSTID(oi)); + CERROR("Too large OID %#llx to set MDT0 " DOSTID "\n", + oid, POSTID(oi)); return; } oi->oi.oi_id = oid; } else if (fid_is_idif(&oi->oi_fid)) { if (oid >= IDIF_MAX_OID) { - CERROR("Bad %llu to set "DOSTID"\n", + CERROR("Too large OID %#llx to set IDIF " DOSTID "\n", oid, POSTID(oi)); return; } @@ -569,7 +570,7 @@ static inline int fid_set_id(struct lu_fid *fid, __u64 oid) if (fid_is_idif(fid)) { if (oid >= IDIF_MAX_OID) { - CERROR("Too large OID %#llx to set IDIF "DFID"\n", + CERROR("Too large OID %#llx to set IDIF " DFID "\n", (unsigned long long)oid, PFID(fid)); return -EBADF; } @@ -578,7 +579,7 @@ static inline int fid_set_id(struct lu_fid *fid, __u64 oid) fid->f_ver = oid >> 48; } else { if (oid >= OBIF_MAX_OID) { - CERROR("Too large OID %#llx to set REG "DFID"\n", + CERROR("Too large OID %#llx to set REG " DFID "\n", (unsigned long long)oid, PFID(fid)); return -EBADF; } diff --git a/drivers/staging/lustre/lustre/include/lustre/lustre_user.h b/drivers/staging/lustre/lustre/include/lustre/lustre_user.h index e393ae3..d964841 100644 --- a/drivers/staging/lustre/lustre/include/lustre/lustre_user.h +++ b/drivers/staging/lustre/lustre/include/lustre/lustre_user.h @@ -528,25 +528,20 @@ static inline void obd_uuid2fsname(char *buf, char *uuid, int buflen) } /* printf display format - * e.g. printf("file FID is "DFID"\n", PFID(fid)); + * * usage: printf("file FID is "DFID"\n", PFID(fid)); */ #define FID_NOBRACE_LEN 40 #define FID_LEN (FID_NOBRACE_LEN + 2) #define DFID_NOBRACE "%#llx:0x%x:0x%x" #define DFID "["DFID_NOBRACE"]" -#define PFID(fid) \ - (fid)->f_seq, \ - (fid)->f_oid, \ - (fid)->f_ver +#define PFID(fid) (unsigned long long)(fid)->f_seq, (fid)->f_oid, (fid)->f_ver -/* scanf input parse format -- strip '[' first. - * e.g. sscanf(fidstr, SFID, RFID(&fid)); +/* scanf input parse format for fids in DFID_NOBRACE format + * Need to strip '[' from DFID format first or use "["SFID"]" at caller. + * usage: sscanf(fidstr, SFID, RFID(&fid)); */ #define SFID "0x%llx:0x%x:0x%x" -#define RFID(fid) \ - &((fid)->f_seq), \ - &((fid)->f_oid), \ - &((fid)->f_ver) +#define RFID(fid) &((fid)->f_seq), &((fid)->f_oid), &((fid)->f_ver) /********* Quotas **********/ -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:30:53 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:30:53 -0500 Subject: [lustre-devel] [PATCH 23/35] staging: lustre: ldlm: improve lock timeout messages In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <1478799065-24841-24-git-send-email-jsimmons@infradead.org> From: John L. Hammond In ldlm_expired_completion_wait() remove the useless LCONSOLE_WARN() message and upgrade the LDLM_DEBUG() statement to LDLM_ERROR(). Signed-off-by: John L. Hammond Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-7296 Reviewed-on: http://review.whamcloud.com/16824 Reviewed-by: Andreas Dilger Reviewed-by: Alex Zhuravlev Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- drivers/staging/lustre/lustre/ldlm/ldlm_request.c | 6 +----- 1 files changed, 1 insertions(+), 5 deletions(-) diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c index 6a96f2c..836789f 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c @@ -93,11 +93,7 @@ static int ldlm_expired_completion_wait(void *data) if (!lock->l_conn_export) { static unsigned long next_dump, last_dump; - LCONSOLE_WARN("lock timed out (enqueued at %lld, %llds ago)\n", - (s64)lock->l_last_activity, - (s64)(ktime_get_real_seconds() - - lock->l_last_activity)); - LDLM_DEBUG(lock, "lock timed out (enqueued at %lld, %llds ago); not entering recovery in server code, just going back to sleep", + LDLM_ERROR(lock, "lock timed out (enqueued at %lld, %llds ago); not entering recovery in server code, just going back to sleep", (s64)lock->l_last_activity, (s64)(ktime_get_real_seconds() - lock->l_last_activity)); -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:30:52 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:30:52 -0500 Subject: [lustre-devel] [PATCH 22/35] staging: lustre: llite: clear dir stripe md in ll_iget In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <1478799065-24841-23-git-send-email-jsimmons@infradead.org> From: wang di If ll_iget fails during inode initialization, especially during striped directory lookup after creation failed, then it should clear stripe MD before make_bad_inode(), because make_bad_inode() will reset the i_mode, which can cause ll_clear_inode() skip freeing those stripe MD. Signed-off-by: wang di Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-7230 Reviewed-on: http://review.whamcloud.com/16677 Reviewed-by: James Simmons Reviewed-by: Andreas Dilger Signed-off-by: James Simmons --- .../staging/lustre/lustre/llite/llite_internal.h | 1 + drivers/staging/lustre/lustre/llite/llite_lib.c | 2 +- drivers/staging/lustre/lustre/llite/namei.c | 10 ++++++++++ 3 files changed, 12 insertions(+), 1 deletions(-) diff --git a/drivers/staging/lustre/lustre/llite/llite_internal.h b/drivers/staging/lustre/lustre/llite/llite_internal.h index f635efa..c0513bf 100644 --- a/drivers/staging/lustre/lustre/llite/llite_internal.h +++ b/drivers/staging/lustre/lustre/llite/llite_internal.h @@ -785,6 +785,7 @@ int ll_revalidate_it_finish(struct ptlrpc_request *request, void ll_put_super(struct super_block *sb); void ll_kill_super(struct super_block *sb); struct inode *ll_inode_from_resource_lock(struct ldlm_lock *lock); +void ll_dir_clear_lsm_md(struct inode *inode); void ll_clear_inode(struct inode *inode); int ll_setattr_raw(struct dentry *dentry, struct iattr *attr, bool hsm_import); int ll_setattr(struct dentry *de, struct iattr *attr); diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c index 609db9b..dcd9240 100644 --- a/drivers/staging/lustre/lustre/llite/llite_lib.c +++ b/drivers/staging/lustre/lustre/llite/llite_lib.c @@ -1058,7 +1058,7 @@ struct inode *ll_inode_from_resource_lock(struct ldlm_lock *lock) return inode; } -static void ll_dir_clear_lsm_md(struct inode *inode) +void ll_dir_clear_lsm_md(struct inode *inode) { struct ll_inode_info *lli = ll_i2info(inode); diff --git a/drivers/staging/lustre/lustre/llite/namei.c b/drivers/staging/lustre/lustre/llite/namei.c index 9ea43e1..b07079c 100644 --- a/drivers/staging/lustre/lustre/llite/namei.c +++ b/drivers/staging/lustre/lustre/llite/namei.c @@ -117,6 +117,14 @@ struct inode *ll_iget(struct super_block *sb, ino_t hash, rc = cl_file_inode_init(inode, md); if (rc) { + /* + * Let's clear directory lsm here, otherwise + * make_bad_inode() will reset the inode mode + * to regular, then ll_clear_inode will not + * be able to clear lsm_md + */ + if (S_ISDIR(inode->i_mode)) + ll_dir_clear_lsm_md(inode); make_bad_inode(inode); unlock_new_inode(inode); iput(inode); @@ -129,6 +137,8 @@ struct inode *ll_iget(struct super_block *sb, ino_t hash, CDEBUG(D_VFSTRACE, "got inode: "DFID"(%p): rc = %d\n", PFID(&md->body->mbo_fid1), inode, rc); if (rc) { + if (S_ISDIR(inode->i_mode)) + ll_dir_clear_lsm_md(inode); iput(inode); inode = ERR_PTR(rc); } -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:30:42 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:30:42 -0500 Subject: [lustre-devel] [PATCH 12/35] staging: lustre: clio: get rid of cl_req In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <1478799065-24841-13-git-send-email-jsimmons@infradead.org> From: Jinshan Xiong Implement cl_req_attr_set with a cl_object operation. Get rid of cl_req and related function and data structures. Signed-off-by: Jinshan Xiong Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6943 Reviewed-on: http://review.whamcloud.com/15833 Reviewed-by: John L. Hammond Reviewed-by: Bobi Jam Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- drivers/staging/lustre/lustre/include/cl_object.h | 242 ++------------------ drivers/staging/lustre/lustre/llite/Makefile | 2 +- .../staging/lustre/lustre/llite/llite_internal.h | 11 - drivers/staging/lustre/lustre/llite/lproc_llite.c | 4 - drivers/staging/lustre/lustre/llite/vvp_dev.c | 12 - drivers/staging/lustre/lustre/llite/vvp_internal.h | 8 - drivers/staging/lustre/lustre/llite/vvp_object.c | 23 ++- drivers/staging/lustre/lustre/llite/vvp_req.c | 115 --------- .../staging/lustre/lustre/lov/lov_cl_internal.h | 26 -- drivers/staging/lustre/lustre/lov/lov_dev.c | 52 ----- drivers/staging/lustre/lustre/lov/lovsub_dev.c | 61 ----- drivers/staging/lustre/lustre/lov/lovsub_object.c | 22 ++- drivers/staging/lustre/lustre/obdclass/cl_io.c | 231 +------------------ drivers/staging/lustre/lustre/obdclass/cl_page.c | 19 +-- .../staging/lustre/lustre/obdecho/echo_client.c | 4 - drivers/staging/lustre/lustre/osc/osc_cache.c | 9 +- .../staging/lustre/lustre/osc/osc_cl_internal.h | 21 +- drivers/staging/lustre/lustre/osc/osc_dev.c | 13 +- drivers/staging/lustre/lustre/osc/osc_io.c | 133 ----------- drivers/staging/lustre/lustre/osc/osc_object.c | 73 ++++++- drivers/staging/lustre/lustre/osc/osc_request.c | 127 ++++------- 21 files changed, 200 insertions(+), 1008 deletions(-) delete mode 100644 drivers/staging/lustre/lustre/llite/vvp_req.c diff --git a/drivers/staging/lustre/lustre/include/cl_object.h b/drivers/staging/lustre/lustre/include/cl_object.h index 3fe26e7..dc68561 100644 --- a/drivers/staging/lustre/lustre/include/cl_object.h +++ b/drivers/staging/lustre/lustre/include/cl_object.h @@ -59,10 +59,6 @@ * read/write system call it is associated with the single user * thread, that issued the system call). * - * - cl_req represents a collection of pages for a transfer. cl_req is - * constructed by req-forming engine that tries to saturate - * transport with large and continuous transfers. - * * Terminology * * - to avoid confusion high-level I/O operation like read or write system @@ -103,11 +99,8 @@ struct inode; struct cl_device; -struct cl_device_operations; struct cl_object; -struct cl_object_page_operations; -struct cl_object_lock_operations; struct cl_page; struct cl_page_slice; @@ -120,27 +113,7 @@ struct cl_io; struct cl_io_slice; -struct cl_req; -struct cl_req_slice; - -/** - * Operations for each data device in the client stack. - * - * \see vvp_cl_ops, lov_cl_ops, lovsub_cl_ops, osc_cl_ops - */ -struct cl_device_operations { - /** - * Initialize cl_req. This method is called top-to-bottom on all - * devices in the stack to get them a chance to allocate layer-private - * data, and to attach them to the cl_req by calling - * cl_req_slice_add(). - * - * \see osc_req_init(), lov_req_init(), lovsub_req_init() - * \see vvp_req_init() - */ - int (*cdo_req_init)(const struct lu_env *env, struct cl_device *dev, - struct cl_req *req); -}; +struct cl_req_attr; /** * Device in the client stack. @@ -150,8 +123,6 @@ struct cl_device_operations { struct cl_device { /** Super-class. */ struct lu_device cd_lu_dev; - /** Per-layer operation vector. */ - const struct cl_device_operations *cd_ops; }; /** \addtogroup cl_object cl_object @@ -435,6 +406,12 @@ struct cl_object_operations { * Get maximum size of the object. */ loff_t (*coo_maxbytes)(struct cl_object *obj); + /** + * Set request attributes. + */ + void (*coo_req_attr_set)(const struct lu_env *env, + struct cl_object *obj, + struct cl_req_attr *attr); }; /** @@ -626,7 +603,7 @@ enum cl_page_state { * * - [cl_page_state::CPS_PAGEOUT] page is dirty, the * req-formation engine decides that it wants to include this page - * into an cl_req being constructed, and yanks it from the cache; + * into an RPC being constructed, and yanks it from the cache; * * - [cl_page_state::CPS_FREEING] VM callback is executed to * evict the page form the memory; @@ -695,7 +672,7 @@ enum cl_page_state { * Page is being read in, as a part of a transfer. This is quite * similar to the cl_page_state::CPS_PAGEOUT state, except that * read-in is always "immediate"---there is no such thing a sudden - * construction of read cl_req from cached, presumably not up to date, + * construction of read request from cached, presumably not up to date, * pages. * * Underlying VM page is locked for the duration of transfer. @@ -749,8 +726,6 @@ struct cl_page { struct list_head cp_batch; /** List of slices. Immutable after creation. */ struct list_head cp_layers; - /** Linkage of pages within cl_req. */ - struct list_head cp_flight; /** * Page state. This field is const to avoid accidental update, it is * modified only internally within cl_page.c. Protected by a VM lock. @@ -767,12 +742,6 @@ struct cl_page { * by sub-io. Protected by a VM lock. */ struct cl_io *cp_owner; - /** - * Owning IO request in cl_page_state::CPS_PAGEOUT and - * cl_page_state::CPS_PAGEIN states. This field is maintained only in - * the top-level pages. Protected by a VM lock. - */ - struct cl_req *cp_req; /** List of references to this page, for debugging. */ struct lu_ref cp_reference; /** Link to an object, for debugging. */ @@ -814,7 +783,6 @@ enum cl_lock_mode { /** * Requested transfer type. - * \ingroup cl_req */ enum cl_req_type { CRT_READ, @@ -930,8 +898,7 @@ struct cl_page_operations { /** * \name transfer * - * Transfer methods. See comment on cl_req for a description of - * transfer formation and life-cycle. + * Transfer methods. * * @{ */ @@ -977,7 +944,7 @@ struct cl_page_operations { int ioret); /** * Called when cached page is about to be added to the - * cl_req as a part of req formation. + * ptlrpc request as a part of req formation. * * \return 0 : proceed with this page; * \return -EAGAIN : skip this page; @@ -1879,179 +1846,20 @@ struct cl_io { /** @} cl_io */ -/** \addtogroup cl_req cl_req - * @{ - */ -/** \struct cl_req - * Transfer. - * - * There are two possible modes of transfer initiation on the client: - * - * - immediate transfer: this is started when a high level io wants a page - * or a collection of pages to be transferred right away. Examples: - * read-ahead, synchronous read in the case of non-page aligned write, - * page write-out as a part of extent lock cancellation, page write-out - * as a part of memory cleansing. Immediate transfer can be both - * cl_req_type::CRT_READ and cl_req_type::CRT_WRITE; - * - * - opportunistic transfer (cl_req_type::CRT_WRITE only), that happens - * when io wants to transfer a page to the server some time later, when - * it can be done efficiently. Example: pages dirtied by the write(2) - * path. - * - * In any case, transfer takes place in the form of a cl_req, which is a - * representation for a network RPC. - * - * Pages queued for an opportunistic transfer are cached until it is decided - * that efficient RPC can be composed of them. This decision is made by "a - * req-formation engine", currently implemented as a part of osc - * layer. Req-formation depends on many factors: the size of the resulting - * RPC, whether or not multi-object RPCs are supported by the server, - * max-rpc-in-flight limitations, size of the dirty cache, etc. - * - * For the immediate transfer io submits a cl_page_list, that req-formation - * engine slices into cl_req's, possibly adding cached pages to some of - * the resulting req's. - * - * Whenever a page from cl_page_list is added to a newly constructed req, its - * cl_page_operations::cpo_prep() layer methods are called. At that moment, - * page state is atomically changed from cl_page_state::CPS_OWNED to - * cl_page_state::CPS_PAGEOUT or cl_page_state::CPS_PAGEIN, cl_page::cp_owner - * is zeroed, and cl_page::cp_req is set to the - * req. cl_page_operations::cpo_prep() method at the particular layer might - * return -EALREADY to indicate that it does not need to submit this page - * at all. This is possible, for example, if page, submitted for read, - * became up-to-date in the meantime; and for write, the page don't have - * dirty bit marked. \see cl_io_submit_rw() - * - * Whenever a cached page is added to a newly constructed req, its - * cl_page_operations::cpo_make_ready() layer methods are called. At that - * moment, page state is atomically changed from cl_page_state::CPS_CACHED to - * cl_page_state::CPS_PAGEOUT, and cl_page::cp_req is set to - * req. cl_page_operations::cpo_make_ready() method at the particular layer - * might return -EAGAIN to indicate that this page is not eligible for the - * transfer right now. - * - * FUTURE - * - * Plan is to divide transfers into "priority bands" (indicated when - * submitting cl_page_list, and queuing a page for the opportunistic transfer) - * and allow glueing of cached pages to immediate transfers only within single - * band. This would make high priority transfers (like lock cancellation or - * memory pressure induced write-out) really high priority. - * - */ - /** * Per-transfer attributes. */ struct cl_req_attr { + enum cl_req_type cra_type; + u64 cra_flags; + struct cl_page *cra_page; + /** Generic attributes for the server consumption. */ struct obdo *cra_oa; /** Jobid */ char cra_jobid[LUSTRE_JOBID_SIZE]; }; -/** - * Transfer request operations definable at every layer. - * - * Concurrency: transfer formation engine synchronizes calls to all transfer - * methods. - */ -struct cl_req_operations { - /** - * Invoked top-to-bottom by cl_req_prep() when transfer formation is - * complete (all pages are added). - * - * \see osc_req_prep() - */ - int (*cro_prep)(const struct lu_env *env, - const struct cl_req_slice *slice); - /** - * Called top-to-bottom to fill in \a oa fields. This is called twice - * with different flags, see bug 10150 and osc_build_req(). - * - * \param obj an object from cl_req which attributes are to be set in - * \a oa. - * - * \param oa struct obdo where attributes are placed - * - * \param flags \a oa fields to be filled. - */ - void (*cro_attr_set)(const struct lu_env *env, - const struct cl_req_slice *slice, - const struct cl_object *obj, - struct cl_req_attr *attr, u64 flags); - /** - * Called top-to-bottom from cl_req_completion() to notify layers that - * transfer completed. Has to free all state allocated by - * cl_device_operations::cdo_req_init(). - */ - void (*cro_completion)(const struct lu_env *env, - const struct cl_req_slice *slice, int ioret); -}; - -/** - * A per-object state that (potentially multi-object) transfer request keeps. - */ -struct cl_req_obj { - /** object itself */ - struct cl_object *ro_obj; - /** reference to cl_req_obj::ro_obj. For debugging. */ - struct lu_ref_link ro_obj_ref; - /* something else? Number of pages for a given object? */ -}; - -/** - * Transfer request. - * - * Transfer requests are not reference counted, because IO sub-system owns - * them exclusively and knows when to free them. - * - * Life cycle. - * - * cl_req is created by cl_req_alloc() that calls - * cl_device_operations::cdo_req_init() device methods to allocate per-req - * state in every layer. - * - * Then pages are added (cl_req_page_add()), req keeps track of all objects it - * contains pages for. - * - * Once all pages were collected, cl_page_operations::cpo_prep() method is - * called top-to-bottom. At that point layers can modify req, let it pass, or - * deny it completely. This is to support things like SNS that have transfer - * ordering requirements invisible to the individual req-formation engine. - * - * On transfer completion (or transfer timeout, or failure to initiate the - * transfer of an allocated req), cl_req_operations::cro_completion() method - * is called, after execution of cl_page_operations::cpo_completion() of all - * req's pages. - */ -struct cl_req { - enum cl_req_type crq_type; - /** A list of pages being transferred */ - struct list_head crq_pages; - /** Number of pages in cl_req::crq_pages */ - unsigned crq_nrpages; - /** An array of objects which pages are in ->crq_pages */ - struct cl_req_obj *crq_o; - /** Number of elements in cl_req::crq_objs[] */ - unsigned crq_nrobjs; - struct list_head crq_layers; -}; - -/** - * Per-layer state for request. - */ -struct cl_req_slice { - struct cl_req *crs_req; - struct cl_device *crs_dev; - struct list_head crs_linkage; - const struct cl_req_operations *crs_ops; -}; - -/* @} cl_req */ - enum cache_stats_item { /** how many cache lookups were performed */ CS_lookup = 0, @@ -2196,9 +2004,6 @@ void cl_lock_slice_add(struct cl_lock *lock, struct cl_lock_slice *slice, const struct cl_lock_operations *ops); void cl_io_slice_add(struct cl_io *io, struct cl_io_slice *slice, struct cl_object *obj, const struct cl_io_operations *ops); -void cl_req_slice_add(struct cl_req *req, struct cl_req_slice *slice, - struct cl_device *dev, - const struct cl_req_operations *ops); /** @} helpers */ /** \defgroup cl_object cl_object @@ -2567,19 +2372,8 @@ void cl_2queue_discard(const struct lu_env *env, /** @} cl_page_list */ -/** \defgroup cl_req cl_req - * @{ - */ -struct cl_req *cl_req_alloc(const struct lu_env *env, struct cl_page *page, - enum cl_req_type crt, int nr_objects); - -void cl_req_page_add(const struct lu_env *env, struct cl_req *req, - struct cl_page *page); -void cl_req_page_done(const struct lu_env *env, struct cl_page *page); -int cl_req_prep(const struct lu_env *env, struct cl_req *req); -void cl_req_attr_set(const struct lu_env *env, struct cl_req *req, - struct cl_req_attr *attr, u64 flags); -void cl_req_completion(const struct lu_env *env, struct cl_req *req, int ioret); +void cl_req_attr_set(const struct lu_env *env, struct cl_object *obj, + struct cl_req_attr *attr); /** \defgroup cl_sync_io cl_sync_io * @{ @@ -2615,8 +2409,6 @@ void cl_sync_io_note(const struct lu_env *env, struct cl_sync_io *anchor, /** @} cl_sync_io */ -/** @} cl_req */ - /** \defgroup cl_env cl_env * * lu_env handling for a client. diff --git a/drivers/staging/lustre/lustre/llite/Makefile b/drivers/staging/lustre/lustre/llite/Makefile index ca9c275..322d4fa 100644 --- a/drivers/staging/lustre/lustre/llite/Makefile +++ b/drivers/staging/lustre/lustre/llite/Makefile @@ -3,5 +3,5 @@ lustre-y := dcache.o dir.o file.o llite_lib.o llite_nfs.o \ rw.o rw26.o namei.o symlink.o llite_mmap.o range_lock.o \ xattr.o xattr_cache.o xattr_security.o \ super25.o statahead.o glimpse.o lcommon_cl.o lcommon_misc.o \ - vvp_dev.o vvp_page.o vvp_lock.o vvp_io.o vvp_object.o vvp_req.o \ + vvp_dev.o vvp_page.o vvp_lock.o vvp_io.o vvp_object.o \ lproc_llite.o diff --git a/drivers/staging/lustre/lustre/llite/llite_internal.h b/drivers/staging/lustre/lustre/llite/llite_internal.h index 8bd1eb8..f635efa 100644 --- a/drivers/staging/lustre/lustre/llite/llite_internal.h +++ b/drivers/staging/lustre/lustre/llite/llite_internal.h @@ -662,8 +662,6 @@ enum { LPROC_LL_WRITE_BYTES, LPROC_LL_BRW_READ, LPROC_LL_BRW_WRITE, - LPROC_LL_OSC_READ, - LPROC_LL_OSC_WRITE, LPROC_LL_IOCTL, LPROC_LL_OPEN, LPROC_LL_RELEASE, @@ -1214,15 +1212,6 @@ struct ll_dio_pages { int ldp_nr; }; -static inline void cl_stats_tally(struct cl_device *dev, enum cl_req_type crt, - int rc) -{ - int opc = (crt == CRT_READ) ? LPROC_LL_OSC_READ : - LPROC_LL_OSC_WRITE; - - ll_stats_ops_tally(ll_s2sbi(cl2vvp_dev(dev)->vdv_sb), opc, rc); -} - ssize_t ll_direct_rw_pages(const struct lu_env *env, struct cl_io *io, int rw, struct inode *inode, struct ll_dio_pages *pv); diff --git a/drivers/staging/lustre/lustre/llite/lproc_llite.c b/drivers/staging/lustre/lustre/llite/lproc_llite.c index 23fda9d..03682c1 100644 --- a/drivers/staging/lustre/lustre/llite/lproc_llite.c +++ b/drivers/staging/lustre/lustre/llite/lproc_llite.c @@ -1060,10 +1060,6 @@ static void llite_sb_release(struct kobject *kobj) "brw_read" }, { LPROC_LL_BRW_WRITE, LPROCFS_CNTR_AVGMINMAX | LPROCFS_TYPE_PAGES, "brw_write" }, - { LPROC_LL_OSC_READ, LPROCFS_CNTR_AVGMINMAX | LPROCFS_TYPE_BYTES, - "osc_read" }, - { LPROC_LL_OSC_WRITE, LPROCFS_CNTR_AVGMINMAX | LPROCFS_TYPE_BYTES, - "osc_write" }, { LPROC_LL_IOCTL, LPROCFS_TYPE_REGS, "ioctl" }, { LPROC_LL_OPEN, LPROCFS_TYPE_REGS, "open" }, { LPROC_LL_RELEASE, LPROCFS_TYPE_REGS, "close" }, diff --git a/drivers/staging/lustre/lustre/llite/vvp_dev.c b/drivers/staging/lustre/lustre/llite/vvp_dev.c index cab95ac..12c129f 100644 --- a/drivers/staging/lustre/lustre/llite/vvp_dev.c +++ b/drivers/staging/lustre/lustre/llite/vvp_dev.c @@ -55,7 +55,6 @@ static struct kmem_cache *ll_thread_kmem; struct kmem_cache *vvp_lock_kmem; struct kmem_cache *vvp_object_kmem; -struct kmem_cache *vvp_req_kmem; static struct kmem_cache *vvp_session_kmem; static struct kmem_cache *vvp_thread_kmem; @@ -76,11 +75,6 @@ .ckd_size = sizeof(struct vvp_object), }, { - .ckd_cache = &vvp_req_kmem, - .ckd_name = "vvp_req_kmem", - .ckd_size = sizeof(struct vvp_req), - }, - { .ckd_cache = &vvp_session_kmem, .ckd_name = "vvp_session_kmem", .ckd_size = sizeof(struct vvp_session) @@ -177,10 +171,6 @@ struct lu_context_key vvp_thread_key = { .ldo_object_alloc = vvp_object_alloc }; -static const struct cl_device_operations vvp_cl_ops = { - .cdo_req_init = vvp_req_init -}; - static struct lu_device *vvp_device_free(const struct lu_env *env, struct lu_device *d) { @@ -213,7 +203,6 @@ struct lu_context_key vvp_thread_key = { lud = &vdv->vdv_cl.cd_lu_dev; cl_device_init(&vdv->vdv_cl, t); vvp2lu_dev(vdv)->ld_ops = &vvp_lu_ops; - vdv->vdv_cl.cd_ops = &vvp_cl_ops; site = kzalloc(sizeof(*site), GFP_NOFS); if (site) { @@ -332,7 +321,6 @@ int cl_sb_init(struct super_block *sb) cl = cl_type_setup(env, NULL, &vvp_device_type, sbi->ll_dt_exp->exp_obd->obd_lu_dev); if (!IS_ERR(cl)) { - cl2vvp_dev(cl)->vdv_sb = sb; sbi->ll_cl = cl; sbi->ll_site = cl2lu_dev(cl)->ld_site; } diff --git a/drivers/staging/lustre/lustre/llite/vvp_internal.h b/drivers/staging/lustre/lustre/llite/vvp_internal.h index afed7db..c60d041 100644 --- a/drivers/staging/lustre/lustre/llite/vvp_internal.h +++ b/drivers/staging/lustre/lustre/llite/vvp_internal.h @@ -120,7 +120,6 @@ struct vvp_io { extern struct kmem_cache *vvp_lock_kmem; extern struct kmem_cache *vvp_object_kmem; -extern struct kmem_cache *vvp_req_kmem; struct vvp_thread_info { struct cl_lock vti_lock; @@ -242,7 +241,6 @@ static inline pgoff_t vvp_index(struct vvp_page *vvp) struct vvp_device { struct cl_device vdv_cl; - struct super_block *vdv_sb; struct cl_device *vdv_next; }; @@ -250,10 +248,6 @@ struct vvp_lock { struct cl_lock_slice vlk_cl; }; -struct vvp_req { - struct cl_req_slice vrq_cl; -}; - void *ccc_key_init(const struct lu_context *ctx, struct lu_context_key *key); void ccc_key_fini(const struct lu_context *ctx, @@ -316,8 +310,6 @@ int vvp_lock_init(const struct lu_env *env, struct cl_object *obj, struct cl_lock *lock, const struct cl_io *io); int vvp_page_init(const struct lu_env *env, struct cl_object *obj, struct cl_page *page, pgoff_t index); -int vvp_req_init(const struct lu_env *env, struct cl_device *dev, - struct cl_req *req); struct lu_object *vvp_object_alloc(const struct lu_env *env, const struct lu_object_header *hdr, struct lu_device *dev); diff --git a/drivers/staging/lustre/lustre/llite/vvp_object.c b/drivers/staging/lustre/lustre/llite/vvp_object.c index cc0f3da..8e18cf8 100644 --- a/drivers/staging/lustre/lustre/llite/vvp_object.c +++ b/drivers/staging/lustre/lustre/llite/vvp_object.c @@ -184,6 +184,26 @@ static int vvp_object_glimpse(const struct lu_env *env, return 0; } +static void vvp_req_attr_set(const struct lu_env *env, struct cl_object *obj, + struct cl_req_attr *attr) +{ + u64 valid_flags = OBD_MD_FLTYPE; + struct inode *inode; + struct obdo *oa; + + oa = attr->cra_oa; + inode = vvp_object_inode(obj); + + if (attr->cra_type == CRT_WRITE) + valid_flags |= OBD_MD_FLMTIME | OBD_MD_FLCTIME | + OBD_MD_FLUID | OBD_MD_FLGID; + obdo_from_inode(oa, inode, valid_flags & attr->cra_flags); + obdo_set_parent_fid(oa, &ll_i2info(inode)->lli_fid); + if (OBD_FAIL_CHECK(OBD_FAIL_LFSCK_INVALID_PFID)) + oa->o_parent_oid++; + memcpy(attr->cra_jobid, ll_i2info(inode)->lli_jobid, LUSTRE_JOBID_SIZE); +} + static const struct cl_object_operations vvp_ops = { .coo_page_init = vvp_page_init, .coo_lock_init = vvp_lock_init, @@ -192,7 +212,8 @@ static int vvp_object_glimpse(const struct lu_env *env, .coo_attr_update = vvp_attr_update, .coo_conf_set = vvp_conf_set, .coo_prune = vvp_prune, - .coo_glimpse = vvp_object_glimpse + .coo_glimpse = vvp_object_glimpse, + .coo_req_attr_set = vvp_req_attr_set }; static int vvp_object_init0(const struct lu_env *env, diff --git a/drivers/staging/lustre/lustre/llite/vvp_req.c b/drivers/staging/lustre/lustre/llite/vvp_req.c deleted file mode 100644 index a8892e4..0000000 --- a/drivers/staging/lustre/lustre/llite/vvp_req.c +++ /dev/null @@ -1,115 +0,0 @@ -/* - * GPL HEADER START - * - * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 only, - * as published by the Free Software Foundation. - * - * This program is distributed in the hope that it will be useful, but - * WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * General Public License version 2 for more details (a copy is included - * in the LICENSE file that accompanied this code). - * - * You should have received a copy of the GNU General Public License - * version 2 along with this program; If not, see - * http://www.gnu.org/licenses/gpl-2.0.html - * - * GPL HEADER END - */ -/* - * Copyright (c) 2008, 2010, Oracle and/or its affiliates. All rights reserved. - * Use is subject to license terms. - * - * Copyright (c) 2011, 2014, Intel Corporation. - */ - -#define DEBUG_SUBSYSTEM S_LLITE - -#include "../include/lustre/lustre_idl.h" -#include "../include/cl_object.h" -#include "../include/obd.h" -#include "../include/obd_support.h" -#include "llite_internal.h" -#include "vvp_internal.h" - -static inline struct vvp_req *cl2vvp_req(const struct cl_req_slice *slice) -{ - return container_of0(slice, struct vvp_req, vrq_cl); -} - -/** - * Implementation of struct cl_req_operations::cro_attr_set() for VVP - * layer. VVP is responsible for - * - * - o_[mac]time - * - * - o_mode - * - * - o_parent_seq - * - * - o_[ug]id - * - * - o_parent_oid - * - * - o_parent_ver - * - */ -static void vvp_req_attr_set(const struct lu_env *env, - const struct cl_req_slice *slice, - const struct cl_object *obj, - struct cl_req_attr *attr, u64 flags) -{ - struct inode *inode; - struct obdo *oa; - u32 valid_flags; - - oa = attr->cra_oa; - inode = vvp_object_inode(obj); - valid_flags = OBD_MD_FLTYPE; - - if (slice->crs_req->crq_type == CRT_WRITE) - valid_flags |= OBD_MD_FLMTIME | OBD_MD_FLCTIME | - OBD_MD_FLUID | OBD_MD_FLGID; - obdo_from_inode(oa, inode, valid_flags & flags); - obdo_set_parent_fid(oa, &ll_i2info(inode)->lli_fid); - if (OBD_FAIL_CHECK(OBD_FAIL_LFSCK_INVALID_PFID)) - oa->o_parent_oid++; - memcpy(attr->cra_jobid, ll_i2info(inode)->lli_jobid, - LUSTRE_JOBID_SIZE); -} - -static void vvp_req_completion(const struct lu_env *env, - const struct cl_req_slice *slice, int ioret) -{ - struct vvp_req *vrq; - - if (ioret > 0) - cl_stats_tally(slice->crs_dev, slice->crs_req->crq_type, ioret); - - vrq = cl2vvp_req(slice); - kmem_cache_free(vvp_req_kmem, vrq); -} - -static const struct cl_req_operations vvp_req_ops = { - .cro_attr_set = vvp_req_attr_set, - .cro_completion = vvp_req_completion -}; - -int vvp_req_init(const struct lu_env *env, struct cl_device *dev, - struct cl_req *req) -{ - struct vvp_req *vrq; - int result; - - vrq = kmem_cache_zalloc(vvp_req_kmem, GFP_NOFS); - if (vrq) { - cl_req_slice_add(req, &vrq->vrq_cl, dev, &vvp_req_ops); - result = 0; - } else { - result = -ENOMEM; - } - return result; -} diff --git a/drivers/staging/lustre/lustre/lov/lov_cl_internal.h b/drivers/staging/lustre/lustre/lov/lov_cl_internal.h index a08e41d..c49a34b 100644 --- a/drivers/staging/lustre/lustre/lov/lov_cl_internal.h +++ b/drivers/staging/lustre/lustre/lov/lov_cl_internal.h @@ -472,20 +472,6 @@ struct lov_session { struct lov_sublock_env ls_subenv; }; -/** - * State of transfer for lov. - */ -struct lov_req { - struct cl_req_slice lr_cl; -}; - -/** - * State of transfer for lovsub. - */ -struct lovsub_req { - struct cl_req_slice lsrq_cl; -}; - extern struct lu_device_type lov_device_type; extern struct lu_device_type lovsub_device_type; @@ -496,11 +482,9 @@ struct lovsub_req { extern struct kmem_cache *lov_object_kmem; extern struct kmem_cache *lov_thread_kmem; extern struct kmem_cache *lov_session_kmem; -extern struct kmem_cache *lov_req_kmem; extern struct kmem_cache *lovsub_lock_kmem; extern struct kmem_cache *lovsub_object_kmem; -extern struct kmem_cache *lovsub_req_kmem; extern struct kmem_cache *lov_lock_link_kmem; @@ -699,11 +683,6 @@ static inline int lovsub_is_object(const struct lu_object *obj) return container_of0(slice, struct lov_page, lps_cl); } -static inline struct lov_req *cl2lov_req(const struct cl_req_slice *slice) -{ - return container_of0(slice, struct lov_req, lr_cl); -} - static inline struct lovsub_page * cl2lovsub_page(const struct cl_page_slice *slice) { @@ -711,11 +690,6 @@ static inline int lovsub_is_object(const struct lu_object *obj) return container_of0(slice, struct lovsub_page, lsb_cl); } -static inline struct lovsub_req *cl2lovsub_req(const struct cl_req_slice *slice) -{ - return container_of0(slice, struct lovsub_req, lsrq_cl); -} - static inline struct lov_io *cl2lov_io(const struct lu_env *env, const struct cl_io_slice *ios) { diff --git a/drivers/staging/lustre/lustre/lov/lov_dev.c b/drivers/staging/lustre/lustre/lov/lov_dev.c index 056ae2e..7301f6e 100644 --- a/drivers/staging/lustre/lustre/lov/lov_dev.c +++ b/drivers/staging/lustre/lustre/lov/lov_dev.c @@ -46,11 +46,9 @@ struct kmem_cache *lov_object_kmem; struct kmem_cache *lov_thread_kmem; struct kmem_cache *lov_session_kmem; -struct kmem_cache *lov_req_kmem; struct kmem_cache *lovsub_lock_kmem; struct kmem_cache *lovsub_object_kmem; -struct kmem_cache *lovsub_req_kmem; struct kmem_cache *lov_lock_link_kmem; @@ -79,11 +77,6 @@ struct lu_kmem_descr lov_caches[] = { .ckd_size = sizeof(struct lov_session) }, { - .ckd_cache = &lov_req_kmem, - .ckd_name = "lov_req_kmem", - .ckd_size = sizeof(struct lov_req) - }, - { .ckd_cache = &lovsub_lock_kmem, .ckd_name = "lovsub_lock_kmem", .ckd_size = sizeof(struct lovsub_lock) @@ -94,11 +87,6 @@ struct lu_kmem_descr lov_caches[] = { .ckd_size = sizeof(struct lovsub_object) }, { - .ckd_cache = &lovsub_req_kmem, - .ckd_name = "lovsub_req_kmem", - .ckd_size = sizeof(struct lovsub_req) - }, - { .ckd_cache = &lov_lock_link_kmem, .ckd_name = "lov_lock_link_kmem", .ckd_size = sizeof(struct lov_lock_link) @@ -110,25 +98,6 @@ struct lu_kmem_descr lov_caches[] = { /***************************************************************************** * - * Lov transfer operations. - * - */ - -static void lov_req_completion(const struct lu_env *env, - const struct cl_req_slice *slice, int ioret) -{ - struct lov_req *lr; - - lr = cl2lov_req(slice); - kmem_cache_free(lov_req_kmem, lr); -} - -static const struct cl_req_operations lov_req_ops = { - .cro_completion = lov_req_completion -}; - -/***************************************************************************** - * * Lov device and device type functions. * */ @@ -248,26 +217,6 @@ static int lov_device_init(const struct lu_env *env, struct lu_device *d, return rc; } -static int lov_req_init(const struct lu_env *env, struct cl_device *dev, - struct cl_req *req) -{ - struct lov_req *lr; - int result; - - lr = kmem_cache_zalloc(lov_req_kmem, GFP_NOFS); - if (lr) { - cl_req_slice_add(req, &lr->lr_cl, dev, &lov_req_ops); - result = 0; - } else { - result = -ENOMEM; - } - return result; -} - -static const struct cl_device_operations lov_cl_ops = { - .cdo_req_init = lov_req_init -}; - static void lov_emerg_free(struct lov_device_emerg **emrg, int nr) { int i; @@ -478,7 +427,6 @@ static int lov_process_config(const struct lu_env *env, cl_device_init(&ld->ld_cl, t); d = lov2lu_dev(ld); d->ld_ops = &lov_lu_ops; - ld->ld_cl.cd_ops = &lov_cl_ops; mutex_init(&ld->ld_mutex); lockdep_set_class(&ld->ld_mutex, &cl_lov_device_mutex_class); diff --git a/drivers/staging/lustre/lustre/lov/lovsub_dev.c b/drivers/staging/lustre/lustre/lov/lovsub_dev.c index b519a19..5d6536f 100644 --- a/drivers/staging/lustre/lustre/lov/lovsub_dev.c +++ b/drivers/staging/lustre/lustre/lov/lovsub_dev.c @@ -44,46 +44,6 @@ /***************************************************************************** * - * Lovsub transfer operations. - * - */ - -static void lovsub_req_completion(const struct lu_env *env, - const struct cl_req_slice *slice, int ioret) -{ - struct lovsub_req *lsr; - - lsr = cl2lovsub_req(slice); - kmem_cache_free(lovsub_req_kmem, lsr); -} - -/** - * Implementation of struct cl_req_operations::cro_attr_set() for lovsub - * layer. Lov and lovsub are responsible only for struct obdo::o_stripe_idx - * field, which is filled there. - */ -static void lovsub_req_attr_set(const struct lu_env *env, - const struct cl_req_slice *slice, - const struct cl_object *obj, - struct cl_req_attr *attr, u64 flags) -{ - struct lovsub_object *subobj; - - subobj = cl2lovsub(obj); - /* - * There is no OBD_MD_* flag for obdo::o_stripe_idx, so set it - * unconditionally. It never changes anyway. - */ - attr->cra_oa->o_stripe_idx = subobj->lso_index; -} - -static const struct cl_req_operations lovsub_req_ops = { - .cro_attr_set = lovsub_req_attr_set, - .cro_completion = lovsub_req_completion -}; - -/***************************************************************************** - * * Lov-sub device and device type functions. * */ @@ -137,32 +97,12 @@ static int lovsub_device_init(const struct lu_env *env, struct lu_device *d, return next; } -static int lovsub_req_init(const struct lu_env *env, struct cl_device *dev, - struct cl_req *req) -{ - struct lovsub_req *lsr; - int result; - - lsr = kmem_cache_zalloc(lovsub_req_kmem, GFP_NOFS); - if (lsr) { - cl_req_slice_add(req, &lsr->lsrq_cl, dev, &lovsub_req_ops); - result = 0; - } else { - result = -ENOMEM; - } - return result; -} - static const struct lu_device_operations lovsub_lu_ops = { .ldo_object_alloc = lovsub_object_alloc, .ldo_process_config = NULL, .ldo_recovery_complete = NULL }; -static const struct cl_device_operations lovsub_cl_ops = { - .cdo_req_init = lovsub_req_init -}; - static struct lu_device *lovsub_device_alloc(const struct lu_env *env, struct lu_device_type *t, struct lustre_cfg *cfg) @@ -178,7 +118,6 @@ static int lovsub_req_init(const struct lu_env *env, struct cl_device *dev, if (result == 0) { d = lovsub2lu_dev(lsd); d->ld_ops = &lovsub_lu_ops; - lsd->acid_cl.cd_ops = &lovsub_cl_ops; } else { d = ERR_PTR(result); } diff --git a/drivers/staging/lustre/lustre/lov/lovsub_object.c b/drivers/staging/lustre/lustre/lov/lovsub_object.c index a2bac7a..011296e 100644 --- a/drivers/staging/lustre/lustre/lov/lovsub_object.c +++ b/drivers/staging/lustre/lustre/lov/lovsub_object.c @@ -116,11 +116,31 @@ static int lovsub_object_glimpse(const struct lu_env *env, return cl_object_glimpse(env, &los->lso_super->lo_cl, lvb); } +/** + * Implementation of struct cl_object_operations::coo_req_attr_set() for lovsub + * layer. Lov and lovsub are responsible only for struct obdo::o_stripe_idx + * field, which is filled there. + */ +static void lovsub_req_attr_set(const struct lu_env *env, struct cl_object *obj, + struct cl_req_attr *attr) +{ + struct lovsub_object *subobj = cl2lovsub(obj); + + cl_req_attr_set(env, &subobj->lso_super->lo_cl, attr); + + /* + * There is no OBD_MD_* flag for obdo::o_stripe_idx, so set it + * unconditionally. It never changes anyway. + */ + attr->cra_oa->o_stripe_idx = subobj->lso_index; +} + static const struct cl_object_operations lovsub_ops = { .coo_page_init = lovsub_page_init, .coo_lock_init = lovsub_lock_init, .coo_attr_update = lovsub_attr_update, - .coo_glimpse = lovsub_object_glimpse + .coo_glimpse = lovsub_object_glimpse, + .coo_req_attr_set = lovsub_req_attr_set }; static const struct lu_object_operations lovsub_lu_obj_ops = { diff --git a/drivers/staging/lustre/lustre/obdclass/cl_io.c b/drivers/staging/lustre/lustre/obdclass/cl_io.c index af8e4d6..3f42457 100644 --- a/drivers/staging/lustre/lustre/obdclass/cl_io.c +++ b/drivers/staging/lustre/lustre/obdclass/cl_io.c @@ -1045,235 +1045,18 @@ struct cl_io *cl_io_top(struct cl_io *io) EXPORT_SYMBOL(cl_io_top); /** - * Adds request slice to the compound request. - * - * This is called by cl_device_operations::cdo_req_init() methods to add a - * per-layer state to the request. New state is added at the end of - * cl_req::crq_layers list, that is, it is at the bottom of the stack. - * - * \see cl_lock_slice_add(), cl_page_slice_add(), cl_io_slice_add() - */ -void cl_req_slice_add(struct cl_req *req, struct cl_req_slice *slice, - struct cl_device *dev, - const struct cl_req_operations *ops) -{ - list_add_tail(&slice->crs_linkage, &req->crq_layers); - slice->crs_dev = dev; - slice->crs_ops = ops; - slice->crs_req = req; -} -EXPORT_SYMBOL(cl_req_slice_add); - -static void cl_req_free(const struct lu_env *env, struct cl_req *req) -{ - unsigned i; - - LASSERT(list_empty(&req->crq_pages)); - LASSERT(req->crq_nrpages == 0); - LINVRNT(list_empty(&req->crq_layers)); - LINVRNT(equi(req->crq_nrobjs > 0, req->crq_o)); - - if (req->crq_o) { - for (i = 0; i < req->crq_nrobjs; ++i) { - struct cl_object *obj = req->crq_o[i].ro_obj; - - if (obj) { - lu_object_ref_del_at(&obj->co_lu, - &req->crq_o[i].ro_obj_ref, - "cl_req", req); - cl_object_put(env, obj); - } - } - kfree(req->crq_o); - } - kfree(req); -} - -static int cl_req_init(const struct lu_env *env, struct cl_req *req, - struct cl_page *page) -{ - struct cl_device *dev; - struct cl_page_slice *slice; - int result; - - result = 0; - list_for_each_entry(slice, &page->cp_layers, cpl_linkage) { - dev = lu2cl_dev(slice->cpl_obj->co_lu.lo_dev); - if (dev->cd_ops->cdo_req_init) { - result = dev->cd_ops->cdo_req_init(env, dev, req); - if (result != 0) - break; - } - } - return result; -} - -/** - * Invokes per-request transfer completion call-backs - * (cl_req_operations::cro_completion()) bottom-to-top. - */ -void cl_req_completion(const struct lu_env *env, struct cl_req *req, int rc) -{ - struct cl_req_slice *slice; - - /* - * for the lack of list_for_each_entry_reverse_safe()... - */ - while (!list_empty(&req->crq_layers)) { - slice = list_entry(req->crq_layers.prev, - struct cl_req_slice, crs_linkage); - list_del_init(&slice->crs_linkage); - if (slice->crs_ops->cro_completion) - slice->crs_ops->cro_completion(env, slice, rc); - } - cl_req_free(env, req); -} -EXPORT_SYMBOL(cl_req_completion); - -/** - * Allocates new transfer request. - */ -struct cl_req *cl_req_alloc(const struct lu_env *env, struct cl_page *page, - enum cl_req_type crt, int nr_objects) -{ - struct cl_req *req; - - LINVRNT(nr_objects > 0); - - req = kzalloc(sizeof(*req), GFP_NOFS); - if (req) { - int result; - - req->crq_type = crt; - INIT_LIST_HEAD(&req->crq_pages); - INIT_LIST_HEAD(&req->crq_layers); - - req->crq_o = kcalloc(nr_objects, sizeof(req->crq_o[0]), - GFP_NOFS); - if (req->crq_o) { - req->crq_nrobjs = nr_objects; - result = cl_req_init(env, req, page); - } else { - result = -ENOMEM; - } - if (result != 0) { - cl_req_completion(env, req, result); - req = ERR_PTR(result); - } - } else { - req = ERR_PTR(-ENOMEM); - } - return req; -} -EXPORT_SYMBOL(cl_req_alloc); - -/** - * Adds a page to a request. - */ -void cl_req_page_add(const struct lu_env *env, - struct cl_req *req, struct cl_page *page) -{ - struct cl_object *obj; - struct cl_req_obj *rqo; - unsigned int i; - - LASSERT(list_empty(&page->cp_flight)); - LASSERT(!page->cp_req); - - CL_PAGE_DEBUG(D_PAGE, env, page, "req %p, %d, %u\n", - req, req->crq_type, req->crq_nrpages); - - list_add_tail(&page->cp_flight, &req->crq_pages); - ++req->crq_nrpages; - page->cp_req = req; - obj = cl_object_top(page->cp_obj); - for (i = 0, rqo = req->crq_o; obj != rqo->ro_obj; ++i, ++rqo) { - if (!rqo->ro_obj) { - rqo->ro_obj = obj; - cl_object_get(obj); - lu_object_ref_add_at(&obj->co_lu, &rqo->ro_obj_ref, - "cl_req", req); - break; - } - } - LASSERT(i < req->crq_nrobjs); -} -EXPORT_SYMBOL(cl_req_page_add); - -/** - * Removes a page from a request. - */ -void cl_req_page_done(const struct lu_env *env, struct cl_page *page) -{ - struct cl_req *req = page->cp_req; - - LASSERT(!list_empty(&page->cp_flight)); - LASSERT(req->crq_nrpages > 0); - - list_del_init(&page->cp_flight); - --req->crq_nrpages; - page->cp_req = NULL; -} -EXPORT_SYMBOL(cl_req_page_done); - -/** - * Notifies layers that request is about to depart by calling - * cl_req_operations::cro_prep() top-to-bottom. - */ -int cl_req_prep(const struct lu_env *env, struct cl_req *req) -{ - unsigned int i; - int result; - const struct cl_req_slice *slice; - - /* - * Check that the caller of cl_req_alloc() didn't lie about the number - * of objects. - */ - for (i = 0; i < req->crq_nrobjs; ++i) - LASSERT(req->crq_o[i].ro_obj); - - result = 0; - list_for_each_entry(slice, &req->crq_layers, crs_linkage) { - if (slice->crs_ops->cro_prep) { - result = slice->crs_ops->cro_prep(env, slice); - if (result != 0) - break; - } - } - return result; -} -EXPORT_SYMBOL(cl_req_prep); - -/** * Fills in attributes that are passed to server together with transfer. Only * attributes from \a flags may be touched. This can be called multiple times * for the same request. */ -void cl_req_attr_set(const struct lu_env *env, struct cl_req *req, - struct cl_req_attr *attr, u64 flags) +void cl_req_attr_set(const struct lu_env *env, struct cl_object *obj, + struct cl_req_attr *attr) { - const struct cl_req_slice *slice; - struct cl_page *page; - unsigned int i; - - LASSERT(!list_empty(&req->crq_pages)); - - /* Take any page to use as a model. */ - page = list_entry(req->crq_pages.next, struct cl_page, cp_flight); - - for (i = 0; i < req->crq_nrobjs; ++i) { - list_for_each_entry(slice, &req->crq_layers, crs_linkage) { - const struct cl_page_slice *scan; - const struct cl_object *obj; - - scan = cl_page_at(page, - slice->crs_dev->cd_lu_dev.ld_type); - obj = scan->cpl_obj; - if (slice->crs_ops->cro_attr_set) - slice->crs_ops->cro_attr_set(env, slice, obj, - attr + i, flags); - } + struct cl_object *scan; + + cl_object_for_each(scan, obj) { + if (scan->co_ops->coo_req_attr_set) + scan->co_ops->coo_req_attr_set(env, scan, attr); } } EXPORT_SYMBOL(cl_req_attr_set); diff --git a/drivers/staging/lustre/lustre/obdclass/cl_page.c b/drivers/staging/lustre/lustre/obdclass/cl_page.c index 40b7bee..bdc0aa5 100644 --- a/drivers/staging/lustre/lustre/obdclass/cl_page.c +++ b/drivers/staging/lustre/lustre/obdclass/cl_page.c @@ -99,7 +99,6 @@ static void cl_page_free(const struct lu_env *env, struct cl_page *page) PASSERT(env, page, list_empty(&page->cp_batch)); PASSERT(env, page, !page->cp_owner); - PASSERT(env, page, !page->cp_req); PASSERT(env, page, page->cp_state == CPS_FREEING); while (!list_empty(&page->cp_layers)) { @@ -150,7 +149,6 @@ struct cl_page *cl_page_alloc(const struct lu_env *env, page->cp_type = type; INIT_LIST_HEAD(&page->cp_layers); INIT_LIST_HEAD(&page->cp_batch); - INIT_LIST_HEAD(&page->cp_flight); lu_ref_init(&page->cp_reference); head = o->co_lu.lo_header; list_for_each_entry(o, &head->loh_layers, co_lu.lo_linkage) { @@ -528,7 +526,6 @@ static int cl_page_own0(const struct lu_env *env, struct cl_io *io, io, nonblock); if (result == 0) { PASSERT(env, pg, !pg->cp_owner); - PASSERT(env, pg, !pg->cp_req); pg->cp_owner = cl_io_top(io); cl_page_owner_set(pg); if (pg->cp_state != CPS_FREEING) { @@ -821,8 +818,6 @@ void cl_page_completion(const struct lu_env *env, struct cl_sync_io *anchor = pg->cp_sync_io; PASSERT(env, pg, crt < CRT_NR); - /* cl_page::cp_req already cleared by the caller (osc_completion()) */ - PASSERT(env, pg, !pg->cp_req); PASSERT(env, pg, pg->cp_state == cl_req_type_state(crt)); CL_PAGE_HEADER(D_TRACE, env, pg, "%d %d\n", crt, ioret); @@ -836,16 +831,8 @@ void cl_page_completion(const struct lu_env *env, if (anchor) { LASSERT(pg->cp_sync_io == anchor); pg->cp_sync_io = NULL; - } - /* - * As page->cp_obj is pinned by a reference from page->cp_req, it is - * safe to call cl_page_put() without risking object destruction in a - * non-blocking context. - */ - cl_page_put(env, pg); - - if (anchor) cl_sync_io_note(env, anchor, ioret); + } } EXPORT_SYMBOL(cl_page_completion); @@ -927,10 +914,10 @@ void cl_page_header_print(const struct lu_env *env, void *cookie, lu_printer_t printer, const struct cl_page *pg) { (*printer)(env, cookie, - "page@%p[%d %p %d %d %p %p]\n", + "page@%p[%d %p %d %d %p]\n", pg, atomic_read(&pg->cp_ref), pg->cp_obj, pg->cp_state, pg->cp_type, - pg->cp_owner, pg->cp_req); + pg->cp_owner); } EXPORT_SYMBOL(cl_page_header_print); diff --git a/drivers/staging/lustre/lustre/obdecho/echo_client.c b/drivers/staging/lustre/lustre/obdecho/echo_client.c index 25ecfb3..5490761 100644 --- a/drivers/staging/lustre/lustre/obdecho/echo_client.c +++ b/drivers/staging/lustre/lustre/obdecho/echo_client.c @@ -505,9 +505,6 @@ static int echo_object_print(const struct lu_env *env, void *cookie, /** @} echo_lu_dev_ops */ -static const struct cl_device_operations echo_device_cl_ops = { -}; - /** \defgroup echo_init Setup and teardown * * Init and fini functions for echo client. @@ -620,7 +617,6 @@ static void echo_session_key_fini(const struct lu_context *ctx, goto out_free; cd->cd_lu_dev.ld_ops = &echo_device_lu_ops; - cd->cd_ops = &echo_device_cl_ops; obd = class_name2obd(lustre_cfg_string(cfg, 0)); LASSERT(obd); diff --git a/drivers/staging/lustre/lustre/osc/osc_cache.c b/drivers/staging/lustre/lustre/osc/osc_cache.c index b939c9c..d17ec32 100644 --- a/drivers/staging/lustre/lustre/osc/osc_cache.c +++ b/drivers/staging/lustre/lustre/osc/osc_cache.c @@ -1337,14 +1337,6 @@ static int osc_completion(const struct lu_env *env, struct osc_async_page *oap, "cp_state:%u, cmd:%d\n", page->cp_state, cmd); LASSERT(opg->ops_transfer_pinned); - /* - * page->cp_req can be NULL if io submission failed before - * cl_req was allocated. - */ - if (page->cp_req) - cl_req_page_done(env, page); - LASSERT(!page->cp_req); - crt = cmd == OBD_BRW_READ ? CRT_READ : CRT_WRITE; /* Clear opg->ops_transfer_pinned before VM lock is released. */ opg->ops_transfer_pinned = 0; @@ -1372,6 +1364,7 @@ static int osc_completion(const struct lu_env *env, struct osc_async_page *oap, lu_ref_del(&page->cp_reference, "transfer", page); cl_page_completion(env, page, crt, rc); + cl_page_put(env, page); return 0; } diff --git a/drivers/staging/lustre/lustre/osc/osc_cl_internal.h b/drivers/staging/lustre/lustre/osc/osc_cl_internal.h index 513397a..e525110 100644 --- a/drivers/staging/lustre/lustre/osc/osc_cl_internal.h +++ b/drivers/staging/lustre/lustre/osc/osc_cl_internal.h @@ -86,13 +86,6 @@ struct osc_io { }; /** - * State of transfer for osc. - */ -struct osc_req { - struct cl_req_slice or_cl; -}; - -/** * State maintained by osc layer for the duration of a system call. */ struct osc_session { @@ -115,6 +108,7 @@ struct osc_thread_info { pgoff_t oti_next_index; pgoff_t oti_fn_index; /* first non-overlapped index */ struct cl_sync_io oti_anchor; + struct cl_req_attr oti_req_attr; }; struct osc_object { @@ -362,7 +356,6 @@ struct osc_page { extern struct kmem_cache *osc_object_kmem; extern struct kmem_cache *osc_thread_kmem; extern struct kmem_cache *osc_session_kmem; -extern struct kmem_cache *osc_req_kmem; extern struct kmem_cache *osc_extent_kmem; extern struct lu_device_type osc_device_type; @@ -376,8 +369,6 @@ int osc_lock_init(const struct lu_env *env, const struct cl_io *io); int osc_io_init(const struct lu_env *env, struct cl_object *obj, struct cl_io *io); -int osc_req_init(const struct lu_env *env, struct cl_device *dev, - struct cl_req *req); struct lu_object *osc_object_alloc(const struct lu_env *env, const struct lu_object_header *hdr, struct lu_device *dev); @@ -534,6 +525,16 @@ static inline pgoff_t osc_index(struct osc_page *opg) return (struct osc_page *)container_of(oap, struct osc_page, ops_oap); } +static inline struct osc_page * +osc_cl_page_osc(struct cl_page *page, struct osc_object *osc) +{ + const struct cl_page_slice *slice; + + LASSERT(osc); + slice = cl_object_page_slice(&osc->oo_cl, page); + return cl2osc_page(slice); +} + static inline struct osc_lock *cl2osc_lock(const struct cl_lock_slice *slice) { LINVRNT(osc_is_object(&slice->cls_obj->co_lu)); diff --git a/drivers/staging/lustre/lustre/osc/osc_dev.c b/drivers/staging/lustre/lustre/osc/osc_dev.c index 83d30c1..89f288b 100644 --- a/drivers/staging/lustre/lustre/osc/osc_dev.c +++ b/drivers/staging/lustre/lustre/osc/osc_dev.c @@ -29,7 +29,7 @@ * This file is part of Lustre, http://www.lustre.org/ * Lustre is a trademark of Sun Microsystems, Inc. * - * Implementation of cl_device, cl_req for OSC layer. + * Implementation of cl_device, for OSC layer. * * Author: Nikita Danilov */ @@ -49,7 +49,6 @@ struct kmem_cache *osc_object_kmem; struct kmem_cache *osc_thread_kmem; struct kmem_cache *osc_session_kmem; -struct kmem_cache *osc_req_kmem; struct kmem_cache *osc_extent_kmem; struct kmem_cache *osc_quota_kmem; @@ -75,11 +74,6 @@ struct lu_kmem_descr osc_caches[] = { .ckd_size = sizeof(struct osc_session) }, { - .ckd_cache = &osc_req_kmem, - .ckd_name = "osc_req_kmem", - .ckd_size = sizeof(struct osc_req) - }, - { .ckd_cache = &osc_extent_kmem, .ckd_name = "osc_extent_kmem", .ckd_size = sizeof(struct osc_extent) @@ -178,10 +172,6 @@ static int osc_cl_process_config(const struct lu_env *env, .ldo_recovery_complete = NULL }; -static const struct cl_device_operations osc_cl_ops = { - .cdo_req_init = osc_req_init -}; - static int osc_device_init(const struct lu_env *env, struct lu_device *d, const char *name, struct lu_device *next) { @@ -220,7 +210,6 @@ static int osc_device_init(const struct lu_env *env, struct lu_device *d, cl_device_init(&od->od_cl, t); d = osc2lu_dev(od); d->ld_ops = &osc_lu_ops; - od->od_cl.cd_ops = &osc_cl_ops; /* Setup OSC OBD */ obd = class_name2obd(lustre_cfg_string(cfg, 0)); diff --git a/drivers/staging/lustre/lustre/osc/osc_io.c b/drivers/staging/lustre/lustre/osc/osc_io.c index e0f81a4..439ba64 100644 --- a/drivers/staging/lustre/lustre/osc/osc_io.c +++ b/drivers/staging/lustre/lustre/osc/osc_io.c @@ -49,12 +49,6 @@ * */ -static struct osc_req *cl2osc_req(const struct cl_req_slice *slice) -{ - LINVRNT(slice->crs_dev->cd_lu_dev.ld_type == &osc_device_type); - return container_of0(slice, struct osc_req, or_cl); -} - static struct osc_io *cl2osc_io(const struct lu_env *env, const struct cl_io_slice *slice) { @@ -64,20 +58,6 @@ return oio; } -static struct osc_page *osc_cl_page_osc(struct cl_page *page, - struct osc_object *osc) -{ - const struct cl_page_slice *slice; - - if (osc) - slice = cl_object_page_slice(&osc->oo_cl, page); - else - slice = cl_page_at(page, &osc_device_type); - LASSERT(slice); - - return cl2osc_page(slice); -} - /***************************************************************************** * * io operations. @@ -894,103 +874,6 @@ static void osc_io_end(const struct lu_env *env, * */ -static int osc_req_prep(const struct lu_env *env, - const struct cl_req_slice *slice) -{ - return 0; -} - -static void osc_req_completion(const struct lu_env *env, - const struct cl_req_slice *slice, int ioret) -{ - struct osc_req *or; - - or = cl2osc_req(slice); - kmem_cache_free(osc_req_kmem, or); -} - -/** - * Implementation of struct cl_req_operations::cro_attr_set() for osc - * layer. osc is responsible for struct obdo::o_id and struct obdo::o_seq - * fields. - */ -static void osc_req_attr_set(const struct lu_env *env, - const struct cl_req_slice *slice, - const struct cl_object *obj, - struct cl_req_attr *attr, u64 flags) -{ - struct lov_oinfo *oinfo; - struct cl_req *clerq; - struct cl_page *apage; /* _some_ page in @clerq */ - struct ldlm_lock *lock; /* _some_ lock protecting @apage */ - struct osc_page *opg; - struct obdo *oa; - struct ost_lvb *lvb; - - oinfo = cl2osc(obj)->oo_oinfo; - lvb = &oinfo->loi_lvb; - oa = attr->cra_oa; - - if ((flags & OBD_MD_FLMTIME) != 0) { - oa->o_mtime = lvb->lvb_mtime; - oa->o_valid |= OBD_MD_FLMTIME; - } - if ((flags & OBD_MD_FLATIME) != 0) { - oa->o_atime = lvb->lvb_atime; - oa->o_valid |= OBD_MD_FLATIME; - } - if ((flags & OBD_MD_FLCTIME) != 0) { - oa->o_ctime = lvb->lvb_ctime; - oa->o_valid |= OBD_MD_FLCTIME; - } - if (flags & OBD_MD_FLGROUP) { - ostid_set_seq(&oa->o_oi, ostid_seq(&oinfo->loi_oi)); - oa->o_valid |= OBD_MD_FLGROUP; - } - if (flags & OBD_MD_FLID) { - ostid_set_id(&oa->o_oi, ostid_id(&oinfo->loi_oi)); - oa->o_valid |= OBD_MD_FLID; - } - if (flags & OBD_MD_FLHANDLE) { - clerq = slice->crs_req; - LASSERT(!list_empty(&clerq->crq_pages)); - apage = container_of(clerq->crq_pages.next, - struct cl_page, cp_flight); - opg = osc_cl_page_osc(apage, NULL); - lock = osc_dlmlock_at_pgoff(env, cl2osc(obj), osc_index(opg), - OSC_DAP_FL_TEST_LOCK | OSC_DAP_FL_CANCELING); - if (!lock && !opg->ops_srvlock) { - struct ldlm_resource *res; - struct ldlm_res_id *resname; - - CL_PAGE_DEBUG(D_ERROR, env, apage, "uncovered page!\n"); - - resname = &osc_env_info(env)->oti_resname; - ostid_build_res_name(&oinfo->loi_oi, resname); - res = ldlm_resource_get( - osc_export(cl2osc(obj))->exp_obd->obd_namespace, - NULL, resname, LDLM_EXTENT, 0); - ldlm_resource_dump(D_ERROR, res); - - dump_stack(); - LBUG(); - } - - /* check for lockless io. */ - if (lock) { - oa->o_handle = lock->l_remote_handle; - oa->o_valid |= OBD_MD_FLHANDLE; - LDLM_LOCK_PUT(lock); - } - } -} - -static const struct cl_req_operations osc_req_ops = { - .cro_prep = osc_req_prep, - .cro_attr_set = osc_req_attr_set, - .cro_completion = osc_req_completion -}; - int osc_io_init(const struct lu_env *env, struct cl_object *obj, struct cl_io *io) { @@ -1001,20 +884,4 @@ int osc_io_init(const struct lu_env *env, return 0; } -int osc_req_init(const struct lu_env *env, struct cl_device *dev, - struct cl_req *req) -{ - struct osc_req *or; - int result; - - or = kmem_cache_zalloc(osc_req_kmem, GFP_NOFS); - if (or) { - cl_req_slice_add(req, &or->or_cl, dev, &osc_req_ops); - result = 0; - } else { - result = -ENOMEM; - } - return result; -} - /** @} osc */ diff --git a/drivers/staging/lustre/lustre/osc/osc_object.c b/drivers/staging/lustre/lustre/osc/osc_object.c index a6dafbf..ca1f8cd 100644 --- a/drivers/staging/lustre/lustre/osc/osc_object.c +++ b/drivers/staging/lustre/lustre/osc/osc_object.c @@ -335,6 +335,76 @@ int osc_object_is_contended(struct osc_object *obj) return 1; } +/** + * Implementation of struct cl_object_operations::coo_req_attr_set() for osc + * layer. osc is responsible for struct obdo::o_id and struct obdo::o_seq + * fields. + */ +static void osc_req_attr_set(const struct lu_env *env, struct cl_object *obj, + struct cl_req_attr *attr) +{ + u64 flags = attr->cra_flags; + struct lov_oinfo *oinfo; + struct ost_lvb *lvb; + struct obdo *oa; + + oinfo = cl2osc(obj)->oo_oinfo; + lvb = &oinfo->loi_lvb; + oa = attr->cra_oa; + + if (flags & OBD_MD_FLMTIME) { + oa->o_mtime = lvb->lvb_mtime; + oa->o_valid |= OBD_MD_FLMTIME; + } + if (flags & OBD_MD_FLATIME) { + oa->o_atime = lvb->lvb_atime; + oa->o_valid |= OBD_MD_FLATIME; + } + if (flags & OBD_MD_FLCTIME) { + oa->o_ctime = lvb->lvb_ctime; + oa->o_valid |= OBD_MD_FLCTIME; + } + if (flags & OBD_MD_FLGROUP) { + ostid_set_seq(&oa->o_oi, ostid_seq(&oinfo->loi_oi)); + oa->o_valid |= OBD_MD_FLGROUP; + } + if (flags & OBD_MD_FLID) { + ostid_set_id(&oa->o_oi, ostid_id(&oinfo->loi_oi)); + oa->o_valid |= OBD_MD_FLID; + } + if (flags & OBD_MD_FLHANDLE) { + struct ldlm_lock *lock; + struct osc_page *opg; + + opg = osc_cl_page_osc(attr->cra_page, cl2osc(obj)); + lock = osc_dlmlock_at_pgoff(env, cl2osc(obj), osc_index(opg), + OSC_DAP_FL_TEST_LOCK | OSC_DAP_FL_CANCELING); + if (!lock && !opg->ops_srvlock) { + struct ldlm_resource *res; + struct ldlm_res_id *resname; + + CL_PAGE_DEBUG(D_ERROR, env, attr->cra_page, + "uncovered page!\n"); + + resname = &osc_env_info(env)->oti_resname; + ostid_build_res_name(&oinfo->loi_oi, resname); + res = ldlm_resource_get( + osc_export(cl2osc(obj))->exp_obd->obd_namespace, + NULL, resname, LDLM_EXTENT, 0); + ldlm_resource_dump(D_ERROR, res); + + LBUG(); + } + + /* check for lockless io. */ + if (lock) { + oa->o_handle = lock->l_remote_handle; + oa->o_valid |= OBD_MD_FLHANDLE; + LDLM_LOCK_PUT(lock); + } + } +} + static const struct cl_object_operations osc_ops = { .coo_page_init = osc_page_init, .coo_lock_init = osc_lock_init, @@ -343,7 +413,8 @@ int osc_object_is_contended(struct osc_object *obj) .coo_attr_update = osc_attr_update, .coo_glimpse = osc_object_glimpse, .coo_prune = osc_object_prune, - .coo_fiemap = osc_object_fiemap, + .coo_fiemap = osc_object_fiemap, + .coo_req_attr_set = osc_req_attr_set }; static const struct lu_object_operations osc_lu_obj_ops = { diff --git a/drivers/staging/lustre/lustre/osc/osc_request.c b/drivers/staging/lustre/lustre/osc/osc_request.c index 3daca63..9b0b94b 100644 --- a/drivers/staging/lustre/lustre/osc/osc_request.c +++ b/drivers/staging/lustre/lustre/osc/osc_request.c @@ -68,7 +68,6 @@ struct osc_brw_async_args { struct client_obd *aa_cli; struct list_head aa_oaps; struct list_head aa_exts; - struct cl_req *aa_clerq; }; struct osc_async_args { @@ -1600,8 +1599,6 @@ static int brw_interpret(const struct lu_env *env, LASSERT(list_empty(&aa->aa_exts)); LASSERT(list_empty(&aa->aa_oaps)); - cl_req_completion(env, aa->aa_clerq, rc < 0 ? rc : - req->rq_bulk->bd_nob_transferred); osc_release_ppga(aa->aa_ppga, aa->aa_page_count); ptlrpc_lprocfs_brw(req, req->rq_bulk->bd_nob_transferred); @@ -1654,9 +1651,7 @@ int osc_build_rpc(const struct lu_env *env, struct client_obd *cli, struct osc_brw_async_args *aa = NULL; struct obdo *oa = NULL; struct osc_async_page *oap; - struct osc_async_page *tmp; - struct cl_req *clerq = NULL; - enum cl_req_type crt = (cmd & OBD_BRW_WRITE) ? CRT_WRITE : CRT_READ; + struct osc_object *obj = NULL; struct cl_req_attr *crattr = NULL; u64 starting_offset = OBD_OBJECT_EOF; u64 ending_offset = 0; @@ -1664,6 +1659,7 @@ int osc_build_rpc(const struct lu_env *env, struct client_obd *cli, int mem_tight = 0; int page_count = 0; bool soft_sync = false; + bool interrupted = false; int i; int rc; struct ost_body *body; @@ -1675,32 +1671,15 @@ int osc_build_rpc(const struct lu_env *env, struct client_obd *cli, list_for_each_entry(ext, ext_list, oe_link) { LASSERT(ext->oe_state == OES_RPC); mem_tight |= ext->oe_memalloc; - list_for_each_entry(oap, &ext->oe_pages, oap_pending_item) { - ++page_count; - list_add_tail(&oap->oap_rpc_item, &rpc_list); - if (starting_offset > oap->oap_obj_off) - starting_offset = oap->oap_obj_off; - else - LASSERT(oap->oap_page_off == 0); - if (ending_offset < oap->oap_obj_off + oap->oap_count) - ending_offset = oap->oap_obj_off + - oap->oap_count; - else - LASSERT(oap->oap_page_off + oap->oap_count == - PAGE_SIZE); - } + page_count += ext->oe_nr_pages; + if (!obj) + obj = ext->oe_obj; } soft_sync = osc_over_unstable_soft_limit(cli); if (mem_tight) mpflag = cfs_memory_pressure_get_and_set(); - crattr = kzalloc(sizeof(*crattr), GFP_NOFS); - if (!crattr) { - rc = -ENOMEM; - goto out; - } - pga = kcalloc(page_count, sizeof(*pga), GFP_NOFS); if (!pga) { rc = -ENOMEM; @@ -1714,40 +1693,43 @@ int osc_build_rpc(const struct lu_env *env, struct client_obd *cli, } i = 0; - list_for_each_entry(oap, &rpc_list, oap_rpc_item) { - struct cl_page *page = oap2cl_page(oap); - - if (!clerq) { - clerq = cl_req_alloc(env, page, crt, - 1 /* only 1-object rpcs for now */); - if (IS_ERR(clerq)) { - rc = PTR_ERR(clerq); - goto out; - } + list_for_each_entry(ext, ext_list, oe_link) { + list_for_each_entry(oap, &ext->oe_pages, oap_pending_item) { + if (mem_tight) + oap->oap_brw_flags |= OBD_BRW_MEMALLOC; + if (soft_sync) + oap->oap_brw_flags |= OBD_BRW_SOFT_SYNC; + pga[i] = &oap->oap_brw_page; + pga[i]->off = oap->oap_obj_off + oap->oap_page_off; + i++; + + list_add_tail(&oap->oap_rpc_item, &rpc_list); + if (starting_offset == OBD_OBJECT_EOF || + starting_offset > oap->oap_obj_off) + starting_offset = oap->oap_obj_off; + else + LASSERT(!oap->oap_page_off); + if (ending_offset < oap->oap_obj_off + oap->oap_count) + ending_offset = oap->oap_obj_off + + oap->oap_count; + else + LASSERT(oap->oap_page_off + oap->oap_count == + PAGE_SIZE); + if (oap->oap_interrupted) + interrupted = true; } - if (mem_tight) - oap->oap_brw_flags |= OBD_BRW_MEMALLOC; - if (soft_sync) - oap->oap_brw_flags |= OBD_BRW_SOFT_SYNC; - pga[i] = &oap->oap_brw_page; - pga[i]->off = oap->oap_obj_off + oap->oap_page_off; - CDEBUG(0, "put page %p index %lu oap %p flg %x to pga\n", - pga[i]->pg, oap->oap_page->index, oap, - pga[i]->flag); - i++; - cl_req_page_add(env, clerq, page); } - /* always get the data for the obdo for the rpc */ - LASSERT(clerq); - crattr->cra_oa = oa; - cl_req_attr_set(env, clerq, crattr, ~0ULL); + /* first page in the list */ + oap = list_entry(rpc_list.next, typeof(*oap), oap_rpc_item); - rc = cl_req_prep(env, clerq); - if (rc != 0) { - CERROR("cl_req_prep failed: %d\n", rc); - goto out; - } + crattr = &osc_env_info(env)->oti_req_attr; + memset(crattr, 0, sizeof(*crattr)); + crattr->cra_type = (cmd & OBD_BRW_WRITE) ? CRT_WRITE : CRT_READ; + crattr->cra_flags = ~0ULL; + crattr->cra_page = oap2cl_page(oap); + crattr->cra_oa = oa; + cl_req_attr_set(env, osc2cl(obj), crattr); sort_brw_pages(pga, page_count); rc = osc_brw_prep_request(cmd, cli, oa, page_count, pga, &req, 1, 0); @@ -1759,8 +1741,10 @@ int osc_build_rpc(const struct lu_env *env, struct client_obd *cli, req->rq_commit_cb = brw_commit; req->rq_interpret_reply = brw_interpret; - if (mem_tight != 0) - req->rq_memalloc = 1; + req->rq_memalloc = mem_tight != 0; + oap->oap_request = ptlrpc_request_addref(req); + if (interrupted && !req->rq_intr) + ptlrpc_mark_interrupted(req); /* Need to update the timestamps after the request is built in case * we race with setattr (locally or in queue at OST). If OST gets @@ -1770,9 +1754,8 @@ int osc_build_rpc(const struct lu_env *env, struct client_obd *cli, */ body = req_capsule_client_get(&req->rq_pill, &RMF_OST_BODY); crattr->cra_oa = &body->oa; - cl_req_attr_set(env, clerq, crattr, - OBD_MD_FLMTIME | OBD_MD_FLCTIME | OBD_MD_FLATIME); - + crattr->cra_flags = OBD_MD_FLMTIME | OBD_MD_FLCTIME | OBD_MD_FLATIME; + cl_req_attr_set(env, osc2cl(obj), crattr); lustre_msg_set_jobid(req->rq_reqmsg, crattr->cra_jobid); CLASSERT(sizeof(*aa) <= sizeof(req->rq_async_args)); @@ -1781,24 +1764,6 @@ int osc_build_rpc(const struct lu_env *env, struct client_obd *cli, list_splice_init(&rpc_list, &aa->aa_oaps); INIT_LIST_HEAD(&aa->aa_exts); list_splice_init(ext_list, &aa->aa_exts); - aa->aa_clerq = clerq; - - /* queued sync pages can be torn down while the pages - * were between the pending list and the rpc - */ - tmp = NULL; - list_for_each_entry(oap, &aa->aa_oaps, oap_rpc_item) { - /* only one oap gets a request reference */ - if (!tmp) - tmp = oap; - if (oap->oap_interrupted && !req->rq_intr) { - CDEBUG(D_INODE, "oap %p in req %p interrupted\n", - oap, req); - ptlrpc_mark_interrupted(req); - } - } - if (tmp) - tmp->oap_request = ptlrpc_request_addref(req); spin_lock(&cli->cl_loi_list_lock); starting_offset >>= PAGE_SHIFT; @@ -1829,8 +1794,6 @@ int osc_build_rpc(const struct lu_env *env, struct client_obd *cli, if (mem_tight != 0) cfs_memory_pressure_restore(mpflag); - kfree(crattr); - if (rc != 0) { LASSERT(!req); @@ -1846,8 +1809,6 @@ int osc_build_rpc(const struct lu_env *env, struct client_obd *cli, list_del_init(&ext->oe_link); osc_extent_finish(env, ext, 0, rc); } - if (clerq && !IS_ERR(clerq)) - cl_req_completion(env, clerq, rc); } return rc; } -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:30:32 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:30:32 -0500 Subject: [lustre-devel] [PATCH 02/35] staging: lustre: obd: rename obd_unpackmd() to md_unpackmd() In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <1478799065-24841-3-git-send-email-jsimmons@infradead.org> From: John L. Hammond obd_unpackmd() is only implemented by LMV so move it from OBD operations to OBD MD operations and update the prototype to reflex the actual usage. Remove the unused function obd_free_memmd(). Signed-off-by: John L. Hammond Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-5814 Signed-off-by: Jinshan Xiong Reviewed-on: http://review.whamcloud.com/13737 Reviewed-by: James Simmons Reviewed-by: Andreas Dilger Signed-off-by: James Simmons --- drivers/staging/lustre/lustre/include/lustre_lmv.h | 13 +----- drivers/staging/lustre/lustre/include/obd.h | 5 +- drivers/staging/lustre/lustre/include/obd_class.h | 51 +++++++------------- drivers/staging/lustre/lustre/lmv/lmv_internal.h | 3 - drivers/staging/lustre/lustre/lmv/lmv_obd.c | 25 ++------- drivers/staging/lustre/lustre/mdc/mdc_request.c | 13 ++--- 6 files changed, 33 insertions(+), 77 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre_lmv.h b/drivers/staging/lustre/lustre/include/lustre_lmv.h index d7f7afa..5aa3645 100644 --- a/drivers/staging/lustre/lustre/include/lustre_lmv.h +++ b/drivers/staging/lustre/lustre/include/lustre_lmv.h @@ -76,18 +76,7 @@ struct lmv_stripe_md { union lmv_mds_md; -int lmv_unpack_md(struct obd_export *exp, struct lmv_stripe_md **lsmp, - const union lmv_mds_md *lmm, int stripe_count); - -static inline int lmv_alloc_memmd(struct lmv_stripe_md **lsmp, int stripe_count) -{ - return lmv_unpack_md(NULL, lsmp, NULL, stripe_count); -} - -static inline void lmv_free_memmd(struct lmv_stripe_md *lsm) -{ - lmv_unpack_md(NULL, &lsm, NULL, 0); -} +void lmv_free_memmd(struct lmv_stripe_md *lsm); static inline void lmv1_le_to_cpu(struct lmv_mds_md_v1 *lmv_dst, const struct lmv_mds_md_v1 *lmv_src) diff --git a/drivers/staging/lustre/lustre/include/obd.h b/drivers/staging/lustre/lustre/include/obd.h index c8a6e23..5ebcfb8 100644 --- a/drivers/staging/lustre/lustre/include/obd.h +++ b/drivers/staging/lustre/lustre/include/obd.h @@ -808,9 +808,6 @@ struct obd_ops { struct obd_statfs *osfs, __u64 max_age, __u32 flags); int (*statfs_async)(struct obd_export *exp, struct obd_info *oinfo, __u64 max_age, struct ptlrpc_request_set *set); - int (*unpackmd)(struct obd_export *exp, - struct lov_stripe_md **mem_tgt, - struct lov_mds_md *disk_src, int disk_len); int (*create)(const struct lu_env *env, struct obd_export *exp, struct obdo *oa); int (*destroy)(const struct lu_env *env, struct obd_export *exp, @@ -978,6 +975,8 @@ struct md_ops { int (*revalidate_lock)(struct obd_export *, struct lookup_intent *, struct lu_fid *, __u64 *bits); + int (*unpackmd)(struct obd_export *exp, struct lmv_stripe_md **plsm, + const union lmv_mds_md *lmv, size_t lmv_size); /* * NOTE: If adding ops, add another LPROCFS_MD_OP_INIT() line to * lprocfs_alloc_md_stats() in obdclass/lprocfs_status.c. Also, add a diff --git a/drivers/staging/lustre/lustre/include/obd_class.h b/drivers/staging/lustre/lustre/include/obd_class.h index 70b355e..f82f37b 100644 --- a/drivers/staging/lustre/lustre/include/obd_class.h +++ b/drivers/staging/lustre/lustre/include/obd_class.h @@ -612,39 +612,6 @@ static inline void obd_cleanup_client_import(struct obd_device *obd) return rc; } -/* Unpack an MD struct from disk to in-memory format. - * Returns +ve size of unpacked MD (0 for free), or -ve error. - * - * If @mem_tgt == NULL, MD size is returned (max size if @disk_src == NULL). - * If @*mem_tgt != NULL and @disk_src == NULL, @*mem_tgt will be freed. - * If @*mem_tgt == NULL, it will be allocated - */ -static inline int obd_unpackmd(struct obd_export *exp, - struct lov_stripe_md **mem_tgt, - struct lov_mds_md *disk_src, - int disk_len) -{ - int rc; - - EXP_CHECK_DT_OP(exp, unpackmd); - EXP_COUNTER_INCREMENT(exp, unpackmd); - - rc = OBP(exp->exp_obd, unpackmd)(exp, mem_tgt, disk_src, disk_len); - return rc; -} - -static inline int obd_free_memmd(struct obd_export *exp, - struct lov_stripe_md **mem_tgt) -{ - int rc; - - LASSERT(mem_tgt); - LASSERT(*mem_tgt); - rc = obd_unpackmd(exp, mem_tgt, NULL, 0); - *mem_tgt = NULL; - return rc; -} - static inline int obd_create(const struct lu_env *env, struct obd_export *exp, struct obdo *obdo) { @@ -1510,6 +1477,24 @@ static inline int md_get_fid_from_lsm(struct obd_export *exp, return rc; } +/* Unpack an MD struct from disk to in-memory format. + * Returns +ve size of unpacked MD (0 for free), or -ve error. + * + * If *plsm != NULL and lmm == NULL then *lsm will be freed. + * If *plsm == NULL then it will be allocated. + */ +static inline int md_unpackmd(struct obd_export *exp, + struct lmv_stripe_md **plsm, + const union lmv_mds_md *lmm, size_t lmm_size) +{ + int rc; + + EXP_CHECK_MD_OP(exp, unpackmd); + EXP_MD_COUNTER_INCREMENT(exp, unpackmd); + rc = MDP(exp->exp_obd, unpackmd)(exp, plsm, lmm, lmm_size); + return rc; +} + /* OBD Metadata Support */ int obd_init_caches(void); diff --git a/drivers/staging/lustre/lustre/lmv/lmv_internal.h b/drivers/staging/lustre/lustre/lmv/lmv_internal.h index 52b0374..12731a1 100644 --- a/drivers/staging/lustre/lustre/lmv/lmv_internal.h +++ b/drivers/staging/lustre/lustre/lmv/lmv_internal.h @@ -54,9 +54,6 @@ int lmv_intent_lock(struct obd_export *exp, struct md_op_data *op_data, int lmv_fid_alloc(const struct lu_env *env, struct obd_export *exp, struct lu_fid *fid, struct md_op_data *op_data); -int lmv_unpack_md(struct obd_export *exp, struct lmv_stripe_md **lsmp, - const union lmv_mds_md *lmm, int stripe_count); - int lmv_revalidate_slaves(struct obd_export *exp, const struct lmv_stripe_md *lsm, ldlm_blocking_callback cb_blocking, diff --git a/drivers/staging/lustre/lustre/lmv/lmv_obd.c b/drivers/staging/lustre/lustre/lmv/lmv_obd.c index 12e8b1e..ca78aff 100644 --- a/drivers/staging/lustre/lustre/lmv/lmv_obd.c +++ b/drivers/staging/lustre/lustre/lmv/lmv_obd.c @@ -2816,8 +2816,8 @@ static int lmv_unpack_md_v1(struct obd_export *exp, struct lmv_stripe_md *lsm, return rc; } -int lmv_unpack_md(struct obd_export *exp, struct lmv_stripe_md **lsmp, - const union lmv_mds_md *lmm, int stripe_count) +static int lmv_unpackmd(struct obd_export *exp, struct lmv_stripe_md **lsmp, + const union lmv_mds_md *lmm, size_t lmm_size) { struct lmv_stripe_md *lsm; bool allocated = false; @@ -2846,17 +2846,6 @@ int lmv_unpack_md(struct obd_export *exp, struct lmv_stripe_md **lsmp, return 0; } - /* Alloc memmd */ - if (!lsm && !lmm) { - lsm_size = lmv_stripe_md_size(stripe_count); - lsm = libcfs_kvzalloc(lsm_size, GFP_NOFS); - if (!lsm) - return -ENOMEM; - lsm->lsm_md_stripe_count = stripe_count; - *lsmp = lsm; - return 0; - } - if (le32_to_cpu(lmm->lmv_magic) == LMV_MAGIC_STRIPE) return -EPERM; @@ -2904,14 +2893,12 @@ int lmv_unpack_md(struct obd_export *exp, struct lmv_stripe_md **lsmp, } return lsm_size; } -EXPORT_SYMBOL(lmv_unpack_md); -static int lmv_unpackmd(struct obd_export *exp, struct lov_stripe_md **lsmp, - struct lov_mds_md *lmm, int disk_len) +void lmv_free_memmd(struct lmv_stripe_md *lsm) { - return lmv_unpack_md(exp, (struct lmv_stripe_md **)lsmp, - (union lmv_mds_md *)lmm, disk_len); + lmv_unpackmd(NULL, &lsm, NULL, 0); } +EXPORT_SYMBOL(lmv_free_memmd); static int lmv_cancel_unused(struct obd_export *exp, const struct lu_fid *fid, ldlm_policy_data_t *policy, enum ldlm_mode mode, @@ -3216,7 +3203,6 @@ static int lmv_merge_attr(struct obd_export *exp, .statfs = lmv_statfs, .get_info = lmv_get_info, .set_info_async = lmv_set_info_async, - .unpackmd = lmv_unpackmd, .notify = lmv_notify, .get_uuid = lmv_get_uuid, .iocontrol = lmv_iocontrol, @@ -3252,6 +3238,7 @@ static int lmv_merge_attr(struct obd_export *exp, .intent_getattr_async = lmv_intent_getattr_async, .revalidate_lock = lmv_revalidate_lock, .get_fid_from_lsm = lmv_get_fid_from_lsm, + .unpackmd = lmv_unpackmd, }; static int __init lmv_init(void) diff --git a/drivers/staging/lustre/lustre/mdc/mdc_request.c b/drivers/staging/lustre/lustre/mdc/mdc_request.c index 215b156..8bf0c5b 100644 --- a/drivers/staging/lustre/lustre/mdc/mdc_request.c +++ b/drivers/staging/lustre/lustre/mdc/mdc_request.c @@ -443,8 +443,8 @@ static int mdc_get_lustre_md(struct obd_export *exp, goto out; } } else if (md->body->mbo_valid & OBD_MD_FLDIREA) { - int lmvsize; - struct lov_mds_md *lmv; + const union lmv_mds_md *lmv; + size_t lmv_size; if (!S_ISDIR(md->body->mbo_mode)) { CDEBUG(D_INFO, @@ -453,22 +453,21 @@ static int mdc_get_lustre_md(struct obd_export *exp, goto out; } - if (md->body->mbo_eadatasize == 0) { + lmv_size = md->body->mbo_eadatasize; + if (!lmv_size) { CDEBUG(D_INFO, "OBD_MD_FLDIREA is set, but eadatasize 0\n"); return -EPROTO; } if (md->body->mbo_valid & OBD_MD_MEA) { - lmvsize = md->body->mbo_eadatasize; lmv = req_capsule_server_sized_get(pill, &RMF_MDT_MD, - lmvsize); + lmv_size); if (!lmv) { rc = -EPROTO; goto out; } - rc = obd_unpackmd(md_exp, (void *)&md->lmv, lmv, - lmvsize); + rc = md_unpackmd(md_exp, &md->lmv, lmv, lmv_size); if (rc < 0) goto out; -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:30:33 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:30:33 -0500 Subject: [lustre-devel] [PATCH 03/35] staging: lustre: ptlrpc: mbits is sent within ptlrpc_body In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <1478799065-24841-4-git-send-email-jsimmons@infradead.org> From: Liang Zhen ptlrpc is using rq_xid as matchbits of bulk data, which means it has to change rq_xid for bulk resend to avoid several bulk data landing into the same buffer from different resends. This patch uses one of reserved __u64 of ptlrpc_body to transfer mbits to peer, matchbits is now separated from xid. With this change, ptlrpc can keep rq_xid unchanged on resend, it only updates matchbits for bulk data. This protocol change is only applied if both sides of connection have OBD_CONNECT_BULK_MBITS, otherwise, ptlrpc still uses old approach and update xid while resending bulk. Signed-off-by: Liang Zhen Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3534 Reviewed-on: http://review.whamcloud.com/15421 Reviewed-by: Andreas Dilger Reviewed-by: Alex Zhuravlev Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- .../lustre/lustre/include/lustre/lustre_idl.h | 12 +++- drivers/staging/lustre/lustre/include/lustre_net.h | 5 +- drivers/staging/lustre/lustre/llite/llite_lib.c | 7 ++- .../lustre/lustre/obdclass/lprocfs_status.c | 1 + drivers/staging/lustre/lustre/obdclass/obd_mount.c | 2 +- drivers/staging/lustre/lustre/ptlrpc/client.c | 72 ++++++++++++------- drivers/staging/lustre/lustre/ptlrpc/niobuf.c | 63 +++++++++-------- .../staging/lustre/lustre/ptlrpc/pack_generic.c | 20 +++++- .../staging/lustre/lustre/ptlrpc/ptlrpc_internal.h | 1 + drivers/staging/lustre/lustre/ptlrpc/wiretest.c | 40 +++++++++-- 10 files changed, 151 insertions(+), 72 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h index 21abab4..33deee6 100644 --- a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h +++ b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h @@ -1006,8 +1006,11 @@ struct ptlrpc_body_v3 { __u64 pb_slv; /* VBR: pre-versions */ __u64 pb_pre_versions[PTLRPC_NUM_VERSIONS]; + __u64 pb_mbits; /**< match bits for bulk request */ /* padding for future needs */ - __u64 pb_padding[4]; + __u64 pb_padding64_0; + __u64 pb_padding64_1; + __u64 pb_padding64_2; char pb_jobid[LUSTRE_JOBID_SIZE]; }; @@ -1036,8 +1039,11 @@ struct ptlrpc_body_v2 { __u64 pb_slv; /* VBR: pre-versions */ __u64 pb_pre_versions[PTLRPC_NUM_VERSIONS]; + __u64 pb_mbits; /**< unused in V2 */ /* padding for future needs */ - __u64 pb_padding[4]; + __u64 pb_padding64_0; + __u64 pb_padding64_1; + __u64 pb_padding64_2; }; void lustre_swab_ptlrpc_body(struct ptlrpc_body *pb); @@ -1182,6 +1188,8 @@ struct ptlrpc_body_v2 { * RPCs in parallel */ #define OBD_CONNECT_DIR_STRIPE 0x400000000000000ULL/* striped DNE dir */ +/** bulk matchbits is sent within ptlrpc_body */ +#define OBD_CONNECT_BULK_MBITS 0x2000000000000000ULL /* XXX README XXX: * Please DO NOT add flag values here before first ensuring that this same diff --git a/drivers/staging/lustre/lustre/include/lustre_net.h b/drivers/staging/lustre/lustre/include/lustre_net.h index a14f1a4..7f549a3 100644 --- a/drivers/staging/lustre/lustre/include/lustre_net.h +++ b/drivers/staging/lustre/lustre/include/lustre_net.h @@ -796,6 +796,8 @@ struct ptlrpc_request { __u64 rq_transno; /** xid */ __u64 rq_xid; + /** bulk match bits */ + u64 rq_mbits; /** * List item to for replay list. Not yet committed requests get linked * there. @@ -1208,7 +1210,7 @@ struct ptlrpc_bulk_desc { int bd_nob; /* # bytes covered */ int bd_nob_transferred; /* # bytes GOT/PUT */ - __u64 bd_last_xid; + u64 bd_last_mbits; struct ptlrpc_cb_id bd_cbid; /* network callback info */ lnet_nid_t bd_sender; /* stash event::sender */ @@ -2080,6 +2082,7 @@ void lustre_msg_set_handle(struct lustre_msg *msg, void lustre_msg_set_service_time(struct lustre_msg *msg, __u32 service_time); void lustre_msg_set_jobid(struct lustre_msg *msg, char *jobid); void lustre_msg_set_cksum(struct lustre_msg *msg, __u32 cksum); +void lustre_msg_set_mbits(struct lustre_msg *msg, u64 mbits); static inline void lustre_shrink_reply(struct ptlrpc_request *req, int segment, diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c index c52653e..c1b646c 100644 --- a/drivers/staging/lustre/lustre/llite/llite_lib.c +++ b/drivers/staging/lustre/lustre/llite/llite_lib.c @@ -191,7 +191,8 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt, OBD_CONNECT_FLOCK_DEAD | OBD_CONNECT_DISP_STRIPE | OBD_CONNECT_LFSCK | OBD_CONNECT_OPEN_BY_FID | - OBD_CONNECT_DIR_STRIPE; + OBD_CONNECT_DIR_STRIPE | + OBD_CONNECT_BULK_MBITS; if (sbi->ll_flags & LL_SBI_LRU_RESIZE) data->ocd_connect_flags |= OBD_CONNECT_LRU_RESIZE; @@ -352,7 +353,9 @@ static int client_common_fill_super(struct super_block *sb, char *md, char *dt, OBD_CONNECT_64BITHASH | OBD_CONNECT_MAXBYTES | OBD_CONNECT_EINPROGRESS | OBD_CONNECT_JOBSTATS | OBD_CONNECT_LVB_TYPE | - OBD_CONNECT_LAYOUTLOCK | OBD_CONNECT_PINGLESS; + OBD_CONNECT_LAYOUTLOCK | + OBD_CONNECT_PINGLESS | OBD_CONNECT_LFSCK | + OBD_CONNECT_BULK_MBITS; if (!OBD_FAIL_CHECK(OBD_FAIL_OSC_CONNECT_CKSUM)) { /* OBD_CONNECT_CKSUM should always be set, even if checksums are diff --git a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c index b520c96..8a2f02f 100644 --- a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c +++ b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c @@ -102,6 +102,7 @@ "unlink_close", "multi_mod_rpcs", "dir_stripe", + "bulk_mbits", "unknown", NULL }; diff --git a/drivers/staging/lustre/lustre/obdclass/obd_mount.c b/drivers/staging/lustre/lustre/obdclass/obd_mount.c index 59fbc29..1d13751 100644 --- a/drivers/staging/lustre/lustre/obdclass/obd_mount.c +++ b/drivers/staging/lustre/lustre/obdclass/obd_mount.c @@ -382,7 +382,7 @@ int lustre_start_mgc(struct super_block *sb) /* We connect to the MGS at setup, and don't disconnect until cleanup */ data->ocd_connect_flags = OBD_CONNECT_VERSION | OBD_CONNECT_AT | OBD_CONNECT_FULL20 | OBD_CONNECT_IMP_RECOV | - OBD_CONNECT_LVB_TYPE; + OBD_CONNECT_LVB_TYPE | OBD_CONNECT_BULK_MBITS; #if OBD_OCD_VERSION(3, 0, 53, 0) > LUSTRE_VERSION_CODE data->ocd_connect_flags |= OBD_CONNECT_MNE_SWAB; diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c index bb7ae4e..b940e24 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/client.c +++ b/drivers/staging/lustre/lustre/ptlrpc/client.c @@ -1304,13 +1304,6 @@ static int after_reply(struct ptlrpc_request *req) spin_unlock(&req->rq_lock); req->rq_nr_resend++; - /* allocate new xid to avoid reply reconstruction */ - if (!req->rq_bulk) { - /* new xid is already allocated for bulk in ptlrpc_check_set() */ - req->rq_xid = ptlrpc_next_xid(); - DEBUG_REQ(D_RPCTRACE, req, "Allocating new xid for resend on EINPROGRESS"); - } - /* Readjust the timeout for current conditions */ ptlrpc_at_set_req_timeout(req); /* @@ -1802,18 +1795,9 @@ int ptlrpc_check_set(const struct lu_env *env, struct ptlrpc_request_set *set) spin_lock(&req->rq_lock); req->rq_resend = 1; spin_unlock(&req->rq_lock); - if (req->rq_bulk) { - __u64 old_xid; - - if (!ptlrpc_unregister_bulk(req, 1)) - continue; - - /* ensure previous bulk fails */ - old_xid = req->rq_xid; - req->rq_xid = ptlrpc_next_xid(); - CDEBUG(D_HA, "resend bulk old x%llu new x%llu\n", - old_xid, req->rq_xid); - } + if (req->rq_bulk && + !ptlrpc_unregister_bulk(req, 1)) + continue; } /* * rq_wait_ctx is only touched by ptlrpcd, @@ -2664,14 +2648,6 @@ void ptlrpc_resend_req(struct ptlrpc_request *req) req->rq_resend = 1; req->rq_net_err = 0; req->rq_timedout = 0; - if (req->rq_bulk) { - __u64 old_xid = req->rq_xid; - - /* ensure previous bulk fails */ - req->rq_xid = ptlrpc_next_xid(); - CDEBUG(D_HA, "resend bulk old x%llu new x%llu\n", - old_xid, req->rq_xid); - } ptlrpc_client_wake_req(req); spin_unlock(&req->rq_lock); } @@ -3068,6 +3044,48 @@ __u64 ptlrpc_next_xid(void) } /** + * If request has a new allocated XID (new request or EINPROGRESS resend), + * use this XID as matchbits of bulk, otherwise allocate a new matchbits for + * request to ensure previous bulk fails and avoid problems with lost replies + * and therefore several transfers landing into the same buffer from different + * sending attempts. + */ +void ptlrpc_set_bulk_mbits(struct ptlrpc_request *req) +{ + struct ptlrpc_bulk_desc *bd = req->rq_bulk; + + LASSERT(bd); + + if (!req->rq_resend || req->rq_nr_resend) { + /* this request has a new xid, just use it as bulk matchbits */ + req->rq_mbits = req->rq_xid; + + } else { /* needs to generate a new matchbits for resend */ + u64 old_mbits = req->rq_mbits; + + if ((bd->bd_import->imp_connect_data.ocd_connect_flags & + OBD_CONNECT_BULK_MBITS)) { + req->rq_mbits = ptlrpc_next_xid(); + } else { + /* old version transfers rq_xid to peer as matchbits */ + req->rq_mbits = ptlrpc_next_xid(); + req->rq_xid = req->rq_mbits; + } + + CDEBUG(D_HA, "resend bulk old x%llu new x%llu\n", + old_mbits, req->rq_mbits); + } + + /* + * For multi-bulk RPCs, rq_mbits is the last mbits needed for bulks so + * that server can infer the number of bulks that were prepared, + * see LU-1431 + */ + req->rq_mbits += ((bd->bd_iov_count + LNET_MAX_IOV - 1) / + LNET_MAX_IOV) - 1; +} + +/** * Get a glimpse at what next xid value might have been. * Returns possible next xid. */ diff --git a/drivers/staging/lustre/lustre/ptlrpc/niobuf.c b/drivers/staging/lustre/lustre/ptlrpc/niobuf.c index 4e80ba9..5810565 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/niobuf.c +++ b/drivers/staging/lustre/lustre/ptlrpc/niobuf.c @@ -114,7 +114,7 @@ static int ptlrpc_register_bulk(struct ptlrpc_request *req) int rc2; int posted_md; int total_md; - __u64 xid; + u64 mbits; lnet_handle_me_t me_h; lnet_md_t md; @@ -142,40 +142,37 @@ static int ptlrpc_register_bulk(struct ptlrpc_request *req) LASSERT(desc->bd_cbid.cbid_fn == client_bulk_callback); LASSERT(desc->bd_cbid.cbid_arg == desc); - /* An XID is only used for a single request from the client. - * For retried bulk transfers, a new XID will be allocated in - * in ptlrpc_check_set() if it needs to be resent, so it is not - * using the same RDMA match bits after an error. - * - * For multi-bulk RPCs, rq_xid is the last XID needed for bulks. The - * first bulk XID is power-of-two aligned before rq_xid. LU-1431 - */ - xid = req->rq_xid & ~((__u64)desc->bd_md_max_brw - 1); + total_md = (desc->bd_iov_count + LNET_MAX_IOV - 1) / LNET_MAX_IOV; + /* rq_mbits is matchbits of the final bulk */ + mbits = req->rq_mbits - total_md + 1; + + LASSERTF(mbits == (req->rq_mbits & PTLRPC_BULK_OPS_MASK), + "first mbits = x%llu, last mbits = x%llu\n", + mbits, req->rq_mbits); LASSERTF(!(desc->bd_registered && req->rq_send_state != LUSTRE_IMP_REPLAY) || - xid != desc->bd_last_xid, - "registered: %d rq_xid: %llu bd_last_xid: %llu\n", - desc->bd_registered, xid, desc->bd_last_xid); + mbits != desc->bd_last_mbits, + "registered: %d rq_mbits: %llu bd_last_mbits: %llu\n", + desc->bd_registered, mbits, desc->bd_last_mbits); - total_md = (desc->bd_iov_count + LNET_MAX_IOV - 1) / LNET_MAX_IOV; desc->bd_registered = 1; - desc->bd_last_xid = xid; + desc->bd_last_mbits = mbits; desc->bd_md_count = total_md; md.user_ptr = &desc->bd_cbid; md.eq_handle = ptlrpc_eq_h; md.threshold = 1; /* PUT or GET */ - for (posted_md = 0; posted_md < total_md; posted_md++, xid++) { + for (posted_md = 0; posted_md < total_md; posted_md++, mbits++) { md.options = PTLRPC_MD_OPTIONS | (ptlrpc_is_bulk_op_get(desc->bd_type) ? LNET_MD_OP_GET : LNET_MD_OP_PUT); ptlrpc_fill_bulk_md(&md, desc, posted_md); - rc = LNetMEAttach(desc->bd_portal, peer, xid, 0, + rc = LNetMEAttach(desc->bd_portal, peer, mbits, 0, LNET_UNLINK, LNET_INS_AFTER, &me_h); if (rc != 0) { CERROR("%s: LNetMEAttach failed x%llu/%d: rc = %d\n", - desc->bd_import->imp_obd->obd_name, xid, + desc->bd_import->imp_obd->obd_name, mbits, posted_md, rc); break; } @@ -185,7 +182,7 @@ static int ptlrpc_register_bulk(struct ptlrpc_request *req) &desc->bd_mds[posted_md]); if (rc != 0) { CERROR("%s: LNetMDAttach failed x%llu/%d: rc = %d\n", - desc->bd_import->imp_obd->obd_name, xid, + desc->bd_import->imp_obd->obd_name, mbits, posted_md, rc); rc2 = LNetMEUnlink(me_h); LASSERT(rc2 == 0); @@ -204,27 +201,19 @@ static int ptlrpc_register_bulk(struct ptlrpc_request *req) return -ENOMEM; } - /* Set rq_xid to matchbits of the final bulk so that server can - * infer the number of bulks that were prepared - */ - req->rq_xid = --xid; - LASSERTF(desc->bd_last_xid == (req->rq_xid & PTLRPC_BULK_OPS_MASK), - "bd_last_xid = x%llu, rq_xid = x%llu\n", - desc->bd_last_xid, req->rq_xid); - spin_lock(&desc->bd_lock); - /* Holler if peer manages to touch buffers before he knows the xid */ + /* Holler if peer manages to touch buffers before he knows the mbits */ if (desc->bd_md_count != total_md) CWARN("%s: Peer %s touched %d buffers while I registered\n", desc->bd_import->imp_obd->obd_name, libcfs_id2str(peer), total_md - desc->bd_md_count); spin_unlock(&desc->bd_lock); - CDEBUG(D_NET, "Setup %u bulk %s buffers: %u pages %u bytes, xid x%#llx-%#llx, portal %u\n", + CDEBUG(D_NET, "Setup %u bulk %s buffers: %u pages %u bytes, mbits x%#llx-%#llx, portal %u\n", desc->bd_md_count, ptlrpc_is_bulk_op_get(desc->bd_type) ? "get-source" : "put-sink", desc->bd_iov_count, desc->bd_nob, - desc->bd_last_xid, req->rq_xid, desc->bd_portal); + desc->bd_last_mbits, req->rq_mbits, desc->bd_portal); return 0; } @@ -520,6 +509,20 @@ int ptl_send_rpc(struct ptlrpc_request *request, int noreply) lustre_msg_set_conn_cnt(request->rq_reqmsg, imp->imp_conn_cnt); lustre_msghdr_set_flags(request->rq_reqmsg, imp->imp_msghdr_flags); + if (request->rq_nr_resend) { + /* + * resend for EINPROGRESS, allocate new xid to avoid reply + * reconstruction + */ + request->rq_xid = ptlrpc_next_xid(); + DEBUG_REQ(D_RPCTRACE, request, "Allocating new xid for resend on EINPROGRESS"); + } + + if (request->rq_bulk) { + ptlrpc_set_bulk_mbits(request); + lustre_msg_set_mbits(request->rq_reqmsg, request->rq_mbits); + } + /** * For enabled AT all request should have AT_SUPPORT in the * FULL import state when OBD_CONNECT_AT is set diff --git a/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c b/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c index 4f63a80..39c4e18 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c +++ b/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c @@ -1423,6 +1423,21 @@ void lustre_msg_set_cksum(struct lustre_msg *msg, __u32 cksum) } } +void lustre_msg_set_mbits(struct lustre_msg *msg, __u64 mbits) +{ + switch (msg->lm_magic) { + case LUSTRE_MSG_MAGIC_V2: { + struct ptlrpc_body *pb = lustre_msg_ptlrpc_body(msg); + + LASSERTF(pb, "invalid msg %p: no ptlrpc body!\n", msg); + pb->pb_mbits = mbits; + return; + } + default: + LASSERTF(0, "incorrect message magic: %08x\n", msg->lm_magic); + } +} + void ptlrpc_request_set_replen(struct ptlrpc_request *req) { int count = req_capsule_filled_sizes(&req->rq_pill, RCL_SERVER); @@ -1506,9 +1521,12 @@ void lustre_swab_ptlrpc_body(struct ptlrpc_body *b) __swab64s(&b->pb_pre_versions[1]); __swab64s(&b->pb_pre_versions[2]); __swab64s(&b->pb_pre_versions[3]); + __swab64s(&b->pb_mbits); CLASSERT(offsetof(typeof(*b), pb_padding0) != 0); CLASSERT(offsetof(typeof(*b), pb_padding1) != 0); - CLASSERT(offsetof(typeof(*b), pb_padding) != 0); + CLASSERT(offsetof(typeof(*b), pb_padding64_0) != 0); + CLASSERT(offsetof(typeof(*b), pb_padding64_1) != 0); + CLASSERT(offsetof(typeof(*b), pb_padding64_2) != 0); /* While we need to maintain compatibility between * clients and servers without ptlrpc_body_v2 (< 2.3) * do not swab any fields beyond pb_jobid, as we are diff --git a/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h b/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h index b848c25..f056c82 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h +++ b/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h @@ -70,6 +70,7 @@ void ptlrpc_set_add_new_req(struct ptlrpcd_ctl *pc, int ptlrpc_expired_set(void *data); int ptlrpc_set_next_timeout(struct ptlrpc_request_set *); void ptlrpc_resend_req(struct ptlrpc_request *request); +void ptlrpc_set_bulk_mbits(struct ptlrpc_request *req); /* events.c */ int ptlrpc_init_portals(void); diff --git a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c index 6c75446..485ac88 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c +++ b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c @@ -668,10 +668,22 @@ void lustre_assert_wire_constants(void) (long long)(int)offsetof(struct ptlrpc_body_v3, pb_pre_versions)); LASSERTF((int)sizeof(((struct ptlrpc_body_v3 *)0)->pb_pre_versions) == 32, "found %lld\n", (long long)(int)sizeof(((struct ptlrpc_body_v3 *)0)->pb_pre_versions)); - LASSERTF((int)offsetof(struct ptlrpc_body_v3, pb_padding) == 120, "found %lld\n", - (long long)(int)offsetof(struct ptlrpc_body_v3, pb_padding)); - LASSERTF((int)sizeof(((struct ptlrpc_body_v3 *)0)->pb_padding) == 32, "found %lld\n", - (long long)(int)sizeof(((struct ptlrpc_body_v3 *)0)->pb_padding)); + LASSERTF((int)offsetof(struct ptlrpc_body_v3, pb_mbits) == 120, "found %lld\n", + (long long)(int)offsetof(struct ptlrpc_body_v3, pb_mbits)); + LASSERTF((int)sizeof(((struct ptlrpc_body_v3 *)0)->pb_mbits) == 8, "found %lld\n", + (long long)(int)sizeof(((struct ptlrpc_body_v3 *)0)->pb_mbits)); + LASSERTF((int)offsetof(struct ptlrpc_body_v3, pb_padding64_0) == 128, "found %lld\n", + (long long)(int)offsetof(struct ptlrpc_body_v3, pb_padding64_0)); + LASSERTF((int)sizeof(((struct ptlrpc_body_v3 *)0)->pb_padding64_0) == 8, "found %lld\n", + (long long)(int)sizeof(((struct ptlrpc_body_v3 *)0)->pb_padding64_0)); + LASSERTF((int)offsetof(struct ptlrpc_body_v3, pb_padding64_1) == 136, "found %lld\n", + (long long)(int)offsetof(struct ptlrpc_body_v3, pb_padding64_1)); + LASSERTF((int)sizeof(((struct ptlrpc_body_v3 *)0)->pb_padding64_1) == 8, "found %lld\n", + (long long)(int)sizeof(((struct ptlrpc_body_v3 *)0)->pb_padding64_1)); + LASSERTF((int)offsetof(struct ptlrpc_body_v3, pb_padding64_2) == 144, "found %lld\n", + (long long)(int)offsetof(struct ptlrpc_body_v3, pb_padding64_2)); + LASSERTF((int)sizeof(((struct ptlrpc_body_v3 *)0)->pb_padding64_2) == 8, "found %lld\n", + (long long)(int)sizeof(((struct ptlrpc_body_v3 *)0)->pb_padding64_2)); CLASSERT(LUSTRE_JOBID_SIZE == 32); LASSERTF((int)offsetof(struct ptlrpc_body_v3, pb_jobid) == 152, "found %lld\n", (long long)(int)offsetof(struct ptlrpc_body_v3, pb_jobid)); @@ -753,10 +765,22 @@ void lustre_assert_wire_constants(void) (int)offsetof(struct ptlrpc_body_v3, pb_pre_versions), (int)offsetof(struct ptlrpc_body_v2, pb_pre_versions)); LASSERTF((int)sizeof(((struct ptlrpc_body_v3 *)0)->pb_pre_versions) == (int)sizeof(((struct ptlrpc_body_v2 *)0)->pb_pre_versions), "%d != %d\n", (int)sizeof(((struct ptlrpc_body_v3 *)0)->pb_pre_versions), (int)sizeof(((struct ptlrpc_body_v2 *)0)->pb_pre_versions)); - LASSERTF((int)offsetof(struct ptlrpc_body_v3, pb_padding) == (int)offsetof(struct ptlrpc_body_v2, pb_padding), "%d != %d\n", - (int)offsetof(struct ptlrpc_body_v3, pb_padding), (int)offsetof(struct ptlrpc_body_v2, pb_padding)); - LASSERTF((int)sizeof(((struct ptlrpc_body_v3 *)0)->pb_padding) == (int)sizeof(((struct ptlrpc_body_v2 *)0)->pb_padding), "%d != %d\n", - (int)sizeof(((struct ptlrpc_body_v3 *)0)->pb_padding), (int)sizeof(((struct ptlrpc_body_v2 *)0)->pb_padding)); + LASSERTF((int)offsetof(struct ptlrpc_body_v3, pb_mbits) == (int)offsetof(struct ptlrpc_body_v2, pb_mbits), "%d != %d\n", + (int)offsetof(struct ptlrpc_body_v3, pb_mbits), (int)offsetof(struct ptlrpc_body_v2, pb_mbits)); + LASSERTF((int)sizeof(((struct ptlrpc_body_v3 *)0)->pb_mbits) == (int)sizeof(((struct ptlrpc_body_v2 *)0)->pb_mbits), "%d != %d\n", + (int)sizeof(((struct ptlrpc_body_v3 *)0)->pb_mbits), (int)sizeof(((struct ptlrpc_body_v2 *)0)->pb_mbits)); + LASSERTF((int)offsetof(struct ptlrpc_body_v3, pb_padding64_0) == (int)offsetof(struct ptlrpc_body_v2, pb_padding64_0), "%d != %d\n", + (int)offsetof(struct ptlrpc_body_v3, pb_padding64_0), (int)offsetof(struct ptlrpc_body_v2, pb_padding64_0)); + LASSERTF((int)sizeof(((struct ptlrpc_body_v3 *)0)->pb_padding64_0) == (int)sizeof(((struct ptlrpc_body_v2 *)0)->pb_padding64_0), "%d != %d\n", + (int)sizeof(((struct ptlrpc_body_v3 *)0)->pb_padding64_0), (int)sizeof(((struct ptlrpc_body_v2 *)0)->pb_padding64_0)); + LASSERTF((int)offsetof(struct ptlrpc_body_v3, pb_padding64_1) == (int)offsetof(struct ptlrpc_body_v2, pb_padding64_1), "%d != %d\n", + (int)offsetof(struct ptlrpc_body_v3, pb_padding64_1), (int)offsetof(struct ptlrpc_body_v2, pb_padding64_1)); + LASSERTF((int)sizeof(((struct ptlrpc_body_v3 *)0)->pb_padding64_1) == (int)sizeof(((struct ptlrpc_body_v2 *)0)->pb_padding64_1), "%d != %d\n", + (int)sizeof(((struct ptlrpc_body_v3 *)0)->pb_padding64_1), (int)sizeof(((struct ptlrpc_body_v2 *)0)->pb_padding64_1)); + LASSERTF((int)offsetof(struct ptlrpc_body_v3, pb_padding64_2) == (int)offsetof(struct ptlrpc_body_v2, pb_padding64_2), "%d != %d\n", + (int)offsetof(struct ptlrpc_body_v3, pb_padding64_2), (int)offsetof(struct ptlrpc_body_v2, pb_padding64_2)); + LASSERTF((int)sizeof(((struct ptlrpc_body_v3 *)0)->pb_padding64_2) == (int)sizeof(((struct ptlrpc_body_v2 *)0)->pb_padding64_2), "%d != %d\n", + (int)sizeof(((struct ptlrpc_body_v3 *)0)->pb_padding64_2), (int)sizeof(((struct ptlrpc_body_v2 *)0)->pb_padding64_2)); LASSERTF(MSG_PTLRPC_BODY_OFF == 0, "found %lld\n", (long long)MSG_PTLRPC_BODY_OFF); LASSERTF(REQ_REC_OFF == 1, "found %lld\n", -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:30:37 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:30:37 -0500 Subject: [lustre-devel] [PATCH 07/35] staging: lustre: lov: avoid infinite loop in lsm_alloc_plain() In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <1478799065-24841-8-git-send-email-jsimmons@infradead.org> From: John L. Hammond In lsm_alloc_plain() use a signed loop index to avoid an infinite loop in the error path. Signed-off-by: John L. Hammond Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6872 Reviewed-on: http://review.whamcloud.com/15644 Reviewed-by: Dmitry Eremin Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- drivers/staging/lustre/lustre/lov/lov_ea.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/drivers/staging/lustre/lustre/lov/lov_ea.c b/drivers/staging/lustre/lustre/lov/lov_ea.c index 218f275..ac0bf64 100644 --- a/drivers/staging/lustre/lustre/lov/lov_ea.c +++ b/drivers/staging/lustre/lustre/lov/lov_ea.c @@ -81,7 +81,7 @@ struct lov_stripe_md *lsm_alloc_plain(u16 stripe_count) size_t oinfo_ptrs_size, lsm_size; struct lov_stripe_md *lsm; struct lov_oinfo *loi; - unsigned int i; + int i; LASSERT(stripe_count <= LOV_MAX_STRIPE_COUNT); -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:30:41 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:30:41 -0500 Subject: [lustre-devel] [PATCH 11/35] staging: lustre: ptlrpc: race at req processing In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <1478799065-24841-12-git-send-email-jsimmons@infradead.org> From: Alexander Boyko Fix: 5c689e689baa ("staging/lustre/ptlrpc: race at req processing") decreased the race window, but does not remove it. Disable rq_resend right after MSG_REPLAY flag set. Import lock protects two threads from race between set/clear MSG_REPLAY and rq_resend flags. Signed-off-by: Alexander Boyko Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-5554 Xyratex-bug-id: MRP-1888 Reviewed-on: http://review.whamcloud.com/10735 Reviewed-by: Niu Yawei Reviewed-by: Mike Pershin Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- drivers/staging/lustre/lustre/ptlrpc/client.c | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c index b940e24..d2f4cd5 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/client.c +++ b/drivers/staging/lustre/lustre/ptlrpc/client.c @@ -2690,6 +2690,10 @@ void ptlrpc_retain_replayable_request(struct ptlrpc_request *req, lustre_msg_add_flags(req->rq_reqmsg, MSG_REPLAY); + spin_lock(&req->rq_lock); + req->rq_resend = 0; + spin_unlock(&req->rq_lock); + LASSERT(imp->imp_replayable); /* Balanced in ptlrpc_free_committed, usually. */ ptlrpc_request_addref(req); -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:30:44 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:30:44 -0500 Subject: [lustre-devel] [PATCH 14/35] staging: lustre: nrs: serialize executions of nrs_policy_stop In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <1478799065-24841-15-git-send-email-jsimmons@infradead.org> From: Henri Doreau Do not release nrs_lock in nrs_policy_stop0 to prevent op_policy_stop() from being executed concurrently. Signed-off-by: Henri Doreau Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-7096 Reviewed-on: http://review.whamcloud.com/16214 Reviewed-by: Lai Siyao Reviewed-by: Emoly Liu Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- drivers/staging/lustre/lustre/ptlrpc/nrs.c | 16 +++------------- 1 files changed, 3 insertions(+), 13 deletions(-) diff --git a/drivers/staging/lustre/lustre/ptlrpc/nrs.c b/drivers/staging/lustre/lustre/ptlrpc/nrs.c index f856632..7b6ffb1 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/nrs.c +++ b/drivers/staging/lustre/lustre/ptlrpc/nrs.c @@ -82,16 +82,9 @@ static int nrs_policy_ctl_locked(struct ptlrpc_nrs_policy *policy, static void nrs_policy_stop0(struct ptlrpc_nrs_policy *policy) { - struct ptlrpc_nrs *nrs = policy->pol_nrs; - - if (policy->pol_desc->pd_ops->op_policy_stop) { - spin_unlock(&nrs->nrs_lock); - + if (policy->pol_desc->pd_ops->op_policy_stop) policy->pol_desc->pd_ops->op_policy_stop(policy); - spin_lock(&nrs->nrs_lock); - } - LASSERT(list_empty(&policy->pol_list_queued)); LASSERT(policy->pol_req_queued == 0 && policy->pol_req_started == 0); @@ -619,11 +612,8 @@ static int nrs_policy_ctl(struct ptlrpc_nrs *nrs, char *name, goto out; } - /** - * Wait for the policy to be fully started before attempting - * to operate it. - */ - if (policy->pol_state == NRS_POL_STATE_STARTING) { + if (policy->pol_state != NRS_POL_STATE_STARTED && + policy->pol_state != NRS_POL_STATE_STOPPED) { rc = -EAGAIN; goto out; } -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:30:35 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:30:35 -0500 Subject: [lustre-devel] [PATCH 05/35] staging: lustre: llog: fix wrong offset in llog_process_thread() In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <1478799065-24841-6-git-send-email-jsimmons@infradead.org> From: Mikhail Pershin - llh_cat_idx may become bigger than llog bitmap size in llog_cat_set_first_idx() function - it is wrong to use previous cur_offset as new buffer offset, new offset should be calculated from value returned by llog_next_block(). - optimize llog_skip_over() to find llog entry offset by index for llog with fixed-size records. Signed-off-by: Mikhail Pershin Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6714 Reviewed-on: http://review.whamcloud.com/15316 Reviewed-by: John L. Hammond Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- drivers/staging/lustre/lustre/obdclass/llog.c | 81 +++++++++++++++++------- 1 files changed, 57 insertions(+), 24 deletions(-) diff --git a/drivers/staging/lustre/lustre/obdclass/llog.c b/drivers/staging/lustre/lustre/obdclass/llog.c index 3bc1789..f69fa32 100644 --- a/drivers/staging/lustre/lustre/obdclass/llog.c +++ b/drivers/staging/lustre/lustre/obdclass/llog.c @@ -218,8 +218,7 @@ static int llog_process_thread(void *arg) struct llog_process_cat_data *cd = lpi->lpi_catdata; char *buf; __u64 cur_offset; - __u64 last_offset; - int chunk_size; + size_t chunk_size; int rc = 0, index = 1, last_index; int saved_index = 0; int last_called_index = 0; @@ -229,6 +228,8 @@ static int llog_process_thread(void *arg) cur_offset = llh->llh_hdr.lrh_len; chunk_size = llh->llh_hdr.lrh_len; + /* expect chunk_size to be power of two */ + LASSERT(is_power_of_2(chunk_size)); buf = libcfs_kvzalloc(chunk_size, GFP_NOFS); if (!buf) { @@ -245,38 +246,49 @@ static int llog_process_thread(void *arg) else last_index = LLOG_HDR_BITMAP_SIZE(llh) - 1; - /* Record is not in this buffer. */ - if (index > last_index) - goto out; - while (rc == 0) { + unsigned int buf_offset = 0; struct llog_rec_hdr *rec; + bool partial_chunk; + off_t chunk_offset; /* skip records not set in bitmap */ while (index <= last_index && !ext2_test_bit(index, LLOG_HDR_BITMAP(llh))) ++index; - LASSERT(index <= last_index + 1); - if (index == last_index + 1) + if (index > last_index) break; -repeat: + CDEBUG(D_OTHER, "index: %d last_index %d\n", index, last_index); - +repeat: /* get the buf with our target record; avoid old garbage */ memset(buf, 0, chunk_size); - last_offset = cur_offset; rc = llog_next_block(lpi->lpi_env, loghandle, &saved_index, index, &cur_offset, buf, chunk_size); if (rc) goto out; + /* + * NB: after llog_next_block() call the cur_offset is the + * offset of the next block after read one. + * The absolute offset of the current chunk is calculated + * from cur_offset value and stored in chunk_offset variable. + */ + if (cur_offset % chunk_size) { + partial_chunk = true; + chunk_offset = cur_offset & ~(chunk_size - 1); + } else { + partial_chunk = false; + chunk_offset = cur_offset - chunk_size; + } + /* NB: when rec->lrh_len is accessed it is already swabbed * since it is used at the "end" of the loop and the rec * swabbing is done at the beginning of the loop. */ - for (rec = (struct llog_rec_hdr *)buf; + for (rec = (struct llog_rec_hdr *)(buf + buf_offset); (char *)rec < buf + chunk_size; rec = llog_rec_hdr_next(rec)) { CDEBUG(D_OTHER, "processing rec 0x%p type %#x\n", @@ -288,13 +300,28 @@ static int llog_process_thread(void *arg) CDEBUG(D_OTHER, "after swabbing, type=%#x idx=%d\n", rec->lrh_type, rec->lrh_index); - if (rec->lrh_index == 0) { - /* probably another rec just got added? */ - rc = 0; - if (index <= loghandle->lgh_last_idx) - goto repeat; - goto out; /* no more records */ + /* + * for partial chunk the end of it is zeroed, check + * for index 0 to distinguish it. + */ + if (partial_chunk && !rec->lrh_index) { + /* concurrent llog_add() might add new records + * while llog_processing, check this is not + * the case and re-read the current chunk + * otherwise. + */ + if (index > loghandle->lgh_last_idx) { + rc = 0; + goto out; + } + CDEBUG(D_OTHER, "Re-read last llog buffer for new records, index %u, last %u\n", + index, loghandle->lgh_last_idx); + /* save offset inside buffer for the re-read */ + buf_offset = (char *)rec - (char *)buf; + cur_offset = chunk_offset; + goto repeat; } + if (!rec->lrh_len || rec->lrh_len > chunk_size) { CWARN("invalid length %d in llog record for index %d/%d\n", rec->lrh_len, @@ -309,6 +336,14 @@ static int llog_process_thread(void *arg) continue; } + if (rec->lrh_index != index) { + CERROR("%s: Invalid record: index %u but expected %u\n", + loghandle->lgh_ctxt->loc_obd->obd_name, + rec->lrh_index, index); + rc = -ERANGE; + goto out; + } + CDEBUG(D_OTHER, "lrh_index: %d lrh_len: %d (%d remains)\n", rec->lrh_index, rec->lrh_len, @@ -316,7 +351,7 @@ static int llog_process_thread(void *arg) loghandle->lgh_cur_idx = rec->lrh_index; loghandle->lgh_cur_offset = (char *)rec - (char *)buf + - last_offset; + chunk_offset; /* if set, process the callback on this record */ if (ext2_test_bit(index, LLOG_HDR_BITMAP(llh))) { @@ -325,16 +360,14 @@ static int llog_process_thread(void *arg) last_called_index = index; if (rc) goto out; - } else { - CDEBUG(D_OTHER, "Skipped index %d\n", index); } - /* next record, still in buffer? */ - ++index; - if (index > last_index) { + /* exit if the last index is reached */ + if (index >= last_index) { rc = 0; goto out; } + index++; } } -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:30:51 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:30:51 -0500 Subject: [lustre-devel] [PATCH 21/35] staging: lustre: statahead: lock leaks if statahead file recreated In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <1478799065-24841-22-git-send-email-jsimmons@infradead.org> From: Lai Siyao During statahead file may be recreated, though this is rare case, current code will leak the lock, this patch will release lock in this case. Signed-off-by: Lai Siyao Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-7232 Reviewed-on: http://review.whamcloud.com/16841 Reviewed-by: Fan Yong Reviewed-by: John L. Hammond Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- drivers/staging/lustre/lustre/llite/statahead.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/drivers/staging/lustre/lustre/llite/statahead.c b/drivers/staging/lustre/lustre/llite/statahead.c index 0677513..b43955f 100644 --- a/drivers/staging/lustre/lustre/llite/statahead.c +++ b/drivers/staging/lustre/lustre/llite/statahead.c @@ -1475,6 +1475,7 @@ static int revalidate_statahead_dentry(struct inode *dir, alias = ll_splice_alias(inode, *dentryp); if (IS_ERR(alias)) { + ll_intent_release(&it); rc = PTR_ERR(alias); goto out_unplug; } @@ -1493,6 +1494,7 @@ static int revalidate_statahead_dentry(struct inode *dir, *dentryp, PFID(ll_inode2fid((*dentryp)->d_inode)), PFID(ll_inode2fid(inode))); + ll_intent_release(&it); rc = -ESTALE; goto out_unplug; } -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:31:03 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:31:03 -0500 Subject: [lustre-devel] [PATCH 33/35] staging: lustre: hsm: prevent migration of HSM archived files In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <1478799065-24841-34-git-send-email-jsimmons@infradead.org> From: John L. Hammond The reference copytool cannot handle migration of HSM archive files. In the MDT migration path check for HSM attributes and fail if they are present. In the LMV layer allow creation of volatile files with any MDT index. Add a test to sanity-hsm to ensure that attempting to migrate an HSM archive file is handled safely. Signed-off-by: John L. Hammond Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6866 Reviewed-on: http://review.whamcloud.com/17511 Reviewed-by: wangdi Reviewed-by: Alex Zhuravlev Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- drivers/staging/lustre/lustre/lmv/lmv_obd.c | 25 +++++++++++++------------ 1 files changed, 13 insertions(+), 12 deletions(-) diff --git a/drivers/staging/lustre/lustre/lmv/lmv_obd.c b/drivers/staging/lustre/lustre/lmv/lmv_obd.c index 9335ffe..0e94a58 100644 --- a/drivers/staging/lustre/lustre/lmv/lmv_obd.c +++ b/drivers/staging/lustre/lustre/lmv/lmv_obd.c @@ -1619,27 +1619,28 @@ struct lmv_tgt_desc* * ct_restore(). */ if (op_data->op_bias & MDS_CREATE_VOLATILE && - (int)op_data->op_mds != -1 && lsm) { + (int)op_data->op_mds != -1) { int i; tgt = lmv_get_target(lmv, op_data->op_mds, NULL); if (IS_ERR(tgt)) return tgt; - /* refill the right parent fid */ - for (i = 0; i < lsm->lsm_md_stripe_count; i++) { - struct lmv_oinfo *oinfo; + if (lsm) { + /* refill the right parent fid */ + for (i = 0; i < lsm->lsm_md_stripe_count; i++) { + struct lmv_oinfo *oinfo; - oinfo = &lsm->lsm_md_oinfo[i]; - if (oinfo->lmo_mds == op_data->op_mds) { - *fid = oinfo->lmo_fid; - break; + oinfo = &lsm->lsm_md_oinfo[i]; + if (oinfo->lmo_mds == op_data->op_mds) { + *fid = oinfo->lmo_fid; + break; + } } - } - /* Hmm, can not find the stripe by mdt_index(op_mds) */ - if (i == lsm->lsm_md_stripe_count) - tgt = ERR_PTR(-EINVAL); + if (i == lsm->lsm_md_stripe_count) + *fid = lsm->lsm_md_oinfo[0].lmo_fid; + } return tgt; } -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:31:05 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:31:05 -0500 Subject: [lustre-devel] [PATCH 35/35] staging: lustre: idl: clean up file attribute flags In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <1478799065-24841-36-git-send-email-jsimmons@infradead.org> From: Andreas Dilger Remove unused file attribute flag LUSTRE_BFLAG_UNCOMMITTED_WRITES that was used internally on the client at one point. Add flags from the kernel which may be useful in the near future. Signed-off-by: Andreas Dilger Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-5017 Reviewed-on: http://review.whamcloud.com/10274 Reviewed-by: Dmitry Eremin Reviewed-by: wangdi Reviewed-by: Jinshan Xiong Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- .../lustre/lustre/include/lustre/lustre_idl.h | 9 ++++++--- drivers/staging/lustre/lustre/ptlrpc/wiretest.c | 12 ++++++++++-- 2 files changed, 16 insertions(+), 5 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h index b8ddebd..5c1fb6c 100644 --- a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h +++ b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h @@ -1924,16 +1924,19 @@ enum { #define MDS_STATUS_CONN 1 #define MDS_STATUS_LOV 2 -#define LUSTRE_BFLAG_UNCOMMITTED_WRITES 0x1 - /* these should be identical to their EXT4_*_FL counterparts, they are * redefined here only to avoid dragging in fs/ext4/ext4.h */ #define LUSTRE_SYNC_FL 0x00000008 /* Synchronous updates */ #define LUSTRE_IMMUTABLE_FL 0x00000010 /* Immutable file */ #define LUSTRE_APPEND_FL 0x00000020 /* writes to file may only append */ +#define LUSTRE_NODUMP_FL 0x00000040 /* do not dump file */ #define LUSTRE_NOATIME_FL 0x00000080 /* do not update atime */ +#define LUSTRE_INDEX_FL 0x00001000 /* hash-indexed directory */ #define LUSTRE_DIRSYNC_FL 0x00010000 /* dirsync behaviour (dir only) */ +#define LUSTRE_TOPDIR_FL 0x00020000 /* Top of directory hierarchies*/ +#define LUSTRE_DIRECTIO_FL 0x00100000 /* Use direct i/o */ +#define LUSTRE_INLINE_DATA_FL 0x10000000 /* Inode has inline data. */ /* Convert wire LUSTRE_*_FL to corresponding client local VFS S_* values * for the client inode i_flags. The LUSTRE_*_FL are the Lustre wire @@ -1986,7 +1989,7 @@ struct mdt_body { __u32 mbo_mode; __u32 mbo_uid; __u32 mbo_gid; - __u32 mbo_flags; + __u32 mbo_flags; /* LUSTRE_*_FL file attributes */ __u32 mbo_rdev; __u32 mbo_nlink; /* #bytes to read in the case of MDS_READPAGE */ __u32 mbo_unused2; /* was "generation" until 2.4.0 */ diff --git a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c index 485ac88..b239563 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c +++ b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c @@ -218,8 +218,6 @@ void lustre_assert_wire_constants(void) (long long)MDS_STATUS_CONN); LASSERTF(MDS_STATUS_LOV == 2, "found %lld\n", (long long)MDS_STATUS_LOV); - LASSERTF(LUSTRE_BFLAG_UNCOMMITTED_WRITES == 1, "found %lld\n", - (long long)LUSTRE_BFLAG_UNCOMMITTED_WRITES); LASSERTF(MDS_ATTR_MODE == 0x0000000000000001ULL, "found 0x%.16llxULL\n", (long long)MDS_ATTR_MODE); LASSERTF(MDS_ATTR_UID == 0x0000000000000002ULL, "found 0x%.16llxULL\n", @@ -1927,10 +1925,20 @@ void lustre_assert_wire_constants(void) LUSTRE_IMMUTABLE_FL); LASSERTF(LUSTRE_APPEND_FL == 0x00000020, "found 0x%.8x\n", LUSTRE_APPEND_FL); + LASSERTF(LUSTRE_NODUMP_FL == 0x00000040, "found 0x%.8x\n", + LUSTRE_NODUMP_FL); LASSERTF(LUSTRE_NOATIME_FL == 0x00000080, "found 0x%.8x\n", LUSTRE_NOATIME_FL); + LASSERTF(LUSTRE_INDEX_FL == 0x00001000, "found 0x%.8x\n", + LUSTRE_INDEX_FL); LASSERTF(LUSTRE_DIRSYNC_FL == 0x00010000, "found 0x%.8x\n", LUSTRE_DIRSYNC_FL); + LASSERTF(LUSTRE_TOPDIR_FL == 0x00020000, "found 0x%.8x\n", + LUSTRE_TOPDIR_FL); + LASSERTF(LUSTRE_DIRECTIO_FL == 0x00100000, "found 0x%.8x\n", + LUSTRE_DIRECTIO_FL); + LASSERTF(LUSTRE_INLINE_DATA_FL == 0x10000000, "found 0x%.8x\n", + LUSTRE_INLINE_DATA_FL); LASSERTF(MDS_INODELOCK_LOOKUP == 0x000001, "found 0x%.8x\n", MDS_INODELOCK_LOOKUP); LASSERTF(MDS_INODELOCK_UPDATE == 0x000002, "found 0x%.8x\n", -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:30:40 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:30:40 -0500 Subject: [lustre-devel] [PATCH 10/35] staging: lustre: lmv: revalidate the dentry for striped dir In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <1478799065-24841-11-git-send-email-jsimmons@infradead.org> From: wang di If there are bad stripe during striped dir revalidation, most likely due the race between close(unlink) and getattr, then let's revalidate the dentry, instead of return error, like normal directory. Signed-off-by: wang di Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6831 Reviewed-on: http://review.whamcloud.com/15720 Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-7078 Reviewed-on: http://review.whamcloud.com/16382 Reviewed-by: Alex Zhuravlev Reviewed-by: James Simmons Reviewed-by: John L. Hammond Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- drivers/staging/lustre/lustre/llite/file.c | 7 +++++ drivers/staging/lustre/lustre/llite/llite_lib.c | 30 ++++++++++++++++++++++- drivers/staging/lustre/lustre/lmv/lmv_intent.c | 11 +++++++- 3 files changed, 45 insertions(+), 3 deletions(-) diff --git a/drivers/staging/lustre/lustre/llite/file.c b/drivers/staging/lustre/lustre/llite/file.c index 7adbf31..28c2501 100644 --- a/drivers/staging/lustre/lustre/llite/file.c +++ b/drivers/staging/lustre/lustre/llite/file.c @@ -2732,6 +2732,13 @@ static int ll_inode_revalidate_fini(struct inode *inode, int rc) /* Already unlinked. Just update nlink and return success */ if (rc == -ENOENT) { clear_nlink(inode); + /* If it is striped directory, and there is bad stripe + * Let's revalidate the dentry again, instead of returning + * error + */ + if (S_ISDIR(inode->i_mode) && ll_i2info(inode)->lli_lsm_md) + return 0; + /* This path cannot be hit for regular files unless in * case of obscure races, so no need to validate size. */ diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c index c1b646c..4a4741b 100644 --- a/drivers/staging/lustre/lustre/llite/llite_lib.c +++ b/drivers/staging/lustre/lustre/llite/llite_lib.c @@ -1194,16 +1194,44 @@ static int ll_update_lsm_md(struct inode *inode, struct lustre_md *md) /* set the directory layout */ if (!lli->lli_lsm_md) { + struct cl_attr *attr; + rc = ll_init_lsm_md(inode, md); if (rc) return rc; - lli->lli_lsm_md = lsm; /* * set lsm_md to NULL, so the following free lustre_md * will not free this lsm */ md->lmv = NULL; + lli->lli_lsm_md = lsm; + + attr = kzalloc(sizeof(*attr), GFP_NOFS); + if (!attr) + return -ENOMEM; + + /* validate the lsm */ + rc = md_merge_attr(ll_i2mdexp(inode), lsm, attr, + ll_md_blocking_ast); + if (rc) { + kfree(attr); + return rc; + } + + if (md->body->mbo_valid & OBD_MD_FLNLINK) + md->body->mbo_nlink = attr->cat_nlink; + if (md->body->mbo_valid & OBD_MD_FLSIZE) + md->body->mbo_size = attr->cat_size; + if (md->body->mbo_valid & OBD_MD_FLATIME) + md->body->mbo_atime = attr->cat_atime; + if (md->body->mbo_valid & OBD_MD_FLCTIME) + md->body->mbo_ctime = attr->cat_ctime; + if (md->body->mbo_valid & OBD_MD_FLMTIME) + md->body->mbo_mtime = attr->cat_mtime; + + kfree(attr); + CDEBUG(D_INODE, "Set lsm %p magic %x to "DFID"\n", lsm, lsm->lsm_md_magic, PFID(ll_inode2fid(inode))); return 0; diff --git a/drivers/staging/lustre/lustre/lmv/lmv_intent.c b/drivers/staging/lustre/lustre/lmv/lmv_intent.c index 9f4e826..b1071cf 100644 --- a/drivers/staging/lustre/lustre/lmv/lmv_intent.c +++ b/drivers/staging/lustre/lustre/lmv/lmv_intent.c @@ -223,7 +223,14 @@ int lmv_revalidate_slaves(struct obd_export *exp, LASSERT(body); if (unlikely(body->mbo_nlink < 2)) { - CERROR("%s: nlink %d < 2 corrupt stripe %d "DFID":" DFID"\n", + /* + * If this is bad stripe, most likely due + * to the race between close(unlink) and + * getattr, let's return -EONENT, so llite + * will revalidate the dentry see + * ll_inode_revalidate_fini() + */ + CDEBUG(D_INODE, "%s: nlink %d < 2 corrupt stripe %d "DFID":" DFID"\n", obd->obd_name, body->mbo_nlink, i, PFID(&lsm->lsm_md_oinfo[i].lmo_fid), PFID(&lsm->lsm_md_oinfo[0].lmo_fid)); @@ -233,7 +240,7 @@ int lmv_revalidate_slaves(struct obd_export *exp, it.it_lock_mode = 0; } - rc = -EIO; + rc = -ENOENT; goto cleanup; } -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:30:56 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:30:56 -0500 Subject: [lustre-devel] [PATCH 26/35] staging: lustre: mdc: remove console spew from mdc_ioc_fid2path In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <1478799065-24841-27-git-send-email-jsimmons@infradead.org> From: Andreas Dilger In some cases with a very long pathname, such as with sanity.sh test_154c, mdc_ioc_fid2path() would spew long debug messages to the log, because libcfs_debug_vmsg2() refuses to log messages over one page in size. Truncate the debug message to only log the last 512 characters of the pathname, which is sufficient for most debugging, saves a bit of space in the debug log, and will prevent the debug logging from printing to the console in the first place. Signed-off-by: Andreas Dilger Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-1095 Reviewed-on: http://review.whamcloud.com/17078 Reviewed-by: Jian Yu Reviewed-by: Niu Yawei Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- drivers/staging/lustre/lustre/mdc/mdc_request.c | 7 +++++-- 1 files changed, 5 insertions(+), 2 deletions(-) diff --git a/drivers/staging/lustre/lustre/mdc/mdc_request.c b/drivers/staging/lustre/lustre/mdc/mdc_request.c index eee848d..6bed5ec 100644 --- a/drivers/staging/lustre/lustre/mdc/mdc_request.c +++ b/drivers/staging/lustre/lustre/mdc/mdc_request.c @@ -1455,8 +1455,11 @@ static int mdc_ioc_fid2path(struct obd_export *exp, struct getinfo_fid2path *gf) goto out; } - CDEBUG(D_IOCTL, "path get "DFID" from %llu #%d\n%s\n", - PFID(&gf->gf_fid), gf->gf_recno, gf->gf_linkno, gf->gf_path); + CDEBUG(D_IOCTL, "path got " DFID " from %llu #%d: %s\n", + PFID(&gf->gf_fid), gf->gf_recno, gf->gf_linkno, + gf->gf_pathlen < 512 ? gf->gf_path : + /* only log the last 512 characters of the path */ + gf->gf_path + gf->gf_pathlen - 512); out: kfree(key); -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:31:02 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:31:02 -0500 Subject: [lustre-devel] [PATCH 32/35] staging: lustre: mount: fix lmd_parse() to handle commas in expr_list In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <1478799065-24841-33-git-send-email-jsimmons@infradead.org> From: Jian Yu The lmd_parse() function parses mount options with comma as delimiter without considering commas in expr_list as follows is a valid LNET nid range syntax: :== '[' [ ',' ] ']' This patch fixes the above issue by using cfs_parse_nidlist() to parse nid range list instead of using class_parse_nid_quiet() to parse only one nid. Signed-off-by: Jian Yu Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-5690 Reviewed-on: http://review.whamcloud.com/17036 Reviewed-by: Niu Yawei Reviewed-by: Bob Glossman Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- drivers/staging/lustre/lustre/obdclass/obd_mount.c | 91 ++++++++++++++++++-- 1 files changed, 85 insertions(+), 6 deletions(-) diff --git a/drivers/staging/lustre/lustre/obdclass/obd_mount.c b/drivers/staging/lustre/lustre/obdclass/obd_mount.c index 2283e92..1eb8e71 100644 --- a/drivers/staging/lustre/lustre/obdclass/obd_mount.c +++ b/drivers/staging/lustre/lustre/obdclass/obd_mount.c @@ -871,6 +871,87 @@ static int lmd_parse_mgs(struct lustre_mount_data *lmd, char **ptr) return 0; } +/** + * Find the first comma delimiter from the specified \a buf and make \a *endh + * point to the string starting with the comma. The commas in expression list + * [...] will be skipped. + * + * \param[in] buf a comma-separated string + * \param[in] endh a pointer to a pointer that will point to the string + * starting with the comma + * + * \retval 0 if comma delimiter is found + * \retval 1 if comma delimiter is not found + */ +static int lmd_find_comma(char *buf, char **endh) +{ + char *c = buf; + int skip = 0; + + if (!buf) + return 1; + + while (*c != '\0') { + if (*c == '[') + skip++; + else if (*c == ']') + skip--; + + if (*c == ',' && !skip) { + if (endh) + *endh = c; + return 0; + } + c++; + } + return 1; +} + +/** + * Find the first valid string delimited by comma from the specified \a buf + # and parse it to see whether it's a valid nid list. If yes, \a *endh will + * point to the next string starting with the comma. + * + * \param[in] buf a comma-separated string + * \param[in] endh a pointer to a pointer that will point to the string + * starting with the comma + * + * \retval 0 if the string is a valid nid list + * \retval 1 if the string is not a valid nid list + */ +static int lmd_parse_nidlist(char *buf, char **endh) +{ + struct list_head nidlist; + char *endp = buf; + int rc = 0; + char tmp; + + if (!buf) + return 1; + while (*buf == ',' || *buf == ':') + buf++; + if (*buf == ' ' || *buf == '/' || *buf == '\0') + return 1; + + if (lmd_find_comma(buf, &endp)) + endp = buf + strlen(buf); + + tmp = *endp; + *endp = '\0'; + + INIT_LIST_HEAD(&nidlist); + if (cfs_parse_nidlist(buf, strlen(buf), &nidlist) <= 0) + rc = 1; + cfs_free_nidlist(&nidlist); + + *endp = tmp; + if (rc) + return rc; + if (endh) + *endh = endp; + return 0; +} + /** Parse mount line options * e.g. mount -v -t lustre -o abort_recov uml1:uml2:/lustre-client /mnt/lustre * dev is passed as device=uml1:/lustre by mount.lustre @@ -987,19 +1068,17 @@ static int lmd_parse(char *options, struct lustre_mount_data *lmd) clear++; } else if (strncmp(s1, "param=", 6) == 0) { size_t length, params_length; - char *tail = strchr(s1 + 6, ','); + char *tail = s1; - if (!tail) { + if (lmd_find_comma(s1 + 6, &tail)) { length = strlen(s1); } else { - lnet_nid_t nid; char *param_str = tail + 1; int supplementary = 1; - while (!class_parse_nid_quiet(param_str, &nid, - ¶m_str)) { + while (!lmd_parse_nidlist(param_str, + ¶m_str)) supplementary = 0; - } length = param_str - s1 - supplementary; } length -= 6; -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:30:47 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:30:47 -0500 Subject: [lustre-devel] [PATCH 17/35] staging: lustre: obd: Remove dead code in precleanup In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <1478799065-24841-18-git-send-email-jsimmons@infradead.org> From: Henri Doreau There used to be several pre-cleanup phases, but only OBD_CLEANUP_EXPORTS is actually used. Thus remove the whole notion of precleanup phases. Signed-off-by: Henri Doreau Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-7034 Reviewed-on: http://review.whamcloud.com/16061 Reviewed-by: John L. Hammond Reviewed-by: Dmitry Eremin Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- drivers/staging/lustre/lustre/include/obd.h | 10 +--- drivers/staging/lustre/lustre/include/obd_class.h | 17 ++--- drivers/staging/lustre/lustre/lmv/lmv_obd.c | 19 +----- drivers/staging/lustre/lustre/lov/lov_obd.c | 24 -------- drivers/staging/lustre/lustre/mdc/mdc_request.c | 25 +++----- drivers/staging/lustre/lustre/mgc/mgc_request.c | 46 +++++++-------- .../staging/lustre/lustre/obdclass/obd_config.c | 2 +- drivers/staging/lustre/lustre/osc/osc_request.c | 62 ++++++++------------ 8 files changed, 67 insertions(+), 138 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/obd.h b/drivers/staging/lustre/lustre/include/obd.h index 2ab7560..ffe09a3 100644 --- a/drivers/staging/lustre/lustre/include/obd.h +++ b/drivers/staging/lustre/lustre/include/obd.h @@ -625,13 +625,6 @@ struct obd_device { struct completion obd_kobj_unregister; }; -enum obd_cleanup_stage { -/* Special case hack for MDS LOVs */ - OBD_CLEANUP_EARLY, -/* can be directly mapped to .ldto_device_fini() */ - OBD_CLEANUP_EXPORTS, -}; - /* get/set_info keys */ #define KEY_ASYNC "async" #define KEY_CHANGELOG_CLEAR "changelog_clear" @@ -794,8 +787,7 @@ struct obd_ops { __u32 vallen, void *val, struct ptlrpc_request_set *set); int (*setup)(struct obd_device *dev, struct lustre_cfg *cfg); - int (*precleanup)(struct obd_device *dev, - enum obd_cleanup_stage cleanup_stage); + int (*precleanup)(struct obd_device *dev); int (*cleanup)(struct obd_device *dev); int (*process_config)(struct obd_device *dev, u32 len, void *data); int (*postrecov)(struct obd_device *dev); diff --git a/drivers/staging/lustre/lustre/include/obd_class.h b/drivers/staging/lustre/lustre/include/obd_class.h index f82f37b..01cd489 100644 --- a/drivers/staging/lustre/lustre/include/obd_class.h +++ b/drivers/staging/lustre/lustre/include/obd_class.h @@ -505,8 +505,7 @@ static inline int obd_setup(struct obd_device *obd, struct lustre_cfg *cfg) return rc; } -static inline int obd_precleanup(struct obd_device *obd, - enum obd_cleanup_stage cleanup_stage) +static inline int obd_precleanup(struct obd_device *obd) { int rc; DECLARE_LU_VARS(ldt, d); @@ -517,20 +516,18 @@ static inline int obd_precleanup(struct obd_device *obd, ldt = obd->obd_type->typ_lu; d = obd->obd_lu_dev; if (ldt && d) { - if (cleanup_stage == OBD_CLEANUP_EXPORTS) { - struct lu_env env; + struct lu_env env; - rc = lu_env_init(&env, ldt->ldt_ctx_tags); - if (rc == 0) { - ldt->ldt_ops->ldto_device_fini(&env, d); - lu_env_fini(&env); - } + rc = lu_env_init(&env, ldt->ldt_ctx_tags); + if (!rc) { + ldt->ldt_ops->ldto_device_fini(&env, d); + lu_env_fini(&env); } } OBD_CHECK_DT_OP(obd, precleanup, 0); OBD_COUNTER_INCREMENT(obd, precleanup); - rc = OBP(obd, precleanup)(obd, cleanup_stage); + rc = OBP(obd, precleanup)(obd); return rc; } diff --git a/drivers/staging/lustre/lustre/lmv/lmv_obd.c b/drivers/staging/lustre/lustre/lmv/lmv_obd.c index 2338556..9335ffe 100644 --- a/drivers/staging/lustre/lustre/lmv/lmv_obd.c +++ b/drivers/staging/lustre/lustre/lmv/lmv_obd.c @@ -2623,23 +2623,10 @@ static int lmv_unlink(struct obd_export *exp, struct md_op_data *op_data, goto retry_unlink; } -static int lmv_precleanup(struct obd_device *obd, enum obd_cleanup_stage stage) +static int lmv_precleanup(struct obd_device *obd) { - struct lmv_obd *lmv = &obd->u.lmv; - - switch (stage) { - case OBD_CLEANUP_EARLY: - /* XXX: here should be calling obd_precleanup() down to - * stack. - */ - break; - case OBD_CLEANUP_EXPORTS: - fld_client_debugfs_fini(&lmv->lmv_fld); - lprocfs_obd_cleanup(obd); - break; - default: - break; - } + fld_client_debugfs_fini(&obd->u.lmv.lmv_fld); + lprocfs_obd_cleanup(obd); return 0; } diff --git a/drivers/staging/lustre/lustre/lov/lov_obd.c b/drivers/staging/lustre/lustre/lov/lov_obd.c index 421a0e2..5bae584 100644 --- a/drivers/staging/lustre/lustre/lov/lov_obd.c +++ b/drivers/staging/lustre/lustre/lov/lov_obd.c @@ -825,29 +825,6 @@ int lov_setup(struct obd_device *obd, struct lustre_cfg *lcfg) return rc; } -static int lov_precleanup(struct obd_device *obd, enum obd_cleanup_stage stage) -{ - struct lov_obd *lov = &obd->u.lov; - - switch (stage) { - case OBD_CLEANUP_EARLY: { - int i; - - for (i = 0; i < lov->desc.ld_tgt_count; i++) { - if (!lov->lov_tgts[i] || !lov->lov_tgts[i]->ltd_active) - continue; - obd_precleanup(class_exp2obd(lov->lov_tgts[i]->ltd_exp), - OBD_CLEANUP_EARLY); - } - break; - } - default: - break; - } - - return 0; -} - static int lov_cleanup(struct obd_device *obd) { struct lov_obd *lov = &obd->u.lov; @@ -1399,7 +1376,6 @@ static int lov_quotactl(struct obd_device *obd, struct obd_export *exp, static struct obd_ops lov_obd_ops = { .owner = THIS_MODULE, .setup = lov_setup, - .precleanup = lov_precleanup, .cleanup = lov_cleanup, /*.process_config = lov_process_config,*/ .connect = lov_connect, diff --git a/drivers/staging/lustre/lustre/mdc/mdc_request.c b/drivers/staging/lustre/lustre/mdc/mdc_request.c index 8bf0c5b..eee848d 100644 --- a/drivers/staging/lustre/lustre/mdc/mdc_request.c +++ b/drivers/staging/lustre/lustre/mdc/mdc_request.c @@ -2659,23 +2659,16 @@ static int mdc_init_ea_size(struct obd_export *exp, u32 easize, u32 def_easize) return 0; } -static int mdc_precleanup(struct obd_device *obd, enum obd_cleanup_stage stage) +static int mdc_precleanup(struct obd_device *obd) { - switch (stage) { - case OBD_CLEANUP_EARLY: - break; - case OBD_CLEANUP_EXPORTS: - /* Failsafe, ok if racy */ - if (obd->obd_type->typ_refcnt <= 1) - libcfs_kkuc_group_rem(0, KUC_GRP_HSM); - - obd_cleanup_client_import(obd); - ptlrpc_lprocfs_unregister_obd(obd); - lprocfs_obd_cleanup(obd); + /* Failsafe, ok if racy */ + if (obd->obd_type->typ_refcnt <= 1) + libcfs_kkuc_group_rem(0, KUC_GRP_HSM); - mdc_llog_finish(obd); - break; - } + obd_cleanup_client_import(obd); + ptlrpc_lprocfs_unregister_obd(obd); + lprocfs_obd_cleanup(obd); + mdc_llog_finish(obd); return 0; } diff --git a/drivers/staging/lustre/lustre/mgc/mgc_request.c b/drivers/staging/lustre/lustre/mgc/mgc_request.c index e248f33..a2cd5dd 100644 --- a/drivers/staging/lustre/lustre/mgc/mgc_request.c +++ b/drivers/staging/lustre/lustre/mgc/mgc_request.c @@ -684,35 +684,33 @@ static int mgc_llog_fini(const struct lu_env *env, struct obd_device *obd) } static atomic_t mgc_count = ATOMIC_INIT(0); -static int mgc_precleanup(struct obd_device *obd, enum obd_cleanup_stage stage) +static int mgc_precleanup(struct obd_device *obd) { int rc = 0; int temp; - switch (stage) { - case OBD_CLEANUP_EARLY: - break; - case OBD_CLEANUP_EXPORTS: - if (atomic_dec_and_test(&mgc_count)) { - LASSERT(rq_state & RQ_RUNNING); - /* stop requeue thread */ - temp = RQ_STOP; - } else { - /* wakeup requeue thread to clean our cld */ - temp = RQ_NOW | RQ_PRECLEANUP; - } - spin_lock(&config_list_lock); - rq_state |= temp; - spin_unlock(&config_list_lock); - wake_up(&rq_waitq); - if (temp & RQ_STOP) - wait_for_completion(&rq_exit); - obd_cleanup_client_import(obd); - rc = mgc_llog_fini(NULL, obd); - if (rc != 0) - CERROR("failed to cleanup llogging subsystems\n"); - break; + if (atomic_dec_and_test(&mgc_count)) { + LASSERT(rq_state & RQ_RUNNING); + /* stop requeue thread */ + temp = RQ_STOP; + } else { + /* wakeup requeue thread to clean our cld */ + temp = RQ_NOW | RQ_PRECLEANUP; } + + spin_lock(&config_list_lock); + rq_state |= temp; + spin_unlock(&config_list_lock); + wake_up(&rq_waitq); + + if (temp & RQ_STOP) + wait_for_completion(&rq_exit); + obd_cleanup_client_import(obd); + + rc = mgc_llog_fini(NULL, obd); + if (rc) + CERROR("failed to cleanup llogging subsystems\n"); + return rc; } diff --git a/drivers/staging/lustre/lustre/obdclass/obd_config.c b/drivers/staging/lustre/lustre/obdclass/obd_config.c index 017bdac..9d5530c 100644 --- a/drivers/staging/lustre/lustre/obdclass/obd_config.c +++ b/drivers/staging/lustre/lustre/obdclass/obd_config.c @@ -446,7 +446,7 @@ static int class_cleanup(struct obd_device *obd, struct lustre_cfg *lcfg) LASSERT(obd->obd_self_export); /* Precleanup, we must make sure all exports get destroyed. */ - err = obd_precleanup(obd, OBD_CLEANUP_EXPORTS); + err = obd_precleanup(obd); if (err) CERROR("Precleanup %s returned %d\n", obd->obd_name, err); diff --git a/drivers/staging/lustre/lustre/osc/osc_request.c b/drivers/staging/lustre/lustre/osc/osc_request.c index 9b0b94b..c1f5e24 100644 --- a/drivers/staging/lustre/lustre/osc/osc_request.c +++ b/drivers/staging/lustre/lustre/osc/osc_request.c @@ -2686,47 +2686,33 @@ int osc_setup(struct obd_device *obd, struct lustre_cfg *lcfg) return rc; } -static int osc_precleanup(struct obd_device *obd, enum obd_cleanup_stage stage) +static int osc_precleanup(struct obd_device *obd) { - switch (stage) { - case OBD_CLEANUP_EARLY: { - struct obd_import *imp; - - imp = obd->u.cli.cl_import; - CDEBUG(D_HA, "Deactivating import %s\n", obd->obd_name); - /* ptlrpc_abort_inflight to stop an mds_lov_synchronize */ - ptlrpc_deactivate_import(imp); - spin_lock(&imp->imp_lock); - imp->imp_pingable = 0; - spin_unlock(&imp->imp_lock); - break; + struct client_obd *cli = &obd->u.cli; + + /* LU-464 + * for echo client, export may be on zombie list, wait for + * zombie thread to cull it, because cli.cl_import will be + * cleared in client_disconnect_export(): + * class_export_destroy() -> obd_cleanup() -> + * echo_device_free() -> echo_client_cleanup() -> + * obd_disconnect() -> osc_disconnect() -> + * client_disconnect_export() + */ + obd_zombie_barrier(); + if (cli->cl_writeback_work) { + ptlrpcd_destroy_work(cli->cl_writeback_work); + cli->cl_writeback_work = NULL; } - case OBD_CLEANUP_EXPORTS: { - struct client_obd *cli = &obd->u.cli; - /* LU-464 - * for echo client, export may be on zombie list, wait for - * zombie thread to cull it, because cli.cl_import will be - * cleared in client_disconnect_export(): - * class_export_destroy() -> obd_cleanup() -> - * echo_device_free() -> echo_client_cleanup() -> - * obd_disconnect() -> osc_disconnect() -> - * client_disconnect_export() - */ - obd_zombie_barrier(); - if (cli->cl_writeback_work) { - ptlrpcd_destroy_work(cli->cl_writeback_work); - cli->cl_writeback_work = NULL; - } - if (cli->cl_lru_work) { - ptlrpcd_destroy_work(cli->cl_lru_work); - cli->cl_lru_work = NULL; - } - obd_cleanup_client_import(obd); - ptlrpc_lprocfs_unregister_obd(obd); - lprocfs_obd_cleanup(obd); - break; - } + + if (cli->cl_lru_work) { + ptlrpcd_destroy_work(cli->cl_lru_work); + cli->cl_lru_work = NULL; } + + obd_cleanup_client_import(obd); + ptlrpc_lprocfs_unregister_obd(obd); + lprocfs_obd_cleanup(obd); return 0; } -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:30:57 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:30:57 -0500 Subject: [lustre-devel] [PATCH 27/35] staging: lustre: ptlrpc: reset imp_replay_cursor In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <1478799065-24841-28-git-send-email-jsimmons@infradead.org> From: Hongchao Zhang At client side, the replay cursor using to speed up the lookup of committed open requests in its obd_import should be resetted for normal connection (not reconnection) during recovery. Signed-off-by: Hongchao Zhang Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6802 Reviewed-on: http://review.whamcloud.com/17351 Reviewed-by: Niu Yawei Reviewed-by: Jinshan Xiong Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- drivers/staging/lustre/lustre/ptlrpc/import.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/drivers/staging/lustre/lustre/ptlrpc/import.c b/drivers/staging/lustre/lustre/ptlrpc/import.c index 05fd92d..babb80d 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/import.c +++ b/drivers/staging/lustre/lustre/ptlrpc/import.c @@ -1130,6 +1130,7 @@ static int ptlrpc_connect_interpret(const struct lu_env *env, imp->imp_remote_handle = *lustre_msg_get_handle(request->rq_repmsg); imp->imp_last_replay_transno = 0; + imp->imp_replay_cursor = &imp->imp_committed_list; IMPORT_SET_STATE(imp, LUSTRE_IMP_REPLAY); } else { DEBUG_REQ(D_HA, request, "%s: evicting (reconnect/recover flags not set: %x)", -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:31:04 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:31:04 -0500 Subject: [lustre-devel] [PATCH 34/35] staging: lustre: lnet: add offset for selftest brw In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <1478799065-24841-35-git-send-email-jsimmons@infradead.org> From: Liang Zhen In current lnet selftest, both client and server side bulk have no offset and we can only test page aligned IO, this patch changed this: - user can set brw offset by lst add_test ... brw off=OFFSET ... - offset is only effective on client side so far - to simply implementation, offset needs to be eight bytes aligned Signed-off-by: Liang Zhen Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-5718 Reviewed-on: http://review.whamcloud.com/12496 Reviewed-by: Doug Oucharek Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- drivers/staging/lustre/include/linux/lnet/lnetst.h | 2 + drivers/staging/lustre/lnet/selftest/brw_test.c | 73 ++++++++++++------- drivers/staging/lustre/lnet/selftest/conrpc.c | 8 ++- drivers/staging/lustre/lnet/selftest/framework.c | 2 +- drivers/staging/lustre/lnet/selftest/rpc.c | 19 +++-- drivers/staging/lustre/lnet/selftest/rpc.h | 2 +- drivers/staging/lustre/lnet/selftest/selftest.h | 5 +- 7 files changed, 69 insertions(+), 42 deletions(-) diff --git a/drivers/staging/lustre/include/linux/lnet/lnetst.h b/drivers/staging/lustre/include/linux/lnet/lnetst.h index 4170445..78f825d 100644 --- a/drivers/staging/lustre/include/linux/lnet/lnetst.h +++ b/drivers/staging/lustre/include/linux/lnet/lnetst.h @@ -490,6 +490,8 @@ int blk_size; /* size (bytes) */ int blk_time; /* time of running the test*/ int blk_flags; /* reserved flags */ + int blk_cli_off; /* bulk offset on client */ + int blk_srv_off; /* reserved: bulk offset on server */ } lst_test_bulk_param_t; typedef struct { diff --git a/drivers/staging/lustre/lnet/selftest/brw_test.c b/drivers/staging/lustre/lnet/selftest/brw_test.c index b20c5d3..67b460f 100644 --- a/drivers/staging/lustre/lnet/selftest/brw_test.c +++ b/drivers/staging/lustre/lnet/selftest/brw_test.c @@ -44,6 +44,10 @@ module_param(brw_inject_errors, int, 0644); MODULE_PARM_DESC(brw_inject_errors, "# data errors to inject randomly, zero by default"); +#define BRW_POISON 0xbeefbeefbeefbeefULL +#define BRW_MAGIC 0xeeb0eeb1eeb2eeb3ULL +#define BRW_MSIZE sizeof(u64) + static void brw_client_fini(struct sfw_test_instance *tsi) { @@ -67,6 +71,7 @@ { struct sfw_session *sn = tsi->tsi_batch->bat_session; int flags; + int off; int npg; int len; int opc; @@ -87,6 +92,7 @@ * but we have to keep it for compatibility */ len = npg * PAGE_SIZE; + off = 0; } else { struct test_bulk_req_v1 *breq = &tsi->tsi_u.bulk_v1; @@ -99,9 +105,13 @@ opc = breq->blk_opc; flags = breq->blk_flags; len = breq->blk_len; - npg = (len + PAGE_SIZE - 1) >> PAGE_SHIFT; + off = breq->blk_offset & ~PAGE_MASK; + npg = (off + len + PAGE_SIZE - 1) >> PAGE_SHIFT; } + if (off % BRW_MSIZE) + return -EINVAL; + if (npg > LNET_MAX_IOV || npg <= 0) return -EINVAL; @@ -114,7 +124,7 @@ list_for_each_entry(tsu, &tsi->tsi_units, tsu_list) { bulk = srpc_alloc_bulk(lnet_cpt_of_nid(tsu->tsu_dest.nid), - npg, len, opc == LST_BRW_READ); + off, npg, len, opc == LST_BRW_READ); if (!bulk) { brw_client_fini(tsi); return -ENOMEM; @@ -126,12 +136,7 @@ return 0; } -#define BRW_POISON 0xbeefbeefbeefbeefULL -#define BRW_MAGIC 0xeeb0eeb1eeb2eeb3ULL -#define BRW_MSIZE sizeof(__u64) - -static int -brw_inject_one_error(void) +int brw_inject_one_error(void) { struct timespec64 ts; @@ -147,12 +152,13 @@ } static void -brw_fill_page(struct page *pg, int pattern, __u64 magic) +brw_fill_page(struct page *pg, int off, int len, int pattern, __u64 magic) { - char *addr = page_address(pg); + char *addr = page_address(pg) + off; int i; LASSERT(addr); + LASSERT(!(off % BRW_MSIZE) && !(len % BRW_MSIZE)); if (pattern == LST_BRW_CHECK_NONE) return; @@ -162,14 +168,16 @@ if (pattern == LST_BRW_CHECK_SIMPLE) { memcpy(addr, &magic, BRW_MSIZE); - addr += PAGE_SIZE - BRW_MSIZE; - memcpy(addr, &magic, BRW_MSIZE); + if (len > BRW_MSIZE) { + addr += PAGE_SIZE - BRW_MSIZE; + memcpy(addr, &magic, BRW_MSIZE); + } return; } if (pattern == LST_BRW_CHECK_FULL) { - for (i = 0; i < PAGE_SIZE / BRW_MSIZE; i++) - memcpy(addr + i * BRW_MSIZE, &magic, BRW_MSIZE); + for (i = 0; i < len; i += BRW_MSIZE) + memcpy(addr + i, &magic, BRW_MSIZE); return; } @@ -177,13 +185,14 @@ } static int -brw_check_page(struct page *pg, int pattern, __u64 magic) +brw_check_page(struct page *pg, int off, int len, int pattern, __u64 magic) { - char *addr = page_address(pg); + char *addr = page_address(pg) + off; __u64 data = 0; /* make compiler happy */ int i; LASSERT(addr); + LASSERT(!(off % BRW_MSIZE) && !(len % BRW_MSIZE)); if (pattern == LST_BRW_CHECK_NONE) return 0; @@ -193,21 +202,21 @@ if (data != magic) goto bad_data; - addr += PAGE_SIZE - BRW_MSIZE; - data = *((__u64 *)addr); - if (data != magic) - goto bad_data; - + if (len > BRW_MSIZE) { + addr += PAGE_SIZE - BRW_MSIZE; + data = *((__u64 *)addr); + if (data != magic) + goto bad_data; + } return 0; } if (pattern == LST_BRW_CHECK_FULL) { - for (i = 0; i < PAGE_SIZE / BRW_MSIZE; i++) { - data = *(((__u64 *)addr) + i); + for (i = 0; i < len; i += BRW_MSIZE) { + data = *(u64 *)(addr + i); if (data != magic) goto bad_data; } - return 0; } @@ -226,8 +235,12 @@ struct page *pg; for (i = 0; i < bk->bk_niov; i++) { + int off, len; + pg = bk->bk_iovs[i].bv_page; - brw_fill_page(pg, pattern, magic); + off = bk->bk_iovs[i].bv_offset; + len = bk->bk_iovs[i].bv_len; + brw_fill_page(pg, off, len, pattern, magic); } } @@ -238,8 +251,12 @@ struct page *pg; for (i = 0; i < bk->bk_niov; i++) { + int off, len; + pg = bk->bk_iovs[i].bv_page; - if (brw_check_page(pg, pattern, magic)) { + off = bk->bk_iovs[i].bv_offset; + len = bk->bk_iovs[i].bv_len; + if (brw_check_page(pg, off, len, pattern, magic)) { CERROR("Bulk page %p (%d/%d) is corrupted!\n", pg, i, bk->bk_niov); return 1; @@ -276,6 +293,7 @@ len = npg * PAGE_SIZE; } else { struct test_bulk_req_v1 *breq = &tsi->tsi_u.bulk_v1; + int off; /* * I should never get this step if it's unknown feature @@ -286,7 +304,8 @@ opc = breq->blk_opc; flags = breq->blk_flags; len = breq->blk_len; - npg = (len + PAGE_SIZE - 1) >> PAGE_SHIFT; + off = breq->blk_offset; + npg = (off + len + PAGE_SIZE - 1) >> PAGE_SHIFT; } rc = sfw_create_test_rpc(tsu, dest, sn->sn_features, npg, len, &rpc); diff --git a/drivers/staging/lustre/lnet/selftest/conrpc.c b/drivers/staging/lustre/lnet/selftest/conrpc.c index 55afb53..8a3a2da 100644 --- a/drivers/staging/lustre/lnet/selftest/conrpc.c +++ b/drivers/staging/lustre/lnet/selftest/conrpc.c @@ -789,14 +789,15 @@ void lstcon_rpc_stat_reply(struct lstcon_rpc_trans *, struct srpc_msg *, } static int -lstcon_bulkrpc_v1_prep(lst_test_bulk_param_t *param, struct srpc_test_reqst *req) +lstcon_bulkrpc_v1_prep(lst_test_bulk_param_t *param, bool is_client, + struct srpc_test_reqst *req) { struct test_bulk_req_v1 *brq = &req->tsr_u.bulk_v1; brq->blk_opc = param->blk_opc; brq->blk_flags = param->blk_flags; brq->blk_len = param->blk_size; - brq->blk_offset = 0; /* reserved */ + brq->blk_offset = is_client ? param->blk_cli_off : param->blk_srv_off; return 0; } @@ -897,7 +898,8 @@ void lstcon_rpc_stat_reply(struct lstcon_rpc_trans *, struct srpc_msg *, &test->tes_param[0], trq); } else { rc = lstcon_bulkrpc_v1_prep((lst_test_bulk_param_t *) - &test->tes_param[0], trq); + &test->tes_param[0], + trq->tsr_is_client, trq); } break; diff --git a/drivers/staging/lustre/lnet/selftest/framework.c b/drivers/staging/lustre/lnet/selftest/framework.c index abbd628..2774327 100644 --- a/drivers/staging/lustre/lnet/selftest/framework.c +++ b/drivers/staging/lustre/lnet/selftest/framework.c @@ -1101,7 +1101,7 @@ LASSERT(!rpc->srpc_bulk); LASSERT(npages > 0 && npages <= LNET_MAX_IOV); - rpc->srpc_bulk = srpc_alloc_bulk(cpt, npages, len, sink); + rpc->srpc_bulk = srpc_alloc_bulk(cpt, 0, npages, len, sink); if (!rpc->srpc_bulk) return -ENOMEM; diff --git a/drivers/staging/lustre/lnet/selftest/rpc.c b/drivers/staging/lustre/lnet/selftest/rpc.c index f5619d8..0498c56 100644 --- a/drivers/staging/lustre/lnet/selftest/rpc.c +++ b/drivers/staging/lustre/lnet/selftest/rpc.c @@ -84,14 +84,13 @@ void srpc_set_counters(const srpc_counters_t *cnt) } static int -srpc_add_bulk_page(struct srpc_bulk *bk, struct page *pg, int i, int nob) +srpc_add_bulk_page(struct srpc_bulk *bk, struct page *pg, int i, int off, + int nob) { - nob = min_t(int, nob, PAGE_SIZE); + LASSERT(off < PAGE_SIZE); + LASSERT(nob > 0 && nob <= PAGE_SIZE); - LASSERT(nob > 0); - LASSERT(i >= 0 && i < bk->bk_niov); - - bk->bk_iovs[i].bv_offset = 0; + bk->bk_iovs[i].bv_offset = off; bk->bk_iovs[i].bv_page = pg; bk->bk_iovs[i].bv_len = nob; return nob; @@ -117,7 +116,8 @@ void srpc_set_counters(const srpc_counters_t *cnt) } struct srpc_bulk * -srpc_alloc_bulk(int cpt, unsigned bulk_npg, unsigned bulk_len, int sink) +srpc_alloc_bulk(int cpt, unsigned int bulk_off, unsigned int bulk_npg, + unsigned int bulk_len, int sink) { struct srpc_bulk *bk; int i; @@ -148,8 +148,11 @@ struct srpc_bulk * return NULL; } - nob = srpc_add_bulk_page(bk, pg, i, bulk_len); + nob = min_t(unsigned int, bulk_off + bulk_len, PAGE_SIZE) - + bulk_off; + srpc_add_bulk_page(bk, pg, i, bulk_off, nob); bulk_len -= nob; + bulk_off = 0; } return bk; diff --git a/drivers/staging/lustre/lnet/selftest/rpc.h b/drivers/staging/lustre/lnet/selftest/rpc.h index 4ab2ee2..8a0c18e 100644 --- a/drivers/staging/lustre/lnet/selftest/rpc.h +++ b/drivers/staging/lustre/lnet/selftest/rpc.h @@ -175,7 +175,7 @@ struct test_bulk_req_v1 { __u16 blk_opc; /* bulk operation code */ __u16 blk_flags; /* data check flags */ __u32 blk_len; /* data length */ - __u32 blk_offset; /* reserved: offset */ + __u32 blk_offset; /* offset */ } WIRE_ATTR; struct test_ping_req { diff --git a/drivers/staging/lustre/lnet/selftest/selftest.h b/drivers/staging/lustre/lnet/selftest/selftest.h index d033ac0..877bb36 100644 --- a/drivers/staging/lustre/lnet/selftest/selftest.h +++ b/drivers/staging/lustre/lnet/selftest/selftest.h @@ -434,8 +434,9 @@ struct srpc_client_rpc * void srpc_post_rpc(struct srpc_client_rpc *rpc); void srpc_abort_rpc(struct srpc_client_rpc *rpc, int why); void srpc_free_bulk(struct srpc_bulk *bk); -struct srpc_bulk *srpc_alloc_bulk(int cpt, unsigned bulk_npg, - unsigned bulk_len, int sink); +struct srpc_bulk *srpc_alloc_bulk(int cpt, unsigned int off, + unsigned int bulk_npg, unsigned int bulk_len, + int sink); int srpc_send_rpc(struct swi_workitem *wi); int srpc_send_reply(struct srpc_server_rpc *rpc); int srpc_add_service(struct srpc_service *sv); -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:30:48 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:30:48 -0500 Subject: [lustre-devel] [PATCH 18/35] staging: lustre: osc: fix max_dirty_mb tunable setting limit In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <1478799065-24841-19-git-send-email-jsimmons@infradead.org> From: Gregoire Pichon The OSC tunable max_dirty_mb must be set to a value strictly lower than 2048, as it is assumed by OSS in ofd_grant_alloc() routine. Signed-off-by: Gregoire Pichon Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-7218 Reviewed-on: http://review.whamcloud.com/16652 Reviewed-by: Andreas Dilger Reviewed-by: Bobi Jam Signed-off-by: James Simmons --- drivers/staging/lustre/lustre/osc/lproc_osc.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/drivers/staging/lustre/lustre/osc/lproc_osc.c b/drivers/staging/lustre/lustre/osc/lproc_osc.c index 4421cfe..575b296 100644 --- a/drivers/staging/lustre/lustre/osc/lproc_osc.c +++ b/drivers/staging/lustre/lustre/osc/lproc_osc.c @@ -162,7 +162,7 @@ static ssize_t max_dirty_mb_store(struct kobject *kobj, pages_number *= 1 << (20 - PAGE_SHIFT); /* MB -> pages */ if (pages_number <= 0 || - pages_number > OSC_MAX_DIRTY_MB_MAX << (20 - PAGE_SHIFT) || + pages_number >= OSC_MAX_DIRTY_MB_MAX << (20 - PAGE_SHIFT) || pages_number > totalram_pages / 4) /* 1/4 of RAM */ return -ERANGE; -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:30:46 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:30:46 -0500 Subject: [lustre-devel] [PATCH 16/35] staging: lustre: llite: support SELinux context labelling In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <1478799065-24841-17-git-send-email-jsimmons@infradead.org> From: Andrew Wellington SELinux contexts are applied by the kernel if mount options are not binary. As we don't use any binary mount options in Lustre, remove the binary mount option flag. Signed-off-by: Andrew Wellington Signed-off-by: John L. Hammond Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6950 Reviewed-on: http://review.whamcloud.com/15840 Reviewed-by: Dmitry Eremin Reviewed-by: Sebastien Buisson Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- drivers/staging/lustre/lustre/llite/llite_lib.c | 12 ++++++++++++ drivers/staging/lustre/lustre/obdclass/obd_mount.c | 3 +-- 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c index 4a4741b..609db9b 100644 --- a/drivers/staging/lustre/lustre/llite/llite_lib.c +++ b/drivers/staging/lustre/lustre/llite/llite_lib.c @@ -720,6 +720,18 @@ static int ll_options(char *options, int *flags) *flags &= ~tmp; goto next; } + tmp = ll_set_opt("context", s1, 1); + if (tmp) + goto next; + tmp = ll_set_opt("fscontext", s1, 1); + if (tmp) + goto next; + tmp = ll_set_opt("defcontext", s1, 1); + if (tmp) + goto next; + tmp = ll_set_opt("rootcontext", s1, 1); + if (tmp) + goto next; tmp = ll_set_opt("user_fid2path", s1, LL_SBI_USER_FID2PATH); if (tmp) { *flags |= tmp; diff --git a/drivers/staging/lustre/lustre/obdclass/obd_mount.c b/drivers/staging/lustre/lustre/obdclass/obd_mount.c index 1d13751..2283e92 100644 --- a/drivers/staging/lustre/lustre/obdclass/obd_mount.c +++ b/drivers/staging/lustre/lustre/obdclass/obd_mount.c @@ -1216,8 +1216,7 @@ static void lustre_kill_super(struct super_block *sb) .name = "lustre", .mount = lustre_mount, .kill_sb = lustre_kill_super, - .fs_flags = FS_BINARY_MOUNTDATA | FS_REQUIRES_DEV | - FS_RENAME_DOES_D_MOVE, + .fs_flags = FS_REQUIRES_DEV | FS_RENAME_DOES_D_MOVE, }; MODULE_ALIAS_FS("lustre"); -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:30:54 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 12:30:54 -0500 Subject: [lustre-devel] [PATCH 24/35] staging: lustre: osc: osc_extent should hold refcount to osc_object In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <1478799065-24841-25-git-send-email-jsimmons@infradead.org> From: Jinshan Xiong To avoid a race that osc_extent and osc_object destroy happens on the same time, which causes kernel crash. Signed-off-by: Jinshan Xiong Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-7164 Reviewed-on: http://review.whamcloud.com/16433 Reviewed-by: John L. Hammond Reviewed-by: Patrick Farrell Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- drivers/staging/lustre/lustre/osc/osc_cache.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/drivers/staging/lustre/lustre/osc/osc_cache.c b/drivers/staging/lustre/lustre/osc/osc_cache.c index d17ec32..0ca01b1 100644 --- a/drivers/staging/lustre/lustre/osc/osc_cache.c +++ b/drivers/staging/lustre/lustre/osc/osc_cache.c @@ -360,6 +360,7 @@ static void osc_extent_state_set(struct osc_extent *ext, int state) RB_CLEAR_NODE(&ext->oe_node); ext->oe_obj = obj; + cl_object_get(osc2cl(obj)); atomic_set(&ext->oe_refc, 1); atomic_set(&ext->oe_users, 0); INIT_LIST_HEAD(&ext->oe_link); @@ -398,6 +399,7 @@ static void osc_extent_put(const struct lu_env *env, struct osc_extent *ext) LDLM_LOCK_PUT(ext->oe_dlmlock); ext->oe_dlmlock = NULL; } + cl_object_put(env, osc2cl(ext->oe_obj)); osc_extent_free(ext); } } -- 1.7.1 From jsimmons at infradead.org Thu Nov 10 17:49:18 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 17:49:18 +0000 (GMT) Subject: [lustre-devel] [patch] staging/lustre/osc: indent an if statement In-Reply-To: <20161110125050.GA18577@mwanda> References: <20161110125050.GA18577@mwanda> Message-ID: > We accidentally removed a tab here. Let's add it back, and some curly > braces as well since this is a muti-line indent. Reviewed-by: James Simmons > Signed-off-by: Dan Carpenter > > diff --git a/drivers/staging/lustre/lustre/osc/osc_lock.c b/drivers/staging/lustre/lustre/osc/osc_lock.c > index e337e87..bfc8d38 100644 > --- a/drivers/staging/lustre/lustre/osc/osc_lock.c > +++ b/drivers/staging/lustre/lustre/osc/osc_lock.c > @@ -607,9 +607,11 @@ static int osc_ldlm_glimpse_ast(struct ldlm_lock *dlmlock, void *data) > lvb = req_capsule_server_get(cap, &RMF_DLM_LVB); > result = cl_object_glimpse(env, obj, lvb); > } > - if (!exp_connect_lvb_type(req->rq_export)) > - req_capsule_shrink(&req->rq_pill, &RMF_DLM_LVB, > - sizeof(struct ost_lvb_v1), RCL_SERVER); > + if (!exp_connect_lvb_type(req->rq_export)) { > + req_capsule_shrink(&req->rq_pill, &RMF_DLM_LVB, > + sizeof(struct ost_lvb_v1), > + RCL_SERVER); > + } > cl_object_put(env, obj); > } else { > /* > From jsimmons at infradead.org Thu Nov 10 17:53:31 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 17:53:31 +0000 (GMT) Subject: [lustre-devel] [PATCH] staging: lustre: ldlm: pl_recalc time handling is wrong In-Reply-To: <5149342.oboLeYkUcs@wuerfel> References: <1478573240-12850-1-git-send-email-jsimmons@infradead.org> <1594836.LAbS3nYoXQ@wuerfel> <20161110122108.GA7580@kroah.com> <5149342.oboLeYkUcs@wuerfel> Message-ID: > On Thursday, November 10, 2016 1:21:08 PM CET Greg Kroah-Hartman wrote: > > > > > > the intention here is simply to have the console log keep the > > > same numbers as "date +%s" for absolute values. The patch that > > > James suggested converting everything to ktime_get_seconds() > > > would result in the same intervals that have always been there > > > (until I broke it by using time domains inconsistently), but > > > would mean we could use a u32 type for pl_recalc_time and > > > pl_recalc_period because that doesn't overflow until 136 years > > > after booting. (signed time_t overflows 68 years after 1970, > > > i.e 2038). > > > > So, is this patch correct and should be merged to the tree, or not? > > > > No, I think it's wrong in a different way as before. After my > patch, there were six instances of ktime_get_real_seconds() and > three instances of ktime_get_seconds(), and that was wrong. > > James's patch converts three of the six instances to > ktime_get_seconds(), which is equally wrong, just different. > > We can either convert the six ktime_get_real_seconds() > to ktime_get_seconds(), or convert the four (one was added > in the meantime) ktime_get_seconds() to ktime_get_real_seconds(). > > I'll follow up with a patch for the latter as it is what I > had originally intended. We can also do what James intended > instead. I tought about it and yes ktime_get_real_seconds() is the right way to go due to the debug message. Especially if the node happens to reboot. Having a debug message after a reboot with a smaller time reported in the debug message compared to a earlier debug message might be confusing to someone. From jsimmons at infradead.org Thu Nov 10 18:59:03 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 18:59:03 +0000 (GMT) Subject: [lustre-devel] [PATCH v2] staging: lustre: ldlm: pl_recalc time handling is wrong In-Reply-To: <4839295.T6k2aHkh3K@wuerfel> References: <1478573240-12850-1-git-send-email-jsimmons@infradead.org> <1594836.LAbS3nYoXQ@wuerfel> <20161110122108.GA7580@kroah.com> <4839295.T6k2aHkh3K@wuerfel> Message-ID: > James Simmons reports: > > The ldlm_pool field pl_recalc_time is set to the current > > monotonic clock value but the interval period is calculated > > with the wall clock. This means the interval period will > > always be far larger than the pl_recalc_period, which is > > just a small interval time period. The correct thing to > > do is to use monotomic clock current value instead of the > > wall clocks value when calculating recalc_interval_sec. > > This broke when I converted the 32-bit get_seconds() into > ktime_get_{real_,}seconds() inconsistently. Either > one of those two would have worked, but mixing them > does not. > > Staying with the original intention of the patch, this > changes the ktime_get_seconds() calls into ktime_get_real_seconds(), > using real time instead of mononic time. Reviewed-by: James Simmons > Cc: stable at vger.kernel.org # v4.4+ > Fixes: 8f83409cf238 ("staging/lustre: use 64-bit time for pl_recalc") > Reported-by: James Simmons > Signed-off-by: Arnd Bergmann > --- > v2: James' patch was similarly incomplete to mine, as it only > addressed some of the calls. With this new version, all ktime > accessors use the same time domain. > > diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c > index 19831c555c49..b820309d70e3 100644 > --- a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c > +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c > @@ -356,10 +356,10 @@ static int ldlm_pool_recalc(struct ldlm_pool *pl) > u32 recalc_interval_sec; > int count; > > - recalc_interval_sec = ktime_get_seconds() - pl->pl_recalc_time; > + recalc_interval_sec = ktime_get_real_seconds() - pl->pl_recalc_time; > if (recalc_interval_sec > 0) { > spin_lock(&pl->pl_lock); > - recalc_interval_sec = ktime_get_seconds() - pl->pl_recalc_time; > + recalc_interval_sec = ktime_get_real_seconds() - pl->pl_recalc_time; > > if (recalc_interval_sec > 0) { > /* > @@ -382,7 +382,7 @@ static int ldlm_pool_recalc(struct ldlm_pool *pl) > count); > } > > - recalc_interval_sec = pl->pl_recalc_time - ktime_get_seconds() + > + recalc_interval_sec = pl->pl_recalc_time - ktime_get_real_seconds() + > pl->pl_recalc_period; > if (recalc_interval_sec <= 0) { > /* DEBUG: should be re-removed after LU-4536 is fixed */ > @@ -657,7 +657,7 @@ int ldlm_pool_init(struct ldlm_pool *pl, struct ldlm_namespace *ns, > > spin_lock_init(&pl->pl_lock); > atomic_set(&pl->pl_granted, 0); > - pl->pl_recalc_time = ktime_get_seconds(); > + pl->pl_recalc_time = ktime_get_real_seconds(); > atomic_set(&pl->pl_lock_volume_factor, 1); > > atomic_set(&pl->pl_grant_rate, 0); > > From dan.carpenter at oracle.com Thu Nov 10 19:41:05 2016 From: dan.carpenter at oracle.com (Dan Carpenter) Date: Thu, 10 Nov 2016 22:41:05 +0300 Subject: [lustre-devel] [patch] staging: lustre: lov: crash in error handling Message-ID: <20161110194105.GB18347@mwanda> We accidentally made this unsigned so now the error handling will loop until it crashes. Fixes: 55051039fdad ('staging: lustre: lov: remove LSM from struct lustre_md') Signed-off-by: Dan Carpenter diff --git a/drivers/staging/lustre/lustre/lov/lov_ea.c b/drivers/staging/lustre/lustre/lov/lov_ea.c index 53db170..2e2abed 100644 --- a/drivers/staging/lustre/lustre/lov/lov_ea.c +++ b/drivers/staging/lustre/lustre/lov/lov_ea.c @@ -81,7 +81,7 @@ struct lov_stripe_md *lsm_alloc_plain(u16 stripe_count) size_t oinfo_ptrs_size, lsm_size; struct lov_stripe_md *lsm; struct lov_oinfo *loi; - unsigned int i; + int i; LASSERT(stripe_count <= LOV_MAX_STRIPE_COUNT); From jsimmons at infradead.org Thu Nov 10 19:44:57 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 10 Nov 2016 19:44:57 +0000 (GMT) Subject: [lustre-devel] [patch] staging: lustre: lov: crash in error handling In-Reply-To: <20161110194105.GB18347@mwanda> References: <20161110194105.GB18347@mwanda> Message-ID: > We accidentally made this unsigned so now the error handling will loop > until it crashes. > > Fixes: 55051039fdad ('staging: lustre: lov: remove LSM from struct lustre_md') > Signed-off-by: Dan Carpenter Yep. I just sent a patch to fix that. > diff --git a/drivers/staging/lustre/lustre/lov/lov_ea.c b/drivers/staging/lustre/lustre/lov/lov_ea.c > index 53db170..2e2abed 100644 > --- a/drivers/staging/lustre/lustre/lov/lov_ea.c > +++ b/drivers/staging/lustre/lustre/lov/lov_ea.c > @@ -81,7 +81,7 @@ struct lov_stripe_md *lsm_alloc_plain(u16 stripe_count) > size_t oinfo_ptrs_size, lsm_size; > struct lov_stripe_md *lsm; > struct lov_oinfo *loi; > - unsigned int i; > + int i; > > LASSERT(stripe_count <= LOV_MAX_STRIPE_COUNT); > > From andreas.dilger at intel.com Thu Nov 10 22:31:08 2016 From: andreas.dilger at intel.com (Dilger, Andreas) Date: Thu, 10 Nov 2016 22:31:08 +0000 Subject: [lustre-devel] [PATCH] staging: lustre: lov: Fix signed wrap around when decrementing index 'i' In-Reply-To: <20161110141910.25095-1-colin.king@canonical.com> References: <20161110141910.25095-1-colin.king@canonical.com> Message-ID: <1E6549EE-799C-45ED-A9FA-B9B362F51351@intel.com> On Nov 10, 2016, at 07:19, Colin King wrote: > > From: Colin Ian King > > Change predecrement compare to post decrement compare to avoid an > unsigned integer wrap-around comparisomn when decrementing in the while > loop. > > Issue found with static analysis with CoverityScan, CID 1375917 Thanks for the patch. Seems this change has gotten a lot of attention, this is the third patch to the list to fix it. My preference is to undo the int->unsigned declaration change, for which James has already submitted a patch. Cheers, Andreas > Signed-off-by: Colin Ian King > --- > drivers/staging/lustre/lustre/lov/lov_ea.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/drivers/staging/lustre/lustre/lov/lov_ea.c b/drivers/staging/lustre/lustre/lov/lov_ea.c > index 53db170..399298c 100644 > --- a/drivers/staging/lustre/lustre/lov/lov_ea.c > +++ b/drivers/staging/lustre/lustre/lov/lov_ea.c > @@ -102,7 +102,7 @@ struct lov_stripe_md *lsm_alloc_plain(u16 stripe_count) > return lsm; > > err: > - while (--i >= 0) > + while (i-- > 0) > kmem_cache_free(lov_oinfo_slab, lsm->lsm_oinfo[i]); > kvfree(lsm); > return NULL; > -- > 2.10.2 > From lkp at intel.com Fri Nov 11 04:35:29 2016 From: lkp at intel.com (kbuild test robot) Date: Fri, 11 Nov 2016 12:35:29 +0800 Subject: [lustre-devel] [PATCH 05/35] staging: lustre: llog: fix wrong offset in llog_process_thread() In-Reply-To: <1478799065-24841-6-git-send-email-jsimmons@infradead.org> Message-ID: <201611111201.gguisz8v%fengguang.wu@intel.com> Hi Mikhail, [auto build test ERROR on staging/staging-testing] [also build test ERROR on next-20161110] [cannot apply to v4.9-rc4] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url: https://github.com/0day-ci/linux/commits/James-Simmons/second-batch-of-missing-lustre-2-8-patches/20161111-021225 config: m68k-allmodconfig (attached as .config) compiler: m68k-linux-gcc (GCC) 4.9.0 reproduce: wget https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/plain/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross # save the attached .config to linux build tree make.cross ARCH=m68k All errors (new ones prefixed by >>): >> ERROR: "__umoddi3" [drivers/staging/lustre/lustre/obdclass/obdclass.ko] undefined! --- 0-DAY kernel test infrastructure Open Source Technology Center https://lists.01.org/pipermail/kbuild-all Intel Corporation -------------- next part -------------- A non-text attachment was scrubbed... Name: .config.gz Type: application/gzip Size: 38321 bytes Desc: not available URL: From lkp at intel.com Fri Nov 11 05:16:58 2016 From: lkp at intel.com (kbuild test robot) Date: Fri, 11 Nov 2016 13:16:58 +0800 Subject: [lustre-devel] [PATCH 06/35] staging: lustre: osc: Performance tune for LRU In-Reply-To: <1478799065-24841-7-git-send-email-jsimmons@infradead.org> Message-ID: <201611111328.fDXHD6zF%fengguang.wu@intel.com> Hi Jinshan, [auto build test ERROR on staging/staging-testing] [also build test ERROR on next-20161110] [cannot apply to v4.9-rc4] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url: https://github.com/0day-ci/linux/commits/James-Simmons/second-batch-of-missing-lustre-2-8-patches/20161111-021225 config: m68k-allmodconfig (attached as .config) compiler: m68k-linux-gcc (GCC) 4.9.0 reproduce: wget https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/plain/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross # save the attached .config to linux build tree make.cross ARCH=m68k All errors (new ones prefixed by >>): >> ERROR: "__divdi3" [drivers/staging/lustre/lustre/osc/osc.ko] undefined! ERROR: "__umoddi3" [drivers/staging/lustre/lustre/obdclass/obdclass.ko] undefined! --- 0-DAY kernel test infrastructure Open Source Technology Center https://lists.01.org/pipermail/kbuild-all Intel Corporation -------------- next part -------------- A non-text attachment was scrubbed... Name: .config.gz Type: application/gzip Size: 38321 bytes Desc: not available URL: From gregkh at linuxfoundation.org Mon Nov 14 14:59:51 2016 From: gregkh at linuxfoundation.org (Greg Kroah-Hartman) Date: Mon, 14 Nov 2016 15:59:51 +0100 Subject: [lustre-devel] [PATCH 31/35] staging: lustre: obdclass: add export for lprocfs_stats_alloc_one() In-Reply-To: <1478799065-24841-32-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> <1478799065-24841-32-git-send-email-jsimmons@infradead.org> Message-ID: <20161114145951.GA4245@kroah.com> On Thu, Nov 10, 2016 at 12:31:01PM -0500, James Simmons wrote: > From: Chennaiah Palla > > When compiling the kernel without optimization, when using GCOV, > the lprocfs_stats_alloc_one() symbol is not properly exported to > other modules and causes the ptlrpc module to fail loading with > an unknown symbol. Added EXPORT_SYMBOL(lprocfs_stats_alloc_one) > so that this works properly. No, let's fix this properly. Please get rid of lprocfs_stats_lock(), or if you _really_ need it, move it to a .c file. Having it in a .h file is just a mess, as this proves. thanks, greg k-h From gregkh at linuxfoundation.org Mon Nov 14 15:12:46 2016 From: gregkh at linuxfoundation.org (Greg Kroah-Hartman) Date: Mon, 14 Nov 2016 16:12:46 +0100 Subject: [lustre-devel] [PATCH 32/35] staging: lustre: mount: fix lmd_parse() to handle commas in expr_list In-Reply-To: <1478799065-24841-33-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> <1478799065-24841-33-git-send-email-jsimmons@infradead.org> Message-ID: <20161114151246.GA29168@kroah.com> On Thu, Nov 10, 2016 at 12:31:02PM -0500, James Simmons wrote: > From: Jian Yu > > The lmd_parse() function parses mount options with comma as > delimiter without considering commas in expr_list as follows > is a valid LNET nid range syntax: > > :== '[' [ ',' ] ']' > > This patch fixes the above issue by using cfs_parse_nidlist() > to parse nid range list instead of using class_parse_nid_quiet() > to parse only one nid. ugh, parsing mount strings in the kernel in odd ways, what could ever go wrong... > > Signed-off-by: Jian Yu > Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-5690 > Reviewed-on: http://review.whamcloud.com/17036 > Reviewed-by: Niu Yawei > Reviewed-by: Bob Glossman > Reviewed-by: Oleg Drokin > Signed-off-by: James Simmons > --- > drivers/staging/lustre/lustre/obdclass/obd_mount.c | 91 ++++++++++++++++++-- > 1 files changed, 85 insertions(+), 6 deletions(-) > > diff --git a/drivers/staging/lustre/lustre/obdclass/obd_mount.c b/drivers/staging/lustre/lustre/obdclass/obd_mount.c > index 2283e92..1eb8e71 100644 > --- a/drivers/staging/lustre/lustre/obdclass/obd_mount.c > +++ b/drivers/staging/lustre/lustre/obdclass/obd_mount.c > @@ -871,6 +871,87 @@ static int lmd_parse_mgs(struct lustre_mount_data *lmd, char **ptr) > return 0; > } > > +/** > + * Find the first comma delimiter from the specified \a buf and make \a *endh > + * point to the string starting with the comma. The commas in expression list > + * [...] will be skipped. > + * > + * \param[in] buf a comma-separated string > + * \param[in] endh a pointer to a pointer that will point to the string > + * starting with the comma Please drop this mess of \param, it's not needed and is not kernel-doc format. > + * > + * \retval 0 if comma delimiter is found > + * \retval 1 if comma delimiter is not found > + */ > +static int lmd_find_comma(char *buf, char **endh) > +{ > + char *c = buf; > + int skip = 0; > + > + if (!buf) > + return 1; > + > + while (*c != '\0') { > + if (*c == '[') > + skip++; > + else if (*c == ']') > + skip--; > + > + if (*c == ',' && !skip) { > + if (endh) > + *endh = c; > + return 0; > + } > + c++; > + } > + return 1; > +} Don't we have a standard string search function for finding a string in a string already in the kernel? Why write another one? Or better yet, why are you using such a crazy string in the first place from userspace? Please fix this up and resend. thanks, greg k-h From gregkh at linuxfoundation.org Mon Nov 14 15:16:50 2016 From: gregkh at linuxfoundation.org (Greg Kroah-Hartman) Date: Mon, 14 Nov 2016 16:16:50 +0100 Subject: [lustre-devel] [PATCH 00/35] second batch of missing lustre 2.8 patches In-Reply-To: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> Message-ID: <20161114151650.GA12605@kroah.com> On Thu, Nov 10, 2016 at 12:30:30PM -0500, James Simmons wrote: > More fixes missing from the upstream client. Also a nice cleanup > with the removal of cl_req which is no longer needed. More cleanup > for lustre_idl.h which is a uapi header. Like the last batch these > patches are independent of order. I didn't apply a few of these (string parsing stuff, and build breakages.) What's the plan for getting this out of staging? I feel like you all are still adding new features along with these "cleanups". Normally that's fine, but I really really want to get this out of staging as it's been there for way too long. When is that going to happen? thanks, greg k-h From gregkh at linuxfoundation.org Mon Nov 14 15:17:37 2016 From: gregkh at linuxfoundation.org (Greg Kroah-Hartman) Date: Mon, 14 Nov 2016 16:17:37 +0100 Subject: [lustre-devel] [PATCH v4] staging: lustre: mdc: manage number of modify RPCs in flight In-Reply-To: <1478793073-21814-1-git-send-email-jsimmons@infradead.org> References: <1478793073-21814-1-git-send-email-jsimmons@infradead.org> Message-ID: <20161114151737.GA18183@kroah.com> On Thu, Nov 10, 2016 at 10:51:13AM -0500, James Simmons wrote: > From: Gregoire Pichon > > This patch is the main client part of a new feature that supports > multiple modify metadata RPCs in parallel. Its goal is to improve > metadata operations performance of a single client, while maintening > the consistency of MDT reply reconstruction and MDT recovery > mechanisms. > > It allows to manage the number of modify RPCs in flight within > the client obd structure and to assign a virtual index (the tag) to > each modify RPC to help server side cleaning of reply data. > > The mdc component uses this feature to send multiple modify RPCs > in parallel. Is this a new feature? Why should we take this now and not just wait until the code is out of staging? thanks, greg k-h From jsimmons at infradead.org Mon Nov 14 16:59:48 2016 From: jsimmons at infradead.org (James Simmons) Date: Mon, 14 Nov 2016 16:59:48 +0000 (GMT) Subject: [lustre-devel] [PATCH v4] staging: lustre: mdc: manage number of modify RPCs in flight In-Reply-To: <20161114151737.GA18183@kroah.com> References: <1478793073-21814-1-git-send-email-jsimmons@infradead.org> <20161114151737.GA18183@kroah.com> Message-ID: > On Thu, Nov 10, 2016 at 10:51:13AM -0500, James Simmons wrote: > > From: Gregoire Pichon > > > > This patch is the main client part of a new feature that supports > > multiple modify metadata RPCs in parallel. Its goal is to improve > > metadata operations performance of a single client, while maintening > > the consistency of MDT reply reconstruction and MDT recovery > > mechanisms. > > > > It allows to manage the number of modify RPCs in flight within > > the client obd structure and to assign a virtual index (the tag) to > > each modify RPC to help server side cleaning of reply data. > > > > The mdc component uses this feature to send multiple modify RPCs > > in parallel. > > Is this a new feature? Why should we take this now and not just wait > until the code is out of staging? Yes on the server side. So the problem on our meta data servers couldn't handle writing mulitiple bits of data to the back end disk at ths same time. One client side the issue was the metadata operations were being serialized by a mutex in the MDC layer. That is what this patch fixed. So for the client it would be a performance improvement patch. From gregkh at linuxfoundation.org Mon Nov 14 17:20:47 2016 From: gregkh at linuxfoundation.org (Greg Kroah-Hartman) Date: Mon, 14 Nov 2016 18:20:47 +0100 Subject: [lustre-devel] [PATCH v4] staging: lustre: mdc: manage number of modify RPCs in flight In-Reply-To: References: <1478793073-21814-1-git-send-email-jsimmons@infradead.org> <20161114151737.GA18183@kroah.com> Message-ID: <20161114172047.GA4989@kroah.com> On Mon, Nov 14, 2016 at 04:59:48PM +0000, James Simmons wrote: > > > On Thu, Nov 10, 2016 at 10:51:13AM -0500, James Simmons wrote: > > > From: Gregoire Pichon > > > > > > This patch is the main client part of a new feature that supports > > > multiple modify metadata RPCs in parallel. Its goal is to improve > > > metadata operations performance of a single client, while maintening > > > the consistency of MDT reply reconstruction and MDT recovery > > > mechanisms. > > > > > > It allows to manage the number of modify RPCs in flight within > > > the client obd structure and to assign a virtual index (the tag) to > > > each modify RPC to help server side cleaning of reply data. > > > > > > The mdc component uses this feature to send multiple modify RPCs > > > in parallel. > > > > Is this a new feature? Why should we take this now and not just wait > > until the code is out of staging? > > Yes on the server side. So the problem on our meta data servers couldn't > handle writing mulitiple bits of data to the back end disk at ths same > time. > > One client side the issue was the metadata operations were being > serialized by a mutex in the MDC layer. That is what this patch fixed. > So for the client it would be a performance improvement patch. So, it's a "performance" patch, which isn't functionality, so why should we merge this to staging now? Why aren't people working on the known coding issues to get this out of staging instead of working on performance stuff? thanks, greg k-h From jsimmons at infradead.org Mon Nov 14 17:27:00 2016 From: jsimmons at infradead.org (James Simmons) Date: Mon, 14 Nov 2016 17:27:00 +0000 (GMT) Subject: [lustre-devel] [PATCH v4] staging: lustre: mdc: manage number of modify RPCs in flight In-Reply-To: <20161114172047.GA4989@kroah.com> References: <1478793073-21814-1-git-send-email-jsimmons@infradead.org> <20161114151737.GA18183@kroah.com> <20161114172047.GA4989@kroah.com> Message-ID: > On Mon, Nov 14, 2016 at 04:59:48PM +0000, James Simmons wrote: > > > > > On Thu, Nov 10, 2016 at 10:51:13AM -0500, James Simmons wrote: > > > > From: Gregoire Pichon > > > > > > > > This patch is the main client part of a new feature that supports > > > > multiple modify metadata RPCs in parallel. Its goal is to improve > > > > metadata operations performance of a single client, while maintening > > > > the consistency of MDT reply reconstruction and MDT recovery > > > > mechanisms. > > > > > > > > It allows to manage the number of modify RPCs in flight within > > > > the client obd structure and to assign a virtual index (the tag) to > > > > each modify RPC to help server side cleaning of reply data. > > > > > > > > The mdc component uses this feature to send multiple modify RPCs > > > > in parallel. > > > > > > Is this a new feature? Why should we take this now and not just wait > > > until the code is out of staging? > > > > Yes on the server side. So the problem on our meta data servers couldn't > > handle writing mulitiple bits of data to the back end disk at ths same > > time. > > > > One client side the issue was the metadata operations were being > > serialized by a mutex in the MDC layer. That is what this patch fixed. > > So for the client it would be a performance improvement patch. > > So, it's a "performance" patch, which isn't functionality, so why should > we merge this to staging now? Why aren't people working on the known > coding issues to get this out of staging instead of working on > performance stuff? Because the primary goal which the people at my company, not Intel by the way, wanted was to get this to what is running in production environments so people would actually use the staging client. When it was old and broken no one would touch it with a 10 foot pole. The currently supported verison in production is lustre 2.8.0 so we are only about 30 patches away from reaching the goal. I'm going to respond to your other email in length about leaving staging. It would be really nice for the user base if we can reach that goal. From jsimmons at infradead.org Mon Nov 14 18:27:57 2016 From: jsimmons at infradead.org (James Simmons) Date: Mon, 14 Nov 2016 18:27:57 +0000 (GMT) Subject: [lustre-devel] [PATCH 00/35] second batch of missing lustre 2.8 patches In-Reply-To: <20161114151650.GA12605@kroah.com> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> <20161114151650.GA12605@kroah.com> Message-ID: > On Thu, Nov 10, 2016 at 12:30:30PM -0500, James Simmons wrote: > > More fixes missing from the upstream client. Also a nice cleanup > > with the removal of cl_req which is no longer needed. More cleanup > > for lustre_idl.h which is a uapi header. Like the last batch these > > patches are independent of order. > > I didn't apply a few of these (string parsing stuff, and build > breakages.) > > What's the plan for getting this out of staging? I feel like you all > are still adding new features along with these "cleanups". Normally > that's fine, but I really really want to get this out of staging as it's > been there for way too long. When is that going to happen? First I should address why the push to bring it into sync with our prouction code base. One was to make it attractive to our user base. In my other email I explained that. Second the feed back here has been so valuable. We are at the point where bugs we haven't found are being reported here and addressed. Also your input has made the Lustre developers to reflect on what we have done. In a way leaving staging will be sad since that will stop :-( Lastly I really didn't want to cleanup the lustre client and then when it left staging do this massive dump of changes without people like Julia, Dan and you looking at it. I just felt that wouldn't of been right. We are super close to reaching a very important mile stone of reaching lustre 2.8.0 level of suppport. At this point we can stop at our lustre 2.8.0 release for the sync since currently most the lustre users out their are staying at that version. Secondly the major of patches landed to our soon to be release 2.9 version was for the patches missing from the staging tree. So before we consider moving out of staging some gaps need to be filled. The zero day system has found bugs on platforms we don't have access too. We really need to hook into that system. Also Julia Lawall Coccinelle work has been really wonderful. Intel does have a mirror of your tree and have started the integration of the test harness. For my work I have using a local private test harness setup. This Intel one is planned for public consumption. What is done here with Coccinelle and zero day needs to be intergrated. So this is what needs to be done from our side. Now for what is required to leave the staging tree. Honestly I can think of many many many things that need to be done. The question becomes what has to be done before leaving staging verses what can be completed after leaving staging. We have the normal style issues and checkpatch issues which is not much anymore. Then their is the uapi header cleanup. What else beyond that? From gregkh at linuxfoundation.org Tue Nov 15 10:00:35 2016 From: gregkh at linuxfoundation.org (Greg Kroah-Hartman) Date: Tue, 15 Nov 2016 11:00:35 +0100 Subject: [lustre-devel] [PATCH 00/35] second batch of missing lustre 2.8 patches In-Reply-To: References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> <20161114151650.GA12605@kroah.com> Message-ID: <20161115100035.GA5376@kroah.com> On Mon, Nov 14, 2016 at 06:27:57PM +0000, James Simmons wrote: > > > On Thu, Nov 10, 2016 at 12:30:30PM -0500, James Simmons wrote: > > > More fixes missing from the upstream client. Also a nice cleanup > > > with the removal of cl_req which is no longer needed. More cleanup > > > for lustre_idl.h which is a uapi header. Like the last batch these > > > patches are independent of order. > > > > I didn't apply a few of these (string parsing stuff, and build > > breakages.) > > > > What's the plan for getting this out of staging? I feel like you all > > are still adding new features along with these "cleanups". Normally > > that's fine, but I really really want to get this out of staging as it's > > been there for way too long. When is that going to happen? > > First I should address why the push to bring it into sync with our > prouction code base. One was to make it attractive to our user base. > In my other email I explained that. Yes, but remember, it's not _our_ fault your have a diverged source code base, that's yours :) > Second the feed back here has been > so valuable. We are at the point where bugs we haven't found are being > reported here and addressed. Also your input has made the Lustre > developers to reflect on what we have done. In a way leaving staging will > be sad since that will stop :-( Why do you think that once the code is out of staging people will stop reviewing your patches? You still have to get patches in through the vfs maintainer, and really, I've never heard anyone say that getting patches reviewed on linux-fsdevel is "easier" than the staging mailing list... > Lastly I really didn't want to cleanup the lustre client and then when > it left staging do this massive dump of changes without people like > Julia, Dan and you looking at it. I just felt that wouldn't of been > right. You wouldn't be allowed to do a "dump", you still have to follow the same rules if you are in staging or in fs/ > We are super close to reaching a very important mile stone of reaching > lustre 2.8.0 level of suppport. At this point we can stop at our lustre > 2.8.0 release for the sync since currently most the lustre users out their > are staying at that version. Secondly the major of patches landed to our > soon to be release 2.9 version was for the patches missing from the > staging tree. I have no idea what your numbering scheme is, nor how it relates to the kernel release numbers, nor do I really want to learn it :) > So before we consider moving out of staging some gaps need to be filled. > The zero day system has found bugs on platforms we don't have access too. Good, you will have access to that once you are in fs/ as well. > We really need to hook into that system. Also Julia Lawall Coccinelle > work has been really wonderful. Intel does have a mirror of your tree and > have started the integration of the test harness. For my work I have > using a local private test harness setup. This Intel one is planned for > public consumption. What is done here with Coccinelle and zero day needs > to be intergrated. So this is what needs to be done from our side. Both of that can happen today with your own trees, no need to be in drivers/staging/ or in fs/, so please don't think that matters. > Now for what is required to leave the staging tree. Honestly I can think > of many many many things that need to be done. The question becomes what > has to be done before leaving staging verses what can be completed after > leaving staging. We have the normal style issues and checkpatch issues > which is not much anymore. Then their is the uapi header cleanup. What > else beyond that? Why not do these basic things first and then we can worry about the rest? The fact that there is still such basic cleanups needed is worrying to me... thanks, greg k-h From gregkh at linuxfoundation.org Tue Nov 15 10:03:06 2016 From: gregkh at linuxfoundation.org (Greg Kroah-Hartman) Date: Tue, 15 Nov 2016 11:03:06 +0100 Subject: [lustre-devel] [PATCH v4] staging: lustre: mdc: manage number of modify RPCs in flight In-Reply-To: References: <1478793073-21814-1-git-send-email-jsimmons@infradead.org> <20161114151737.GA18183@kroah.com> <20161114172047.GA4989@kroah.com> Message-ID: <20161115100306.GA23274@kroah.com> On Mon, Nov 14, 2016 at 05:27:00PM +0000, James Simmons wrote: > > > On Mon, Nov 14, 2016 at 04:59:48PM +0000, James Simmons wrote: > > > > > > > On Thu, Nov 10, 2016 at 10:51:13AM -0500, James Simmons wrote: > > > > > From: Gregoire Pichon > > > > > > > > > > This patch is the main client part of a new feature that supports > > > > > multiple modify metadata RPCs in parallel. Its goal is to improve > > > > > metadata operations performance of a single client, while maintening > > > > > the consistency of MDT reply reconstruction and MDT recovery > > > > > mechanisms. > > > > > > > > > > It allows to manage the number of modify RPCs in flight within > > > > > the client obd structure and to assign a virtual index (the tag) to > > > > > each modify RPC to help server side cleaning of reply data. > > > > > > > > > > The mdc component uses this feature to send multiple modify RPCs > > > > > in parallel. > > > > > > > > Is this a new feature? Why should we take this now and not just wait > > > > until the code is out of staging? > > > > > > Yes on the server side. So the problem on our meta data servers couldn't > > > handle writing mulitiple bits of data to the back end disk at ths same > > > time. > > > > > > One client side the issue was the metadata operations were being > > > serialized by a mutex in the MDC layer. That is what this patch fixed. > > > So for the client it would be a performance improvement patch. > > > > So, it's a "performance" patch, which isn't functionality, so why should > > we merge this to staging now? Why aren't people working on the known > > coding issues to get this out of staging instead of working on > > performance stuff? > > Because the primary goal which the people at my company, not Intel by > the way, wanted was to get this to what is running in production > environments so people would actually use the staging client. When it was > old and broken no one would touch it with a 10 foot pole. The currently > supported verison in production is lustre 2.8.0 so we are only about 30 > patches away from reaching the goal. I'm going to respond to your other > email in length about leaving staging. It would be really nice for the > user base if we can reach that goal. Ugh, ok, I'll take this, but PLEASE start working to get this tree out of staging and stop working on adding new features. The longer you delay, the more you have to deal with this external/internal tree mess. greg k-h From green at linuxhacker.ru Thu Nov 17 01:05:29 2016 From: green at linuxhacker.ru (Oleg Drokin) Date: Wed, 16 Nov 2016 20:05:29 -0500 Subject: [lustre-devel] [PATCH 1/2] staging/lustre/ptlrpc: track unreplied requests In-Reply-To: <1479344730-1131617-1-git-send-email-green@linuxhacker.ru> References: <1479344730-1131617-1-git-send-email-green@linuxhacker.ru> Message-ID: <1479344730-1131617-2-git-send-email-green@linuxhacker.ru> From: Niu Yawei The request xid was used to make sure the ost object timestamps being updated by the out of order setattr/punch/write requests properly. However, this mechanism is broken by the multiple rcvd slot feature, where we deferred the xid assignment from request packing to request sending. This patch moved back the xid assignment to request packing, and the manner of finding lowest unreplied xid is changed from scan sending & delay list to scan a unreplied requests list. This patch also skipped packing the known replied XID in connect and disconnect request, so that we can make sure the known replied XID is increased only on both server & client side. Signed-off-by: Niu Yawei Reviewed-on: http://review.whamcloud.com/16759 Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-5951 Reviewed-by: Gregoire Pichon Reviewed-by: Alex Zhuravlev Signed-off-by: Oleg Drokin --- .../staging/lustre/lustre/include/lustre_import.h | 5 + drivers/staging/lustre/lustre/include/lustre_net.h | 3 + drivers/staging/lustre/lustre/obdclass/genops.c | 2 + drivers/staging/lustre/lustre/ptlrpc/client.c | 112 +++++++++++++++------ drivers/staging/lustre/lustre/ptlrpc/import.c | 34 +++++++ drivers/staging/lustre/lustre/ptlrpc/niobuf.c | 29 +++++- .../staging/lustre/lustre/ptlrpc/ptlrpc_internal.h | 24 +++++ drivers/staging/lustre/lustre/ptlrpc/recover.c | 12 +++ 8 files changed, 185 insertions(+), 36 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre_import.h b/drivers/staging/lustre/lustre/include/lustre_import.h index 5461ba3..4499c69 100644 --- a/drivers/staging/lustre/lustre/include/lustre_import.h +++ b/drivers/staging/lustre/lustre/include/lustre_import.h @@ -185,6 +185,11 @@ struct obd_import { struct list_head *imp_replay_cursor; /** @} */ + /** List of not replied requests */ + struct list_head imp_unreplied_list; + /** Known maximal replied XID */ + __u64 imp_known_replied_xid; + /** obd device for this import */ struct obd_device *imp_obd; diff --git a/drivers/staging/lustre/lustre/include/lustre_net.h b/drivers/staging/lustre/lustre/include/lustre_net.h index d2cbec3..2be135d 100644 --- a/drivers/staging/lustre/lustre/include/lustre_net.h +++ b/drivers/staging/lustre/lustre/include/lustre_net.h @@ -596,6 +596,8 @@ struct ptlrpc_cli_req { union ptlrpc_async_args cr_async_args; /** Opaq data for replay and commit callbacks. */ void *cr_cb_data; + /** Link to the imp->imp_unreplied_list */ + struct list_head cr_unreplied_list; /** * Commit callback, called when request is committed and about to be * freed. @@ -635,6 +637,7 @@ struct ptlrpc_cli_req { #define rq_interpret_reply rq_cli.cr_reply_interp #define rq_async_args rq_cli.cr_async_args #define rq_cb_data rq_cli.cr_cb_data +#define rq_unreplied_list rq_cli.cr_unreplied_list #define rq_commit_cb rq_cli.cr_commit_cb #define rq_replay_cb rq_cli.cr_replay_cb diff --git a/drivers/staging/lustre/lustre/obdclass/genops.c b/drivers/staging/lustre/lustre/obdclass/genops.c index 438d619..fa0d38d 100644 --- a/drivers/staging/lustre/lustre/obdclass/genops.c +++ b/drivers/staging/lustre/lustre/obdclass/genops.c @@ -907,6 +907,8 @@ struct obd_import *class_new_import(struct obd_device *obd) INIT_LIST_HEAD(&imp->imp_sending_list); INIT_LIST_HEAD(&imp->imp_delayed_list); INIT_LIST_HEAD(&imp->imp_committed_list); + INIT_LIST_HEAD(&imp->imp_unreplied_list); + imp->imp_known_replied_xid = 0; imp->imp_replay_cursor = &imp->imp_committed_list; spin_lock_init(&imp->imp_lock); imp->imp_last_success_conn = 0; diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c index d2f4cd5..ac959ef 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/client.c +++ b/drivers/staging/lustre/lustre/ptlrpc/client.c @@ -652,6 +652,42 @@ static void __ptlrpc_free_req_to_pool(struct ptlrpc_request *request) spin_unlock(&pool->prp_lock); } +void ptlrpc_add_unreplied(struct ptlrpc_request *req) +{ + struct obd_import *imp = req->rq_import; + struct list_head *tmp; + struct ptlrpc_request *iter; + + assert_spin_locked(&imp->imp_lock); + LASSERT(list_empty(&req->rq_unreplied_list)); + + /* unreplied list is sorted by xid in ascending order */ + list_for_each_prev(tmp, &imp->imp_unreplied_list) { + iter = list_entry(tmp, struct ptlrpc_request, + rq_unreplied_list); + + LASSERT(req->rq_xid != iter->rq_xid); + if (req->rq_xid < iter->rq_xid) + continue; + list_add(&req->rq_unreplied_list, &iter->rq_unreplied_list); + return; + } + list_add(&req->rq_unreplied_list, &imp->imp_unreplied_list); +} + +void ptlrpc_assign_next_xid_nolock(struct ptlrpc_request *req) +{ + req->rq_xid = ptlrpc_next_xid(); + ptlrpc_add_unreplied(req); +} + +static inline void ptlrpc_assign_next_xid(struct ptlrpc_request *req) +{ + spin_lock(&req->rq_import->imp_lock); + ptlrpc_assign_next_xid_nolock(req); + spin_unlock(&req->rq_import->imp_lock); +} + int ptlrpc_request_bufs_pack(struct ptlrpc_request *request, __u32 version, int opcode, char **bufs, struct ptlrpc_cli_ctx *ctx) @@ -701,6 +737,7 @@ int ptlrpc_request_bufs_pack(struct ptlrpc_request *request, ptlrpc_at_set_req_timeout(request); lustre_msg_set_opc(request->rq_reqmsg, opcode); + ptlrpc_assign_next_xid(request); /* Let's setup deadline for req/reply/bulk unlink for opcode. */ if (cfs_fail_val == opcode) { @@ -1230,6 +1267,24 @@ static void ptlrpc_save_versions(struct ptlrpc_request *req) versions[0], versions[1]); } +__u64 ptlrpc_known_replied_xid(struct obd_import *imp) +{ + struct ptlrpc_request *req; + + assert_spin_locked(&imp->imp_lock); + if (list_empty(&imp->imp_unreplied_list)) + return 0; + + req = list_entry(imp->imp_unreplied_list.next, struct ptlrpc_request, + rq_unreplied_list); + LASSERTF(req->rq_xid >= 1, "XID:%llu\n", req->rq_xid); + + if (imp->imp_known_replied_xid < req->rq_xid - 1) + imp->imp_known_replied_xid = req->rq_xid - 1; + + return req->rq_xid - 1; +} + /** * Callback function called when client receives RPC reply for \a req. * Returns 0 on success or error code. @@ -1317,6 +1372,11 @@ static int after_reply(struct ptlrpc_request *req) else req->rq_sent = now + req->rq_nr_resend; + /* Resend for EINPROGRESS will use a new XID */ + spin_lock(&imp->imp_lock); + list_del_init(&req->rq_unreplied_list); + spin_unlock(&imp->imp_lock); + return 0; } @@ -1430,8 +1490,7 @@ static int after_reply(struct ptlrpc_request *req) static int ptlrpc_send_new_req(struct ptlrpc_request *req) { struct obd_import *imp = req->rq_import; - struct list_head *tmp; - u64 min_xid = ~0ULL; + u64 min_xid = 0; int rc; LASSERT(req->rq_phase == RQ_PHASE_NEW); @@ -1451,17 +1510,8 @@ static int ptlrpc_send_new_req(struct ptlrpc_request *req) spin_lock(&imp->imp_lock); - /* - * the very first time we assign XID. it's important to assign XID - * and put it on the list atomically, so that the lowest assigned - * XID is always known. this is vital for multislot last_rcvd - */ - if (req->rq_send_state == LUSTRE_IMP_REPLAY) { - LASSERT(req->rq_xid); - } else { - LASSERT(!req->rq_xid); - req->rq_xid = ptlrpc_next_xid(); - } + LASSERT(req->rq_xid); + LASSERT(!list_empty(&req->rq_unreplied_list)); if (!req->rq_generation_set) req->rq_import_generation = imp->imp_generation; @@ -1493,25 +1543,23 @@ static int ptlrpc_send_new_req(struct ptlrpc_request *req) list_add_tail(&req->rq_list, &imp->imp_sending_list); atomic_inc(&req->rq_import->imp_inflight); - /* find the lowest unreplied XID */ - list_for_each(tmp, &imp->imp_delayed_list) { - struct ptlrpc_request *r; - - r = list_entry(tmp, struct ptlrpc_request, rq_list); - if (r->rq_xid < min_xid) - min_xid = r->rq_xid; - } - list_for_each(tmp, &imp->imp_sending_list) { - struct ptlrpc_request *r; - - r = list_entry(tmp, struct ptlrpc_request, rq_list); - if (r->rq_xid < min_xid) - min_xid = r->rq_xid; - } + /* find the known replied XID from the unreplied list, CONNECT + * and DISCONNECT requests are skipped to make the sanity check + * on server side happy. see process_req_last_xid(). + * + * For CONNECT: Because replay requests have lower XID, it'll + * break the sanity check if CONNECT bump the exp_last_xid on + * server. + * + * For DISCONNECT: Since client will abort inflight RPC before + * sending DISCONNECT, DISCONNECT may carry an XID which higher + * than the inflight RPC. + */ + if (!ptlrpc_req_is_connect(req) && !ptlrpc_req_is_disconnect(req)) + min_xid = ptlrpc_known_replied_xid(imp); spin_unlock(&imp->imp_lock); - if (likely(min_xid != ~0ULL)) - lustre_msg_set_last_xid(req->rq_reqmsg, min_xid - 1); + lustre_msg_set_last_xid(req->rq_reqmsg, min_xid); lustre_msg_set_status(req->rq_reqmsg, current_pid()); @@ -1956,6 +2004,7 @@ int ptlrpc_check_set(const struct lu_env *env, struct ptlrpc_request_set *set) list_del_init(&req->rq_list); atomic_dec(&imp->imp_inflight); } + list_del_init(&req->rq_unreplied_list); spin_unlock(&imp->imp_lock); atomic_dec(&set->set_remaining); @@ -2353,6 +2402,7 @@ static void __ptlrpc_free_req(struct ptlrpc_request *request, int locked) if (!locked) spin_lock(&request->rq_import->imp_lock); list_del_init(&request->rq_replay_list); + list_del_init(&request->rq_unreplied_list); if (!locked) spin_unlock(&request->rq_import->imp_lock); } @@ -3060,7 +3110,7 @@ void ptlrpc_set_bulk_mbits(struct ptlrpc_request *req) LASSERT(bd); - if (!req->rq_resend || req->rq_nr_resend) { + if (!req->rq_resend) { /* this request has a new xid, just use it as bulk matchbits */ req->rq_mbits = req->rq_xid; diff --git a/drivers/staging/lustre/lustre/ptlrpc/import.c b/drivers/staging/lustre/lustre/ptlrpc/import.c index babb80d..66f5b49 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/import.c +++ b/drivers/staging/lustre/lustre/ptlrpc/import.c @@ -904,6 +904,39 @@ static int ptlrpc_connect_set_flags(struct obd_import *imp, } /** + * Add all replay requests back to unreplied list before start replay, + * so that we can make sure the known replied XID is always increased + * only even if when replaying requests. + */ +static void ptlrpc_prepare_replay(struct obd_import *imp) +{ + struct ptlrpc_request *req; + + if (imp->imp_state != LUSTRE_IMP_REPLAY || + imp->imp_resend_replay) + return; + + /* + * If the server was restart during repaly, the requests may + * have been added to the unreplied list in former replay. + */ + spin_lock(&imp->imp_lock); + + list_for_each_entry(req, &imp->imp_committed_list, rq_replay_list) { + if (list_empty(&req->rq_unreplied_list)) + ptlrpc_add_unreplied(req); + } + + list_for_each_entry(req, &imp->imp_replay_list, rq_replay_list) { + if (list_empty(&req->rq_unreplied_list)) + ptlrpc_add_unreplied(req); + } + + imp->imp_known_replied_xid = ptlrpc_known_replied_xid(imp); + spin_unlock(&imp->imp_lock); +} + +/** * interpret_reply callback for connect RPCs. * Looks into returned status of connect operation and decides * what to do with the import - i.e enter recovery, promote it to @@ -1154,6 +1187,7 @@ static int ptlrpc_connect_interpret(const struct lu_env *env, } finish: + ptlrpc_prepare_replay(imp); rc = ptlrpc_import_recovery_state_machine(imp); if (rc == -ENOTCONN) { CDEBUG(D_HA, "evicted/aborted by %s@%s during recovery; invalidating and reconnecting\n", diff --git a/drivers/staging/lustre/lustre/ptlrpc/niobuf.c b/drivers/staging/lustre/lustre/ptlrpc/niobuf.c index 5810565..da1209e 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/niobuf.c +++ b/drivers/staging/lustre/lustre/ptlrpc/niobuf.c @@ -509,20 +509,39 @@ int ptl_send_rpc(struct ptlrpc_request *request, int noreply) lustre_msg_set_conn_cnt(request->rq_reqmsg, imp->imp_conn_cnt); lustre_msghdr_set_flags(request->rq_reqmsg, imp->imp_msghdr_flags); - if (request->rq_nr_resend) { + /* + * If it's the first time to resend the request for EINPROGRESS, + * we need to allocate a new XID (see after_reply()), it's different + * from the resend for reply timeout. + */ + if (request->rq_nr_resend && list_empty(&request->rq_unreplied_list)) { + __u64 min_xid = 0; /* * resend for EINPROGRESS, allocate new xid to avoid reply * reconstruction */ - request->rq_xid = ptlrpc_next_xid(); - DEBUG_REQ(D_RPCTRACE, request, "Allocating new xid for resend on EINPROGRESS"); - } + spin_lock(&imp->imp_lock); + ptlrpc_assign_next_xid_nolock(request); + request->rq_mbits = request->rq_xid; + min_xid = ptlrpc_known_replied_xid(imp); + spin_unlock(&imp->imp_lock); - if (request->rq_bulk) { + lustre_msg_set_last_xid(request->rq_reqmsg, min_xid); + DEBUG_REQ(D_RPCTRACE, request, "Allocating new xid for resend on EINPROGRESS"); + } else if (request->rq_bulk) { ptlrpc_set_bulk_mbits(request); lustre_msg_set_mbits(request->rq_reqmsg, request->rq_mbits); } + if (list_empty(&request->rq_unreplied_list) || + request->rq_xid <= imp->imp_known_replied_xid) { + DEBUG_REQ(D_ERROR, request, + "xid: %llu, replied: %llu, list_empty:%d\n", + request->rq_xid, imp->imp_known_replied_xid, + list_empty(&request->rq_unreplied_list)); + LBUG(); + } + /** * For enabled AT all request should have AT_SUPPORT in the * FULL import state when OBD_CONNECT_AT is set diff --git a/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h b/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h index f056c82..e0f859c 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h +++ b/drivers/staging/lustre/lustre/ptlrpc/ptlrpc_internal.h @@ -71,6 +71,9 @@ int ptlrpc_expired_set(void *data); int ptlrpc_set_next_timeout(struct ptlrpc_request_set *); void ptlrpc_resend_req(struct ptlrpc_request *request); void ptlrpc_set_bulk_mbits(struct ptlrpc_request *req); +void ptlrpc_assign_next_xid_nolock(struct ptlrpc_request *req); +__u64 ptlrpc_known_replied_xid(struct obd_import *imp); +void ptlrpc_add_unreplied(struct ptlrpc_request *req); /* events.c */ int ptlrpc_init_portals(void); @@ -324,6 +327,7 @@ static inline void ptlrpc_cli_req_init(struct ptlrpc_request *req) INIT_LIST_HEAD(&cr->cr_set_chain); INIT_LIST_HEAD(&cr->cr_ctx_chain); + INIT_LIST_HEAD(&cr->cr_unreplied_list); init_waitqueue_head(&cr->cr_reply_waitq); init_waitqueue_head(&cr->cr_set_waitq); } @@ -340,4 +344,24 @@ static inline void ptlrpc_srv_req_init(struct ptlrpc_request *req) INIT_LIST_HEAD(&sr->sr_hist_list); } +static inline bool ptlrpc_req_is_connect(struct ptlrpc_request *req) +{ + if (lustre_msg_get_opc(req->rq_reqmsg) == MDS_CONNECT || + lustre_msg_get_opc(req->rq_reqmsg) == OST_CONNECT || + lustre_msg_get_opc(req->rq_reqmsg) == MGS_CONNECT) + return true; + else + return false; +} + +static inline bool ptlrpc_req_is_disconnect(struct ptlrpc_request *req) +{ + if (lustre_msg_get_opc(req->rq_reqmsg) == MDS_DISCONNECT || + lustre_msg_get_opc(req->rq_reqmsg) == OST_DISCONNECT || + lustre_msg_get_opc(req->rq_reqmsg) == MGS_DISCONNECT) + return true; + else + return false; +} + #endif /* PTLRPC_INTERNAL_H */ diff --git a/drivers/staging/lustre/lustre/ptlrpc/recover.c b/drivers/staging/lustre/lustre/ptlrpc/recover.c index 9144cd8..344aedd 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/recover.c +++ b/drivers/staging/lustre/lustre/ptlrpc/recover.c @@ -157,10 +157,22 @@ int ptlrpc_replay_next(struct obd_import *imp, int *inflight) lustre_msg_add_flags(req->rq_reqmsg, MSG_RESENT); spin_lock(&imp->imp_lock); + /* The resend replay request may have been removed from the + * unreplied list. + */ + if (req && imp->imp_resend_replay && + list_empty(&req->rq_unreplied_list)) + ptlrpc_add_unreplied(req); + imp->imp_resend_replay = 0; spin_unlock(&imp->imp_lock); if (req) { + /* The request should have been added back in unreplied list + * by ptlrpc_prepare_replay(). + */ + LASSERT(!list_empty(&req->rq_unreplied_list)); + rc = ptlrpc_replay_req(req); if (rc) { CERROR("recovery replay error %d for req %llu\n", -- 2.7.4 From green at linuxhacker.ru Thu Nov 17 01:05:28 2016 From: green at linuxhacker.ru (Oleg Drokin) Date: Wed, 16 Nov 2016 20:05:28 -0500 Subject: [lustre-devel] [PATCH 0/2] Lustre fixes Message-ID: <1479344730-1131617-1-git-send-email-green@linuxhacker.ru> With multiple metadata RPCs code in now, these two fixes become important. Please consider. Niu Yawei (2): staging/lustre/ptlrpc: track unreplied requests staging/lustre/ptlrpc: update imp_known_replied_xid on resend-replay .../staging/lustre/lustre/include/lustre_import.h | 5 + drivers/staging/lustre/lustre/include/lustre_net.h | 3 + drivers/staging/lustre/lustre/obdclass/genops.c | 2 + drivers/staging/lustre/lustre/ptlrpc/client.c | 112 +++++++++++++++------ drivers/staging/lustre/lustre/ptlrpc/import.c | 34 +++++++ drivers/staging/lustre/lustre/ptlrpc/niobuf.c | 29 +++++- .../staging/lustre/lustre/ptlrpc/ptlrpc_internal.h | 24 +++++ drivers/staging/lustre/lustre/ptlrpc/recover.c | 14 +++ 8 files changed, 187 insertions(+), 36 deletions(-) -- 2.7.4 From green at linuxhacker.ru Thu Nov 17 01:05:30 2016 From: green at linuxhacker.ru (Oleg Drokin) Date: Wed, 16 Nov 2016 20:05:30 -0500 Subject: [lustre-devel] [PATCH 2/2] staging/lustre/ptlrpc: update imp_known_replied_xid on resend-replay In-Reply-To: <1479344730-1131617-1-git-send-email-green@linuxhacker.ru> References: <1479344730-1131617-1-git-send-email-green@linuxhacker.ru> Message-ID: <1479344730-1131617-3-git-send-email-green@linuxhacker.ru> From: Niu Yawei The imp_known_replied_xid should be updated when try to resend an already replied replay request, because the xid of this replay request could be less than current imp_known_replied_xid. Signed-off-by: Niu Yawei Reviewed-on: http://review.whamcloud.com/22776 Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-8645 Reviewed-by: Alex Zhuravlev Reviewed-by: Fan Yong Signed-off-by: Oleg Drokin --- drivers/staging/lustre/lustre/ptlrpc/recover.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/staging/lustre/lustre/ptlrpc/recover.c b/drivers/staging/lustre/lustre/ptlrpc/recover.c index 344aedd..c004490 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/recover.c +++ b/drivers/staging/lustre/lustre/ptlrpc/recover.c @@ -161,8 +161,10 @@ int ptlrpc_replay_next(struct obd_import *imp, int *inflight) * unreplied list. */ if (req && imp->imp_resend_replay && - list_empty(&req->rq_unreplied_list)) + list_empty(&req->rq_unreplied_list)) { ptlrpc_add_unreplied(req); + imp->imp_known_replied_xid = ptlrpc_known_replied_xid(imp); + } imp->imp_resend_replay = 0; spin_unlock(&imp->imp_lock); -- 2.7.4 From jsimmons at infradead.org Thu Nov 17 19:35:38 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 17 Nov 2016 14:35:38 -0500 Subject: [lustre-devel] [PATCH 00/19] LNet/libcfs checkpatch and style cleanups Message-ID: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> Sweep of the LNet/libcfs layer to cleanup various checkpatch issues. Also more cleanups to match the kernel coding style. James Simmons (19): staging: lustre: lnet: don't use bare unsigned staging: lustre: lnet: resolve trailing */ checkpatch errors staging: lustre: libcfs: use static const char const * for a suffixes array staging: lustre: libcfs: remove explict test of NULL variable staging: lustre: libcfs: fix aligment issue staging: lustre: lnet: fix misspelled word destroy staging: lustre: lnet: remove unused lib_me_dump staging: lustre: lnet: missing blank line after declaration staging: lustre: lnet: fill in real lnet_md_t staging: lustre: lnet: replace uninitialized_var staging: lustre: libcfs: remove != 0 testing staging: lustre: libcfs: remove == 0 testing staging: lustre: libcfs: white space cleanup staging: lustre: libcfs: fix misspelling of descriptor staging: lustre: libcfs: Make char * array envp static constant staging: lustre: libcfs: use pr_* instead of printk staging: lustre: libcfs: remove use of __u* types staging: lustre: lnet: use BIT macro in LNet selftest staging: lustre: lnet: cleanup some of the > 80 line issues .../staging/lustre/lnet/klnds/socklnd/socklnd.c | 31 +++-- .../staging/lustre/lnet/klnds/socklnd/socklnd.h | 132 +++++++++++++------- .../staging/lustre/lnet/klnds/socklnd/socklnd_cb.c | 40 ++++-- .../lustre/lnet/klnds/socklnd/socklnd_lib.c | 6 +- .../lustre/lnet/klnds/socklnd/socklnd_proto.c | 26 +++- drivers/staging/lustre/lnet/libcfs/debug.c | 36 +++--- drivers/staging/lustre/lnet/libcfs/fail.c | 4 +- drivers/staging/lustre/lnet/libcfs/hash.c | 117 +++++++++--------- drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c | 2 +- drivers/staging/lustre/lnet/libcfs/libcfs_lock.c | 20 ++-- drivers/staging/lustre/lnet/libcfs/libcfs_mem.c | 22 ++-- drivers/staging/lustre/lnet/libcfs/libcfs_string.c | 58 +++++----- .../staging/lustre/lnet/libcfs/linux/linux-cpu.c | 132 ++++++++++---------- .../lustre/lnet/libcfs/linux/linux-crypto.c | 32 +++--- .../lustre/lnet/libcfs/linux/linux-crypto.h | 2 +- .../staging/lustre/lnet/libcfs/linux/linux-debug.c | 20 ++-- .../lustre/lnet/libcfs/linux/linux-module.c | 2 +- .../staging/lustre/lnet/libcfs/linux/linux-prim.c | 10 +- .../lustre/lnet/libcfs/linux/linux-tracefile.c | 28 ++-- drivers/staging/lustre/lnet/libcfs/module.c | 18 ++-- drivers/staging/lustre/lnet/libcfs/prng.c | 2 +- drivers/staging/lustre/lnet/libcfs/tracefile.c | 108 ++++++++-------- drivers/staging/lustre/lnet/libcfs/tracefile.h | 28 ++-- drivers/staging/lustre/lnet/libcfs/workitem.c | 50 ++++---- drivers/staging/lustre/lnet/lnet/api-ni.c | 8 +- drivers/staging/lustre/lnet/lnet/lib-me.c | 18 --- drivers/staging/lustre/lnet/lnet/lib-move.c | 1 + drivers/staging/lustre/lnet/lnet/nidstrings.c | 4 +- drivers/staging/lustre/lnet/lnet/router.c | 17 ++-- drivers/staging/lustre/lnet/selftest/conctl.c | 2 +- drivers/staging/lustre/lnet/selftest/conrpc.c | 28 +++-- drivers/staging/lustre/lnet/selftest/conrpc.h | 23 ++-- drivers/staging/lustre/lnet/selftest/console.c | 27 +++-- drivers/staging/lustre/lnet/selftest/console.h | 25 ++-- drivers/staging/lustre/lnet/selftest/framework.c | 23 ++-- drivers/staging/lustre/lnet/selftest/ping_test.c | 4 +- drivers/staging/lustre/lnet/selftest/rpc.c | 3 +- drivers/staging/lustre/lnet/selftest/rpc.h | 6 +- drivers/staging/lustre/lnet/selftest/selftest.h | 25 +++-- drivers/staging/lustre/lnet/selftest/timer.c | 7 +- 40 files changed, 621 insertions(+), 526 deletions(-) From jsimmons at infradead.org Thu Nov 17 19:35:49 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 17 Nov 2016 14:35:49 -0500 Subject: [lustre-devel] [PATCH 11/19] staging: lustre: libcfs: remove != 0 testing In-Reply-To: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> References: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479411357-28979-12-git-send-email-jsimmons@infradead.org> Testing != 0 is not kernel style so remove this type of testing from libcfs. Signed-off-by: James Simmons --- drivers/staging/lustre/lnet/libcfs/debug.c | 4 +- drivers/staging/lustre/lnet/libcfs/hash.c | 24 ++++++++++---------- drivers/staging/lustre/lnet/libcfs/libcfs_string.c | 6 ++-- .../staging/lustre/lnet/libcfs/linux/linux-cpu.c | 14 +++++----- .../lustre/lnet/libcfs/linux/linux-crypto.c | 6 ++-- .../lustre/lnet/libcfs/linux/linux-tracefile.c | 10 ++++---- drivers/staging/lustre/lnet/libcfs/module.c | 4 +- drivers/staging/lustre/lnet/libcfs/tracefile.c | 14 +++++----- drivers/staging/lustre/lnet/libcfs/workitem.c | 6 ++-- 9 files changed, 44 insertions(+), 44 deletions(-) diff --git a/drivers/staging/lustre/lnet/libcfs/debug.c b/drivers/staging/lustre/lnet/libcfs/debug.c index f95764c..57cdc76 100644 --- a/drivers/staging/lustre/lnet/libcfs/debug.c +++ b/drivers/staging/lustre/lnet/libcfs/debug.c @@ -278,7 +278,7 @@ static int param_set_uintpos(const char *val, const struct kernel_param *kp) len++; } - while (*token != 0) { + while (*token) { if (len < size) str[len] = *token; token++; @@ -315,7 +315,7 @@ static int param_set_uintpos(const char *val, const struct kernel_param *kp) t = sscanf(str, "%i%n", &m, &matched); if (t >= 1 && matched == n) { /* don't print warning for lctl set_param debug=0 or -1 */ - if (m != 0 && m != -1) + if (m && m != -1) CWARN("You are trying to use a numerical value for the mask - this will be deprecated in a future release.\n"); *mask = m; return 0; diff --git a/drivers/staging/lustre/lnet/libcfs/hash.c b/drivers/staging/lustre/lnet/libcfs/hash.c index 530def6..1b67536 100644 --- a/drivers/staging/lustre/lnet/libcfs/hash.c +++ b/drivers/staging/lustre/lnet/libcfs/hash.c @@ -492,7 +492,7 @@ struct cfs_hash_dhead_dep { cfs_hash_bd_from_key(hs, hs->hs_buckets, hs->hs_cur_bits, key, bd); } else { - LASSERT(hs->hs_rehash_bits != 0); + LASSERT(hs->hs_rehash_bits); cfs_hash_bd_from_key(hs, hs->hs_rehash_buckets, hs->hs_rehash_bits, key, bd); } @@ -629,7 +629,7 @@ enum cfs_hash_lookup_intent { struct hlist_head *hhead = cfs_hash_bd_hhead(hs, bd); struct hlist_node *ehnode; struct hlist_node *match; - int intent_add = (intent & CFS_HS_LOOKUP_MASK_ADD) != 0; + int intent_add = intent & CFS_HS_LOOKUP_MASK_ADD; /* with this function, we can avoid a lot of useless refcount ops, * which are expensive atomic operations most time. @@ -643,13 +643,13 @@ enum cfs_hash_lookup_intent { continue; /* match and ... */ - if ((intent & CFS_HS_LOOKUP_MASK_DEL) != 0) { + if (intent & CFS_HS_LOOKUP_MASK_DEL) { cfs_hash_bd_del_locked(hs, bd, ehnode); return ehnode; } /* caller wants refcount? */ - if ((intent & CFS_HS_LOOKUP_MASK_REF) != 0) + if (intent & CFS_HS_LOOKUP_MASK_REF) cfs_hash_get(hs, ehnode); return ehnode; } @@ -815,7 +815,7 @@ struct hlist_node * return; } - LASSERT(hs->hs_rehash_bits != 0); + LASSERT(hs->hs_rehash_bits); cfs_hash_bd_from_key(hs, hs->hs_rehash_buckets, hs->hs_rehash_bits, key, &bds[1]); @@ -976,7 +976,7 @@ static void cfs_hash_depth_wi_cancel(struct cfs_hash *hs) return; spin_lock(&hs->hs_dep_lock); - while (hs->hs_dep_bits != 0) { + while (hs->hs_dep_bits) { spin_unlock(&hs->hs_dep_lock); cond_resched(); spin_lock(&hs->hs_dep_lock); @@ -1010,16 +1010,16 @@ struct cfs_hash * LASSERT(ops->hs_get); LASSERT(ops->hs_put_locked); - if ((flags & CFS_HASH_REHASH) != 0) + if (flags & CFS_HASH_REHASH) flags |= CFS_HASH_COUNTER; /* must have counter */ LASSERT(cur_bits > 0); LASSERT(cur_bits >= bkt_bits); LASSERT(max_bits >= cur_bits && max_bits < 31); LASSERT(ergo((flags & CFS_HASH_REHASH) == 0, cur_bits == max_bits)); - LASSERT(ergo((flags & CFS_HASH_REHASH) != 0, + LASSERT(ergo(flags & CFS_HASH_REHASH, (flags & CFS_HASH_NO_LOCK) == 0)); - LASSERT(ergo((flags & CFS_HASH_REHASH_KEY) != 0, ops->hs_keycpy)); + LASSERT(ergo(flags & CFS_HASH_REHASH_KEY, ops->hs_keycpy)); len = (flags & CFS_HASH_BIGNAME) == 0 ? CFS_HASH_NAME_LEN : CFS_HASH_BIGNAME_LEN; @@ -1948,7 +1948,7 @@ struct cfs_hash_cond_arg { /* can't refer to @hs anymore because it could be destroyed */ if (bkts) cfs_hash_buckets_free(bkts, bsize, new_size, old_size); - if (rc != 0) + if (rc) CDEBUG(D_INFO, "early quit of rehashing: %d\n", rc); /* return 1 only if cfs_wi_exit is called */ return rc == -ESRCH; @@ -2017,7 +2017,7 @@ void cfs_hash_debug_header(struct seq_file *m) if (!hs->hs_rehash_buckets) return hs->hs_buckets; - LASSERT(hs->hs_rehash_bits != 0); + LASSERT(hs->hs_rehash_bits); return hs->hs_rehash_bits > hs->hs_cur_bits ? hs->hs_rehash_buckets : hs->hs_buckets; } @@ -2029,7 +2029,7 @@ void cfs_hash_debug_header(struct seq_file *m) if (!hs->hs_rehash_buckets) return CFS_HASH_NBKT(hs); - LASSERT(hs->hs_rehash_bits != 0); + LASSERT(hs->hs_rehash_bits); return hs->hs_rehash_bits > hs->hs_cur_bits ? CFS_HASH_RH_NBKT(hs) : CFS_HASH_NBKT(hs); } diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_string.c b/drivers/staging/lustre/lnet/libcfs/libcfs_string.c index 8db9157..9c58f82 100644 --- a/drivers/staging/lustre/lnet/libcfs/libcfs_string.c +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_string.c @@ -391,7 +391,7 @@ int cfs_str2mask(const char *str, const char *(*bit2str)(int bit), i += scnprintf(buffer + i, count - i, "["); list_for_each_entry(expr, &expr_list->el_exprs, re_link) { - if (j++ != 0) + if (j++) i += scnprintf(buffer + i, count - i, ","); i += cfs_range_expr_print(buffer + i, count - i, expr, numexprs > 1); @@ -533,7 +533,7 @@ int cfs_str2mask(const char *str, const char *(*bit2str)(int bit), } rc = cfs_range_expr_parse(&tok, min, max, 1, &expr); - if (rc != 0) + if (rc) break; list_add_tail(&expr->re_link, &expr_list->el_exprs); @@ -544,7 +544,7 @@ int cfs_str2mask(const char *str, const char *(*bit2str)(int bit), list_add_tail(&expr->re_link, &expr_list->el_exprs); } - if (rc != 0) + if (rc) cfs_expr_list_free(expr_list); else *elpp = expr_list; diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c index 268a8ef..44a8790 100644 --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c +++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c @@ -712,7 +712,7 @@ struct cfs_cpt_table * */ ncpt = min(2U, ncpt); #endif - while (ncpu % ncpt != 0) + while (ncpu % ncpt) ncpt--; /* worst case is 1 */ return ncpt; @@ -737,7 +737,7 @@ struct cfs_cpt_table * ncpt, rc); } - if (num_online_cpus() % ncpt != 0) { + if (num_online_cpus() % ncpt) { CERROR("CPU number %d is not multiple of cpu_npartition %d, please try different cpu_npartitions value or set pattern string by cpu_pattern=STRING\n", (int)num_online_cpus(), ncpt); goto failed; @@ -888,7 +888,7 @@ struct cfs_cpt_table * int n; if (!bracket) { - if (*str != 0) { + if (*str) { CERROR("Invalid pattern %s\n", str); goto failed; } @@ -911,7 +911,7 @@ struct cfs_cpt_table * goto failed; } - if (cfs_cpt_weight(cptab, cpt) != 0) { + if (cfs_cpt_weight(cptab, cpt)) { CERROR("Partition %d has already been set.\n", cpt); goto failed; } @@ -930,14 +930,14 @@ struct cfs_cpt_table * } if (cfs_expr_list_parse(str, (bracket - str) + 1, - 0, high, &el) != 0) { + 0, high, &el)) { CERROR("Can't parse number range: %s\n", str); goto failed; } list_for_each_entry(range, &el->el_exprs, re_link) { for (i = range->re_lo; i <= range->re_hi; i++) { - if ((i - range->re_lo) % range->re_stride != 0) + if ((i - range->re_lo) % range->re_stride) continue; rc = node ? cfs_cpt_set_node(cptab, cpt, i) : @@ -1044,7 +1044,7 @@ struct cfs_cpt_table * register_hotcpu_notifier(&cfs_cpu_notifier); #endif - if (*cpu_pattern != 0) { + if (*cpu_pattern) { cfs_cpt_table = cfs_cpt_table_create_pattern(cpu_pattern); if (!cfs_cpt_table) { CERROR("Failed to create cptab from pattern %s\n", diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-crypto.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-crypto.c index 7f56d2c..3d85cc5 100644 --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-crypto.c +++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-crypto.c @@ -93,12 +93,12 @@ static int cfs_crypto_hash_alloc(enum cfs_crypto_hash_alg hash_alg, if (key) err = crypto_ahash_setkey(tfm, key, key_len); - else if ((*type)->cht_key != 0) + else if ((*type)->cht_key) err = crypto_ahash_setkey(tfm, (unsigned char *)&((*type)->cht_key), (*type)->cht_size); - if (err != 0) { + if (err) { ahash_request_free(*req); crypto_free_ahash(tfm); return err; @@ -156,7 +156,7 @@ int cfs_crypto_hash_digest(enum cfs_crypto_hash_alg hash_alg, return -EINVAL; err = cfs_crypto_hash_alloc(hash_alg, &type, &req, key, key_len); - if (err != 0) + if (err) return err; if (!hash || *hash_len < type->cht_size) { diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-tracefile.c index 8b551d2..404c2cc 100644 --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-tracefile.c +++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-tracefile.c @@ -224,21 +224,21 @@ void cfs_print_to_console(struct ptldebug_header *hdr, int mask, { char *prefix = "Lustre", *ptype = NULL; - if ((mask & D_EMERG) != 0) { + if (mask & D_EMERG) { prefix = dbghdr_to_err_string(hdr); ptype = KERN_EMERG; - } else if ((mask & D_ERROR) != 0) { + } else if (mask & D_ERROR) { prefix = dbghdr_to_err_string(hdr); ptype = KERN_ERR; - } else if ((mask & D_WARNING) != 0) { + } else if (mask & D_WARNING) { prefix = dbghdr_to_info_string(hdr); ptype = KERN_WARNING; - } else if ((mask & (D_CONSOLE | libcfs_printk)) != 0) { + } else if (mask & (D_CONSOLE | libcfs_printk)) { prefix = dbghdr_to_info_string(hdr); ptype = KERN_INFO; } - if ((mask & D_CONSOLE) != 0) { + if (mask & D_CONSOLE) { printk("%s%s: %.*s", ptype, prefix, len, buf); } else { printk("%s%s: %d:%d:(%s:%d:%s()) %.*s", ptype, prefix, diff --git a/drivers/staging/lustre/lnet/libcfs/module.c b/drivers/staging/lustre/lnet/libcfs/module.c index 86b4d25..cd057f1 100644 --- a/drivers/staging/lustre/lnet/libcfs/module.c +++ b/drivers/staging/lustre/lnet/libcfs/module.c @@ -547,7 +547,7 @@ static int libcfs_init(void) } rc = cfs_cpu_init(); - if (rc != 0) + if (rc) goto cleanup_debug; rc = misc_register(&libcfs_dev); @@ -566,7 +566,7 @@ static int libcfs_init(void) rc = min(cfs_cpt_weight(cfs_cpt_table, CFS_CPT_ANY), 4); rc = cfs_wi_sched_create("cfs_rh", cfs_cpt_table, CFS_CPT_ANY, rc, &cfs_sched_rehash); - if (rc != 0) { + if (rc) { CERROR("Startup workitem scheduler: error: %d\n", rc); goto cleanup_deregister; } diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c index 1c7efdf..9f5eeb3 100644 --- a/drivers/staging/lustre/lnet/libcfs/tracefile.c +++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c @@ -432,7 +432,7 @@ int libcfs_debug_vmsg2(struct libcfs_debug_msg_data *msgdata, if (cdls) { if (libcfs_console_ratelimit && - cdls->cdls_next != 0 && /* not first time ever */ + cdls->cdls_next && /* not first time ever */ !cfs_time_after(cfs_time_current(), cdls->cdls_next)) { /* skipping a console message */ cdls->cdls_count++; @@ -489,7 +489,7 @@ int libcfs_debug_vmsg2(struct libcfs_debug_msg_data *msgdata, put_cpu(); } - if (cdls && cdls->cdls_count != 0) { + if (cdls && cdls->cdls_count) { string_buf = cfs_trace_get_console_buffer(); needed = snprintf(string_buf, CFS_TRACE_CONSOLE_BUFFER_SIZE, @@ -847,12 +847,12 @@ int cfs_trace_dump_debug_buffer_usrstr(void __user *usr_str, int usr_str_nob) int rc; rc = cfs_trace_allocate_string_buffer(&str, usr_str_nob + 1); - if (rc != 0) + if (rc) return rc; rc = cfs_trace_copyin_string(str, usr_str_nob + 1, usr_str, usr_str_nob); - if (rc != 0) + if (rc) goto out; if (str[0] != '/') { @@ -912,7 +912,7 @@ int cfs_trace_daemon_command_usrstr(void __user *usr_str, int usr_str_nob) int rc; rc = cfs_trace_allocate_string_buffer(&str, usr_str_nob + 1); - if (rc != 0) + if (rc) return rc; rc = cfs_trace_copyin_string(str, usr_str_nob + 1, @@ -1003,7 +1003,7 @@ static int tracefiled(void *arg) filp = NULL; cfs_tracefile_read_lock(); - if (cfs_tracefile[0] != 0) { + if (cfs_tracefile[0]) { filp = filp_open(cfs_tracefile, O_CREAT | O_RDWR | O_LARGEFILE, 0600); @@ -1141,7 +1141,7 @@ int cfs_tracefile_init(int max_pages) int factor; rc = cfs_tracefile_init_arch(); - if (rc != 0) + if (rc) return rc; cfs_tcd_for_each(tcd, i, j) { diff --git a/drivers/staging/lustre/lnet/libcfs/workitem.c b/drivers/staging/lustre/lnet/libcfs/workitem.c index 84118b5..b9eda17 100644 --- a/drivers/staging/lustre/lnet/libcfs/workitem.c +++ b/drivers/staging/lustre/lnet/libcfs/workitem.c @@ -208,7 +208,7 @@ static int cfs_wi_scheduler(void *arg) /* CPT affinity scheduler? */ if (sched->ws_cptab) - if (cfs_cpt_bind(sched->ws_cptab, sched->ws_cpt) != 0) + if (cfs_cpt_bind(sched->ws_cptab, sched->ws_cpt)) CWARN("Failed to bind %s on CPT %d\n", sched->ws_name, sched->ws_cpt); @@ -247,7 +247,7 @@ static int cfs_wi_scheduler(void *arg) rc = (*wi->wi_action) (wi); spin_lock(&sched->ws_lock); - if (rc != 0) /* WI should be dead, even be freed! */ + if (rc) /* WI should be dead, even be freed! */ continue; wi->wi_running = 0; @@ -447,7 +447,7 @@ static int cfs_wi_scheduler(void *arg) list_for_each_entry(sched, &cfs_wi_data.wi_scheds, ws_list) { spin_lock(&cfs_wi_data.wi_glock); - while (sched->ws_nthreads != 0) { + while (sched->ws_nthreads) { spin_unlock(&cfs_wi_data.wi_glock); set_current_state(TASK_UNINTERRUPTIBLE); schedule_timeout(cfs_time_seconds(1) / 20); -- 1.7.1 From jsimmons at infradead.org Thu Nov 17 19:35:57 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 17 Nov 2016 14:35:57 -0500 Subject: [lustre-devel] [PATCH 19/19] staging: lustre: lnet: cleanup some of the > 80 line issues In-Reply-To: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> References: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479411357-28979-20-git-send-email-jsimmons@infradead.org> Cleanup some of the checkpatch over 80 character lines reported. Signed-off-by: James Simmons --- .../staging/lustre/lnet/klnds/socklnd/socklnd.c | 31 ++++++++++----- .../staging/lustre/lnet/klnds/socklnd/socklnd_cb.c | 40 +++++++++++++------- .../lustre/lnet/klnds/socklnd/socklnd_lib.c | 6 ++- .../lustre/lnet/klnds/socklnd/socklnd_proto.c | 26 ++++++++++--- drivers/staging/lustre/lnet/selftest/conrpc.c | 10 +++-- drivers/staging/lustre/lnet/selftest/conrpc.h | 3 +- drivers/staging/lustre/lnet/selftest/console.c | 17 +++++--- drivers/staging/lustre/lnet/selftest/framework.c | 15 +++++-- drivers/staging/lustre/lnet/selftest/rpc.c | 3 +- 9 files changed, 101 insertions(+), 50 deletions(-) diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c index cbc9a9c..b74cf63 100644 --- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c @@ -96,7 +96,8 @@ } static int -ksocknal_create_peer(struct ksock_peer **peerp, lnet_ni_t *ni, lnet_process_id_t id) +ksocknal_create_peer(struct ksock_peer **peerp, lnet_ni_t *ni, + lnet_process_id_t id) { int cpt = lnet_cpt_of_nid(id.nid); struct ksock_net *net = ni->ni_data; @@ -319,7 +320,8 @@ struct ksock_peer * } static void -ksocknal_associate_route_conn_locked(struct ksock_route *route, struct ksock_conn *conn) +ksocknal_associate_route_conn_locked(struct ksock_route *route, + struct ksock_conn *conn) { struct ksock_peer *peer = route->ksnr_peer; int type = conn->ksnc_type; @@ -821,7 +823,8 @@ struct ksock_peer * if (k < peer->ksnp_n_passive_ips) /* using it already */ continue; - k = ksocknal_match_peerip(iface, peerips, n_peerips); + k = ksocknal_match_peerip(iface, peerips, + n_peerips); xor = ip ^ peerips[k]; this_netmatch = !(xor & iface->ksni_netmask) ? 1 : 0; @@ -1302,8 +1305,11 @@ struct ksock_peer * /* Take packets blocking for this connection. */ list_for_each_entry_safe(tx, txtmp, &peer->ksnp_tx_queue, tx_list) { - if (conn->ksnc_proto->pro_match_tx(conn, tx, tx->tx_nonblk) == SOCKNAL_MATCH_NO) - continue; + int match = conn->ksnc_proto->pro_match_tx(conn, tx, + tx->tx_nonblk); + + if (match == SOCKNAL_MATCH_NO) + continue; list_del(&tx->tx_list); ksocknal_queue_tx_locked(tx, conn); @@ -1493,8 +1499,8 @@ struct ksock_peer * spin_unlock_bh(&conn->ksnc_scheduler->kss_lock); } - peer->ksnp_proto = NULL; /* renegotiate protocol version */ - peer->ksnp_error = error; /* stash last conn close reason */ + peer->ksnp_proto = NULL; /* renegotiate protocol version */ + peer->ksnp_error = error; /* stash last conn close reason */ if (list_empty(&peer->ksnp_routes)) { /* @@ -1786,7 +1792,8 @@ struct ksock_peer * (id.pid == LNET_PID_ANY || id.pid == peer->ksnp_id.pid))) continue; - count += ksocknal_close_peer_conns_locked(peer, ipaddr, 0); + count += ksocknal_close_peer_conns_locked(peer, ipaddr, + 0); } } @@ -2026,7 +2033,10 @@ static int ksocknal_push(lnet_ni_t *ni, lnet_process_id_t id) } rc = 0; - /* NB only new connections will pay attention to the new interface! */ + /* + * NB only new connections will pay attention to the + * new interface! + */ } write_unlock_bh(&ksocknal_data.ksnd_global_lock); @@ -2200,8 +2210,9 @@ static int ksocknal_push(lnet_ni_t *ni, lnet_process_id_t id) int txmem; int rxmem; int nagle; - struct ksock_conn *conn = ksocknal_get_conn_by_idx(ni, data->ioc_count); + struct ksock_conn *conn; + conn = ksocknal_get_conn_by_idx(ni, data->ioc_count); if (!conn) return -ENOENT; diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c index f31f4a1..972f609 100644 --- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c @@ -620,7 +620,8 @@ struct ksock_tx * } struct ksock_conn * -ksocknal_find_conn_locked(struct ksock_peer *peer, struct ksock_tx *tx, int nonblk) +ksocknal_find_conn_locked(struct ksock_peer *peer, struct ksock_tx *tx, + int nonblk) { struct list_head *tmp; struct ksock_conn *conn; @@ -630,10 +631,12 @@ struct ksock_conn * int fnob = 0; list_for_each(tmp, &peer->ksnp_conns) { - struct ksock_conn *c = list_entry(tmp, struct ksock_conn, ksnc_list); - int nob = atomic_read(&c->ksnc_tx_nob) + - c->ksnc_sock->sk->sk_wmem_queued; - int rc; + struct ksock_conn *c; + int nob, rc; + + c = list_entry(tmp, struct ksock_conn, ksnc_list); + nob = atomic_read(&c->ksnc_tx_nob) + + c->ksnc_sock->sk->sk_wmem_queued; LASSERT(!c->ksnc_closing); LASSERT(c->ksnc_proto && @@ -752,9 +755,9 @@ struct ksock_conn * LASSERT(msg->ksm_zc_cookies[1]); LASSERT(conn->ksnc_proto->pro_queue_tx_zcack); + /* ZC ACK piggybacked on ztx release tx later */ if (conn->ksnc_proto->pro_queue_tx_zcack(conn, tx, 0)) - ztx = tx; /* ZC ACK piggybacked on ztx release tx later */ - + ztx = tx; } else { /* * It's a normal packet - can it piggback a noop zc-ack that @@ -796,7 +799,8 @@ struct ksock_route * LASSERT(!route->ksnr_connecting || route->ksnr_scheduled); - if (route->ksnr_scheduled) /* connections being established */ + /* connections being established */ + if (route->ksnr_scheduled) continue; /* all route types connected ? */ @@ -1514,7 +1518,10 @@ int ksocknal_scheduler(void *arg) rc = ksocknal_process_transmit(conn, tx); if (rc == -ENOMEM || rc == -EAGAIN) { - /* Incomplete send: replace tx on HEAD of tx_queue */ + /* + * Incomplete send: replace tx on HEAD of + * tx_queue + */ spin_lock_bh(&sched->kss_lock); list_add(&tx->tx_list, &conn->ksnc_tx_queue); } else { @@ -1724,7 +1731,8 @@ void ksocknal_write_callback(struct ksock_conn *conn) timeout = active ? *ksocknal_tunables.ksnd_timeout : lnet_acceptor_timeout(); - rc = lnet_sock_read(sock, &hello->kshm_magic, sizeof(hello->kshm_magic), timeout); + rc = lnet_sock_read(sock, &hello->kshm_magic, + sizeof(hello->kshm_magic), timeout); if (rc) { CERROR("Error %d reading HELLO from %pI4h\n", rc, &conn->ksnc_ipaddr); @@ -1798,7 +1806,8 @@ void ksocknal_write_callback(struct ksock_conn *conn) conn->ksnc_port > LNET_ACCEPTOR_MAX_RESERVED_PORT) { /* Userspace NAL assigns peer process ID from socket */ recv_id.pid = conn->ksnc_port | LNET_PID_USERFLAG; - recv_id.nid = LNET_MKNID(LNET_NIDNET(ni->ni_nid), conn->ksnc_ipaddr); + recv_id.nid = LNET_MKNID(LNET_NIDNET(ni->ni_nid), + conn->ksnc_ipaddr); } else { recv_id.nid = hello->kshm_src_nid; recv_id.pid = hello->kshm_src_pid; @@ -1882,7 +1891,8 @@ void ksocknal_write_callback(struct ksock_conn *conn) if (peer->ksnp_accepting > 0) { CDEBUG(D_NET, "peer %s(%d) already connecting to me, retry later.\n", - libcfs_nid2str(peer->ksnp_id.nid), peer->ksnp_accepting); + libcfs_nid2str(peer->ksnp_id.nid), + peer->ksnp_accepting); retry_later = 1; } @@ -2241,7 +2251,8 @@ void ksocknal_write_callback(struct ksock_conn *conn) /* Nothing to do for 'timeout' */ set_current_state(TASK_INTERRUPTIBLE); - add_wait_queue_exclusive(&ksocknal_data.ksnd_connd_waitq, &wait); + add_wait_queue_exclusive(&ksocknal_data.ksnd_connd_waitq, + &wait); spin_unlock_bh(connd_lock); nloops = 0; @@ -2371,7 +2382,8 @@ void ksocknal_write_callback(struct ksock_conn *conn) struct ksock_conn *conn; struct ksock_tx *tx; - if (list_empty(&peer->ksnp_conns)) /* last_alive will be updated by create_conn */ + /* last_alive will be updated by create_conn */ + if (list_empty(&peer->ksnp_conns)) return 0; if (peer->ksnp_proto != &ksocknal_protocol_v3x) diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_lib.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_lib.c index 6c95e98..4bcab4b 100644 --- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_lib.c +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_lib.c @@ -202,7 +202,8 @@ fragnob = sum; conn->ksnc_rx_csum = ksocknal_csum(conn->ksnc_rx_csum, - iov[i].iov_base, fragnob); + iov[i].iov_base, + fragnob); } conn->ksnc_msg.ksm_csum = saved_csum; } @@ -291,7 +292,8 @@ } int -ksocknal_lib_get_conn_tunables(struct ksock_conn *conn, int *txmem, int *rxmem, int *nagle) +ksocknal_lib_get_conn_tunables(struct ksock_conn *conn, int *txmem, + int *rxmem, int *nagle) { struct socket *sock = conn->ksnc_sock; int len; diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_proto.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_proto.c index 82e174f..8f0ff6c 100644 --- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_proto.c +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_proto.c @@ -194,7 +194,10 @@ } if (!tx->tx_msg.ksm_zc_cookies[0]) { - /* NOOP tx has only one ZC-ACK cookie, can carry at least one more */ + /* + * NOOP tx has only one ZC-ACK cookie, + * can carry at least one more + */ if (tx->tx_msg.ksm_zc_cookies[1] > cookie) { tx->tx_msg.ksm_zc_cookies[0] = tx->tx_msg.ksm_zc_cookies[1]; tx->tx_msg.ksm_zc_cookies[1] = cookie; @@ -203,7 +206,10 @@ } if (tx->tx_msg.ksm_zc_cookies[0] - tx->tx_msg.ksm_zc_cookies[1] > 2) { - /* not likely to carry more ACKs, skip it to simplify logic */ + /* + * not likely to carry more ACKs, skip it + * to simplify logic + */ ksocknal_next_tx_carrier(conn); } @@ -237,7 +243,10 @@ } } else { - /* ksm_zc_cookies[0] < ksm_zc_cookies[1], it is range of cookies */ + /* + * ksm_zc_cookies[0] < ksm_zc_cookies[1], + * it is range of cookies + */ if (cookie >= tx->tx_msg.ksm_zc_cookies[0] && cookie <= tx->tx_msg.ksm_zc_cookies[1]) { CWARN("%s: duplicated ZC cookie: %llu\n", @@ -425,7 +434,8 @@ tx_zc_list) { __u64 c = tx->tx_msg.ksm_zc_cookies[0]; - if (c == cookie1 || c == cookie2 || (cookie1 < c && c < cookie2)) { + if (c == cookie1 || c == cookie2 || + (cookie1 < c && c < cookie2)) { tx->tx_msg.ksm_zc_cookies[0] = 0; list_del(&tx->tx_zc_list); list_add(&tx->tx_zc_list, &zlist); @@ -639,7 +649,8 @@ } static int -ksocknal_recv_hello_v2(struct ksock_conn *conn, ksock_hello_msg_t *hello, int timeout) +ksocknal_recv_hello_v2(struct ksock_conn *conn, ksock_hello_msg_t *hello, + int timeout) { struct socket *sock = conn->ksnc_sock; int rc; @@ -737,7 +748,10 @@ tx->tx_nob = offsetof(ksock_msg_t, ksm_u.lnetmsg.ksnm_hdr); tx->tx_resid = offsetof(ksock_msg_t, ksm_u.lnetmsg.ksnm_hdr); } - /* Don't checksum before start sending, because packet can be piggybacked with ACK */ + /* + * Don't checksum before start sending, because packet can be + * piggybacked with ACK + */ } static void diff --git a/drivers/staging/lustre/lnet/selftest/conrpc.c b/drivers/staging/lustre/lnet/selftest/conrpc.c index 3d071b4..994422c 100644 --- a/drivers/staging/lustre/lnet/selftest/conrpc.c +++ b/drivers/staging/lustre/lnet/selftest/conrpc.c @@ -87,7 +87,8 @@ void lstcon_rpc_stat_reply(struct lstcon_rpc_trans *, struct srpc_msg *, static int lstcon_rpc_init(struct lstcon_node *nd, int service, unsigned int feats, - int bulk_npg, int bulk_len, int embedded, struct lstcon_rpc *crpc) + int bulk_npg, int bulk_len, int embedded, + struct lstcon_rpc *crpc) { crpc->crp_rpc = sfw_create_rpc(nd->nd_id, service, feats, bulk_npg, bulk_len, @@ -292,8 +293,8 @@ void lstcon_rpc_stat_reply(struct lstcon_rpc_trans *, struct srpc_msg *, spin_lock(&rpc->crpc_lock); - if (!crpc->crp_posted || /* not posted */ - crpc->crp_stamp) { /* rpc done or aborted already */ + if (!crpc->crp_posted || /* not posted */ + crpc->crp_stamp) { /* rpc done or aborted already */ if (!crpc->crp_stamp) { crpc->crp_stamp = cfs_time_current(); crpc->crp_status = -EINTR; @@ -778,7 +779,8 @@ void lstcon_rpc_stat_reply(struct lstcon_rpc_trans *, struct srpc_msg *, } static int -lstcon_bulkrpc_v0_prep(lst_test_bulk_param_t *param, struct srpc_test_reqst *req) +lstcon_bulkrpc_v0_prep(lst_test_bulk_param_t *param, + struct srpc_test_reqst *req) { struct test_bulk_req *brq = &req->tsr_u.bulk_v0; diff --git a/drivers/staging/lustre/lnet/selftest/conrpc.h b/drivers/staging/lustre/lnet/selftest/conrpc.h index 27ba639..e629e87 100644 --- a/drivers/staging/lustre/lnet/selftest/conrpc.h +++ b/drivers/staging/lustre/lnet/selftest/conrpc.h @@ -131,7 +131,8 @@ int lstcon_rpc_trans_interpreter(struct lstcon_rpc_trans *trans, lstcon_rpc_readent_func_t readent); void lstcon_rpc_trans_abort(struct lstcon_rpc_trans *trans, int error); void lstcon_rpc_trans_destroy(struct lstcon_rpc_trans *trans); -void lstcon_rpc_trans_addreq(struct lstcon_rpc_trans *trans, struct lstcon_rpc *req); +void lstcon_rpc_trans_addreq(struct lstcon_rpc_trans *trans, + struct lstcon_rpc *req); int lstcon_rpc_trans_postwait(struct lstcon_rpc_trans *trans, int timeout); int lstcon_rpc_pinger_start(void); void lstcon_rpc_pinger_stop(void); diff --git a/drivers/staging/lustre/lnet/selftest/console.c b/drivers/staging/lustre/lnet/selftest/console.c index faa5320..f2d11fa 100644 --- a/drivers/staging/lustre/lnet/selftest/console.c +++ b/drivers/staging/lustre/lnet/selftest/console.c @@ -86,7 +86,7 @@ if (!create) return -ENOENT; - LIBCFS_ALLOC(*ndpp, sizeof(struct lstcon_node) + sizeof(struct lstcon_ndlink)); + LIBCFS_ALLOC(*ndpp, sizeof(*ndpp) + sizeof(*ndl)); if (!*ndpp) return -ENOMEM; @@ -131,12 +131,12 @@ list_del(&ndl->ndl_link); list_del(&ndl->ndl_hlink); - LIBCFS_FREE(nd, sizeof(struct lstcon_node) + sizeof(struct lstcon_ndlink)); + LIBCFS_FREE(nd, sizeof(*nd) + sizeof(*ndl)); } static int -lstcon_ndlink_find(struct list_head *hash, - lnet_process_id_t id, struct lstcon_ndlink **ndlpp, int create) +lstcon_ndlink_find(struct list_head *hash, lnet_process_id_t id, + struct lstcon_ndlink **ndlpp, int create) { unsigned int idx = LNET_NIDADDR(id.nid) % LST_NODE_HASHSIZE; struct lstcon_ndlink *ndl; @@ -230,7 +230,8 @@ grp->grp_ref++; } -static void lstcon_group_ndlink_release(struct lstcon_group *, struct lstcon_ndlink *); +static void lstcon_group_ndlink_release(struct lstcon_group *, + struct lstcon_ndlink *); static void lstcon_group_drain(struct lstcon_group *grp, int keep) @@ -1183,7 +1184,8 @@ } static int -lstcon_test_nodes_add(struct lstcon_test *test, struct list_head __user *result_up) +lstcon_test_nodes_add(struct lstcon_test *test, + struct list_head __user *result_up) { struct lstcon_rpc_trans *trans; struct lstcon_group *grp; @@ -1366,7 +1368,8 @@ } static int -lstcon_test_find(struct lstcon_batch *batch, int idx, struct lstcon_test **testpp) +lstcon_test_find(struct lstcon_batch *batch, int idx, + struct lstcon_test **testpp) { struct lstcon_test *test; diff --git a/drivers/staging/lustre/lnet/selftest/framework.c b/drivers/staging/lustre/lnet/selftest/framework.c index 8f9ee3f..48dcc33 100644 --- a/drivers/staging/lustre/lnet/selftest/framework.c +++ b/drivers/staging/lustre/lnet/selftest/framework.c @@ -131,7 +131,8 @@ } static int -sfw_register_test(struct srpc_service *service, struct sfw_test_client_ops *cliops) +sfw_register_test(struct srpc_service *service, + struct sfw_test_client_ops *cliops) { struct sfw_test_case *tsc; @@ -469,7 +470,8 @@ } static int -sfw_remove_session(struct srpc_rmsn_reqst *request, struct srpc_rmsn_reply *reply) +sfw_remove_session(struct srpc_rmsn_reqst *request, + struct srpc_rmsn_reply *reply) { struct sfw_session *sn = sfw_data.fw_session; @@ -501,7 +503,8 @@ } static int -sfw_debug_session(struct srpc_debug_reqst *request, struct srpc_debug_reply *reply) +sfw_debug_session(struct srpc_debug_reqst *request, + struct srpc_debug_reply *reply) { struct sfw_session *sn = sfw_data.fw_session; @@ -1064,7 +1067,8 @@ } static int -sfw_query_batch(struct sfw_batch *tsb, int testidx, struct srpc_batch_reply *reply) +sfw_query_batch(struct sfw_batch *tsb, int testidx, + struct srpc_batch_reply *reply) { struct sfw_test_instance *tsi; @@ -1179,7 +1183,8 @@ } static int -sfw_control_batch(struct srpc_batch_reqst *request, struct srpc_batch_reply *reply) +sfw_control_batch(struct srpc_batch_reqst *request, + struct srpc_batch_reply *reply) { struct sfw_session *sn = sfw_data.fw_session; int rc = 0; diff --git a/drivers/staging/lustre/lnet/selftest/rpc.c b/drivers/staging/lustre/lnet/selftest/rpc.c index 0498c56..ce9de8c 100644 --- a/drivers/staging/lustre/lnet/selftest/rpc.c +++ b/drivers/staging/lustre/lnet/selftest/rpc.c @@ -696,7 +696,8 @@ struct srpc_bulk * /* called with sv->sv_lock held */ static void -srpc_service_recycle_buffer(struct srpc_service_cd *scd, struct srpc_buffer *buf) +srpc_service_recycle_buffer(struct srpc_service_cd *scd, + struct srpc_buffer *buf) __must_hold(&scd->scd_lock) { if (!scd->scd_svc->sv_shuttingdown && scd->scd_buf_adjust >= 0) { -- 1.7.1 From jsimmons at infradead.org Thu Nov 17 19:35:50 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 17 Nov 2016 14:35:50 -0500 Subject: [lustre-devel] [PATCH 12/19] staging: lustre: libcfs: remove == 0 testing In-Reply-To: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> References: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479411357-28979-13-git-send-email-jsimmons@infradead.org> Testing == 0 is not kernel style so remove this type of testing from libcfs. Signed-off-by: James Simmons --- drivers/staging/lustre/lnet/libcfs/debug.c | 8 +++--- drivers/staging/lustre/lnet/libcfs/hash.c | 27 +++++++++---------- drivers/staging/lustre/lnet/libcfs/libcfs_lock.c | 4 +- drivers/staging/lustre/lnet/libcfs/libcfs_string.c | 16 ++++++------ .../staging/lustre/lnet/libcfs/linux/linux-cpu.c | 12 ++++---- .../lustre/lnet/libcfs/linux/linux-crypto.c | 4 +- drivers/staging/lustre/lnet/libcfs/tracefile.c | 14 +++++----- drivers/staging/lustre/lnet/libcfs/workitem.c | 2 +- 8 files changed, 43 insertions(+), 44 deletions(-) diff --git a/drivers/staging/lustre/lnet/libcfs/debug.c b/drivers/staging/lustre/lnet/libcfs/debug.c index 57cdc76..67f043e 100644 --- a/drivers/staging/lustre/lnet/libcfs/debug.c +++ b/drivers/staging/lustre/lnet/libcfs/debug.c @@ -259,13 +259,13 @@ static int param_set_uintpos(const char *val, const struct kernel_param *kp) const char *token; int i; - if (mask == 0) { /* "0" */ + if (!mask) { /* "0" */ if (size > 0) str[0] = '0'; len = 1; } else { /* space-separated tokens */ for (i = 0; i < 32; i++) { - if ((mask & (1 << i)) == 0) + if (!(mask & (1 << i))) continue; token = fn(i); @@ -416,9 +416,9 @@ int libcfs_debug_init(unsigned long bufsize) max = max / num_possible_cpus(); max <<= (20 - PAGE_SHIFT); } - rc = cfs_tracefile_init(max); - if (rc == 0) { + rc = cfs_tracefile_init(max); + if (!rc) { libcfs_register_panic_notifier(); libcfs_debug_mb = cfs_trace_get_debug_mb(); } diff --git a/drivers/staging/lustre/lnet/libcfs/hash.c b/drivers/staging/lustre/lnet/libcfs/hash.c index 1b67536..22e1e25 100644 --- a/drivers/staging/lustre/lnet/libcfs/hash.c +++ b/drivers/staging/lustre/lnet/libcfs/hash.c @@ -507,7 +507,7 @@ struct cfs_hash_dhead_dep { bd->bd_bucket->hsb_depmax = dep_cur; # if CFS_HASH_DEBUG_LEVEL >= CFS_HASH_DEBUG_1 - if (likely(warn_on_depth == 0 || + if (likely(!warn_on_depth || max(warn_on_depth, hs->hs_dep_max) >= dep_cur)) return; @@ -531,7 +531,7 @@ struct cfs_hash_dhead_dep { rc = hs->hs_hops->hop_hnode_add(hs, bd, hnode); cfs_hash_bd_dep_record(hs, bd, rc); bd->bd_bucket->hsb_version++; - if (unlikely(bd->bd_bucket->hsb_version == 0)) + if (unlikely(!bd->bd_bucket->hsb_version)) bd->bd_bucket->hsb_version++; bd->bd_bucket->hsb_count++; @@ -551,7 +551,7 @@ struct cfs_hash_dhead_dep { LASSERT(bd->bd_bucket->hsb_count > 0); bd->bd_bucket->hsb_count--; bd->bd_bucket->hsb_version++; - if (unlikely(bd->bd_bucket->hsb_version == 0)) + if (unlikely(!bd->bd_bucket->hsb_version)) bd->bd_bucket->hsb_version++; if (cfs_hash_with_counter(hs)) { @@ -571,7 +571,7 @@ struct cfs_hash_dhead_dep { struct cfs_hash_bucket *nbkt = bd_new->bd_bucket; int rc; - if (cfs_hash_bd_compare(bd_old, bd_new) == 0) + if (!cfs_hash_bd_compare(bd_old, bd_new)) return; /* use cfs_hash_bd_hnode_add/del, to avoid atomic & refcount ops @@ -584,11 +584,11 @@ struct cfs_hash_dhead_dep { LASSERT(obkt->hsb_count > 0); obkt->hsb_count--; obkt->hsb_version++; - if (unlikely(obkt->hsb_version == 0)) + if (unlikely(!obkt->hsb_version)) obkt->hsb_version++; nbkt->hsb_count++; nbkt->hsb_version++; - if (unlikely(nbkt->hsb_version == 0)) + if (unlikely(!nbkt->hsb_version)) nbkt->hsb_version++; } @@ -883,7 +883,7 @@ struct hlist_node * struct cfs_hash_bucket **new_bkts; int i; - LASSERT(old_size == 0 || old_bkts); + LASSERT(!old_size || old_bkts); if (old_bkts && old_size == new_size) return old_bkts; @@ -1016,12 +1016,11 @@ struct cfs_hash * LASSERT(cur_bits > 0); LASSERT(cur_bits >= bkt_bits); LASSERT(max_bits >= cur_bits && max_bits < 31); - LASSERT(ergo((flags & CFS_HASH_REHASH) == 0, cur_bits == max_bits)); - LASSERT(ergo(flags & CFS_HASH_REHASH, - (flags & CFS_HASH_NO_LOCK) == 0)); + LASSERT(ergo(!(flags & CFS_HASH_REHASH), cur_bits == max_bits)); + LASSERT(ergo(flags & CFS_HASH_REHASH, !(flags & CFS_HASH_NO_LOCK))); LASSERT(ergo(flags & CFS_HASH_REHASH_KEY, ops->hs_keycpy)); - len = (flags & CFS_HASH_BIGNAME) == 0 ? + len = !(flags & CFS_HASH_BIGNAME) ? CFS_HASH_NAME_LEN : CFS_HASH_BIGNAME_LEN; LIBCFS_ALLOC(hs, offsetof(struct cfs_hash, hs_name[len])); if (!hs) @@ -1107,12 +1106,12 @@ struct cfs_hash * cfs_hash_exit(hs, hnode); } } - LASSERT(bd.bd_bucket->hsb_count == 0); + LASSERT(!bd.bd_bucket->hsb_count); cfs_hash_bd_unlock(hs, &bd, 1); cond_resched(); } - LASSERT(atomic_read(&hs->hs_count) == 0); + LASSERT(!atomic_read(&hs->hs_count)); cfs_hash_buckets_free(hs->hs_buckets, cfs_hash_bkt_size(hs), 0, CFS_HASH_NBKT(hs)); @@ -1388,7 +1387,7 @@ void cfs_hash_putref(struct cfs_hash *hs) bits = cfs_hash_rehash_bits(hs); cfs_hash_unlock(hs, 1); /* NB: it's race on cfs_has_t::hs_iterating, see above */ - if (remained == 0) + if (!remained) hs->hs_iterating = 0; if (bits > 0) { cfs_hash_rehash(hs, atomic_read(&hs->hs_count) < diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_lock.c b/drivers/staging/lustre/lnet/libcfs/libcfs_lock.c index 1deea76..66f020b 100644 --- a/drivers/staging/lustre/lnet/libcfs/libcfs_lock.c +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_lock.c @@ -114,7 +114,7 @@ struct cfs_percpt_lock * /* exclusive lock request */ for (i = 0; i < ncpt; i++) { spin_lock(pcl->pcl_locks[i]); - if (i == 0) { + if (!i) { LASSERT(!pcl->pcl_locked); /* nobody should take private lock after this * so I wouldn't starve for too long time @@ -141,7 +141,7 @@ struct cfs_percpt_lock * } for (i = ncpt - 1; i >= 0; i--) { - if (i == 0) { + if (!i) { LASSERT(pcl->pcl_locked); pcl->pcl_locked = 0; } diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_string.c b/drivers/staging/lustre/lnet/libcfs/libcfs_string.c index 9c58f82..6f6f240 100644 --- a/drivers/staging/lustre/lnet/libcfs/libcfs_string.c +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_string.c @@ -79,7 +79,7 @@ int cfs_str2mask(const char *str, const char *(*bit2str)(int bit), for (i = 0; i < 32; i++) { debugstr = bit2str(i); if (debugstr && strlen(debugstr) == len && - strncasecmp(str, debugstr, len) == 0) { + !strncasecmp(str, debugstr, len)) { if (op == '-') newmask &= ~(1 << i); else @@ -89,7 +89,7 @@ int cfs_str2mask(const char *str, const char *(*bit2str)(int bit), } } if (!found && len == 3 && - (strncasecmp(str, "ALL", len) == 0)) { + !strncasecmp(str, "ALL", len)) { if (op == '-') newmask = minmask; else @@ -182,7 +182,7 @@ int cfs_str2mask(const char *str, const char *(*bit2str)(int bit), next->ls_len--; } - if (next->ls_len == 0) /* whitespaces only */ + if (!next->ls_len) /* whitespaces only */ return 0; if (*next->ls_str == delim) { @@ -417,7 +417,7 @@ int cfs_str2mask(const char *str, const char *(*bit2str)(int bit), list_for_each_entry(expr, &expr_list->el_exprs, re_link) { if (value >= expr->re_lo && value <= expr->re_hi && - ((value - expr->re_lo) % expr->re_stride) == 0) + !((value - expr->re_lo) % expr->re_stride)) return 1; } @@ -442,12 +442,12 @@ int cfs_str2mask(const char *str, const char *(*bit2str)(int bit), list_for_each_entry(expr, &expr_list->el_exprs, re_link) { for (i = expr->re_lo; i <= expr->re_hi; i++) { - if (((i - expr->re_lo) % expr->re_stride) == 0) + if (!((i - expr->re_lo) % expr->re_stride)) count++; } } - if (count == 0) /* empty expression list */ + if (!count) /* empty expression list */ return 0; if (count > max) { @@ -463,7 +463,7 @@ int cfs_str2mask(const char *str, const char *(*bit2str)(int bit), count = 0; list_for_each_entry(expr, &expr_list->el_exprs, re_link) { for (i = expr->re_lo; i <= expr->re_hi; i++) { - if (((i - expr->re_lo) % expr->re_stride) == 0) + if (!((i - expr->re_lo) % expr->re_stride)) val[count++] = i; } } @@ -540,7 +540,7 @@ int cfs_str2mask(const char *str, const char *(*bit2str)(int bit), } } else { rc = cfs_range_expr_parse(&src, min, max, 0, &expr); - if (rc == 0) + if (!rc) list_add_tail(&expr->re_link, &expr_list->el_exprs); } diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c index 44a8790..8c3fcdb 100644 --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c +++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c @@ -375,7 +375,7 @@ struct cfs_cpt_table * { int i; - if (cpumask_weight(mask) == 0 || + if (!cpumask_weight(mask) || cpumask_any_and(mask, cpu_online_mask) >= nr_cpu_ids) { CDEBUG(D_INFO, "No online CPU is found in the CPU mask for CPU partition %d\n", cpt); @@ -516,7 +516,7 @@ struct cfs_cpt_table * rotor %= weight; for_each_node_mask(node, *mask) { - if (rotor-- == 0) + if (!rotor--) return node; } @@ -584,7 +584,7 @@ struct cfs_cpt_table * rc = set_cpus_allowed_ptr(current, cpumask); set_mems_allowed(*nodemask); - if (rc == 0) + if (!rc) schedule(); /* switch to allowed CPU */ return rc; @@ -658,7 +658,7 @@ struct cfs_cpt_table * goto out; } - if (--number == 0) + if (!--number) goto out; } cpu = cpumask_first(socket); @@ -750,7 +750,7 @@ struct cfs_cpt_table * } num = num_online_cpus() / ncpt; - if (num == 0) { + if (!num) { CERROR("CPU changed while setting CPU partition\n"); goto failed; } @@ -848,7 +848,7 @@ struct cfs_cpt_table * } } - if (ncpt == 0 || + if (!ncpt || (node && ncpt > num_online_nodes()) || (!node && ncpt > num_online_cpus())) { CERROR("Invalid pattern %s, or too many partitions %d\n", diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-crypto.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-crypto.c index 3d85cc5..ac2e055 100644 --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-crypto.c +++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-crypto.c @@ -152,7 +152,7 @@ int cfs_crypto_hash_digest(enum cfs_crypto_hash_alg hash_alg, int err; const struct cfs_crypto_hash_type *type; - if (!buf || buf_len == 0 || !hash_len) + if (!buf || !buf_len || !hash_len) return -EINVAL; err = cfs_crypto_hash_alloc(hash_alg, &type, &req, key, key_len); @@ -440,6 +440,6 @@ int cfs_crypto_register(void) */ void cfs_crypto_unregister(void) { - if (adler32 == 0) + if (!adler32) cfs_crypto_adler32_unregister(); } diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c index 9f5eeb3..a15cedd 100644 --- a/drivers/staging/lustre/lnet/libcfs/tracefile.c +++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c @@ -236,7 +236,7 @@ static void cfs_tcd_shrink(struct cfs_trace_cpu_data *tcd) INIT_LIST_HEAD(&pc.pc_pages); list_for_each_entry_safe(tage, tmp, &tcd->tcd_pages, linkage) { - if (pgcount-- == 0) + if (!pgcount--) break; list_move_tail(&tage->linkage, &pc.pc_pages); @@ -320,7 +320,7 @@ int libcfs_debug_vmsg2(struct libcfs_debug_msg_data *msgdata, if (!tcd) /* arch may not log in IRQ context */ goto console; - if (tcd->tcd_cur_pages == 0) + if (!tcd->tcd_cur_pages) header.ph_flags |= PH_FLAG_FIRST_RECORD; if (tcd->tcd_shutting_down) { @@ -423,7 +423,7 @@ int libcfs_debug_vmsg2(struct libcfs_debug_msg_data *msgdata, __LASSERT(tage->used <= PAGE_SIZE); console: - if ((mask & libcfs_printk) == 0) { + if (!(mask & libcfs_printk)) { /* no console output requested */ if (tcd) cfs_trace_put_tcd(tcd); @@ -871,13 +871,13 @@ int cfs_trace_daemon_command(char *str) cfs_tracefile_write_lock(); - if (strcmp(str, "stop") == 0) { + if (!strcmp(str, "stop")) { cfs_tracefile_write_unlock(); cfs_trace_stop_thread(); cfs_tracefile_write_lock(); memset(cfs_tracefile, 0, sizeof(cfs_tracefile)); - } else if (strncmp(str, "size=", 5) == 0) { + } else if (!strncmp(str, "size=", 5)) { unsigned long tmp; rc = kstrtoul(str + 5, 10, &tmp); @@ -917,7 +917,7 @@ int cfs_trace_daemon_command_usrstr(void __user *usr_str, int usr_str_nob) rc = cfs_trace_copyin_string(str, usr_str_nob + 1, usr_str, usr_str_nob); - if (rc == 0) + if (!rc) rc = cfs_trace_daemon_command(str); kfree(str); @@ -1072,7 +1072,7 @@ static int tracefiled(void *arg) __LASSERT(list_empty(&pc.pc_pages)); end_loop: if (atomic_read(&tctl->tctl_shutdown)) { - if (last_loop == 0) { + if (!last_loop) { last_loop = 1; continue; } else { diff --git a/drivers/staging/lustre/lnet/libcfs/workitem.c b/drivers/staging/lustre/lnet/libcfs/workitem.c index b9eda17..f5f1e9e 100644 --- a/drivers/staging/lustre/lnet/libcfs/workitem.c +++ b/drivers/staging/lustre/lnet/libcfs/workitem.c @@ -325,7 +325,7 @@ static int cfs_wi_scheduler(void *arg) list_del(&sched->ws_list); spin_unlock(&cfs_wi_data.wi_glock); - LASSERT(sched->ws_nscheduled == 0); + LASSERT(!sched->ws_nscheduled); LIBCFS_FREE(sched, sizeof(*sched)); } -- 1.7.1 From jsimmons at infradead.org Thu Nov 17 19:35:45 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 17 Nov 2016 14:35:45 -0500 Subject: [lustre-devel] [PATCH 07/19] staging: lustre: lnet: remove unused lib_me_dump In-Reply-To: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> References: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479411357-28979-8-git-send-email-jsimmons@infradead.org> Remove unused lib_me_dump. Signed-off-by: James Simmons --- drivers/staging/lustre/lnet/lnet/lib-me.c | 18 ------------------ 1 files changed, 0 insertions(+), 18 deletions(-) diff --git a/drivers/staging/lustre/lnet/lnet/lib-me.c b/drivers/staging/lustre/lnet/lnet/lib-me.c index b430046..eb796a8 100644 --- a/drivers/staging/lustre/lnet/lnet/lib-me.c +++ b/drivers/staging/lustre/lnet/lnet/lib-me.c @@ -271,21 +271,3 @@ lnet_res_lh_invalidate(&me->me_lh); lnet_me_free(me); } - -#if 0 -static void -lib_me_dump(lnet_me_t *me) -{ - CWARN("Match Entry %p (%#llx)\n", me, - me->me_lh.lh_cookie); - - CWARN("\tMatch/Ignore\t= %016lx / %016lx\n", - me->me_match_bits, me->me_ignore_bits); - - CWARN("\tMD\t= %p\n", me->md); - CWARN("\tprev\t= %p\n", - list_entry(me->me_list.prev, lnet_me_t, me_list)); - CWARN("\tnext\t= %p\n", - list_entry(me->me_list.next, lnet_me_t, me_list)); -} -#endif -- 1.7.1 From jsimmons at infradead.org Thu Nov 17 19:35:55 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 17 Nov 2016 14:35:55 -0500 Subject: [lustre-devel] [PATCH 17/19] staging: lustre: libcfs: remove use of __u* types In-Reply-To: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> References: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479411357-28979-18-git-send-email-jsimmons@infradead.org> The __[u|s]* types are for UAPI headers or user space. They shouldn't be used in core kernel code. Signed-off-by: James Simmons --- drivers/staging/lustre/lnet/libcfs/fail.c | 4 ++-- drivers/staging/lustre/lnet/libcfs/hash.c | 16 ++++++++-------- drivers/staging/lustre/lnet/libcfs/libcfs_string.c | 6 +++--- .../lustre/lnet/libcfs/linux/linux-module.c | 2 +- 4 files changed, 14 insertions(+), 14 deletions(-) diff --git a/drivers/staging/lustre/lnet/libcfs/fail.c b/drivers/staging/lustre/lnet/libcfs/fail.c index 3a9c8dd..12dd50a 100644 --- a/drivers/staging/lustre/lnet/libcfs/fail.c +++ b/drivers/staging/lustre/lnet/libcfs/fail.c @@ -46,7 +46,7 @@ int cfs_race_state; EXPORT_SYMBOL(cfs_race_state); -int __cfs_fail_check_set(__u32 id, __u32 value, int set) +int __cfs_fail_check_set(u32 id, u32 value, int set) { static atomic_t cfs_fail_count = ATOMIC_INIT(0); @@ -124,7 +124,7 @@ int __cfs_fail_check_set(__u32 id, __u32 value, int set) } EXPORT_SYMBOL(__cfs_fail_check_set); -int __cfs_fail_timeout_set(__u32 id, __u32 value, int ms, int set) +int __cfs_fail_timeout_set(u32 id, u32 value, int ms, int set) { int ret; diff --git a/drivers/staging/lustre/lnet/libcfs/hash.c b/drivers/staging/lustre/lnet/libcfs/hash.c index 12ed71c..b90dfb9 100644 --- a/drivers/staging/lustre/lnet/libcfs/hash.c +++ b/drivers/staging/lustre/lnet/libcfs/hash.c @@ -1035,10 +1035,10 @@ struct cfs_hash * cfs_hash_lock_setup(hs); cfs_hash_hlist_setup(hs); - hs->hs_cur_bits = (__u8)cur_bits; - hs->hs_min_bits = (__u8)cur_bits; - hs->hs_max_bits = (__u8)max_bits; - hs->hs_bkt_bits = (__u8)bkt_bits; + hs->hs_cur_bits = (u8)cur_bits; + hs->hs_min_bits = (u8)cur_bits; + hs->hs_max_bits = (u8)max_bits; + hs->hs_bkt_bits = (u8)bkt_bits; hs->hs_ops = ops; hs->hs_extra_bytes = extra_bytes; @@ -1405,14 +1405,14 @@ void cfs_hash_putref(struct cfs_hash *hs) * . if @removal_safe is true, use can remove current item by * cfs_hash_bd_del_locked */ -static __u64 +static u64 cfs_hash_for_each_tight(struct cfs_hash *hs, cfs_hash_for_each_cb_t func, void *data, int remove_safe) { struct hlist_node *hnode; struct hlist_node *pos; struct cfs_hash_bd bd; - __u64 count = 0; + u64 count = 0; int excl = !!remove_safe; int loop = 0; int i; @@ -1525,7 +1525,7 @@ struct cfs_hash_cond_arg { } EXPORT_SYMBOL(cfs_hash_is_empty); -__u64 +u64 cfs_hash_size_get(struct cfs_hash *hs) { return cfs_hash_with_counter(hs) ? @@ -1556,7 +1556,7 @@ struct cfs_hash_cond_arg { struct hlist_node *hnode; struct hlist_node *tmp; struct cfs_hash_bd bd; - __u32 version; + u32 version; int count = 0; int stop_on_change; int end = -1; diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_string.c b/drivers/staging/lustre/lnet/libcfs/libcfs_string.c index ccebeb4..02de1ee 100644 --- a/drivers/staging/lustre/lnet/libcfs/libcfs_string.c +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_string.c @@ -411,7 +411,7 @@ int cfs_str2mask(const char *str, const char *(*bit2str)(int bit), * \retval 0 otherwise */ int -cfs_expr_list_match(__u32 value, struct cfs_expr_list *expr_list) +cfs_expr_list_match(u32 value, struct cfs_expr_list *expr_list) { struct cfs_range_expr *expr; @@ -433,10 +433,10 @@ int cfs_str2mask(const char *str, const char *(*bit2str)(int bit), * \retval < 0 for failure */ int -cfs_expr_list_values(struct cfs_expr_list *expr_list, int max, __u32 **valpp) +cfs_expr_list_values(struct cfs_expr_list *expr_list, int max, u32 **valpp) { struct cfs_range_expr *expr; - __u32 *val; + u32 *val; int count = 0; int i; diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-module.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-module.c index 38308f8..3f5d58b 100644 --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-module.c +++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-module.c @@ -83,7 +83,7 @@ static inline bool libcfs_ioctl_is_invalid(struct libcfs_ioctl_data *data) CERROR("LIBCFS ioctl: plen2 nonzero but no pbuf2 pointer\n"); return true; } - if ((__u32)libcfs_ioctl_packlen(data) != data->ioc_hdr.ioc_len) { + if ((u32)libcfs_ioctl_packlen(data) != data->ioc_hdr.ioc_len) { CERROR("LIBCFS ioctl: packlen != ioc_len\n"); return true; } -- 1.7.1 From jsimmons at infradead.org Thu Nov 17 19:35:48 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 17 Nov 2016 14:35:48 -0500 Subject: [lustre-devel] [PATCH 10/19] staging: lustre: lnet: replace uninitialized_var In-Reply-To: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> References: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479411357-28979-11-git-send-email-jsimmons@infradead.org> Checkpatch for some reason doesn't like the way libcfs_str2net_internal. Lets just replace it with nf being set to NULL instead Signed-off-by: James Simmons --- drivers/staging/lustre/lnet/lnet/nidstrings.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/drivers/staging/lustre/lnet/lnet/nidstrings.c b/drivers/staging/lustre/lnet/lnet/nidstrings.c index 63379f7..e1f1ca8 100644 --- a/drivers/staging/lustre/lnet/lnet/nidstrings.c +++ b/drivers/staging/lustre/lnet/lnet/nidstrings.c @@ -1156,7 +1156,7 @@ static void cfs_ip_min_max(struct list_head *nidlist, __u32 *min_nid, static struct netstrfns * libcfs_str2net_internal(const char *str, __u32 *net) { - struct netstrfns *uninitialized_var(nf); + struct netstrfns *nf = NULL; int nob; unsigned int netnum; int i; -- 1.7.1 From jsimmons at infradead.org Thu Nov 17 19:35:53 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 17 Nov 2016 14:35:53 -0500 Subject: [lustre-devel] [PATCH 15/19] staging: lustre: libcfs: Make char * array envp static constant In-Reply-To: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> References: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479411357-28979-16-git-send-email-jsimmons@infradead.org> The envp char array can be made static constant. Signed-off-by: James Simmons --- .../staging/lustre/lnet/libcfs/linux/linux-debug.c | 14 ++++++++------ 1 files changed, 8 insertions(+), 6 deletions(-) diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-debug.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-debug.c index 8d37938..d8a9894 100644 --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-debug.c +++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-debug.c @@ -69,10 +69,11 @@ void libcfs_run_debug_log_upcall(char *file) { char *argv[3]; int rc; - char *envp[] = { + static const char * const envp[] = { "HOME=/", "PATH=/sbin:/bin:/usr/sbin:/usr/bin", - NULL}; + NULL + }; argv[0] = lnet_debug_log_upcall; @@ -81,7 +82,7 @@ void libcfs_run_debug_log_upcall(char *file) argv[2] = NULL; - rc = call_usermodehelper(argv[0], argv, envp, 1); + rc = call_usermodehelper(argv[0], argv, (char **)envp, 1); if (rc < 0 && rc != -ENOENT) { CERROR("Error %d invoking LNET debug log upcall %s %s; check /sys/kernel/debug/lnet/debug_log_upcall\n", rc, argv[0], argv[1]); @@ -95,10 +96,11 @@ void libcfs_run_upcall(char **argv) { int rc; int argc; - char *envp[] = { + static const char * const envp[] = { "HOME=/", "PATH=/sbin:/bin:/usr/sbin:/usr/bin", - NULL}; + NULL + }; argv[0] = lnet_upcall; argc = 1; @@ -107,7 +109,7 @@ void libcfs_run_upcall(char **argv) LASSERT(argc >= 2); - rc = call_usermodehelper(argv[0], argv, envp, 1); + rc = call_usermodehelper(argv[0], argv, (char **)envp, 1); if (rc < 0 && rc != -ENOENT) { CERROR("Error %d invoking LNET upcall %s %s%s%s%s%s%s%s%s; check /sys/kernel/debug/lnet/upcall\n", rc, argv[0], argv[1], -- 1.7.1 From jsimmons at infradead.org Thu Nov 17 19:35:41 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 17 Nov 2016 14:35:41 -0500 Subject: [lustre-devel] [PATCH 03/19] staging: lustre: libcfs: use static const char const * for a suffixes array In-Reply-To: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> References: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479411357-28979-4-git-send-email-jsimmons@infradead.org> Change the static const array libcfs_debug_subsystems to use static const char const * as pointed out by checkpatch. Signed-off-by: James Simmons --- drivers/staging/lustre/lnet/libcfs/debug.c | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-) diff --git a/drivers/staging/lustre/lnet/libcfs/debug.c b/drivers/staging/lustre/lnet/libcfs/debug.c index 9fe4227..f95764c 100644 --- a/drivers/staging/lustre/lnet/libcfs/debug.c +++ b/drivers/staging/lustre/lnet/libcfs/debug.c @@ -228,7 +228,8 @@ static int param_set_uintpos(const char *val, const struct kernel_param *kp) static const char * libcfs_debug_subsys2str(int subsys) { - static const char *libcfs_debug_subsystems[] = LIBCFS_DEBUG_SUBSYS_NAMES; + static const char * const libcfs_debug_subsystems[] = + LIBCFS_DEBUG_SUBSYS_NAMES; if (subsys >= ARRAY_SIZE(libcfs_debug_subsystems)) return NULL; @@ -240,7 +241,8 @@ static int param_set_uintpos(const char *val, const struct kernel_param *kp) static const char * libcfs_debug_dbg2str(int debug) { - static const char *libcfs_debug_masks[] = LIBCFS_DEBUG_MASKS_NAMES; + static const char * const libcfs_debug_masks[] = + LIBCFS_DEBUG_MASKS_NAMES; if (debug >= ARRAY_SIZE(libcfs_debug_masks)) return NULL; -- 1.7.1 From jsimmons at infradead.org Thu Nov 17 19:35:40 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 17 Nov 2016 14:35:40 -0500 Subject: [lustre-devel] [PATCH 02/19] staging: lustre: lnet: resolve trailing */ errors In-Reply-To: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> References: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479411357-28979-3-git-send-email-jsimmons@infradead.org> This patch resolves the checkpatch error: Block comments use a trailing */ on a separate line for the LNet/libcfs layer Signed-off-by: James Simmons --- .../staging/lustre/lnet/klnds/socklnd/socklnd.h | 128 +++++++++++++------- drivers/staging/lustre/lnet/libcfs/hash.c | 2 +- .../lustre/lnet/libcfs/linux/linux-crypto.h | 2 +- drivers/staging/lustre/lnet/libcfs/prng.c | 2 +- drivers/staging/lustre/lnet/selftest/console.h | 9 +- drivers/staging/lustre/lnet/selftest/rpc.h | 6 +- drivers/staging/lustre/lnet/selftest/selftest.h | 21 ++- drivers/staging/lustre/lnet/selftest/timer.c | 3 +- 8 files changed, 112 insertions(+), 61 deletions(-) diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.h b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.h index c51c16b..2978014 100644 --- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.h +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.h @@ -84,7 +84,8 @@ struct ksock_sched { /* per scheduler state */ struct list_head kss_zombie_noop_txs; /* zombie noop tx list */ wait_queue_head_t kss_waitq; /* where scheduler sleeps */ int kss_nconns; /* # connections assigned to - * this scheduler */ + * this scheduler + */ struct ksock_sched_info *kss_info; /* owner of it */ }; @@ -110,15 +111,19 @@ struct ksock_interface { /* in-use interface */ struct ksock_tunables { int *ksnd_timeout; /* "stuck" socket timeout - * (seconds) */ + * (seconds) + */ int *ksnd_nscheds; /* # scheduler threads in each - * pool while starting */ + * pool while starting + */ int *ksnd_nconnds; /* # connection daemons */ int *ksnd_nconnds_max; /* max # connection daemons */ int *ksnd_min_reconnectms; /* first connection retry after - * (ms)... */ + * (ms)... + */ int *ksnd_max_reconnectms; /* ...exponentially increasing to - * this */ + * this + */ int *ksnd_eager_ack; /* make TCP ack eagerly? */ int *ksnd_typed_conns; /* drive sockets by type? */ int *ksnd_min_bulk; /* smallest "large" message */ @@ -126,9 +131,11 @@ struct ksock_tunables { int *ksnd_rx_buffer_size; /* socket rx buffer size */ int *ksnd_nagle; /* enable NAGLE? */ int *ksnd_round_robin; /* round robin for multiple - * interfaces */ + * interfaces + */ int *ksnd_keepalive; /* # secs for sending keepalive - * NOOP */ + * NOOP + */ int *ksnd_keepalive_idle; /* # idle secs before 1st probe */ int *ksnd_keepalive_count; /* # probes */ @@ -137,20 +144,26 @@ struct ksock_tunables { int *ksnd_peertxcredits; /* # concurrent sends to 1 peer */ int *ksnd_peerrtrcredits; /* # per-peer router buffer - * credits */ + * credits + */ int *ksnd_peertimeout; /* seconds to consider peer dead */ int *ksnd_enable_csum; /* enable check sum */ int *ksnd_inject_csum_error; /* set non-zero to inject - * checksum error */ + * checksum error + */ int *ksnd_nonblk_zcack; /* always send zc-ack on - * non-blocking connection */ + * non-blocking connection + */ unsigned int *ksnd_zc_min_payload; /* minimum zero copy payload - * size */ + * size + */ int *ksnd_zc_recv; /* enable ZC receive (for - * Chelsio TOE) */ + * Chelsio TOE) + */ int *ksnd_zc_recv_min_nfrags; /* minimum # of fragments to - * enable ZC receive */ + * enable ZC receive + */ }; struct ksock_net { @@ -174,9 +187,11 @@ struct ksock_nal_data { int ksnd_nnets; /* # networks set up */ struct list_head ksnd_nets; /* list of nets */ rwlock_t ksnd_global_lock; /* stabilize peer/conn - * ops */ + * ops + */ struct list_head *ksnd_peers; /* hash table of all my - * known peers */ + * known peers + */ int ksnd_peer_hash_size; /* size of ksnd_peers */ int ksnd_nthreads; /* # live threads */ @@ -187,11 +202,14 @@ struct ksock_nal_data { atomic_t ksnd_nactive_txs; /* #active txs */ struct list_head ksnd_deathrow_conns; /* conns to close: - * reaper_lock*/ + * reaper_lock + */ struct list_head ksnd_zombie_conns; /* conns to free: - * reaper_lock */ + * reaper_lock + */ struct list_head ksnd_enomem_conns; /* conns to retry: - * reaper_lock*/ + * reaper_lock + */ wait_queue_head_t ksnd_reaper_waitq; /* reaper sleeps here */ unsigned long ksnd_reaper_waketime; /* when reaper will wake */ @@ -201,18 +219,21 @@ struct ksock_nal_data { int ksnd_stall_tx; /* test sluggish sender */ int ksnd_stall_rx; /* test sluggish - * receiver */ - + * receiver + */ struct list_head ksnd_connd_connreqs; /* incoming connection - * requests */ + * requests + */ struct list_head ksnd_connd_routes; /* routes waiting to be - * connected */ + * connected + */ wait_queue_head_t ksnd_connd_waitq; /* connds sleep here */ int ksnd_connd_connecting; /* # connds connecting */ time64_t ksnd_connd_failed_stamp;/* time stamp of the * last failed - * connecting attempt */ + * connecting attempt + */ time64_t ksnd_connd_starting_stamp;/* time stamp of the * last starting connd */ @@ -221,10 +242,11 @@ struct ksock_nal_data { spinlock_t ksnd_connd_lock; /* serialise */ struct list_head ksnd_idle_noop_txs; /* list head for freed - * noop tx */ + * noop tx + */ spinlock_t ksnd_tx_lock; /* serialise, g_lock - * unsafe */ - + * unsafe + */ }; #define SOCKNAL_INIT_NOTHING 0 @@ -304,9 +326,11 @@ struct ksock_conn { struct list_head ksnc_list; /* stash on peer's conn list */ struct socket *ksnc_sock; /* actual socket */ void *ksnc_saved_data_ready; /* socket's original - * data_ready() callback */ + * data_ready() callback + */ void *ksnc_saved_write_space; /* socket's original - * write_space() callback */ + * write_space() callback + */ atomic_t ksnc_conn_refcount;/* conn refcount */ atomic_t ksnc_sock_refcount;/* sock refcount */ struct ksock_sched *ksnc_scheduler; /* who schedules this connection @@ -315,7 +339,8 @@ struct ksock_conn { __u32 ksnc_ipaddr; /* peer's IP */ int ksnc_port; /* peer's port */ signed int ksnc_type:3; /* type of connection, should be - * signed value */ + * signed value + */ unsigned int ksnc_closing:1; /* being shut down */ unsigned int ksnc_flip:1; /* flip or not, only for V2.x */ unsigned int ksnc_zc_capable:1; /* enable to ZC */ @@ -323,9 +348,11 @@ struct ksock_conn { /* reader */ struct list_head ksnc_rx_list; /* where I enq waiting input or a - * forwarding descriptor */ + * forwarding descriptor + */ unsigned long ksnc_rx_deadline; /* when (in jiffies) receive times - * out */ + * out + */ __u8 ksnc_rx_started; /* started receiving a message */ __u8 ksnc_rx_ready; /* data ready to read */ __u8 ksnc_rx_scheduled; /* being progressed */ @@ -338,7 +365,8 @@ struct ksock_conn { lnet_kiov_t *ksnc_rx_kiov; /* the page frags */ union ksock_rxiovspace ksnc_rx_iov_space; /* space for frag descriptors */ __u32 ksnc_rx_csum; /* partial checksum for incoming - * data */ + * data + */ void *ksnc_cookie; /* rx lnet_finalize passthru arg */ ksock_msg_t ksnc_msg; /* incoming message buffer: @@ -346,14 +374,16 @@ struct ksock_conn { * whole struct * V1.x message is a bare * lnet_hdr_t, it's stored in - * ksnc_msg.ksm_u.lnetmsg */ - + * ksnc_msg.ksm_u.lnetmsg + */ /* WRITER */ struct list_head ksnc_tx_list; /* where I enq waiting for output - * space */ + * space + */ struct list_head ksnc_tx_queue; /* packets waiting to be sent */ - struct ksock_tx *ksnc_tx_carrier; /* next TX that can carry a LNet - * message or ZC-ACK */ + struct ksock_tx *ksnc_tx_carrier; /* next TX that can carry a LNet + * message or ZC-ACK + */ unsigned long ksnc_tx_deadline; /* when (in jiffies) tx times out */ int ksnc_tx_bufnob; /* send buffer marker */ @@ -361,7 +391,8 @@ struct ksock_conn { int ksnc_tx_ready; /* write space */ int ksnc_tx_scheduled; /* being progressed */ unsigned long ksnc_tx_last_post; /* time stamp of the last posted - * TX */ + * TX + */ }; struct ksock_route { @@ -370,20 +401,24 @@ struct ksock_route { struct ksock_peer *ksnr_peer; /* owning peer */ atomic_t ksnr_refcount; /* # users */ unsigned long ksnr_timeout; /* when (in jiffies) reconnection - * can happen next */ + * can happen next + */ long ksnr_retry_interval; /* how long between retries */ __u32 ksnr_myipaddr; /* my IP */ __u32 ksnr_ipaddr; /* IP address to connect to */ int ksnr_port; /* port to connect to */ unsigned int ksnr_scheduled:1; /* scheduled for attention */ unsigned int ksnr_connecting:1; /* connection establishment in - * progress */ + * progress + */ unsigned int ksnr_connected:4; /* connections established by - * type */ + * type + */ unsigned int ksnr_deleted:1; /* been removed from peer? */ unsigned int ksnr_share_count; /* created explicitly? */ int ksnr_conn_count; /* # conns established by this - * route */ + * route + */ }; #define SOCKNAL_KEEPALIVE_PING 1 /* cookie for keepalive ping */ @@ -391,7 +426,8 @@ struct ksock_route { struct ksock_peer { struct list_head ksnp_list; /* stash on global peer list */ unsigned long ksnp_last_alive; /* when (in jiffies) I was last - * alive */ + * alive + */ lnet_process_id_t ksnp_id; /* who's on the other end(s) */ atomic_t ksnp_refcount; /* # users */ int ksnp_sharecount; /* lconf usage counter */ @@ -408,7 +444,8 @@ struct ksock_peer { struct list_head ksnp_tx_queue; /* waiting packets */ spinlock_t ksnp_lock; /* serialize, g_lock unsafe */ struct list_head ksnp_zc_req_list; /* zero copy requests wait for - * ACK */ + * ACK + */ unsigned long ksnp_send_keepalive; /* time to send keepalive */ lnet_ni_t *ksnp_ni; /* which network */ int ksnp_n_passive_ips; /* # of... */ @@ -429,7 +466,8 @@ struct ksock_connreq { #define SOCKNAL_MATCH_NO 0 /* TX can't match type of connection */ #define SOCKNAL_MATCH_YES 1 /* TX matches type of connection */ #define SOCKNAL_MATCH_MAY 2 /* TX can be sent on the connection, but not - * preferred */ + * preferred + */ struct ksock_proto { /* version number of protocol */ diff --git a/drivers/staging/lustre/lnet/libcfs/hash.c b/drivers/staging/lustre/lnet/libcfs/hash.c index 3137f12..530def6 100644 --- a/drivers/staging/lustre/lnet/libcfs/hash.c +++ b/drivers/staging/lustre/lnet/libcfs/hash.c @@ -1716,7 +1716,7 @@ struct cfs_hash_cond_arg { * the passed callback @func and pass to it as an argument each hash * item and the private @data. During the callback the bucket lock * is held so the callback must never sleep. - */ + */ void cfs_hash_for_each_key(struct cfs_hash *hs, const void *key, cfs_hash_for_each_cb_t func, void *data) diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-crypto.h b/drivers/staging/lustre/lnet/libcfs/linux/linux-crypto.h index 18e8cd4..d0b3aa8 100644 --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-crypto.h +++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-crypto.h @@ -1,4 +1,4 @@ - /* +/* * GPL HEADER START * * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. diff --git a/drivers/staging/lustre/lnet/libcfs/prng.c b/drivers/staging/lustre/lnet/libcfs/prng.c index a9bdb28..21d5a39 100644 --- a/drivers/staging/lustre/lnet/libcfs/prng.c +++ b/drivers/staging/lustre/lnet/libcfs/prng.c @@ -33,7 +33,7 @@ * x(n)=a*x(n-1)+carry mod 2^16 and y(n)=b*y(n-1)+carry mod 2^16, * number and carry packed within the same 32 bit integer. * algorithm recommended by Marsaglia -*/ + */ #include "../../include/linux/libcfs/libcfs.h" diff --git a/drivers/staging/lustre/lnet/selftest/console.h b/drivers/staging/lustre/lnet/selftest/console.h index 5b11b2c..5dc1de4 100644 --- a/drivers/staging/lustre/lnet/selftest/console.h +++ b/drivers/staging/lustre/lnet/selftest/console.h @@ -92,14 +92,16 @@ struct lstcon_batch { int bat_ntest; /* # of test */ int bat_state; /* state of the batch */ int bat_arg; /* parameter for run|stop, timeout - * for run, force for stop */ + * for run, force for stop + */ char bat_name[LST_NAME_SIZE];/* name of batch */ struct list_head bat_test_list; /* list head of tests (struct lstcon_test) */ struct list_head bat_trans_list; /* list head of transaction */ struct list_head bat_cli_list; /* list head of client nodes - * (struct lstcon_node) */ + * (struct lstcon_node) + */ struct list_head *bat_cli_hash; /* hash table of client nodes */ struct list_head bat_srv_list; /* list head of server nodes */ struct list_head *bat_srv_hash; /* hash table of server nodes */ @@ -147,7 +149,8 @@ struct lstcon_session { unsigned int ses_features; /* tests features of the session */ unsigned int ses_feats_updated:1; /* features are synced with - * remote test nodes */ + * remote test nodes + */ unsigned int ses_force:1; /* force creating */ unsigned int ses_shutdown:1; /* session is shutting down */ unsigned int ses_expired:1; /* console is timedout */ diff --git a/drivers/staging/lustre/lnet/selftest/rpc.h b/drivers/staging/lustre/lnet/selftest/rpc.h index 8a0c18e..f353a63 100644 --- a/drivers/staging/lustre/lnet/selftest/rpc.h +++ b/drivers/staging/lustre/lnet/selftest/rpc.h @@ -113,7 +113,8 @@ struct srpc_join_reply { __u32 join_status; /* returned status */ lst_sid_t join_sid; /* session id */ __u32 join_timeout; /* # seconds' inactivity to - * expire */ + * expire + */ char join_session[LST_NAME_SIZE]; /* session name */ } WIRE_ATTR; @@ -190,7 +191,8 @@ struct srpc_test_reqst { lst_bid_t tsr_bid; /* batch id */ __u32 tsr_service; /* test type: bulk|ping|... */ __u32 tsr_loop; /* test client loop count or - * # server buffers needed */ + * # server buffers needed + */ __u32 tsr_concur; /* concurrency of test */ __u8 tsr_is_client; /* is test client or not */ __u8 tsr_stop_onerr; /* stop on error */ diff --git a/drivers/staging/lustre/lnet/selftest/selftest.h b/drivers/staging/lustre/lnet/selftest/selftest.h index 4a1eb29..c8833a0 100644 --- a/drivers/staging/lustre/lnet/selftest/selftest.h +++ b/drivers/staging/lustre/lnet/selftest/selftest.h @@ -131,7 +131,8 @@ enum srpc_event_type { SRPC_BULK_REQ_RCVD = 1, /* passive bulk request(PUT sink/GET source) - * received */ + * received + */ SRPC_BULK_PUT_SENT = 2, /* active bulk PUT sent (source) */ SRPC_BULK_GET_RPLD = 3, /* active bulk GET replied (sink) */ SRPC_REPLY_RCVD = 4, /* incoming reply received */ @@ -295,7 +296,8 @@ struct srpc_service_cd { #define SFW_TEST_WI_MIN 256 #define SFW_TEST_WI_MAX 2048 /* extra buffers for tolerating buggy peers, or unbalanced number - * of peers between partitions */ + * of peers between partitions + */ #define SFW_TEST_WI_EXTRA 64 /* number of server workitems (mini-thread) for framework service */ @@ -347,9 +349,11 @@ struct sfw_batch { struct sfw_test_client_ops { int (*tso_init)(struct sfw_test_instance *tsi); /* initialize test - * client */ + * client + */ void (*tso_fini)(struct sfw_test_instance *tsi); /* finalize test - * client */ + * client + */ int (*tso_prep_rpc)(struct sfw_test_unit *tsu, lnet_process_id_t dest, struct srpc_client_rpc **rpc); /* prep a tests rpc */ @@ -374,7 +378,8 @@ struct sfw_test_instance { spinlock_t tsi_lock; /* serialize */ unsigned int tsi_stopping:1; /* test is stopping */ atomic_t tsi_nactive; /* # of active test - * unit */ + * unit + */ struct list_head tsi_units; /* test units */ struct list_head tsi_free_rpcs; /* free rpcs */ struct list_head tsi_active_rpcs; /* active rpcs */ @@ -386,8 +391,10 @@ struct sfw_test_instance { } tsi_u; }; -/* XXX: trailing (PAGE_SIZE % sizeof(lnet_process_id_t)) bytes at the end of - * pages are not used */ +/* + * XXX: trailing (PAGE_SIZE % sizeof(lnet_process_id_t)) bytes at the end of + * pages are not used + */ #define SFW_MAX_CONCUR LST_MAX_CONCUR #define SFW_ID_PER_PAGE (PAGE_SIZE / sizeof(lnet_process_id_packed_t)) #define SFW_MAX_NDESTS (LNET_MAX_IOV * SFW_ID_PER_PAGE) diff --git a/drivers/staging/lustre/lnet/selftest/timer.c b/drivers/staging/lustre/lnet/selftest/timer.c index dcd2258..a4b723b 100644 --- a/drivers/staging/lustre/lnet/selftest/timer.c +++ b/drivers/staging/lustre/lnet/selftest/timer.c @@ -55,7 +55,8 @@ static struct st_timer_data { spinlock_t stt_lock; unsigned long stt_prev_slot; /* start time of the slot processed - * previously */ + * previously + */ struct list_head stt_hash[STTIMER_NSLOTS]; int stt_shuttingdown; wait_queue_head_t stt_waitq; -- 1.7.1 From jsimmons at infradead.org Thu Nov 17 19:35:39 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 17 Nov 2016 14:35:39 -0500 Subject: [lustre-devel] [PATCH 01/19] staging: lustre: lnet: don't use bare unsigned In-Reply-To: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> References: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479411357-28979-2-git-send-email-jsimmons@infradead.org> Turn all bare unsigned to unsigned int that were detected by checkpatch in the LNet/libcfs layer. Signed-off-by: James Simmons --- .../staging/lustre/lnet/klnds/socklnd/socklnd.h | 4 +- drivers/staging/lustre/lnet/libcfs/debug.c | 2 +- drivers/staging/lustre/lnet/libcfs/hash.c | 26 ++++++++++---------- drivers/staging/lustre/lnet/libcfs/libcfs_string.c | 8 +++--- .../staging/lustre/lnet/libcfs/linux/linux-cpu.c | 6 ++-- drivers/staging/lustre/lnet/lnet/nidstrings.c | 2 +- drivers/staging/lustre/lnet/selftest/conctl.c | 2 +- drivers/staging/lustre/lnet/selftest/conrpc.c | 18 +++++++------ drivers/staging/lustre/lnet/selftest/conrpc.h | 20 ++++++++------- drivers/staging/lustre/lnet/selftest/console.c | 10 ++++--- drivers/staging/lustre/lnet/selftest/console.h | 16 ++++++------ drivers/staging/lustre/lnet/selftest/framework.c | 8 +++--- drivers/staging/lustre/lnet/selftest/ping_test.c | 4 +- drivers/staging/lustre/lnet/selftest/selftest.h | 4 +- 14 files changed, 68 insertions(+), 62 deletions(-) diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.h b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.h index e6ca0cf..c51c16b 100644 --- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.h +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.h @@ -216,8 +216,8 @@ struct ksock_nal_data { time64_t ksnd_connd_starting_stamp;/* time stamp of the * last starting connd */ - unsigned ksnd_connd_starting; /* # starting connd */ - unsigned ksnd_connd_running; /* # running connd */ + unsigned int ksnd_connd_starting; /* # starting connd */ + unsigned int ksnd_connd_running; /* # running connd */ spinlock_t ksnd_connd_lock; /* serialise */ struct list_head ksnd_idle_noop_txs; /* list head for freed diff --git a/drivers/staging/lustre/lnet/libcfs/debug.c b/drivers/staging/lustre/lnet/libcfs/debug.c index 23b36b8..9fe4227 100644 --- a/drivers/staging/lustre/lnet/libcfs/debug.c +++ b/drivers/staging/lustre/lnet/libcfs/debug.c @@ -57,7 +57,7 @@ static int libcfs_param_debug_mb_set(const char *val, const struct kernel_param *kp) { int rc; - unsigned num; + unsigned int num; rc = kstrtouint(val, 0, &num); if (rc < 0) diff --git a/drivers/staging/lustre/lnet/libcfs/hash.c b/drivers/staging/lustre/lnet/libcfs/hash.c index 997b8a5..3137f12 100644 --- a/drivers/staging/lustre/lnet/libcfs/hash.c +++ b/drivers/staging/lustre/lnet/libcfs/hash.c @@ -682,7 +682,7 @@ struct hlist_node * static void cfs_hash_multi_bd_lock(struct cfs_hash *hs, struct cfs_hash_bd *bds, - unsigned n, int excl) + unsigned int n, int excl) { struct cfs_hash_bucket *prev = NULL; int i; @@ -704,7 +704,7 @@ struct hlist_node * static void cfs_hash_multi_bd_unlock(struct cfs_hash *hs, struct cfs_hash_bd *bds, - unsigned n, int excl) + unsigned int n, int excl) { struct cfs_hash_bucket *prev = NULL; int i; @@ -719,10 +719,10 @@ struct hlist_node * static struct hlist_node * cfs_hash_multi_bd_lookup_locked(struct cfs_hash *hs, struct cfs_hash_bd *bds, - unsigned n, const void *key) + unsigned int n, const void *key) { struct hlist_node *ehnode; - unsigned i; + unsigned int i; cfs_hash_for_each_bd(bds, n, i) { ehnode = cfs_hash_bd_lookup_intent(hs, &bds[i], key, NULL, @@ -735,12 +735,12 @@ struct hlist_node * static struct hlist_node * cfs_hash_multi_bd_findadd_locked(struct cfs_hash *hs, struct cfs_hash_bd *bds, - unsigned n, const void *key, + unsigned int n, const void *key, struct hlist_node *hnode, int noref) { struct hlist_node *ehnode; int intent; - unsigned i; + unsigned int i; LASSERT(hnode); intent = (!noref * CFS_HS_LOOKUP_MASK_REF) | CFS_HS_LOOKUP_IT_PEEK; @@ -766,7 +766,7 @@ struct hlist_node * static struct hlist_node * cfs_hash_multi_bd_finddel_locked(struct cfs_hash *hs, struct cfs_hash_bd *bds, - unsigned n, const void *key, + unsigned int n, const void *key, struct hlist_node *hnode) { struct hlist_node *ehnode; @@ -992,10 +992,10 @@ static inline void cfs_hash_depth_wi_cancel(struct cfs_hash *hs) {} #endif /* CFS_HASH_DEBUG_LEVEL >= CFS_HASH_DEBUG_1 */ struct cfs_hash * -cfs_hash_create(char *name, unsigned cur_bits, unsigned max_bits, - unsigned bkt_bits, unsigned extra_bytes, - unsigned min_theta, unsigned max_theta, - struct cfs_hash_ops *ops, unsigned flags) +cfs_hash_create(char *name, unsigned int cur_bits, unsigned int max_bits, + unsigned int bkt_bits, unsigned int extra_bytes, + unsigned int min_theta, unsigned int max_theta, + struct cfs_hash_ops *ops, unsigned int flags) { struct cfs_hash *hs; int len; @@ -1664,7 +1664,7 @@ struct cfs_hash_cond_arg { cfs_hash_for_each_empty(struct cfs_hash *hs, cfs_hash_for_each_cb_t func, void *data) { - unsigned i = 0; + unsigned int i = 0; if (cfs_hash_with_no_lock(hs)) return -EOPNOTSUPP; @@ -1684,7 +1684,7 @@ struct cfs_hash_cond_arg { EXPORT_SYMBOL(cfs_hash_for_each_empty); void -cfs_hash_hlist_for_each(struct cfs_hash *hs, unsigned hindex, +cfs_hash_hlist_for_each(struct cfs_hash *hs, unsigned int hindex, cfs_hash_for_each_cb_t func, void *data) { struct hlist_head *hhead; diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_string.c b/drivers/staging/lustre/lnet/libcfs/libcfs_string.c index 56a614d..8db9157 100644 --- a/drivers/staging/lustre/lnet/libcfs/libcfs_string.c +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_string.c @@ -222,8 +222,8 @@ int cfs_str2mask(const char *str, const char *(*bit2str)(int bit), * \retval 0 otherwise */ int -cfs_str2num_check(char *str, int nob, unsigned *num, - unsigned min, unsigned max) +cfs_str2num_check(char *str, int nob, unsigned int *num, + unsigned int min, unsigned int max) { bool all_numbers = true; char *endp, cache; @@ -273,7 +273,7 @@ int cfs_str2mask(const char *str, const char *(*bit2str)(int bit), * -ENOMEM will be returned. */ static int -cfs_range_expr_parse(struct cfs_lstr *src, unsigned min, unsigned max, +cfs_range_expr_parse(struct cfs_lstr *src, unsigned int min, unsigned int max, int bracketed, struct cfs_range_expr **expr) { struct cfs_range_expr *re; @@ -501,7 +501,7 @@ int cfs_str2mask(const char *str, const char *(*bit2str)(int bit), * \retval -errno otherwise */ int -cfs_expr_list_parse(char *str, int len, unsigned min, unsigned max, +cfs_expr_list_parse(char *str, int len, unsigned int min, unsigned int max, struct cfs_expr_list **elpp) { struct cfs_expr_list *expr_list; diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c index 6a31521..268a8ef 100644 --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c +++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c @@ -678,9 +678,9 @@ struct cfs_cpt_table * static unsigned int cfs_cpt_num_estimate(void) { - unsigned nnode = num_online_nodes(); - unsigned ncpu = num_online_cpus(); - unsigned ncpt; + unsigned int nnode = num_online_nodes(); + unsigned int ncpu = num_online_cpus(); + unsigned int ncpt; if (ncpu <= CPT_WEIGHT_MIN) { ncpt = 1; diff --git a/drivers/staging/lustre/lnet/lnet/nidstrings.c b/drivers/staging/lustre/lnet/lnet/nidstrings.c index a6d7a61..63379f7 100644 --- a/drivers/staging/lustre/lnet/lnet/nidstrings.c +++ b/drivers/staging/lustre/lnet/lnet/nidstrings.c @@ -193,7 +193,7 @@ struct addrrange { struct netstrfns *nf; struct nidrange *nr; int endlen; - unsigned netnum; + unsigned int netnum; if (src->ls_len >= LNET_NIDSTR_SIZE) return NULL; diff --git a/drivers/staging/lustre/lnet/selftest/conctl.c b/drivers/staging/lustre/lnet/selftest/conctl.c index b786f8b..02847bf 100644 --- a/drivers/staging/lustre/lnet/selftest/conctl.c +++ b/drivers/staging/lustre/lnet/selftest/conctl.c @@ -315,7 +315,7 @@ static int lst_nodes_add_ioctl(lstio_group_nodes_args_t *args) { - unsigned feats; + unsigned int feats; int rc; char *name; diff --git a/drivers/staging/lustre/lnet/selftest/conrpc.c b/drivers/staging/lustre/lnet/selftest/conrpc.c index 8a3a2da..3d071b4 100644 --- a/drivers/staging/lustre/lnet/selftest/conrpc.c +++ b/drivers/staging/lustre/lnet/selftest/conrpc.c @@ -86,7 +86,7 @@ void lstcon_rpc_stat_reply(struct lstcon_rpc_trans *, struct srpc_msg *, } static int -lstcon_rpc_init(struct lstcon_node *nd, int service, unsigned feats, +lstcon_rpc_init(struct lstcon_node *nd, int service, unsigned int feats, int bulk_npg, int bulk_len, int embedded, struct lstcon_rpc *crpc) { crpc->crp_rpc = sfw_create_rpc(nd->nd_id, service, @@ -111,7 +111,7 @@ void lstcon_rpc_stat_reply(struct lstcon_rpc_trans *, struct srpc_msg *, } static int -lstcon_rpc_prep(struct lstcon_node *nd, int service, unsigned feats, +lstcon_rpc_prep(struct lstcon_node *nd, int service, unsigned int feats, int bulk_npg, int bulk_len, struct lstcon_rpc **crpcpp) { struct lstcon_rpc *crpc = NULL; @@ -589,7 +589,7 @@ void lstcon_rpc_stat_reply(struct lstcon_rpc_trans *, struct srpc_msg *, int lstcon_sesrpc_prep(struct lstcon_node *nd, int transop, - unsigned feats, struct lstcon_rpc **crpc) + unsigned int feats, struct lstcon_rpc **crpc) { struct srpc_mksn_reqst *msrq; struct srpc_rmsn_reqst *rsrq; @@ -627,7 +627,8 @@ void lstcon_rpc_stat_reply(struct lstcon_rpc_trans *, struct srpc_msg *, } int -lstcon_dbgrpc_prep(struct lstcon_node *nd, unsigned feats, struct lstcon_rpc **crpc) +lstcon_dbgrpc_prep(struct lstcon_node *nd, unsigned int feats, + struct lstcon_rpc **crpc) { struct srpc_debug_reqst *drq; int rc; @@ -645,7 +646,7 @@ void lstcon_rpc_stat_reply(struct lstcon_rpc_trans *, struct srpc_msg *, } int -lstcon_batrpc_prep(struct lstcon_node *nd, int transop, unsigned feats, +lstcon_batrpc_prep(struct lstcon_node *nd, int transop, unsigned int feats, struct lstcon_tsb_hdr *tsb, struct lstcon_rpc **crpc) { struct lstcon_batch *batch; @@ -678,7 +679,8 @@ void lstcon_rpc_stat_reply(struct lstcon_rpc_trans *, struct srpc_msg *, } int -lstcon_statrpc_prep(struct lstcon_node *nd, unsigned feats, struct lstcon_rpc **crpc) +lstcon_statrpc_prep(struct lstcon_node *nd, unsigned int feats, + struct lstcon_rpc **crpc) { struct srpc_stat_reqst *srq; int rc; @@ -803,7 +805,7 @@ void lstcon_rpc_stat_reply(struct lstcon_rpc_trans *, struct srpc_msg *, } int -lstcon_testrpc_prep(struct lstcon_node *nd, int transop, unsigned feats, +lstcon_testrpc_prep(struct lstcon_node *nd, int transop, unsigned int feats, struct lstcon_test *test, struct lstcon_rpc **crpc) { struct lstcon_group *sgrp = test->tes_src_grp; @@ -1086,7 +1088,7 @@ void lstcon_rpc_stat_reply(struct lstcon_rpc_trans *, struct srpc_msg *, struct lstcon_ndlink *ndl; struct lstcon_node *nd; struct lstcon_rpc *rpc; - unsigned feats; + unsigned int feats; int rc; /* Creating session RPG for list of nodes */ diff --git a/drivers/staging/lustre/lnet/selftest/conrpc.h b/drivers/staging/lustre/lnet/selftest/conrpc.h index 7ec6fc9..27ba639 100644 --- a/drivers/staging/lustre/lnet/selftest/conrpc.h +++ b/drivers/staging/lustre/lnet/selftest/conrpc.h @@ -78,8 +78,8 @@ struct lstcon_rpc_trans { struct list_head tas_olink; /* link chain on owner list */ struct list_head tas_link; /* link chain on global list */ int tas_opc; /* operation code of transaction */ - unsigned tas_feats_updated; /* features mask is uptodate */ - unsigned tas_features; /* test features mask */ + unsigned int tas_feats_updated; /* features mask is uptodate */ + unsigned int tas_features; /* test features mask */ wait_queue_head_t tas_waitq; /* wait queue head */ atomic_t tas_remaining; /* # of un-scheduled rpcs */ struct list_head tas_rpcs_list; /* queued requests */ @@ -106,14 +106,16 @@ typedef int (*lstcon_rpc_readent_func_t)(int, struct srpc_msg *, lstcon_rpc_ent_t __user *); int lstcon_sesrpc_prep(struct lstcon_node *nd, int transop, - unsigned version, struct lstcon_rpc **crpc); + unsigned int version, struct lstcon_rpc **crpc); int lstcon_dbgrpc_prep(struct lstcon_node *nd, - unsigned version, struct lstcon_rpc **crpc); -int lstcon_batrpc_prep(struct lstcon_node *nd, int transop, unsigned version, - struct lstcon_tsb_hdr *tsb, struct lstcon_rpc **crpc); -int lstcon_testrpc_prep(struct lstcon_node *nd, int transop, unsigned version, - struct lstcon_test *test, struct lstcon_rpc **crpc); -int lstcon_statrpc_prep(struct lstcon_node *nd, unsigned version, + unsigned int version, struct lstcon_rpc **crpc); +int lstcon_batrpc_prep(struct lstcon_node *nd, int transop, + unsigned int version, struct lstcon_tsb_hdr *tsb, + struct lstcon_rpc **crpc); +int lstcon_testrpc_prep(struct lstcon_node *nd, int transop, + unsigned int version, struct lstcon_test *test, + struct lstcon_rpc **crpc); +int lstcon_statrpc_prep(struct lstcon_node *nd, unsigned int version, struct lstcon_rpc **crpc); void lstcon_rpc_put(struct lstcon_rpc *crpc); int lstcon_rpc_trans_prep(struct list_head *translist, diff --git a/drivers/staging/lustre/lnet/selftest/console.c b/drivers/staging/lustre/lnet/selftest/console.c index a0fcbf3..faa5320 100644 --- a/drivers/staging/lustre/lnet/selftest/console.c +++ b/drivers/staging/lustre/lnet/selftest/console.c @@ -397,7 +397,8 @@ static int lstcon_group_nodes_add(struct lstcon_group *grp, int count, lnet_process_id_t __user *ids_up, - unsigned *featp, struct list_head __user *result_up) + unsigned int *featp, + struct list_head __user *result_up) { struct lstcon_rpc_trans *trans; struct lstcon_ndlink *ndl; @@ -542,7 +543,8 @@ int lstcon_nodes_add(char *name, int count, lnet_process_id_t __user *ids_up, - unsigned *featp, struct list_head __user *result_up) + unsigned int *featp, + struct list_head __user *result_up) { struct lstcon_group *grp; int rc; @@ -1702,7 +1704,7 @@ } int -lstcon_session_new(char *name, int key, unsigned feats, +lstcon_session_new(char *name, int key, unsigned int feats, int timeout, int force, lst_sid_t __user *sid_up) { int rc = 0; @@ -1868,7 +1870,7 @@ } int -lstcon_session_feats_check(unsigned feats) +lstcon_session_feats_check(unsigned int feats) { int rc = 0; diff --git a/drivers/staging/lustre/lnet/selftest/console.h b/drivers/staging/lustre/lnet/selftest/console.h index 78388a6..5b11b2c 100644 --- a/drivers/staging/lustre/lnet/selftest/console.h +++ b/drivers/staging/lustre/lnet/selftest/console.h @@ -144,13 +144,13 @@ struct lstcon_session { int ses_timeout; /* timeout in seconds */ time64_t ses_laststamp; /* last operation stamp (seconds) */ - unsigned ses_features; /* tests features of the session + unsigned int ses_features; /* tests features of the session */ - unsigned ses_feats_updated:1; /* features are synced with + unsigned int ses_feats_updated:1; /* features are synced with * remote test nodes */ - unsigned ses_force:1; /* force creating */ - unsigned ses_shutdown:1; /* session is shutting down */ - unsigned ses_expired:1; /* console is timedout */ + unsigned int ses_force:1; /* force creating */ + unsigned int ses_shutdown:1; /* session is shutting down */ + unsigned int ses_expired:1; /* console is timedout */ __u64 ses_id_cookie; /* batch id cookie */ char ses_name[LST_NAME_SIZE];/* session name */ struct lstcon_rpc_trans *ses_ping; /* session pinger */ @@ -188,14 +188,14 @@ struct lstcon_session { int lstcon_console_init(void); int lstcon_console_fini(void); int lstcon_session_match(lst_sid_t sid); -int lstcon_session_new(char *name, int key, unsigned version, +int lstcon_session_new(char *name, int key, unsigned int version, int timeout, int flags, lst_sid_t __user *sid_up); int lstcon_session_info(lst_sid_t __user *sid_up, int __user *key, unsigned __user *verp, lstcon_ndlist_ent_t __user *entp, char __user *name_up, int len); int lstcon_session_end(void); int lstcon_session_debug(int timeout, struct list_head __user *result_up); -int lstcon_session_feats_check(unsigned feats); +int lstcon_session_feats_check(unsigned int feats); int lstcon_batch_debug(int timeout, char *name, int client, struct list_head __user *result_up); int lstcon_group_debug(int timeout, char *name, @@ -207,7 +207,7 @@ int lstcon_nodes_debug(int timeout, int nnd, lnet_process_id_t __user *nds_up, int lstcon_group_clean(char *name, int args); int lstcon_group_refresh(char *name, struct list_head __user *result_up); int lstcon_nodes_add(char *name, int nnd, lnet_process_id_t __user *nds_up, - unsigned *featp, struct list_head __user *result_up); + unsigned int *featp, struct list_head __user *result_up); int lstcon_nodes_remove(char *name, int nnd, lnet_process_id_t __user *nds_up, struct list_head __user *result_up); int lstcon_group_info(char *name, lstcon_ndlist_ent_t __user *gent_up, diff --git a/drivers/staging/lustre/lnet/selftest/framework.c b/drivers/staging/lustre/lnet/selftest/framework.c index 2774327..8f9ee3f 100644 --- a/drivers/staging/lustre/lnet/selftest/framework.c +++ b/drivers/staging/lustre/lnet/selftest/framework.c @@ -254,7 +254,7 @@ static inline void sfw_init_session(struct sfw_session *sn, lst_sid_t sid, - unsigned features, const char *name) + unsigned int features, const char *name) { struct stt_timer *timer = &sn->sn_timer; @@ -897,7 +897,7 @@ int sfw_create_test_rpc(struct sfw_test_unit *tsu, lnet_process_id_t peer, - unsigned features, int nblk, int blklen, + unsigned int features, int nblk, int blklen, struct srpc_client_rpc **rpcpp) { struct srpc_client_rpc *rpc = NULL; @@ -1225,7 +1225,7 @@ struct srpc_service *sv = rpc->srpc_scd->scd_svc; struct srpc_msg *reply = &rpc->srpc_replymsg; struct srpc_msg *request = &rpc->srpc_reqstbuf->buf_msg; - unsigned features = LST_FEATS_MASK; + unsigned int features = LST_FEATS_MASK; int rc = 0; LASSERT(!sfw_data.fw_active_srpc); @@ -1375,7 +1375,7 @@ struct srpc_client_rpc * sfw_create_rpc(lnet_process_id_t peer, int service, - unsigned features, int nbulkiov, int bulklen, + unsigned int features, int nbulkiov, int bulklen, void (*done)(struct srpc_client_rpc *), void *priv) { struct srpc_client_rpc *rpc = NULL; diff --git a/drivers/staging/lustre/lnet/selftest/ping_test.c b/drivers/staging/lustre/lnet/selftest/ping_test.c index 9331ca4..b9601b0 100644 --- a/drivers/staging/lustre/lnet/selftest/ping_test.c +++ b/drivers/staging/lustre/lnet/selftest/ping_test.c @@ -159,8 +159,8 @@ struct lst_ping_data { ktime_get_real_ts64(&ts); CDEBUG(D_NET, "%d reply in %u usec\n", reply->pnr_seq, - (unsigned)((ts.tv_sec - reqst->pnr_time_sec) * 1000000 + - (ts.tv_nsec / NSEC_PER_USEC - reqst->pnr_time_usec))); + (unsigned int)((ts.tv_sec - reqst->pnr_time_sec) * 1000000 + + (ts.tv_nsec / NSEC_PER_USEC - reqst->pnr_time_usec))); } static int diff --git a/drivers/staging/lustre/lnet/selftest/selftest.h b/drivers/staging/lustre/lnet/selftest/selftest.h index 877bb36..4a1eb29 100644 --- a/drivers/staging/lustre/lnet/selftest/selftest.h +++ b/drivers/staging/lustre/lnet/selftest/selftest.h @@ -410,10 +410,10 @@ struct sfw_test_case { struct srpc_client_rpc * sfw_create_rpc(lnet_process_id_t peer, int service, - unsigned features, int nbulkiov, int bulklen, + unsigned int features, int nbulkiov, int bulklen, void (*done)(struct srpc_client_rpc *), void *priv); int sfw_create_test_rpc(struct sfw_test_unit *tsu, - lnet_process_id_t peer, unsigned features, + lnet_process_id_t peer, unsigned int features, int nblk, int blklen, struct srpc_client_rpc **rpc); void sfw_abort_rpc(struct srpc_client_rpc *rpc); void sfw_post_rpc(struct srpc_client_rpc *rpc); -- 1.7.1 From jsimmons at infradead.org Thu Nov 17 19:35:46 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 17 Nov 2016 14:35:46 -0500 Subject: [lustre-devel] [PATCH 08/19] staging: lustre: lnet: missing blank line after declaration In-Reply-To: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> References: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479411357-28979-9-git-send-email-jsimmons@infradead.org> Add in missing blank line in lnet_copy_iov2iter() after variable declarations. Signed-off-by: James Simmons --- drivers/staging/lustre/lnet/lnet/lib-move.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/drivers/staging/lustre/lnet/lnet/lib-move.c b/drivers/staging/lustre/lnet/lnet/lib-move.c index 48e6f8f..f3dd6e4 100644 --- a/drivers/staging/lustre/lnet/lnet/lib-move.c +++ b/drivers/staging/lustre/lnet/lnet/lib-move.c @@ -192,6 +192,7 @@ left = siov->iov_len - soffset; do { size_t n, copy = left; + LASSERT(nsiov > 0); if (copy > nob) -- 1.7.1 From jsimmons at infradead.org Thu Nov 17 19:35:56 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 17 Nov 2016 14:35:56 -0500 Subject: [lustre-devel] [PATCH 18/19] staging: lustre: lnet: use BIT macro in LNet selftest In-Reply-To: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> References: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479411357-28979-19-git-send-email-jsimmons@infradead.org> Some of the defines for lnet selftest can use the BIT macros. Signed-off-by: James Simmons --- drivers/staging/lustre/lnet/selftest/timer.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/staging/lustre/lnet/selftest/timer.c b/drivers/staging/lustre/lnet/selftest/timer.c index a4b723b..2fe692d 100644 --- a/drivers/staging/lustre/lnet/selftest/timer.c +++ b/drivers/staging/lustre/lnet/selftest/timer.c @@ -46,9 +46,9 @@ * to cover a time period of 1024 seconds into the future before wrapping. */ #define STTIMER_MINPOLL 3 /* log2 min poll interval (8 s) */ -#define STTIMER_SLOTTIME (1 << STTIMER_MINPOLL) +#define STTIMER_SLOTTIME BIT(STTIMER_MINPOLL) #define STTIMER_SLOTTIMEMASK (~(STTIMER_SLOTTIME - 1)) -#define STTIMER_NSLOTS (1 << 7) +#define STTIMER_NSLOTS BIT(7) #define STTIMER_SLOT(t) (&stt_data.stt_hash[(((t) >> STTIMER_MINPOLL) & \ (STTIMER_NSLOTS - 1))]) -- 1.7.1 From jsimmons at infradead.org Thu Nov 17 19:35:44 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 17 Nov 2016 14:35:44 -0500 Subject: [lustre-devel] [PATCH 06/19] staging: lustre: lnet: fix misspelled word destroy In-Reply-To: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> References: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479411357-28979-7-git-send-email-jsimmons@infradead.org> Fix misspelling of destroy in LNet core. Signed-off-by: James Simmons --- drivers/staging/lustre/lnet/lnet/api-ni.c | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/drivers/staging/lustre/lnet/lnet/api-ni.c b/drivers/staging/lustre/lnet/lnet/api-ni.c index 4daf828..b2ba10d 100644 --- a/drivers/staging/lustre/lnet/lnet/api-ni.c +++ b/drivers/staging/lustre/lnet/lnet/api-ni.c @@ -1551,16 +1551,16 @@ void lnet_lib_exit(void) rc = lnet_check_routes(); if (rc) - goto err_destory_routes; + goto err_destroy_routes; rc = lnet_rtrpools_alloc(im_a_router); if (rc) - goto err_destory_routes; + goto err_destroy_routes; } rc = lnet_acceptor_start(); if (rc) - goto err_destory_routes; + goto err_destroy_routes; the_lnet.ln_refcount = 1; /* Now I may use my own API functions... */ @@ -1587,7 +1587,7 @@ void lnet_lib_exit(void) err_acceptor_stop: the_lnet.ln_refcount = 0; lnet_acceptor_stop(); -err_destory_routes: +err_destroy_routes: if (!the_lnet.ln_nis_from_mod_params) lnet_destroy_routes(); err_shutdown_lndnis: -- 1.7.1 From jsimmons at infradead.org Thu Nov 17 19:35:52 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 17 Nov 2016 14:35:52 -0500 Subject: [lustre-devel] [PATCH 14/19] staging: lustre: libcfs: fix misspelling of descriptor In-Reply-To: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> References: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479411357-28979-15-git-send-email-jsimmons@infradead.org> Fix typo for correct spelling. Signed-off-by: James Simmons --- .../lustre/lnet/libcfs/linux/linux-crypto.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-crypto.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-crypto.c index 53b1bf7..68e34b4 100644 --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-crypto.c +++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-crypto.c @@ -177,7 +177,7 @@ int cfs_crypto_hash_digest(enum cfs_crypto_hash_alg hash_alg, EXPORT_SYMBOL(cfs_crypto_hash_digest); /** - * Allocate and initialize desriptor for hash algorithm. + * Allocate and initialize descriptor for hash algorithm. * * This should be used to initialize a hash descriptor for multiple calls * to a single hash function when computing the hash across multiple -- 1.7.1 From jsimmons at infradead.org Thu Nov 17 19:35:47 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 17 Nov 2016 14:35:47 -0500 Subject: [lustre-devel] [PATCH 09/19] staging: lustre: lnet: fill in real lnet_md_t In-Reply-To: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> References: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479411357-28979-10-git-send-email-jsimmons@infradead.org> While checkpatch reported an alignment issue its just ugly to fill in a data structure being passed to a function. Instead fill in a lnet_md_t on the stack and pass that to LNetMDBind. Signed-off-by: James Simmons --- drivers/staging/lustre/lnet/lnet/router.c | 17 +++++++++-------- 1 files changed, 9 insertions(+), 8 deletions(-) diff --git a/drivers/staging/lustre/lnet/lnet/router.c b/drivers/staging/lustre/lnet/lnet/router.c index 063ad55..8afa0ab 100644 --- a/drivers/staging/lustre/lnet/lnet/router.c +++ b/drivers/staging/lustre/lnet/lnet/router.c @@ -903,6 +903,7 @@ int lnet_get_rtr_pool_cfg(int idx, struct lnet_ioctl_pool_cfg *pool_cfg) { lnet_rc_data_t *rcd = NULL; lnet_ping_info_t *pi; + lnet_md_t md; int rc; int i; @@ -925,15 +926,15 @@ int lnet_get_rtr_pool_cfg(int idx, struct lnet_ioctl_pool_cfg *pool_cfg) } rcd->rcd_pinginfo = pi; + md.start = pi; + md.user_ptr = rcd; + md.length = LNET_PINGINFO_SIZE; + md.threshold = LNET_MD_THRESH_INF; + md.options = LNET_MD_TRUNCATE; + md.eq_handle = the_lnet.ln_rc_eqh; + LASSERT(!LNetHandleIsInvalid(the_lnet.ln_rc_eqh)); - rc = LNetMDBind((lnet_md_t){.start = pi, - .user_ptr = rcd, - .length = LNET_PINGINFO_SIZE, - .threshold = LNET_MD_THRESH_INF, - .options = LNET_MD_TRUNCATE, - .eq_handle = the_lnet.ln_rc_eqh}, - LNET_UNLINK, - &rcd->rcd_mdh); + rc = LNetMDBind(md, LNET_UNLINK, &rcd->rcd_mdh); if (rc < 0) { CERROR("Can't bind MD: %d\n", rc); goto out; -- 1.7.1 From jsimmons at infradead.org Thu Nov 17 19:35:54 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 17 Nov 2016 14:35:54 -0500 Subject: [lustre-devel] [PATCH 16/19] staging: lustre: libcfs: use pr_* instead of printk In-Reply-To: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> References: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479411357-28979-17-git-send-email-jsimmons@infradead.org> Checkpatch recommended that we use pr_*() instead of printk directly. Signed-off-by: James Simmons --- .../lustre/lnet/libcfs/linux/linux-tracefile.c | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-) diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-tracefile.c index a3cabb9..75eb84e 100644 --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-tracefile.c +++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-tracefile.c @@ -85,7 +85,7 @@ int cfs_tracefile_init_arch(void) out: cfs_tracefile_fini_arch(); - printk(KERN_ERR "lnet: Not enough memory\n"); + pr_err("lnet: Not enough memory\n"); return -ENOMEM; } @@ -239,11 +239,11 @@ void cfs_print_to_console(struct ptldebug_header *hdr, int mask, } if (mask & D_CONSOLE) { - printk("%s%s: %.*s", ptype, prefix, len, buf); + pr_info("%s%s: %.*s", ptype, prefix, len, buf); } else { - printk("%s%s: %d:%d:(%s:%d:%s()) %.*s", ptype, prefix, - hdr->ph_pid, hdr->ph_extern_pid, file, hdr->ph_line_num, - fn, len, buf); + pr_info("%s%s: %d:%d:(%s:%d:%s()) %.*s", ptype, prefix, + hdr->ph_pid, hdr->ph_extern_pid, file, + hdr->ph_line_num, fn, len, buf); } } -- 1.7.1 From jsimmons at infradead.org Thu Nov 17 19:35:42 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 17 Nov 2016 14:35:42 -0500 Subject: [lustre-devel] [PATCH 04/19] staging: lustre: libcfs: remove explicit test of NULL variable In-Reply-To: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> References: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479411357-28979-5-git-send-email-jsimmons@infradead.org> Remove != NULL which is not needed to test key existence. Signed-off-by: James Simmons --- drivers/staging/lustre/lnet/libcfs/libcfs_lock.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_lock.c b/drivers/staging/lustre/lnet/libcfs/libcfs_lock.c index 83543f9..1deea76 100644 --- a/drivers/staging/lustre/lnet/libcfs/libcfs_lock.c +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_lock.c @@ -73,7 +73,7 @@ struct cfs_percpt_lock * cfs_percpt_for_each(lock, i, pcl->pcl_locks) { spin_lock_init(lock); - if (keys != NULL) + if (keys) lockdep_set_class(lock, &keys[i]); } -- 1.7.1 From jsimmons at infradead.org Thu Nov 17 19:35:43 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 17 Nov 2016 14:35:43 -0500 Subject: [lustre-devel] [PATCH 05/19] staging: lustre: libcfs: fix aligment issue In-Reply-To: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> References: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479411357-28979-6-git-send-email-jsimmons@infradead.org> Make alignment match open parenthesis for parameters to wait_event_interruptible_exclusive() call. Signed-off-by: James Simmons --- drivers/staging/lustre/lnet/libcfs/workitem.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/drivers/staging/lustre/lnet/libcfs/workitem.c b/drivers/staging/lustre/lnet/libcfs/workitem.c index e98c818..84118b5 100644 --- a/drivers/staging/lustre/lnet/libcfs/workitem.c +++ b/drivers/staging/lustre/lnet/libcfs/workitem.c @@ -273,7 +273,7 @@ static int cfs_wi_scheduler(void *arg) spin_unlock(&sched->ws_lock); rc = wait_event_interruptible_exclusive(sched->ws_waitq, - !cfs_wi_sched_cansleep(sched)); + !cfs_wi_sched_cansleep(sched)); spin_lock(&sched->ws_lock); } -- 1.7.1 From jsimmons at infradead.org Thu Nov 17 19:35:51 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 17 Nov 2016 14:35:51 -0500 Subject: [lustre-devel] [PATCH 13/19] staging: lustre: libcfs: white space cleanup In-Reply-To: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> References: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479411357-28979-14-git-send-email-jsimmons@infradead.org> Remove white space present for variable declarations or initialization. Cleanup structs was strange alignments due to white spacing. Signed-off-by: James Simmons --- drivers/staging/lustre/lnet/libcfs/debug.c | 16 ++-- drivers/staging/lustre/lnet/libcfs/hash.c | 24 +++--- drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c | 2 +- drivers/staging/lustre/lnet/libcfs/libcfs_lock.c | 14 ++-- drivers/staging/lustre/lnet/libcfs/libcfs_mem.c | 22 ++-- drivers/staging/lustre/lnet/libcfs/libcfs_string.c | 24 +++--- .../staging/lustre/lnet/libcfs/linux/linux-cpu.c | 100 ++++++++++---------- .../lustre/lnet/libcfs/linux/linux-crypto.c | 20 ++-- .../staging/lustre/lnet/libcfs/linux/linux-debug.c | 6 +- .../staging/lustre/lnet/libcfs/linux/linux-prim.c | 10 +- .../lustre/lnet/libcfs/linux/linux-tracefile.c | 8 +- drivers/staging/lustre/lnet/libcfs/module.c | 14 ++-- drivers/staging/lustre/lnet/libcfs/tracefile.c | 80 ++++++++-------- drivers/staging/lustre/lnet/libcfs/tracefile.h | 28 +++--- drivers/staging/lustre/lnet/libcfs/workitem.c | 40 ++++---- 15 files changed, 204 insertions(+), 204 deletions(-) diff --git a/drivers/staging/lustre/lnet/libcfs/debug.c b/drivers/staging/lustre/lnet/libcfs/debug.c index 67f043e..a38db23 100644 --- a/drivers/staging/lustre/lnet/libcfs/debug.c +++ b/drivers/staging/lustre/lnet/libcfs/debug.c @@ -255,9 +255,9 @@ static int param_set_uintpos(const char *val, const struct kernel_param *kp) { const char *(*fn)(int bit) = is_subsys ? libcfs_debug_subsys2str : libcfs_debug_dbg2str; - int len = 0; - const char *token; - int i; + int len = 0; + const char *token; + int i; if (!mask) { /* "0" */ if (size > 0) @@ -301,10 +301,10 @@ static int param_set_uintpos(const char *val, const struct kernel_param *kp) { const char *(*fn)(int bit) = is_subsys ? libcfs_debug_subsys2str : libcfs_debug_dbg2str; - int m = 0; - int matched; - int n; - int t; + int m = 0; + int matched; + int n; + int t; /* Allow a number for backwards compatibility */ @@ -389,8 +389,8 @@ void libcfs_debug_dumplog(void) int libcfs_debug_init(unsigned long bufsize) { - int rc = 0; unsigned int max = libcfs_debug_mb; + int rc = 0; init_waitqueue_head(&debug_ctlwq); diff --git a/drivers/staging/lustre/lnet/libcfs/hash.c b/drivers/staging/lustre/lnet/libcfs/hash.c index 22e1e25..12ed71c 100644 --- a/drivers/staging/lustre/lnet/libcfs/hash.c +++ b/drivers/staging/lustre/lnet/libcfs/hash.c @@ -289,7 +289,7 @@ struct cfs_hash_head_dep { static struct hlist_head * cfs_hash_hd_hhead(struct cfs_hash *hs, struct cfs_hash_bd *bd) { - struct cfs_hash_head_dep *head; + struct cfs_hash_head_dep *head; head = (struct cfs_hash_head_dep *)&bd->bd_bucket->hsb_head[0]; return &head[bd->bd_offset].hd_head; @@ -512,9 +512,9 @@ struct cfs_hash_dhead_dep { return; spin_lock(&hs->hs_dep_lock); - hs->hs_dep_max = dep_cur; - hs->hs_dep_bkt = bd->bd_bucket->hsb_index; - hs->hs_dep_off = bd->bd_offset; + hs->hs_dep_max = dep_cur; + hs->hs_dep_bkt = bd->bd_bucket->hsb_index; + hs->hs_dep_off = bd->bd_offset; hs->hs_dep_bits = hs->hs_cur_bits; spin_unlock(&hs->hs_dep_lock); @@ -908,9 +908,9 @@ struct hlist_node * return NULL; } - new_bkts[i]->hsb_index = i; - new_bkts[i]->hsb_version = 1; /* shouldn't be zero */ - new_bkts[i]->hsb_depmax = -1; /* unknown */ + new_bkts[i]->hsb_index = i; + new_bkts[i]->hsb_version = 1; /* shouldn't be zero */ + new_bkts[i]->hsb_depmax = -1; /* unknown */ bd.bd_bucket = new_bkts[i]; cfs_hash_bd_for_each_hlist(hs, &bd, hhead) INIT_HLIST_HEAD(hhead); @@ -950,9 +950,9 @@ static int cfs_hash_dep_print(struct cfs_workitem *wi) int bits; spin_lock(&hs->hs_dep_lock); - dep = hs->hs_dep_max; - bkt = hs->hs_dep_bkt; - off = hs->hs_dep_off; + dep = hs->hs_dep_max; + bkt = hs->hs_dep_bkt; + off = hs->hs_dep_off; bits = hs->hs_dep_bits; spin_unlock(&hs->hs_dep_lock); @@ -1040,7 +1040,7 @@ struct cfs_hash * hs->hs_max_bits = (__u8)max_bits; hs->hs_bkt_bits = (__u8)bkt_bits; - hs->hs_ops = ops; + hs->hs_ops = ops; hs->hs_extra_bytes = extra_bytes; hs->hs_rehash_bits = 0; cfs_wi_init(&hs->hs_rehash_wi, hs, cfs_hash_rehash_worker); @@ -1292,7 +1292,7 @@ void cfs_hash_putref(struct cfs_hash *hs) } if (hnode) { - obj = cfs_hash_object(hs, hnode); + obj = cfs_hash_object(hs, hnode); bits = cfs_hash_rehash_bits(hs); } diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c index 33352af..55caa19 100644 --- a/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_cpu.c @@ -74,7 +74,7 @@ struct cfs_cpt_table * int cfs_cpt_table_print(struct cfs_cpt_table *cptab, char *buf, int len) { - int rc; + int rc; rc = snprintf(buf, len, "%d\t: %d\n", 0, 0); len -= rc; diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_lock.c b/drivers/staging/lustre/lnet/libcfs/libcfs_lock.c index 66f020b..1967b97 100644 --- a/drivers/staging/lustre/lnet/libcfs/libcfs_lock.c +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_lock.c @@ -52,9 +52,9 @@ struct cfs_percpt_lock * cfs_percpt_lock_create(struct cfs_cpt_table *cptab, struct lock_class_key *keys) { - struct cfs_percpt_lock *pcl; - spinlock_t *lock; - int i; + struct cfs_percpt_lock *pcl; + spinlock_t *lock; + int i; /* NB: cptab can be NULL, pcl will be for HW CPUs on that case */ LIBCFS_ALLOC(pcl, sizeof(*pcl)); @@ -94,8 +94,8 @@ struct cfs_percpt_lock * cfs_percpt_lock(struct cfs_percpt_lock *pcl, int index) __acquires(pcl->pcl_locks) { - int ncpt = cfs_cpt_number(pcl->pcl_cptab); - int i; + int ncpt = cfs_cpt_number(pcl->pcl_cptab); + int i; LASSERT(index >= CFS_PERCPT_LOCK_EX && index < ncpt); @@ -130,8 +130,8 @@ struct cfs_percpt_lock * cfs_percpt_unlock(struct cfs_percpt_lock *pcl, int index) __releases(pcl->pcl_locks) { - int ncpt = cfs_cpt_number(pcl->pcl_cptab); - int i; + int ncpt = cfs_cpt_number(pcl->pcl_cptab); + int i; index = ncpt == 1 ? 0 : index; diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_mem.c b/drivers/staging/lustre/lnet/libcfs/libcfs_mem.c index d0e81bb..ef085ba 100644 --- a/drivers/staging/lustre/lnet/libcfs/libcfs_mem.c +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_mem.c @@ -43,8 +43,8 @@ struct cfs_var_array { void cfs_percpt_free(void *vars) { - struct cfs_var_array *arr; - int i; + struct cfs_var_array *arr; + int i; arr = container_of(vars, struct cfs_var_array, va_ptrs[0]); @@ -72,9 +72,9 @@ struct cfs_var_array { void * cfs_percpt_alloc(struct cfs_cpt_table *cptab, unsigned int size) { - struct cfs_var_array *arr; - int count; - int i; + struct cfs_var_array *arr; + int count; + int i; count = cfs_cpt_number(cptab); @@ -120,8 +120,8 @@ struct cfs_var_array { void cfs_array_free(void *vars) { - struct cfs_var_array *arr; - int i; + struct cfs_var_array *arr; + int i; arr = container_of(vars, struct cfs_var_array, va_ptrs[0]); @@ -144,15 +144,15 @@ struct cfs_var_array { void * cfs_array_alloc(int count, unsigned int size) { - struct cfs_var_array *arr; - int i; + struct cfs_var_array *arr; + int i; LIBCFS_ALLOC(arr, offsetof(struct cfs_var_array, va_ptrs[count])); if (!arr) return NULL; - arr->va_count = count; - arr->va_size = size; + arr->va_count = count; + arr->va_size = size; for (i = 0; i < count; i++) { LIBCFS_ALLOC(arr->va_ptrs[i], size); diff --git a/drivers/staging/lustre/lnet/libcfs/libcfs_string.c b/drivers/staging/lustre/lnet/libcfs/libcfs_string.c index 6f6f240..ccebeb4 100644 --- a/drivers/staging/lustre/lnet/libcfs/libcfs_string.c +++ b/drivers/staging/lustre/lnet/libcfs/libcfs_string.c @@ -112,7 +112,7 @@ int cfs_str2mask(const char *str, const char *(*bit2str)(int bit), char *cfs_firststr(char *str, size_t size) { size_t i = 0; - char *end; + char *end; /* trim leading spaces */ while (i < size && *str && isspace(*str)) { @@ -276,8 +276,8 @@ int cfs_str2mask(const char *str, const char *(*bit2str)(int bit), cfs_range_expr_parse(struct cfs_lstr *src, unsigned int min, unsigned int max, int bracketed, struct cfs_range_expr **expr) { - struct cfs_range_expr *re; - struct cfs_lstr tok; + struct cfs_range_expr *re; + struct cfs_lstr tok; LIBCFS_ALLOC(re, sizeof(*re)); if (!re) @@ -413,7 +413,7 @@ int cfs_str2mask(const char *str, const char *(*bit2str)(int bit), int cfs_expr_list_match(__u32 value, struct cfs_expr_list *expr_list) { - struct cfs_range_expr *expr; + struct cfs_range_expr *expr; list_for_each_entry(expr, &expr_list->el_exprs, re_link) { if (value >= expr->re_lo && value <= expr->re_hi && @@ -435,10 +435,10 @@ int cfs_str2mask(const char *str, const char *(*bit2str)(int bit), int cfs_expr_list_values(struct cfs_expr_list *expr_list, int max, __u32 **valpp) { - struct cfs_range_expr *expr; - __u32 *val; - int count = 0; - int i; + struct cfs_range_expr *expr; + __u32 *val; + int count = 0; + int i; list_for_each_entry(expr, &expr_list->el_exprs, re_link) { for (i = expr->re_lo; i <= expr->re_hi; i++) { @@ -504,10 +504,10 @@ int cfs_str2mask(const char *str, const char *(*bit2str)(int bit), cfs_expr_list_parse(char *str, int len, unsigned int min, unsigned int max, struct cfs_expr_list **elpp) { - struct cfs_expr_list *expr_list; - struct cfs_range_expr *expr; - struct cfs_lstr src; - int rc; + struct cfs_expr_list *expr_list; + struct cfs_range_expr *expr; + struct cfs_lstr src; + int rc; LIBCFS_ALLOC(expr_list, sizeof(*expr_list)); if (!expr_list) diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c index 8c3fcdb..6b9cf06 100644 --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c +++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c @@ -90,7 +90,7 @@ struct cfs_cpt_data { void cfs_cpt_table_free(struct cfs_cpt_table *cptab) { - int i; + int i; if (cptab->ctb_cpu2cpt) { LIBCFS_FREE(cptab->ctb_cpu2cpt, @@ -128,7 +128,7 @@ struct cfs_cpt_table * cfs_cpt_table_alloc(unsigned int ncpt) { struct cfs_cpt_table *cptab; - int i; + int i; LIBCFS_ALLOC(cptab, sizeof(*cptab)); if (!cptab) @@ -179,10 +179,10 @@ struct cfs_cpt_table * int cfs_cpt_table_print(struct cfs_cpt_table *cptab, char *buf, int len) { - char *tmp = buf; - int rc = 0; - int i; - int j; + char *tmp = buf; + int rc = 0; + int i; + int j; for (i = 0; i < cptab->ctb_nparts; i++) { if (len > 0) { @@ -273,7 +273,7 @@ struct cfs_cpt_table * int cfs_cpt_set_cpu(struct cfs_cpt_table *cptab, int cpt, int cpu) { - int node; + int node; LASSERT(cpt >= 0 && cpt < cptab->ctb_nparts); @@ -313,8 +313,8 @@ struct cfs_cpt_table * void cfs_cpt_unset_cpu(struct cfs_cpt_table *cptab, int cpt, int cpu) { - int node; - int i; + int node; + int i; LASSERT(cpt == CFS_CPT_ANY || (cpt >= 0 && cpt < cptab->ctb_nparts)); @@ -373,7 +373,7 @@ struct cfs_cpt_table * int cfs_cpt_set_cpumask(struct cfs_cpt_table *cptab, int cpt, cpumask_t *mask) { - int i; + int i; if (!cpumask_weight(mask) || cpumask_any_and(mask, cpu_online_mask) >= nr_cpu_ids) { @@ -394,7 +394,7 @@ struct cfs_cpt_table * void cfs_cpt_unset_cpumask(struct cfs_cpt_table *cptab, int cpt, cpumask_t *mask) { - int i; + int i; for_each_cpu(i, mask) cfs_cpt_unset_cpu(cptab, cpt, i); @@ -404,8 +404,8 @@ struct cfs_cpt_table * int cfs_cpt_set_node(struct cfs_cpt_table *cptab, int cpt, int node) { - cpumask_t *mask; - int rc; + cpumask_t *mask; + int rc; if (node < 0 || node >= MAX_NUMNODES) { CDEBUG(D_INFO, @@ -451,7 +451,7 @@ struct cfs_cpt_table * int cfs_cpt_set_nodemask(struct cfs_cpt_table *cptab, int cpt, nodemask_t *mask) { - int i; + int i; for_each_node_mask(i, *mask) { if (!cfs_cpt_set_node(cptab, cpt, i)) @@ -465,7 +465,7 @@ struct cfs_cpt_table * void cfs_cpt_unset_nodemask(struct cfs_cpt_table *cptab, int cpt, nodemask_t *mask) { - int i; + int i; for_each_node_mask(i, *mask) cfs_cpt_unset_node(cptab, cpt, i); @@ -475,8 +475,8 @@ struct cfs_cpt_table * void cfs_cpt_clear(struct cfs_cpt_table *cptab, int cpt) { - int last; - int i; + int last; + int i; if (cpt == CFS_CPT_ANY) { last = cptab->ctb_nparts - 1; @@ -495,10 +495,10 @@ struct cfs_cpt_table * int cfs_cpt_spread_node(struct cfs_cpt_table *cptab, int cpt) { - nodemask_t *mask; - int weight; - int rotor; - int node; + nodemask_t *mask; + int weight; + int rotor; + int node; /* convert CPU partition ID to HW node id */ @@ -528,8 +528,8 @@ struct cfs_cpt_table * int cfs_cpt_current(struct cfs_cpt_table *cptab, int remap) { - int cpu = smp_processor_id(); - int cpt = cptab->ctb_cpu2cpt[cpu]; + int cpu = smp_processor_id(); + int cpt = cptab->ctb_cpu2cpt[cpu]; if (cpt < 0) { if (!remap) @@ -557,10 +557,10 @@ struct cfs_cpt_table * int cfs_cpt_bind(struct cfs_cpt_table *cptab, int cpt) { - cpumask_t *cpumask; - nodemask_t *nodemask; - int rc; - int i; + cpumask_t *cpumask; + nodemask_t *nodemask; + int rc; + int i; LASSERT(cpt == CFS_CPT_ANY || (cpt >= 0 && cpt < cptab->ctb_nparts)); @@ -603,10 +603,10 @@ struct cfs_cpt_table * cfs_cpt_choose_ncpus(struct cfs_cpt_table *cptab, int cpt, cpumask_t *node, int number) { - cpumask_t *socket = NULL; - cpumask_t *core = NULL; - int rc = 0; - int cpu; + cpumask_t *socket = NULL; + cpumask_t *core = NULL; + int rc = 0; + int cpu; LASSERT(number > 0); @@ -640,7 +640,7 @@ struct cfs_cpt_table * LASSERT(!cpumask_empty(socket)); while (!cpumask_empty(socket)) { - int i; + int i; /* get cpumask for hts in the same core */ cpumask_copy(core, topology_sibling_cpumask(cpu)); @@ -665,7 +665,7 @@ struct cfs_cpt_table * } } - out: +out: if (socket) LIBCFS_FREE(socket, cpumask_size()); if (core) @@ -705,7 +705,7 @@ struct cfs_cpt_table * ncpt = nnode; - out: +out: #if (BITS_PER_LONG == 32) /* config many CPU partitions on 32-bit system could consume * too much memory @@ -722,11 +722,11 @@ struct cfs_cpt_table * cfs_cpt_table_create(int ncpt) { struct cfs_cpt_table *cptab = NULL; - cpumask_t *mask = NULL; - int cpt = 0; - int num; - int rc; - int i; + cpumask_t *mask = NULL; + int cpt = 0; + int num; + int rc; + int i; rc = cfs_cpt_num_estimate(); if (ncpt <= 0) @@ -766,7 +766,7 @@ struct cfs_cpt_table * while (!cpumask_empty(mask)) { struct cfs_cpu_partition *part; - int n; + int n; /* * Each emulated NUMA node has all allowed CPUs in @@ -819,14 +819,14 @@ struct cfs_cpt_table * static struct cfs_cpt_table * cfs_cpt_table_create_pattern(char *pattern) { - struct cfs_cpt_table *cptab; + struct cfs_cpt_table *cptab; char *str; - int node = 0; - int high; + int node = 0; + int high; int ncpt = 0; int cpt; int rc; - int c; + int c; int i; str = cfs_trimwhite(pattern); @@ -882,10 +882,10 @@ struct cfs_cpt_table * high = node ? MAX_NUMNODES - 1 : nr_cpu_ids - 1; for (str = cfs_trimwhite(pattern), c = 0;; c++) { - struct cfs_range_expr *range; - struct cfs_expr_list *el; - char *bracket = strchr(str, '['); - int n; + struct cfs_range_expr *range; + struct cfs_expr_list *el; + char *bracket = strchr(str, '['); + int n; if (!bracket) { if (*str) { @@ -970,8 +970,8 @@ struct cfs_cpt_table * static int cfs_cpu_notify(struct notifier_block *self, unsigned long action, void *hcpu) { - unsigned int cpu = (unsigned long)hcpu; - bool warn; + unsigned int cpu = (unsigned long)hcpu; + bool warn; switch (action) { case CPU_DEAD: diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-crypto.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-crypto.c index ac2e055..53b1bf7 100644 --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-crypto.c +++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-crypto.c @@ -64,7 +64,7 @@ static int cfs_crypto_hash_alloc(enum cfs_crypto_hash_alg hash_alg, unsigned int key_len) { struct crypto_ahash *tfm; - int err = 0; + int err = 0; *type = cfs_crypto_hash_type(hash_alg); @@ -147,10 +147,10 @@ int cfs_crypto_hash_digest(enum cfs_crypto_hash_alg hash_alg, unsigned char *key, unsigned int key_len, unsigned char *hash, unsigned int *hash_len) { - struct scatterlist sl; + struct scatterlist sl; struct ahash_request *req; - int err; - const struct cfs_crypto_hash_type *type; + int err; + const struct cfs_crypto_hash_type *type; if (!buf || !buf_len || !hash_len) return -EINVAL; @@ -198,8 +198,8 @@ struct cfs_crypto_hash_desc * unsigned char *key, unsigned int key_len) { struct ahash_request *req; - int err; - const struct cfs_crypto_hash_type *type; + int err; + const struct cfs_crypto_hash_type *type; err = cfs_crypto_hash_alloc(hash_alg, &type, &req, key, key_len); @@ -273,7 +273,7 @@ int cfs_crypto_hash_update(struct cfs_crypto_hash_desc *hdesc, int cfs_crypto_hash_final(struct cfs_crypto_hash_desc *hdesc, unsigned char *hash, unsigned int *hash_len) { - int err; + int err; struct ahash_request *req = (void *)hdesc; int size = crypto_ahash_digestsize(crypto_ahash_reqtfm(req)); @@ -312,8 +312,8 @@ static void cfs_crypto_performance_test(enum cfs_crypto_hash_alg hash_alg) { int buf_len = max(PAGE_SIZE, 1048576UL); void *buf; - unsigned long start, end; - int bcount, err = 0; + unsigned long start, end; + int bcount, err = 0; struct page *page; unsigned char hash[CFS_CRYPTO_HASH_DIGESTSIZE_MAX]; unsigned int hash_len = sizeof(hash); @@ -358,7 +358,7 @@ static void cfs_crypto_performance_test(enum cfs_crypto_hash_alg hash_alg) CDEBUG(D_INFO, "Crypto hash algorithm %s test error: rc = %d\n", cfs_crypto_hash_name(hash_alg), err); } else { - unsigned long tmp; + unsigned long tmp; tmp = ((bcount * buf_len / jiffies_to_msecs(end - start)) * 1000) / (1024 * 1024); diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-debug.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-debug.c index 435b784..8d37938 100644 --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-debug.c +++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-debug.c @@ -68,7 +68,7 @@ void libcfs_run_debug_log_upcall(char *file) { char *argv[3]; - int rc; + int rc; char *envp[] = { "HOME=/", "PATH=/sbin:/bin:/usr/sbin:/usr/bin", @@ -93,8 +93,8 @@ void libcfs_run_debug_log_upcall(char *file) void libcfs_run_upcall(char **argv) { - int rc; - int argc; + int rc; + int argc; char *envp[] = { "HOME=/", "PATH=/sbin:/bin:/usr/sbin:/usr/bin", diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-prim.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-prim.c index 291d286..cf90215 100644 --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-prim.c +++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-prim.c @@ -45,8 +45,8 @@ sigset_t cfs_block_allsigs(void) { - unsigned long flags; - sigset_t old; + unsigned long flags; + sigset_t old; spin_lock_irqsave(¤t->sighand->siglock, flags); old = current->blocked; @@ -60,8 +60,8 @@ sigset_t cfs_block_sigs(unsigned long sigs) { - unsigned long flags; - sigset_t old; + unsigned long flags; + sigset_t old; spin_lock_irqsave(¤t->sighand->siglock, flags); old = current->blocked; @@ -91,7 +91,7 @@ sigset_t cfs_block_sigsinv(unsigned long sigs) void cfs_restore_sigs(sigset_t old) { - unsigned long flags; + unsigned long flags; spin_lock_irqsave(¤t->sighand->siglock, flags); current->blocked = old; diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-tracefile.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-tracefile.c index 404c2cc..a3cabb9 100644 --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-tracefile.c +++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-tracefile.c @@ -49,8 +49,8 @@ int cfs_tracefile_init_arch(void) { - int i; - int j; + int i; + int j; struct cfs_trace_cpu_data *tcd; /* initialize trace_data */ @@ -91,8 +91,8 @@ int cfs_tracefile_init_arch(void) void cfs_tracefile_fini_arch(void) { - int i; - int j; + int i; + int j; for (i = 0; i < num_possible_cpus(); i++) for (j = 0; j < 3; j++) { diff --git a/drivers/staging/lustre/lnet/libcfs/module.c b/drivers/staging/lustre/lnet/libcfs/module.c index cd057f1..5884a33 100644 --- a/drivers/staging/lustre/lnet/libcfs/module.c +++ b/drivers/staging/lustre/lnet/libcfs/module.c @@ -183,12 +183,12 @@ int lprocfs_call_handler(void *data, int write, loff_t *ppos, static int __proc_dobitmasks(void *data, int write, loff_t pos, void __user *buffer, int nob) { - const int tmpstrlen = 512; - char *tmpstr; - int rc; + const int tmpstrlen = 512; + char *tmpstr; + int rc; unsigned int *mask = data; - int is_subsys = (mask == &libcfs_subsystem_debug) ? 1 : 0; - int is_printk = (mask == &libcfs_printk) ? 1 : 0; + int is_subsys = (mask == &libcfs_subsystem_debug) ? 1 : 0; + int is_printk = (mask == &libcfs_printk) ? 1 : 0; rc = cfs_trace_allocate_string_buffer(&tmpstr, tmpstrlen); if (rc < 0) @@ -293,8 +293,8 @@ static int __proc_cpt_table(void *data, int write, loff_t pos, void __user *buffer, int nob) { char *buf = NULL; - int len = 4096; - int rc = 0; + int len = 4096; + int rc = 0; if (write) return -EPERM; diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.c b/drivers/staging/lustre/lnet/libcfs/tracefile.c index a15cedd..d7b29f8 100644 --- a/drivers/staging/lustre/lnet/libcfs/tracefile.c +++ b/drivers/staging/lustre/lnet/libcfs/tracefile.c @@ -59,13 +59,13 @@ struct page_collection { * ->tcd_daemon_pages and ->tcd_pages to the ->pc_pages. Otherwise, * only ->tcd_pages are spilled. */ - int pc_want_daemon_pages; + int pc_want_daemon_pages; }; struct tracefiled_ctl { struct completion tctl_start; struct completion tctl_stop; - wait_queue_head_t tctl_waitq; + wait_queue_head_t tctl_waitq; pid_t tctl_pid; atomic_t tctl_shutdown; }; @@ -77,24 +77,24 @@ struct cfs_trace_page { /* * page itself */ - struct page *page; + struct page *page; /* * linkage into one of the lists in trace_data_union or * page_collection */ - struct list_head linkage; + struct list_head linkage; /* * number of bytes used within this page */ - unsigned int used; + unsigned int used; /* * cpu that owns this page */ - unsigned short cpu; + unsigned short cpu; /* * type(context) of this page */ - unsigned short type; + unsigned short type; }; static void put_pages_on_tcd_daemon_list(struct page_collection *pc, @@ -108,7 +108,7 @@ static void put_pages_on_tcd_daemon_list(struct page_collection *pc, static struct cfs_trace_page *cfs_tage_alloc(gfp_t gfp) { - struct page *page; + struct page *page; struct cfs_trace_page *tage; /* My caller is trying to free memory */ @@ -278,7 +278,7 @@ int libcfs_debug_msg(struct libcfs_debug_msg_data *msgdata, const char *format, ...) { va_list args; - int rc; + int rc; va_start(args, format); rc = libcfs_debug_vmsg2(msgdata, format, args, NULL); @@ -293,21 +293,21 @@ int libcfs_debug_vmsg2(struct libcfs_debug_msg_data *msgdata, const char *format2, ...) { struct cfs_trace_cpu_data *tcd = NULL; - struct ptldebug_header header = {0}; - struct cfs_trace_page *tage; + struct ptldebug_header header = { 0 }; + struct cfs_trace_page *tage; /* string_buf is used only if tcd != NULL, and is always set then */ - char *string_buf = NULL; - char *debug_buf; - int known_size; - int needed = 85; /* average message length */ - int max_nob; - va_list ap; - int depth; - int i; - int remain; - int mask = msgdata->msg_mask; - const char *file = kbasename(msgdata->msg_file); - struct cfs_debug_limit_state *cdls = msgdata->msg_cdls; + char *string_buf = NULL; + char *debug_buf; + int known_size; + int needed = 85; /* average message length */ + int max_nob; + va_list ap; + int depth; + int i; + int remain; + int mask = msgdata->msg_mask; + const char *file = kbasename(msgdata->msg_file); + struct cfs_debug_limit_state *cdls = msgdata->msg_cdls; tcd = cfs_trace_get_tcd(); @@ -535,9 +535,9 @@ int libcfs_debug_vmsg2(struct libcfs_debug_msg_data *msgdata, * CPUs have been stopped during a panic. If this isn't true for some * arch, this will have to be implemented separately in each arch. */ - int i; - int j; struct cfs_trace_cpu_data *tcd; + int i; + int j; INIT_LIST_HEAD(&pc->pc_pages); @@ -698,11 +698,11 @@ void cfs_trace_debug_print(void) int cfs_tracefile_dump_all_pages(char *filename) { - struct page_collection pc; - struct file *filp; - struct cfs_trace_page *tage; - struct cfs_trace_page *tmp; - char *buf; + struct page_collection pc; + struct file *filp; + struct cfs_trace_page *tage; + struct cfs_trace_page *tmp; + char *buf; mm_segment_t __oldfs; int rc; @@ -778,7 +778,7 @@ void cfs_trace_flush_pages(void) int cfs_trace_copyin_string(char *knl_buffer, int knl_buffer_nob, const char __user *usr_buffer, int usr_buffer_nob) { - int nob; + int nob; if (usr_buffer_nob > knl_buffer_nob) return -EOVERFLOW; @@ -810,7 +810,7 @@ int cfs_trace_copyout_string(char __user *usr_buffer, int usr_buffer_nob, * NB if 'append' != NULL, it's a single character to append to the * copied out string - usually "\n" or "" (i.e. a terminating zero byte) */ - int nob = strlen(knl_buffer); + int nob = strlen(knl_buffer); if (nob > usr_buffer_nob) nob = usr_buffer_nob; @@ -843,8 +843,8 @@ int cfs_trace_allocate_string_buffer(char **str, int nob) int cfs_trace_dump_debug_buffer_usrstr(void __user *usr_str, int usr_str_nob) { - char *str; - int rc; + char *str; + int rc; rc = cfs_trace_allocate_string_buffer(&str, usr_str_nob + 1); if (rc) @@ -867,7 +867,7 @@ int cfs_trace_dump_debug_buffer_usrstr(void __user *usr_str, int usr_str_nob) int cfs_trace_daemon_command(char *str) { - int rc = 0; + int rc = 0; cfs_tracefile_write_lock(); @@ -909,7 +909,7 @@ int cfs_trace_daemon_command(char *str) int cfs_trace_daemon_command_usrstr(void __user *usr_str, int usr_str_nob) { char *str; - int rc; + int rc; rc = cfs_trace_allocate_string_buffer(&str, usr_str_nob + 1); if (rc) @@ -1135,10 +1135,10 @@ void cfs_trace_stop_thread(void) int cfs_tracefile_init(int max_pages) { struct cfs_trace_cpu_data *tcd; - int i; - int j; - int rc; - int factor; + int i; + int j; + int rc; + int factor; rc = cfs_tracefile_init_arch(); if (rc) diff --git a/drivers/staging/lustre/lnet/libcfs/tracefile.h b/drivers/staging/lustre/lnet/libcfs/tracefile.h index d878676..f644cbc 100644 --- a/drivers/staging/lustre/lnet/libcfs/tracefile.h +++ b/drivers/staging/lustre/lnet/libcfs/tracefile.h @@ -45,7 +45,7 @@ enum cfs_trace_buf_type { /* trace file lock routines */ #define TRACEFILE_NAME_SIZE 1024 -extern char cfs_tracefile[TRACEFILE_NAME_SIZE]; +extern char cfs_tracefile[TRACEFILE_NAME_SIZE]; extern long long cfs_tracefile_size; void libcfs_run_debug_log_upcall(char *file); @@ -80,7 +80,7 @@ int cfs_trace_copyout_string(char __user *usr_buffer, int usr_buffer_nob, void libcfs_debug_dumplog_internal(void *arg); void libcfs_register_panic_notifier(void); void libcfs_unregister_panic_notifier(void); -extern int libcfs_panic_in_progress; +extern int libcfs_panic_in_progress; int cfs_trace_max_debug_mb(void); #define TCD_MAX_PAGES (5 << (20 - PAGE_SHIFT)) @@ -113,14 +113,14 @@ int cfs_trace_copyout_string(char __user *usr_buffer, int usr_buffer_nob, * tcd_for_each_type_lock */ spinlock_t tcd_lock; - unsigned long tcd_lock_flags; + unsigned long tcd_lock_flags; /* * pages with trace records not yet processed by tracefiled. */ - struct list_head tcd_pages; + struct list_head tcd_pages; /* number of pages on ->tcd_pages */ - unsigned long tcd_cur_pages; + unsigned long tcd_cur_pages; /* * pages with trace records already processed by @@ -132,9 +132,9 @@ int cfs_trace_copyout_string(char __user *usr_buffer, int usr_buffer_nob, * (put_pages_on_daemon_list()). LRU pages from this list are * discarded when list grows too large. */ - struct list_head tcd_daemon_pages; + struct list_head tcd_daemon_pages; /* number of pages on ->tcd_daemon_pages */ - unsigned long tcd_cur_daemon_pages; + unsigned long tcd_cur_daemon_pages; /* * Maximal number of pages allowed on ->tcd_pages and @@ -142,7 +142,7 @@ int cfs_trace_copyout_string(char __user *usr_buffer, int usr_buffer_nob, * Always TCD_MAX_PAGES * tcd_pages_factor / 100 in current * implementation. */ - unsigned long tcd_max_pages; + unsigned long tcd_max_pages; /* * preallocated pages to write trace records into. Pages from @@ -166,15 +166,15 @@ int cfs_trace_copyout_string(char __user *usr_buffer, int usr_buffer_nob, * TCD_STOCK_PAGES pagesful are consumed by trace records all * emitted in non-blocking contexts. Which is quite unlikely. */ - struct list_head tcd_stock_pages; + struct list_head tcd_stock_pages; /* number of pages on ->tcd_stock_pages */ - unsigned long tcd_cur_stock_pages; + unsigned long tcd_cur_stock_pages; - unsigned short tcd_shutting_down; - unsigned short tcd_cpu; - unsigned short tcd_type; + unsigned short tcd_shutting_down; + unsigned short tcd_cpu; + unsigned short tcd_type; /* The factors to share debug memory. */ - unsigned short tcd_pages_factor; + unsigned short tcd_pages_factor; } tcd; char __pad[L1_CACHE_ALIGN(sizeof(struct cfs_trace_cpu_data))]; }; diff --git a/drivers/staging/lustre/lnet/libcfs/workitem.c b/drivers/staging/lustre/lnet/libcfs/workitem.c index f5f1e9e..d0512da 100644 --- a/drivers/staging/lustre/lnet/libcfs/workitem.c +++ b/drivers/staging/lustre/lnet/libcfs/workitem.c @@ -45,7 +45,7 @@ struct cfs_wi_sched { /* chain on global list */ struct list_head ws_list; /** serialised workitems */ - spinlock_t ws_lock; + spinlock_t ws_lock; /** where schedulers sleep */ wait_queue_head_t ws_waitq; /** concurrent workitems */ @@ -59,26 +59,26 @@ struct cfs_wi_sched { */ struct list_head ws_rerunq; /** CPT-table for this scheduler */ - struct cfs_cpt_table *ws_cptab; + struct cfs_cpt_table *ws_cptab; /** CPT id for affinity */ - int ws_cpt; + int ws_cpt; /** number of scheduled workitems */ - int ws_nscheduled; + int ws_nscheduled; /** started scheduler thread, protected by cfs_wi_data::wi_glock */ - unsigned int ws_nthreads:30; + unsigned int ws_nthreads:30; /** shutting down, protected by cfs_wi_data::wi_glock */ - unsigned int ws_stopping:1; + unsigned int ws_stopping:1; /** serialize starting thread, protected by cfs_wi_data::wi_glock */ - unsigned int ws_starting:1; + unsigned int ws_starting:1; /** scheduler name */ - char ws_name[CFS_WS_NAME_LEN]; + char ws_name[CFS_WS_NAME_LEN]; }; static struct cfs_workitem_data { /** serialize */ spinlock_t wi_glock; /** list of all schedulers */ - struct list_head wi_scheds; + struct list_head wi_scheds; /** WI module is initialized */ int wi_init; /** shutting down the whole WI module */ @@ -136,7 +136,7 @@ struct cfs_wi_sched { int cfs_wi_deschedule(struct cfs_wi_sched *sched, struct cfs_workitem *wi) { - int rc; + int rc; LASSERT(!in_interrupt()); /* because we use plain spinlock */ LASSERT(!sched->ws_stopping); @@ -202,7 +202,7 @@ struct cfs_wi_sched { static int cfs_wi_scheduler(void *arg) { - struct cfs_wi_sched *sched = (struct cfs_wi_sched *)arg; + struct cfs_wi_sched *sched = (struct cfs_wi_sched *)arg; cfs_block_allsigs(); @@ -223,8 +223,8 @@ static int cfs_wi_scheduler(void *arg) spin_lock(&sched->ws_lock); while (!sched->ws_stopping) { - int nloops = 0; - int rc; + int nloops = 0; + int rc; struct cfs_workitem *wi; while (!list_empty(&sched->ws_runq) && @@ -238,13 +238,13 @@ static int cfs_wi_scheduler(void *arg) LASSERT(sched->ws_nscheduled > 0); sched->ws_nscheduled--; - wi->wi_running = 1; + wi->wi_running = 1; wi->wi_scheduled = 0; spin_unlock(&sched->ws_lock); nloops++; - rc = (*wi->wi_action) (wi); + rc = (*wi->wi_action)(wi); spin_lock(&sched->ws_lock); if (rc) /* WI should be dead, even be freed! */ @@ -289,7 +289,7 @@ static int cfs_wi_scheduler(void *arg) void cfs_wi_sched_destroy(struct cfs_wi_sched *sched) { - int i; + int i; LASSERT(cfs_wi_data.wi_init); LASSERT(!cfs_wi_data.wi_stopping); @@ -335,8 +335,8 @@ static int cfs_wi_scheduler(void *arg) cfs_wi_sched_create(char *name, struct cfs_cpt_table *cptab, int cpt, int nthrs, struct cfs_wi_sched **sched_pp) { - struct cfs_wi_sched *sched; - int rc; + struct cfs_wi_sched *sched; + int rc; LASSERT(cfs_wi_data.wi_init); LASSERT(!cfs_wi_data.wi_stopping); @@ -364,7 +364,7 @@ static int cfs_wi_scheduler(void *arg) rc = 0; while (nthrs > 0) { - char name[16]; + char name[16]; struct task_struct *task; spin_lock(&cfs_wi_data.wi_glock); @@ -431,7 +431,7 @@ static int cfs_wi_scheduler(void *arg) void cfs_wi_shutdown(void) { - struct cfs_wi_sched *sched; + struct cfs_wi_sched *sched; struct cfs_wi_sched *temp; spin_lock(&cfs_wi_data.wi_glock); -- 1.7.1 From jsimmons at infradead.org Thu Nov 17 19:43:38 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 17 Nov 2016 14:43:38 -0500 Subject: [lustre-devel] [PATCH v2] staging: lustre: llog: fix wrong offset in llog_process_thread() Message-ID: <1479411818-30863-1-git-send-email-jsimmons@infradead.org> From: Mikhail Pershin - llh_cat_idx may become bigger than llog bitmap size in llog_cat_set_first_idx() function - it is wrong to use previous cur_offset as new buffer offset, new offset should be calculated from value returned by llog_next_block(). - optimize llog_skip_over() to find llog entry offset by index for llog with fixed-size records. Signed-off-by: Mikhail Pershin Signed-off-by: Bob Glossman Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6714 Reviewed-on: http://review.whamcloud.com/15316 Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6163 Reviewed-on: http://review.whamcloud.com/18819 Reviewed-by: John L. Hammond Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- ChangeLog: v1) Initial patch with umoddi issue v2) Included fix from patch LU-6163 that fixed umoddi problem drivers/staging/lustre/lustre/obdclass/llog.c | 81 ++++++++++++++++++------- 1 files changed, 58 insertions(+), 23 deletions(-) diff --git a/drivers/staging/lustre/lustre/obdclass/llog.c b/drivers/staging/lustre/lustre/obdclass/llog.c index 3bc1789..1d0c5a3 100644 --- a/drivers/staging/lustre/lustre/obdclass/llog.c +++ b/drivers/staging/lustre/lustre/obdclass/llog.c @@ -217,7 +217,7 @@ static int llog_process_thread(void *arg) struct llog_log_hdr *llh = loghandle->lgh_hdr; struct llog_process_cat_data *cd = lpi->lpi_catdata; char *buf; - __u64 cur_offset; + u64 cur_offset, tmp_offset; __u64 last_offset; int chunk_size; int rc = 0, index = 1, last_index; @@ -229,6 +229,8 @@ static int llog_process_thread(void *arg) cur_offset = llh->llh_hdr.lrh_len; chunk_size = llh->llh_hdr.lrh_len; + /* expect chunk_size to be power of two */ + LASSERT(is_power_of_2(chunk_size)); buf = libcfs_kvzalloc(chunk_size, GFP_NOFS); if (!buf) { @@ -245,38 +247,50 @@ static int llog_process_thread(void *arg) else last_index = LLOG_HDR_BITMAP_SIZE(llh) - 1; - /* Record is not in this buffer. */ - if (index > last_index) - goto out; - while (rc == 0) { + unsigned int buf_offset = 0; struct llog_rec_hdr *rec; + bool partial_chunk; + off_t chunk_offset; /* skip records not set in bitmap */ while (index <= last_index && !ext2_test_bit(index, LLOG_HDR_BITMAP(llh))) ++index; - LASSERT(index <= last_index + 1); - if (index == last_index + 1) + if (index > last_index) break; -repeat: + CDEBUG(D_OTHER, "index: %d last_index %d\n", index, last_index); - +repeat: /* get the buf with our target record; avoid old garbage */ memset(buf, 0, chunk_size); - last_offset = cur_offset; rc = llog_next_block(lpi->lpi_env, loghandle, &saved_index, index, &cur_offset, buf, chunk_size); if (rc) goto out; + /* + * NB: after llog_next_block() call the cur_offset is the + * offset of the next block after read one. + * The absolute offset of the current chunk is calculated + * from cur_offset value and stored in chunk_offset variable. + */ + tmp_offset = cur_offset; + if (do_div(tmp_offset, chunk_size)) { + partial_chunk = true; + chunk_offset = cur_offset & ~(chunk_size - 1); + } else { + partial_chunk = false; + chunk_offset = cur_offset - chunk_size; + } + /* NB: when rec->lrh_len is accessed it is already swabbed * since it is used at the "end" of the loop and the rec * swabbing is done at the beginning of the loop. */ - for (rec = (struct llog_rec_hdr *)buf; + for (rec = (struct llog_rec_hdr *)(buf + buf_offset); (char *)rec < buf + chunk_size; rec = llog_rec_hdr_next(rec)) { CDEBUG(D_OTHER, "processing rec 0x%p type %#x\n", @@ -288,13 +302,28 @@ static int llog_process_thread(void *arg) CDEBUG(D_OTHER, "after swabbing, type=%#x idx=%d\n", rec->lrh_type, rec->lrh_index); - if (rec->lrh_index == 0) { - /* probably another rec just got added? */ - rc = 0; - if (index <= loghandle->lgh_last_idx) - goto repeat; - goto out; /* no more records */ + /* + * for partial chunk the end of it is zeroed, check + * for index 0 to distinguish it. + */ + if (partial_chunk && !rec->lrh_index) { + /* concurrent llog_add() might add new records + * while llog_processing, check this is not + * the case and re-read the current chunk + * otherwise. + */ + if (index > loghandle->lgh_last_idx) { + rc = 0; + goto out; + } + CDEBUG(D_OTHER, "Re-read last llog buffer for new records, index %u, last %u\n", + index, loghandle->lgh_last_idx); + /* save offset inside buffer for the re-read */ + buf_offset = (char *)rec - (char *)buf; + cur_offset = chunk_offset; + goto repeat; } + if (!rec->lrh_len || rec->lrh_len > chunk_size) { CWARN("invalid length %d in llog record for index %d/%d\n", rec->lrh_len, @@ -309,6 +338,14 @@ static int llog_process_thread(void *arg) continue; } + if (rec->lrh_index != index) { + CERROR("%s: Invalid record: index %u but expected %u\n", + loghandle->lgh_ctxt->loc_obd->obd_name, + rec->lrh_index, index); + rc = -ERANGE; + goto out; + } + CDEBUG(D_OTHER, "lrh_index: %d lrh_len: %d (%d remains)\n", rec->lrh_index, rec->lrh_len, @@ -316,7 +353,7 @@ static int llog_process_thread(void *arg) loghandle->lgh_cur_idx = rec->lrh_index; loghandle->lgh_cur_offset = (char *)rec - (char *)buf + - last_offset; + chunk_offset; /* if set, process the callback on this record */ if (ext2_test_bit(index, LLOG_HDR_BITMAP(llh))) { @@ -325,16 +362,14 @@ static int llog_process_thread(void *arg) last_called_index = index; if (rc) goto out; - } else { - CDEBUG(D_OTHER, "Skipped index %d\n", index); } - /* next record, still in buffer? */ - ++index; - if (index > last_index) { + /* exit if the last index is reached */ + if (index >= last_index) { rc = 0; goto out; } + index++; } } -- 1.7.1 From lkp at intel.com Thu Nov 17 21:46:13 2016 From: lkp at intel.com (kbuild test robot) Date: Fri, 18 Nov 2016 05:46:13 +0800 Subject: [lustre-devel] [PATCH v2] staging: lustre: llog: fix wrong offset in llog_process_thread() In-Reply-To: <1479411818-30863-1-git-send-email-jsimmons@infradead.org> Message-ID: <201611180531.NI3A2Y05%fengguang.wu@intel.com> Hi Mikhail, [auto build test WARNING on staging/staging-testing] [also build test WARNING on next-20161117] [cannot apply to v4.9-rc5] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url: https://github.com/0day-ci/linux/commits/James-Simmons/staging-lustre-llog-fix-wrong-offset-in-llog_process_thread/20161118-042856 config: i386-allmodconfig (attached as .config) compiler: gcc-6 (Debian 6.2.0-3) 6.2.0 20160901 reproduce: # save the attached .config to linux build tree make ARCH=i386 All warnings (new ones prefixed by >>): drivers/staging/lustre/lustre/obdclass/llog.c: In function 'llog_process_thread': >> drivers/staging/lustre/lustre/obdclass/llog.c:221:12: warning: unused variable 'last_offset' [-Wunused-variable] __u64 last_offset; ^~~~~~~~~~~ vim +/last_offset +221 drivers/staging/lustre/lustre/obdclass/llog.c d7e09d039 Peng Tao 2013-05-02 205 if (rc) { 11c647caf Andreas Dilger 2016-10-27 206 kvfree(llh); d7e09d039 Peng Tao 2013-05-02 207 handle->lgh_hdr = NULL; d7e09d039 Peng Tao 2013-05-02 208 } 0a3bdb007 Greg Kroah-Hartman 2013-08-03 209 return rc; d7e09d039 Peng Tao 2013-05-02 210 } d7e09d039 Peng Tao 2013-05-02 211 EXPORT_SYMBOL(llog_init_handle); d7e09d039 Peng Tao 2013-05-02 212 d7e09d039 Peng Tao 2013-05-02 213 static int llog_process_thread(void *arg) d7e09d039 Peng Tao 2013-05-02 214 { d7e09d039 Peng Tao 2013-05-02 215 struct llog_process_info *lpi = arg; d7e09d039 Peng Tao 2013-05-02 216 struct llog_handle *loghandle = lpi->lpi_loghandle; d7e09d039 Peng Tao 2013-05-02 217 struct llog_log_hdr *llh = loghandle->lgh_hdr; d7e09d039 Peng Tao 2013-05-02 218 struct llog_process_cat_data *cd = lpi->lpi_catdata; d7e09d039 Peng Tao 2013-05-02 219 char *buf; ede733b28 Mikhail Pershin 2016-11-17 220 u64 cur_offset, tmp_offset; d7e09d039 Peng Tao 2013-05-02 @221 __u64 last_offset; 11c647caf Andreas Dilger 2016-10-27 222 int chunk_size; d7e09d039 Peng Tao 2013-05-02 223 int rc = 0, index = 1, last_index; d7e09d039 Peng Tao 2013-05-02 224 int saved_index = 0; d7e09d039 Peng Tao 2013-05-02 225 int last_called_index = 0; d7e09d039 Peng Tao 2013-05-02 226 11c647caf Andreas Dilger 2016-10-27 227 if (!llh) 11c647caf Andreas Dilger 2016-10-27 228 return -EINVAL; 11c647caf Andreas Dilger 2016-10-27 229 :::::: The code at line 221 was first introduced by commit :::::: d7e09d0397e84eefbabfd9cb353221f3c6448d83 staging: add Lustre file system client support :::::: TO: Peng Tao :::::: CC: Greg Kroah-Hartman --- 0-DAY kernel test infrastructure Open Source Technology Center https://lists.01.org/pipermail/kbuild-all Intel Corporation -------------- next part -------------- A non-text attachment was scrubbed... Name: .config.gz Type: application/gzip Size: 56934 bytes Desc: not available URL: From jsimmons at infradead.org Thu Nov 17 23:29:08 2016 From: jsimmons at infradead.org (James Simmons) Date: Thu, 17 Nov 2016 18:29:08 -0500 Subject: [lustre-devel] [PATCH v3] staging: lustre: llog: fix wrong offset in llog_process_thread() Message-ID: <1479425348-21854-1-git-send-email-jsimmons@infradead.org> From: Mikhail Pershin - llh_cat_idx may become bigger than llog bitmap size in llog_cat_set_first_idx() function - it is wrong to use previous cur_offset as new buffer offset, new offset should be calculated from value returned by llog_next_block(). - optimize llog_skip_over() to find llog entry offset by index for llog with fixed-size records. Signed-off-by: Mikhail Pershin Signed-off-by: Bob Glossman Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6714 Reviewed-on: http://review.whamcloud.com/15316 Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6163 Reviewed-on: http://review.whamcloud.com/18819 Reviewed-by: John L. Hammond Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- ChangeLog: v1) Initial patch with umoddi issue v2) Included fix from patch LU-6163 that fixed umoddi problem v3) Remove no longer needed last_offset variable drivers/staging/lustre/lustre/obdclass/llog.c | 82 +++++++++++++++++------- include/linux/fs.h | 2 +- 2 files changed, 59 insertions(+), 25 deletions(-) diff --git a/drivers/staging/lustre/lustre/obdclass/llog.c b/drivers/staging/lustre/lustre/obdclass/llog.c index 3bc1789..ae63047 100644 --- a/drivers/staging/lustre/lustre/obdclass/llog.c +++ b/drivers/staging/lustre/lustre/obdclass/llog.c @@ -217,8 +217,7 @@ static int llog_process_thread(void *arg) struct llog_log_hdr *llh = loghandle->lgh_hdr; struct llog_process_cat_data *cd = lpi->lpi_catdata; char *buf; - __u64 cur_offset; - __u64 last_offset; + u64 cur_offset, tmp_offset; int chunk_size; int rc = 0, index = 1, last_index; int saved_index = 0; @@ -229,6 +228,8 @@ static int llog_process_thread(void *arg) cur_offset = llh->llh_hdr.lrh_len; chunk_size = llh->llh_hdr.lrh_len; + /* expect chunk_size to be power of two */ + LASSERT(is_power_of_2(chunk_size)); buf = libcfs_kvzalloc(chunk_size, GFP_NOFS); if (!buf) { @@ -245,38 +246,50 @@ static int llog_process_thread(void *arg) else last_index = LLOG_HDR_BITMAP_SIZE(llh) - 1; - /* Record is not in this buffer. */ - if (index > last_index) - goto out; - while (rc == 0) { + unsigned int buf_offset = 0; struct llog_rec_hdr *rec; + bool partial_chunk; + off_t chunk_offset; /* skip records not set in bitmap */ while (index <= last_index && !ext2_test_bit(index, LLOG_HDR_BITMAP(llh))) ++index; - LASSERT(index <= last_index + 1); - if (index == last_index + 1) + if (index > last_index) break; -repeat: + CDEBUG(D_OTHER, "index: %d last_index %d\n", index, last_index); - +repeat: /* get the buf with our target record; avoid old garbage */ memset(buf, 0, chunk_size); - last_offset = cur_offset; rc = llog_next_block(lpi->lpi_env, loghandle, &saved_index, index, &cur_offset, buf, chunk_size); if (rc) goto out; + /* + * NB: after llog_next_block() call the cur_offset is the + * offset of the next block after read one. + * The absolute offset of the current chunk is calculated + * from cur_offset value and stored in chunk_offset variable. + */ + tmp_offset = cur_offset; + if (do_div(tmp_offset, chunk_size)) { + partial_chunk = true; + chunk_offset = cur_offset & ~(chunk_size - 1); + } else { + partial_chunk = false; + chunk_offset = cur_offset - chunk_size; + } + /* NB: when rec->lrh_len is accessed it is already swabbed * since it is used at the "end" of the loop and the rec * swabbing is done at the beginning of the loop. */ - for (rec = (struct llog_rec_hdr *)buf; + for (rec = (struct llog_rec_hdr *)(buf + buf_offset); (char *)rec < buf + chunk_size; rec = llog_rec_hdr_next(rec)) { CDEBUG(D_OTHER, "processing rec 0x%p type %#x\n", @@ -288,13 +301,28 @@ static int llog_process_thread(void *arg) CDEBUG(D_OTHER, "after swabbing, type=%#x idx=%d\n", rec->lrh_type, rec->lrh_index); - if (rec->lrh_index == 0) { - /* probably another rec just got added? */ - rc = 0; - if (index <= loghandle->lgh_last_idx) - goto repeat; - goto out; /* no more records */ + /* + * for partial chunk the end of it is zeroed, check + * for index 0 to distinguish it. + */ + if (partial_chunk && !rec->lrh_index) { + /* concurrent llog_add() might add new records + * while llog_processing, check this is not + * the case and re-read the current chunk + * otherwise. + */ + if (index > loghandle->lgh_last_idx) { + rc = 0; + goto out; + } + CDEBUG(D_OTHER, "Re-read last llog buffer for new records, index %u, last %u\n", + index, loghandle->lgh_last_idx); + /* save offset inside buffer for the re-read */ + buf_offset = (char *)rec - (char *)buf; + cur_offset = chunk_offset; + goto repeat; } + if (!rec->lrh_len || rec->lrh_len > chunk_size) { CWARN("invalid length %d in llog record for index %d/%d\n", rec->lrh_len, @@ -309,6 +337,14 @@ static int llog_process_thread(void *arg) continue; } + if (rec->lrh_index != index) { + CERROR("%s: Invalid record: index %u but expected %u\n", + loghandle->lgh_ctxt->loc_obd->obd_name, + rec->lrh_index, index); + rc = -ERANGE; + goto out; + } + CDEBUG(D_OTHER, "lrh_index: %d lrh_len: %d (%d remains)\n", rec->lrh_index, rec->lrh_len, @@ -316,7 +352,7 @@ static int llog_process_thread(void *arg) loghandle->lgh_cur_idx = rec->lrh_index; loghandle->lgh_cur_offset = (char *)rec - (char *)buf + - last_offset; + chunk_offset; /* if set, process the callback on this record */ if (ext2_test_bit(index, LLOG_HDR_BITMAP(llh))) { @@ -325,16 +361,14 @@ static int llog_process_thread(void *arg) last_called_index = index; if (rc) goto out; - } else { - CDEBUG(D_OTHER, "Skipped index %d\n", index); } - /* next record, still in buffer? */ - ++index; - if (index > last_index) { + /* exit if the last index is reached */ + if (index >= last_index) { rc = 0; goto out; } + index++; } } diff --git a/include/linux/fs.h b/include/linux/fs.h index dc0478c..3f70ec3 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -2690,7 +2690,7 @@ enum kernel_read_file_id { static inline const char *kernel_read_file_id_str(enum kernel_read_file_id id) { - if (id < 0 || id >= READING_MAX_ID) + if (id >= READING_MAX_ID) return kernel_read_file_str[READING_UNKNOWN]; return kernel_read_file_str[id]; -- 1.7.1 From gregkh at linuxfoundation.org Fri Nov 18 07:53:02 2016 From: gregkh at linuxfoundation.org (Greg Kroah-Hartman) Date: Fri, 18 Nov 2016 08:53:02 +0100 Subject: [lustre-devel] [PATCH v3] staging: lustre: llog: fix wrong offset in llog_process_thread() In-Reply-To: <1479425348-21854-1-git-send-email-jsimmons@infradead.org> References: <1479425348-21854-1-git-send-email-jsimmons@infradead.org> Message-ID: <20161118075302.GA666@kroah.com> On Thu, Nov 17, 2016 at 06:29:08PM -0500, James Simmons wrote: > From: Mikhail Pershin > > - llh_cat_idx may become bigger than llog bitmap size in > llog_cat_set_first_idx() function > - it is wrong to use previous cur_offset as new buffer offset, > new offset should be calculated from value returned by > llog_next_block(). > - optimize llog_skip_over() to find llog entry offset by index > for llog with fixed-size records. > > Signed-off-by: Mikhail Pershin > Signed-off-by: Bob Glossman > Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6714 > Reviewed-on: http://review.whamcloud.com/15316 > Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6163 > Reviewed-on: http://review.whamcloud.com/18819 > Reviewed-by: John L. Hammond > Reviewed-by: James Simmons > Reviewed-by: Oleg Drokin > Signed-off-by: James Simmons > --- > > ChangeLog: > > v1) Initial patch with umoddi issue > v2) Included fix from patch LU-6163 that fixed umoddi problem > v3) Remove no longer needed last_offset variable > > drivers/staging/lustre/lustre/obdclass/llog.c | 82 +++++++++++++++++------- > include/linux/fs.h | 2 +- > 2 files changed, 59 insertions(+), 25 deletions(-) > > diff --git a/drivers/staging/lustre/lustre/obdclass/llog.c b/drivers/staging/lustre/lustre/obdclass/llog.c > index 3bc1789..ae63047 100644 > --- a/drivers/staging/lustre/lustre/obdclass/llog.c > +++ b/drivers/staging/lustre/lustre/obdclass/llog.c > @@ -217,8 +217,7 @@ static int llog_process_thread(void *arg) > struct llog_log_hdr *llh = loghandle->lgh_hdr; > struct llog_process_cat_data *cd = lpi->lpi_catdata; > char *buf; > - __u64 cur_offset; > - __u64 last_offset; > + u64 cur_offset, tmp_offset; > int chunk_size; > int rc = 0, index = 1, last_index; > int saved_index = 0; > @@ -229,6 +228,8 @@ static int llog_process_thread(void *arg) > > cur_offset = llh->llh_hdr.lrh_len; > chunk_size = llh->llh_hdr.lrh_len; > + /* expect chunk_size to be power of two */ > + LASSERT(is_power_of_2(chunk_size)); > > buf = libcfs_kvzalloc(chunk_size, GFP_NOFS); > if (!buf) { > @@ -245,38 +246,50 @@ static int llog_process_thread(void *arg) > else > last_index = LLOG_HDR_BITMAP_SIZE(llh) - 1; > > - /* Record is not in this buffer. */ > - if (index > last_index) > - goto out; > - > while (rc == 0) { > + unsigned int buf_offset = 0; > struct llog_rec_hdr *rec; > + bool partial_chunk; > + off_t chunk_offset; > > /* skip records not set in bitmap */ > while (index <= last_index && > !ext2_test_bit(index, LLOG_HDR_BITMAP(llh))) > ++index; > > - LASSERT(index <= last_index + 1); > - if (index == last_index + 1) > + if (index > last_index) > break; > -repeat: > + > CDEBUG(D_OTHER, "index: %d last_index %d\n", > index, last_index); > - > +repeat: > /* get the buf with our target record; avoid old garbage */ > memset(buf, 0, chunk_size); > - last_offset = cur_offset; > rc = llog_next_block(lpi->lpi_env, loghandle, &saved_index, > index, &cur_offset, buf, chunk_size); > if (rc) > goto out; > > + /* > + * NB: after llog_next_block() call the cur_offset is the > + * offset of the next block after read one. > + * The absolute offset of the current chunk is calculated > + * from cur_offset value and stored in chunk_offset variable. > + */ > + tmp_offset = cur_offset; > + if (do_div(tmp_offset, chunk_size)) { > + partial_chunk = true; > + chunk_offset = cur_offset & ~(chunk_size - 1); > + } else { > + partial_chunk = false; > + chunk_offset = cur_offset - chunk_size; > + } > + > /* NB: when rec->lrh_len is accessed it is already swabbed > * since it is used at the "end" of the loop and the rec > * swabbing is done at the beginning of the loop. > */ > - for (rec = (struct llog_rec_hdr *)buf; > + for (rec = (struct llog_rec_hdr *)(buf + buf_offset); > (char *)rec < buf + chunk_size; > rec = llog_rec_hdr_next(rec)) { > CDEBUG(D_OTHER, "processing rec 0x%p type %#x\n", > @@ -288,13 +301,28 @@ static int llog_process_thread(void *arg) > CDEBUG(D_OTHER, "after swabbing, type=%#x idx=%d\n", > rec->lrh_type, rec->lrh_index); > > - if (rec->lrh_index == 0) { > - /* probably another rec just got added? */ > - rc = 0; > - if (index <= loghandle->lgh_last_idx) > - goto repeat; > - goto out; /* no more records */ > + /* > + * for partial chunk the end of it is zeroed, check > + * for index 0 to distinguish it. > + */ > + if (partial_chunk && !rec->lrh_index) { > + /* concurrent llog_add() might add new records > + * while llog_processing, check this is not > + * the case and re-read the current chunk > + * otherwise. > + */ > + if (index > loghandle->lgh_last_idx) { > + rc = 0; > + goto out; > + } > + CDEBUG(D_OTHER, "Re-read last llog buffer for new records, index %u, last %u\n", > + index, loghandle->lgh_last_idx); > + /* save offset inside buffer for the re-read */ > + buf_offset = (char *)rec - (char *)buf; > + cur_offset = chunk_offset; > + goto repeat; > } > + > if (!rec->lrh_len || rec->lrh_len > chunk_size) { > CWARN("invalid length %d in llog record for index %d/%d\n", > rec->lrh_len, > @@ -309,6 +337,14 @@ static int llog_process_thread(void *arg) > continue; > } > > + if (rec->lrh_index != index) { > + CERROR("%s: Invalid record: index %u but expected %u\n", > + loghandle->lgh_ctxt->loc_obd->obd_name, > + rec->lrh_index, index); > + rc = -ERANGE; > + goto out; > + } > + > CDEBUG(D_OTHER, > "lrh_index: %d lrh_len: %d (%d remains)\n", > rec->lrh_index, rec->lrh_len, > @@ -316,7 +352,7 @@ static int llog_process_thread(void *arg) > > loghandle->lgh_cur_idx = rec->lrh_index; > loghandle->lgh_cur_offset = (char *)rec - (char *)buf + > - last_offset; > + chunk_offset; > > /* if set, process the callback on this record */ > if (ext2_test_bit(index, LLOG_HDR_BITMAP(llh))) { > @@ -325,16 +361,14 @@ static int llog_process_thread(void *arg) > last_called_index = index; > if (rc) > goto out; > - } else { > - CDEBUG(D_OTHER, "Skipped index %d\n", index); > } > > - /* next record, still in buffer? */ > - ++index; > - if (index > last_index) { > + /* exit if the last index is reached */ > + if (index >= last_index) { > rc = 0; > goto out; > } > + index++; > } > } > > diff --git a/include/linux/fs.h b/include/linux/fs.h > index dc0478c..3f70ec3 100644 > --- a/include/linux/fs.h > +++ b/include/linux/fs.h > @@ -2690,7 +2690,7 @@ enum kernel_read_file_id { > > static inline const char *kernel_read_file_id_str(enum kernel_read_file_id id) > { > - if (id < 0 || id >= READING_MAX_ID) > + if (id >= READING_MAX_ID) > return kernel_read_file_str[READING_UNKNOWN]; > > return kernel_read_file_str[id]; Why are you modifying this fs.h function in this patch? Your changelog does not talk about it at all. Nor does lustre even call this function, only a security module, loadpin. totally confused, greg k-h From dan.carpenter at oracle.com Fri Nov 18 11:01:27 2016 From: dan.carpenter at oracle.com (Dan Carpenter) Date: Fri, 18 Nov 2016 14:01:27 +0300 Subject: [lustre-devel] [PATCH 12/19] staging: lustre: libcfs: remove == 0 testing In-Reply-To: <1479411357-28979-13-git-send-email-jsimmons@infradead.org> References: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> <1479411357-28979-13-git-send-email-jsimmons@infradead.org> Message-ID: <20161118105235.GB6266@mwanda> On Thu, Nov 17, 2016 at 02:35:50PM -0500, James Simmons wrote: > Testing == 0 is not kernel style so remove this > type of testing from libcfs. > That's not true. Checkpatch.pl won't complain. Not everyone has thought about it or agrees with me but there are time when == 0 is idiomatic. For example, if you are talking about the *number* zero then it should probably be == 0. If it's like "rc == 0" then "rc" doesn't mean the number it means success/fail so it should be "if (!rc) ". But it's intuitive to say "if (len == 0) ". The other place is strcmp() and friends where it's much more intuitive: strcmp(a, b) == 0 means a == b strcmp(a, b) < 0 means a < b strcmp(a, b) != 0 means a != b regards, dan carpenter From jsimmons at infradead.org Fri Nov 18 16:42:26 2016 From: jsimmons at infradead.org (James Simmons) Date: Fri, 18 Nov 2016 16:42:26 +0000 (GMT) Subject: [lustre-devel] [PATCH v3] staging: lustre: llog: fix wrong offset in llog_process_thread() In-Reply-To: <20161118075302.GA666@kroah.com> References: <1479425348-21854-1-git-send-email-jsimmons@infradead.org> <20161118075302.GA666@kroah.com> Message-ID: > On Thu, Nov 17, 2016 at 06:29:08PM -0500, James Simmons wrote: > > From: Mikhail Pershin > > > > - llh_cat_idx may become bigger than llog bitmap size in > > llog_cat_set_first_idx() function > > - it is wrong to use previous cur_offset as new buffer offset, > > new offset should be calculated from value returned by > > llog_next_block(). > > - optimize llog_skip_over() to find llog entry offset by index > > for llog with fixed-size records. > > > > Signed-off-by: Mikhail Pershin > > Signed-off-by: Bob Glossman > > Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6714 > > Reviewed-on: http://review.whamcloud.com/15316 > > Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6163 > > Reviewed-on: http://review.whamcloud.com/18819 > > Reviewed-by: John L. Hammond > > Reviewed-by: James Simmons > > Reviewed-by: Oleg Drokin > > Signed-off-by: James Simmons > > --- > > > > ChangeLog: > > > > v1) Initial patch with umoddi issue > > v2) Included fix from patch LU-6163 that fixed umoddi problem > > v3) Remove no longer needed last_offset variable > > > > drivers/staging/lustre/lustre/obdclass/llog.c | 82 +++++++++++++++++------- > > include/linux/fs.h | 2 +- > > 2 files changed, 59 insertions(+), 25 deletions(-) > > > > diff --git a/drivers/staging/lustre/lustre/obdclass/llog.c b/drivers/staging/lustre/lustre/obdclass/llog.c > > index 3bc1789..ae63047 100644 > > --- a/drivers/staging/lustre/lustre/obdclass/llog.c > > +++ b/drivers/staging/lustre/lustre/obdclass/llog.c > > @@ -217,8 +217,7 @@ static int llog_process_thread(void *arg) > > struct llog_log_hdr *llh = loghandle->lgh_hdr; > > struct llog_process_cat_data *cd = lpi->lpi_catdata; > > char *buf; > > - __u64 cur_offset; > > - __u64 last_offset; > > + u64 cur_offset, tmp_offset; > > int chunk_size; > > int rc = 0, index = 1, last_index; > > int saved_index = 0; > > @@ -229,6 +228,8 @@ static int llog_process_thread(void *arg) > > > > cur_offset = llh->llh_hdr.lrh_len; > > chunk_size = llh->llh_hdr.lrh_len; > > + /* expect chunk_size to be power of two */ > > + LASSERT(is_power_of_2(chunk_size)); > > > > buf = libcfs_kvzalloc(chunk_size, GFP_NOFS); > > if (!buf) { > > @@ -245,38 +246,50 @@ static int llog_process_thread(void *arg) > > else > > last_index = LLOG_HDR_BITMAP_SIZE(llh) - 1; > > > > - /* Record is not in this buffer. */ > > - if (index > last_index) > > - goto out; > > - > > while (rc == 0) { > > + unsigned int buf_offset = 0; > > struct llog_rec_hdr *rec; > > + bool partial_chunk; > > + off_t chunk_offset; > > > > /* skip records not set in bitmap */ > > while (index <= last_index && > > !ext2_test_bit(index, LLOG_HDR_BITMAP(llh))) > > ++index; > > > > - LASSERT(index <= last_index + 1); > > - if (index == last_index + 1) > > + if (index > last_index) > > break; > > -repeat: > > + > > CDEBUG(D_OTHER, "index: %d last_index %d\n", > > index, last_index); > > - > > +repeat: > > /* get the buf with our target record; avoid old garbage */ > > memset(buf, 0, chunk_size); > > - last_offset = cur_offset; > > rc = llog_next_block(lpi->lpi_env, loghandle, &saved_index, > > index, &cur_offset, buf, chunk_size); > > if (rc) > > goto out; > > > > + /* > > + * NB: after llog_next_block() call the cur_offset is the > > + * offset of the next block after read one. > > + * The absolute offset of the current chunk is calculated > > + * from cur_offset value and stored in chunk_offset variable. > > + */ > > + tmp_offset = cur_offset; > > + if (do_div(tmp_offset, chunk_size)) { > > + partial_chunk = true; > > + chunk_offset = cur_offset & ~(chunk_size - 1); > > + } else { > > + partial_chunk = false; > > + chunk_offset = cur_offset - chunk_size; > > + } > > + > > /* NB: when rec->lrh_len is accessed it is already swabbed > > * since it is used at the "end" of the loop and the rec > > * swabbing is done at the beginning of the loop. > > */ > > - for (rec = (struct llog_rec_hdr *)buf; > > + for (rec = (struct llog_rec_hdr *)(buf + buf_offset); > > (char *)rec < buf + chunk_size; > > rec = llog_rec_hdr_next(rec)) { > > CDEBUG(D_OTHER, "processing rec 0x%p type %#x\n", > > @@ -288,13 +301,28 @@ static int llog_process_thread(void *arg) > > CDEBUG(D_OTHER, "after swabbing, type=%#x idx=%d\n", > > rec->lrh_type, rec->lrh_index); > > > > - if (rec->lrh_index == 0) { > > - /* probably another rec just got added? */ > > - rc = 0; > > - if (index <= loghandle->lgh_last_idx) > > - goto repeat; > > - goto out; /* no more records */ > > + /* > > + * for partial chunk the end of it is zeroed, check > > + * for index 0 to distinguish it. > > + */ > > + if (partial_chunk && !rec->lrh_index) { > > + /* concurrent llog_add() might add new records > > + * while llog_processing, check this is not > > + * the case and re-read the current chunk > > + * otherwise. > > + */ > > + if (index > loghandle->lgh_last_idx) { > > + rc = 0; > > + goto out; > > + } > > + CDEBUG(D_OTHER, "Re-read last llog buffer for new records, index %u, last %u\n", > > + index, loghandle->lgh_last_idx); > > + /* save offset inside buffer for the re-read */ > > + buf_offset = (char *)rec - (char *)buf; > > + cur_offset = chunk_offset; > > + goto repeat; > > } > > + > > if (!rec->lrh_len || rec->lrh_len > chunk_size) { > > CWARN("invalid length %d in llog record for index %d/%d\n", > > rec->lrh_len, > > @@ -309,6 +337,14 @@ static int llog_process_thread(void *arg) > > continue; > > } > > > > + if (rec->lrh_index != index) { > > + CERROR("%s: Invalid record: index %u but expected %u\n", > > + loghandle->lgh_ctxt->loc_obd->obd_name, > > + rec->lrh_index, index); > > + rc = -ERANGE; > > + goto out; > > + } > > + > > CDEBUG(D_OTHER, > > "lrh_index: %d lrh_len: %d (%d remains)\n", > > rec->lrh_index, rec->lrh_len, > > @@ -316,7 +352,7 @@ static int llog_process_thread(void *arg) > > > > loghandle->lgh_cur_idx = rec->lrh_index; > > loghandle->lgh_cur_offset = (char *)rec - (char *)buf + > > - last_offset; > > + chunk_offset; > > > > /* if set, process the callback on this record */ > > if (ext2_test_bit(index, LLOG_HDR_BITMAP(llh))) { > > @@ -325,16 +361,14 @@ static int llog_process_thread(void *arg) > > last_called_index = index; > > if (rc) > > goto out; > > - } else { > > - CDEBUG(D_OTHER, "Skipped index %d\n", index); > > } > > > > - /* next record, still in buffer? */ > > - ++index; > > - if (index > last_index) { > > + /* exit if the last index is reached */ > > + if (index >= last_index) { > > rc = 0; > > goto out; > > } > > + index++; > > } > > } > > > > diff --git a/include/linux/fs.h b/include/linux/fs.h > > index dc0478c..3f70ec3 100644 > > --- a/include/linux/fs.h > > +++ b/include/linux/fs.h > > @@ -2690,7 +2690,7 @@ enum kernel_read_file_id { > > > > static inline const char *kernel_read_file_id_str(enum kernel_read_file_id id) > > { > > - if (id < 0 || id >= READING_MAX_ID) > > + if (id >= READING_MAX_ID) > > return kernel_read_file_str[READING_UNKNOWN]; > > > > return kernel_read_file_str[id]; > > Why are you modifying this fs.h function in this patch? Your changelog > does not talk about it at all. Nor does lustre even call this function, > only a security module, loadpin. > > totally confused, Oh that is my mistake. So when I build lustre with W=1 it spits out warnings about id being unsigned and testing for < 0. I patched it to remove the noise to see the real errors. I included it by mistake. From jsimmons at infradead.org Fri Nov 18 16:46:40 2016 From: jsimmons at infradead.org (James Simmons) Date: Fri, 18 Nov 2016 16:46:40 +0000 (GMT) Subject: [lustre-devel] [PATCH 12/19] staging: lustre: libcfs: remove == 0 testing In-Reply-To: <20161118105235.GB6266@mwanda> References: <1479411357-28979-1-git-send-email-jsimmons@infradead.org> <1479411357-28979-13-git-send-email-jsimmons@infradead.org> <20161118105235.GB6266@mwanda> Message-ID: > On Thu, Nov 17, 2016 at 02:35:50PM -0500, James Simmons wrote: > > Testing == 0 is not kernel style so remove this > > type of testing from libcfs. > > > > That's not true. Checkpatch.pl won't complain. > It got lumped in with the checkpatch fixes. Also the __uXX problem is not reported by checkpatch so this patch series covers checkpatch + style issues. > Not everyone has thought about it or agrees with me but there are time > when == 0 is idiomatic. For example, if you are talking about the > *number* zero then it should probably be == 0. If it's like "rc == 0" > then "rc" doesn't mean the number it means success/fail so it should be > "if (!rc) ". But it's intuitive to say "if (len == 0) ". > > The other place is strcmp() and friends where it's much more intuitive: > > strcmp(a, b) == 0 means a == b > strcmp(a, b) < 0 means a < b > strcmp(a, b) != 0 means a != b Oh this I didn't know. I will keep this in mind for the future. From jsimmons at infradead.org Fri Nov 18 16:48:34 2016 From: jsimmons at infradead.org (James Simmons) Date: Fri, 18 Nov 2016 11:48:34 -0500 Subject: [lustre-devel] [PATCH 00/10] checkpatch fixes and style cleanups for libcfs headers Message-ID: <1479487724-20386-1-git-send-email-jsimmons@infradead.org> This patch set contains various cleanups that were reported with checkpatch. Also remove the NULL and 0 comparisons which is not kernel style. Lastly the uXX types are sorted out in the libcfs headers. Only the macro mess is left that needs cleaning up. James Simmons (10): staging: lustre: libcfs: fixup all header block comments staging: lustre: libcfs: remove header's bare unsigned use staging: lustre: libcfs: name parameters for function prototypes staging: lustre: libcfs: remove blank line in header staging: lustre: libcfs: correct spelling in libcfs_cpu.h staging: lustre: libcfs: use bit macro in libcfs headers staging: lustre: libcfs: remove whitespace in libcfs_fail.h staging: lustre: libcfs: remove NULL comparisons in headers staging: lustre: libcfs: remove zero comparisons in headers staging: lustre: libcfs: use uXX instead of __uXX types in headers .../staging/lustre/include/linux/libcfs/curproc.h | 20 +- .../staging/lustre/include/linux/libcfs/libcfs.h | 4 +- .../lustre/include/linux/libcfs/libcfs_cpu.h | 4 +- .../lustre/include/linux/libcfs/libcfs_crypto.h | 12 +- .../lustre/include/linux/libcfs/libcfs_fail.h | 53 ++++--- .../lustre/include/linux/libcfs/libcfs_hash.h | 183 ++++++++++--------- .../lustre/include/linux/libcfs/libcfs_private.h | 13 +- .../lustre/include/linux/libcfs/libcfs_string.h | 24 ++-- .../lustre/include/linux/libcfs/libcfs_workitem.h | 4 +- .../lustre/include/linux/libcfs/linux/linux-cpu.h | 8 +- .../lustre/include/linux/libcfs/linux/linux-time.h | 8 +- drivers/staging/lustre/lustre/obdclass/lu_object.c | 2 +- 12 files changed, 179 insertions(+), 156 deletions(-) From jsimmons at infradead.org Fri Nov 18 16:48:40 2016 From: jsimmons at infradead.org (James Simmons) Date: Fri, 18 Nov 2016 11:48:40 -0500 Subject: [lustre-devel] [PATCH 06/10] staging: lustre: libcfs: use bit macro in libcfs headers In-Reply-To: <1479487724-20386-1-git-send-email-jsimmons@infradead.org> References: <1479487724-20386-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479487724-20386-7-git-send-email-jsimmons@infradead.org> Use the BIT macros instead of (1 << ***) in libcfs headers. Signed-off-by: James Simmons --- .../staging/lustre/include/linux/libcfs/curproc.h | 18 ++++---- .../lustre/include/linux/libcfs/libcfs_fail.h | 4 +- .../lustre/include/linux/libcfs/libcfs_hash.h | 42 ++++++++++---------- drivers/staging/lustre/lustre/obdclass/lu_object.c | 2 +- 4 files changed, 33 insertions(+), 33 deletions(-) diff --git a/drivers/staging/lustre/include/linux/libcfs/curproc.h b/drivers/staging/lustre/include/linux/libcfs/curproc.h index be0675d..6469a67 100644 --- a/drivers/staging/lustre/include/linux/libcfs/curproc.h +++ b/drivers/staging/lustre/include/linux/libcfs/curproc.h @@ -65,15 +65,15 @@ #define CFS_CAP_SYS_BOOT 23 #define CFS_CAP_SYS_RESOURCE 24 -#define CFS_CAP_FS_MASK ((1 << CFS_CAP_CHOWN) | \ - (1 << CFS_CAP_DAC_OVERRIDE) | \ - (1 << CFS_CAP_DAC_READ_SEARCH) | \ - (1 << CFS_CAP_FOWNER) | \ - (1 << CFS_CAP_FSETID) | \ - (1 << CFS_CAP_LINUX_IMMUTABLE) | \ - (1 << CFS_CAP_SYS_ADMIN) | \ - (1 << CFS_CAP_SYS_BOOT) | \ - (1 << CFS_CAP_SYS_RESOURCE)) +#define CFS_CAP_FS_MASK (BIT(CFS_CAP_CHOWN) | \ + BIT(CFS_CAP_DAC_OVERRIDE) | \ + BIT(CFS_CAP_DAC_READ_SEARCH) | \ + BIT(CFS_CAP_FOWNER) | \ + BIT(CFS_CAP_FSETID) | \ + BIT(CFS_CAP_LINUX_IMMUTABLE) | \ + BIT(CFS_CAP_SYS_ADMIN) | \ + BIT(CFS_CAP_SYS_BOOT) | \ + BIT(CFS_CAP_SYS_RESOURCE)) void cfs_cap_raise(cfs_cap_t cap); void cfs_cap_lower(cfs_cap_t cap); diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h index f06b4aa..6f019a4 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h @@ -55,11 +55,11 @@ enum { #define CFS_FAILED_BIT 30 /* CFS_FAILED is 0x40000000 */ -#define CFS_FAILED (1 << CFS_FAILED_BIT) +#define CFS_FAILED BIT(CFS_FAILED_BIT) #define CFS_FAIL_ONCE_BIT 31 /* CFS_FAIL_ONCE is 0x80000000 */ -#define CFS_FAIL_ONCE (1 << CFS_FAIL_ONCE_BIT) +#define CFS_FAIL_ONCE BIT(CFS_FAIL_ONCE_BIT) /* The following flags aren't made to be combined */ #define CFS_FAIL_SKIP 0x20000000 /* skip N times then fail */ diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h index 2f56707..85661b3 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h @@ -125,36 +125,36 @@ enum cfs_hash_tag { * . Some functions will be disabled with this flag, i.e: * cfs_hash_for_each_empty, cfs_hash_rehash */ - CFS_HASH_NO_LOCK = 1 << 0, + CFS_HASH_NO_LOCK = BIT(0), /** no bucket lock, use one spinlock to protect the whole hash */ - CFS_HASH_NO_BKTLOCK = 1 << 1, + CFS_HASH_NO_BKTLOCK = BIT(1), /** rwlock to protect bucket */ - CFS_HASH_RW_BKTLOCK = 1 << 2, + CFS_HASH_RW_BKTLOCK = BIT(2), /** spinlock to protect bucket */ - CFS_HASH_SPIN_BKTLOCK = 1 << 3, + CFS_HASH_SPIN_BKTLOCK = BIT(3), /** always add new item to tail */ - CFS_HASH_ADD_TAIL = 1 << 4, + CFS_HASH_ADD_TAIL = BIT(4), /** hash-table doesn't have refcount on item */ - CFS_HASH_NO_ITEMREF = 1 << 5, + CFS_HASH_NO_ITEMREF = BIT(5), /** big name for param-tree */ - CFS_HASH_BIGNAME = 1 << 6, + CFS_HASH_BIGNAME = BIT(6), /** track global count */ - CFS_HASH_COUNTER = 1 << 7, + CFS_HASH_COUNTER = BIT(7), /** rehash item by new key */ - CFS_HASH_REHASH_KEY = 1 << 8, + CFS_HASH_REHASH_KEY = BIT(8), /** Enable dynamic hash resizing */ - CFS_HASH_REHASH = 1 << 9, + CFS_HASH_REHASH = BIT(9), /** can shrink hash-size */ - CFS_HASH_SHRINK = 1 << 10, + CFS_HASH_SHRINK = BIT(10), /** assert hash is empty on exit */ - CFS_HASH_ASSERT_EMPTY = 1 << 11, + CFS_HASH_ASSERT_EMPTY = BIT(11), /** record hlist depth */ - CFS_HASH_DEPTH = 1 << 12, + CFS_HASH_DEPTH = BIT(12), /** * rehash is always scheduled in a different thread, so current * change on hash table is non-blocking */ - CFS_HASH_NBLK_CHANGE = 1 << 13, + CFS_HASH_NBLK_CHANGE = BIT(13), /** * NB, we typed hs_flags as __u16, please change it * if you need to extend >=16 flags @@ -323,20 +323,20 @@ struct cfs_hash_ops { /** total number of buckets in @hs */ #define CFS_HASH_NBKT(hs) \ - (1U << ((hs)->hs_cur_bits - (hs)->hs_bkt_bits)) + BIT((hs)->hs_cur_bits - (hs)->hs_bkt_bits) /** total number of buckets in @hs while rehashing */ #define CFS_HASH_RH_NBKT(hs) \ - (1U << ((hs)->hs_rehash_bits - (hs)->hs_bkt_bits)) + BIT((hs)->hs_rehash_bits - (hs)->hs_bkt_bits) /** number of hlist for in bucket */ -#define CFS_HASH_BKT_NHLIST(hs) (1U << (hs)->hs_bkt_bits) +#define CFS_HASH_BKT_NHLIST(hs) BIT((hs)->hs_bkt_bits) /** total number of hlist in @hs */ -#define CFS_HASH_NHLIST(hs) (1U << (hs)->hs_cur_bits) +#define CFS_HASH_NHLIST(hs) BIT((hs)->hs_cur_bits) /** total number of hlist in @hs while rehashing */ -#define CFS_HASH_RH_NHLIST(hs) (1U << (hs)->hs_rehash_bits) +#define CFS_HASH_RH_NHLIST(hs) BIT((hs)->hs_rehash_bits) static inline int cfs_hash_with_no_lock(struct cfs_hash *hs) @@ -775,8 +775,8 @@ void cfs_hash_rehash_key(struct cfs_hash *hs, const void *old_key, #endif /* CFS_HASH_DEBUG_LEVEL */ #define CFS_HASH_THETA_BITS 10 -#define CFS_HASH_MIN_THETA (1U << (CFS_HASH_THETA_BITS - 1)) -#define CFS_HASH_MAX_THETA (1U << (CFS_HASH_THETA_BITS + 1)) +#define CFS_HASH_MIN_THETA BIT(CFS_HASH_THETA_BITS - 1) +#define CFS_HASH_MAX_THETA BIT(CFS_HASH_THETA_BITS + 1) /* Return integer component of theta */ static inline int __cfs_hash_theta_int(int theta) diff --git a/drivers/staging/lustre/lustre/obdclass/lu_object.c b/drivers/staging/lustre/lustre/obdclass/lu_object.c index 14ca82d..43868ed 100644 --- a/drivers/staging/lustre/lustre/obdclass/lu_object.c +++ b/drivers/staging/lustre/lustre/obdclass/lu_object.c @@ -1981,7 +1981,7 @@ int lu_site_stats_print(const struct lu_site *s, struct seq_file *m) memset(&stats, 0, sizeof(stats)); lu_site_stats_get(s->ls_obj_hash, &stats, 1); - seq_printf(m, "%d/%d %d/%d %d %d %d %d %d %d %d %d\n", + seq_printf(m, "%d/%d %d/%ld %d %d %d %d %d %d %d %d\n", stats.lss_busy, stats.lss_total, stats.lss_populated, -- 1.7.1 From jsimmons at infradead.org Fri Nov 18 16:48:35 2016 From: jsimmons at infradead.org (James Simmons) Date: Fri, 18 Nov 2016 11:48:35 -0500 Subject: [lustre-devel] [PATCH 01/10] staging: lustre: libcfs: fixup all header block comments In-Reply-To: <1479487724-20386-1-git-send-email-jsimmons@infradead.org> References: <1479487724-20386-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479487724-20386-2-git-send-email-jsimmons@infradead.org> Properly format the incorrect comments sections that were reported by checkpatch. Signed-off-by: James Simmons --- .../lustre/include/linux/libcfs/libcfs_crypto.h | 10 ++++-- .../lustre/include/linux/libcfs/libcfs_fail.h | 30 +++++++++++++------ .../lustre/include/linux/libcfs/libcfs_hash.h | 24 ++++++++++----- .../lustre/include/linux/libcfs/libcfs_private.h | 5 ++- .../lustre/include/linux/libcfs/libcfs_string.h | 6 ++- 5 files changed, 49 insertions(+), 26 deletions(-) diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_crypto.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_crypto.h index 02be7d7..a0865e5 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_crypto.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_crypto.h @@ -29,10 +29,12 @@ #define _LIBCFS_CRYPTO_H struct cfs_crypto_hash_type { - char *cht_name; /**< hash algorithm name, equal to - * format name for crypto api */ - unsigned int cht_key; /**< init key by default (valid for - * 4 bytes context like crc32, adler */ + char *cht_name; /*< hash algorithm name, equal to + * format name for crypto api + */ + unsigned int cht_key; /*< init key by default (valid for + * 4 bytes context like crc32, adler + */ unsigned int cht_size; /**< hash digest size */ }; diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h index bdbbe93..b1658c8 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h @@ -103,22 +103,28 @@ static inline int cfs_fail_check_set(__u32 id, __u32 value, #define CFS_FAIL_CHECK_QUIET(id) \ cfs_fail_check_set(id, 0, CFS_FAIL_LOC_NOSET, 1) -/* If id hit cfs_fail_loc and cfs_fail_val == (-1 or value) return 1, - * otherwise return 0 */ +/* + * If id hit cfs_fail_loc and cfs_fail_val == (-1 or value) return 1, + * otherwise return 0 + */ #define CFS_FAIL_CHECK_VALUE(id, value) \ cfs_fail_check_set(id, value, CFS_FAIL_LOC_VALUE, 0) #define CFS_FAIL_CHECK_VALUE_QUIET(id, value) \ cfs_fail_check_set(id, value, CFS_FAIL_LOC_VALUE, 1) -/* If id hit cfs_fail_loc, cfs_fail_loc |= value and return 1, - * otherwise return 0 */ +/* + * If id hit cfs_fail_loc, cfs_fail_loc |= value and return 1, + * otherwise return 0 + */ #define CFS_FAIL_CHECK_ORSET(id, value) \ cfs_fail_check_set(id, value, CFS_FAIL_LOC_ORSET, 0) #define CFS_FAIL_CHECK_ORSET_QUIET(id, value) \ cfs_fail_check_set(id, value, CFS_FAIL_LOC_ORSET, 1) -/* If id hit cfs_fail_loc, cfs_fail_loc = value and return 1, - * otherwise return 0 */ +/* + * If id hit cfs_fail_loc, cfs_fail_loc = value and return 1, + * otherwise return 0 + */ #define CFS_FAIL_CHECK_RESET(id, value) \ cfs_fail_check_set(id, value, CFS_FAIL_LOC_RESET, 0) #define CFS_FAIL_CHECK_RESET_QUIET(id, value) \ @@ -138,8 +144,10 @@ static inline int cfs_fail_timeout_set(__u32 id, __u32 value, int ms, int set) #define CFS_FAIL_TIMEOUT_MS(id, ms) \ cfs_fail_timeout_set(id, 0, ms, CFS_FAIL_LOC_NOSET) -/* If id hit cfs_fail_loc, cfs_fail_loc |= value and - * sleep seconds or milliseconds */ +/* + * If id hit cfs_fail_loc, cfs_fail_loc |= value and + * sleep seconds or milliseconds + */ #define CFS_FAIL_TIMEOUT_ORSET(id, value, secs) \ cfs_fail_timeout_set(id, value, secs * 1000, CFS_FAIL_LOC_ORSET) @@ -152,10 +160,12 @@ static inline int cfs_fail_timeout_set(__u32 id, __u32 value, int ms, int set) #define CFS_FAULT_CHECK(id) \ CFS_FAIL_CHECK(CFS_FAULT | (id)) -/* The idea here is to synchronise two threads to force a race. The +/* + * The idea here is to synchronise two threads to force a race. The * first thread that calls this with a matching fail_loc is put to * sleep. The next thread that calls with the same fail_loc wakes up - * the first and continues. */ + * the first and continues. + */ static inline void cfs_race(__u32 id) { diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h index f2b4399..a4ca488 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h @@ -57,8 +57,10 @@ /** disable debug */ #define CFS_HASH_DEBUG_NONE 0 -/** record hash depth and output to console when it's too deep, - * computing overhead is low but consume more memory */ +/* + * record hash depth and output to console when it's too deep, + * computing overhead is low but consume more memory + */ #define CFS_HASH_DEBUG_1 1 /** expensive, check key validation */ #define CFS_HASH_DEBUG_2 2 @@ -153,8 +155,10 @@ enum cfs_hash_tag { * change on hash table is non-blocking */ CFS_HASH_NBLK_CHANGE = 1 << 13, - /** NB, we typed hs_flags as __u16, please change it - * if you need to extend >=16 flags */ + /** + * NB, we typed hs_flags as __u16, please change it + * if you need to extend >=16 flags + */ }; /** most used attributes */ @@ -201,8 +205,10 @@ enum cfs_hash_tag { */ struct cfs_hash { - /** serialize with rehash, or serialize all operations if - * the hash-table has CFS_HASH_NO_BKTLOCK */ + /** + * serialize with rehash, or serialize all operations if + * the hash-table has CFS_HASH_NO_BKTLOCK + */ union cfs_hash_lock hs_lock; /** hash operations */ struct cfs_hash_ops *hs_ops; @@ -369,9 +375,11 @@ struct cfs_hash_ops { static inline int cfs_hash_with_no_itemref(struct cfs_hash *hs) { - /* hash-table doesn't keep refcount on item, + /* + * hash-table doesn't keep refcount on item, * item can't be removed from hash unless it's - * ZERO refcount */ + * ZERO refcount + */ return (hs->hs_flags & CFS_HASH_NO_ITEMREF) != 0; } diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h index e0e1a5d..523e8f4 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h @@ -280,7 +280,7 @@ #define CFS_FREE_PTR(ptr) LIBCFS_FREE(ptr, sizeof(*(ptr))) /** Compile-time assertion. - + * * Check an invariant described by a constant expression at compile time by * forcing a compiler error if it does not hold. \a cond must be a constant * expression as defined by the ISO C Standard: @@ -306,7 +306,8 @@ /* -------------------------------------------------------------------- * Light-weight trace * Support for temporary event tracing with minimal Heisenberg effect. - * -------------------------------------------------------------------- */ + * -------------------------------------------------------------------- + */ #define MKSTR(ptr) ((ptr)) ? (ptr) : "" diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_string.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_string.h index 0ee60ff..56341d9 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_string.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_string.h @@ -84,9 +84,11 @@ int cfs_expr_list_values(struct cfs_expr_list *expr_list, static inline void cfs_expr_list_values_free(__u32 *values, int num) { - /* This array is allocated by LIBCFS_ALLOC(), so it shouldn't be freed + /* + * This array is allocated by LIBCFS_ALLOC(), so it shouldn't be freed * by OBD_FREE() if it's called by module other than libcfs & LNet, - * otherwise we will see fake memory leak */ + * otherwise we will see fake memory leak + */ LIBCFS_FREE(values, num * sizeof(values[0])); } -- 1.7.1 From jsimmons at infradead.org Fri Nov 18 16:48:38 2016 From: jsimmons at infradead.org (James Simmons) Date: Fri, 18 Nov 2016 11:48:38 -0500 Subject: [lustre-devel] [PATCH 04/10] staging: lustre: libcfs: remove blank line in header In-Reply-To: <1479487724-20386-1-git-send-email-jsimmons@infradead.org> References: <1479487724-20386-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479487724-20386-5-git-send-email-jsimmons@infradead.org> The header libcfs_fail.h has a extra blank line that is not needed. Signed-off-by: James Simmons --- .../lustre/include/linux/libcfs/libcfs_fail.h | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h index b1658c8..f06b4aa 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h @@ -168,7 +168,6 @@ static inline int cfs_fail_timeout_set(__u32 id, __u32 value, int ms, int set) */ static inline void cfs_race(__u32 id) { - if (CFS_FAIL_PRECHECK(id)) { if (unlikely(__cfs_fail_check_set(id, 0, CFS_FAIL_LOC_NOSET))) { int rc; -- 1.7.1 From jsimmons at infradead.org Fri Nov 18 16:48:39 2016 From: jsimmons at infradead.org (James Simmons) Date: Fri, 18 Nov 2016 11:48:39 -0500 Subject: [lustre-devel] [PATCH 05/10] staging: lustre: libcfs: correct spelling in libcfs_cpu.h In-Reply-To: <1479487724-20386-1-git-send-email-jsimmons@infradead.org> References: <1479487724-20386-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479487724-20386-6-git-send-email-jsimmons@infradead.org> Spell the word destroy correctly. Signed-off-by: James Simmons --- .../lustre/include/linux/libcfs/libcfs_cpu.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h index 81d8079..61bd951 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h @@ -211,7 +211,7 @@ void cfs_cpt_unset_nodemask(struct cfs_cpt_table *cptab, */ void *cfs_percpt_alloc(struct cfs_cpt_table *cptab, unsigned int size); /* - * destory per-cpu-partition variable + * destroy per-cpu-partition variable */ void cfs_percpt_free(void *vars); int cfs_percpt_number(void *vars); -- 1.7.1 From jsimmons at infradead.org Fri Nov 18 16:48:36 2016 From: jsimmons at infradead.org (James Simmons) Date: Fri, 18 Nov 2016 11:48:36 -0500 Subject: [lustre-devel] [PATCH 02/10] staging: lustre: libcfs: remove header's bare unsigned use In-Reply-To: <1479487724-20386-1-git-send-email-jsimmons@infradead.org> References: <1479487724-20386-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479487724-20386-3-git-send-email-jsimmons@infradead.org> Fixup the libcfs headers to use the proper unsigned int instead of raw unsigned. Signed-off-by: James Simmons --- .../lustre/include/linux/libcfs/libcfs_hash.h | 28 ++++++++++---------- .../lustre/include/linux/libcfs/libcfs_string.h | 6 ++-- .../lustre/include/linux/libcfs/linux/linux-cpu.h | 8 +++--- 3 files changed, 21 insertions(+), 21 deletions(-) diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h index a4ca488..28937d5 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h @@ -297,8 +297,8 @@ struct cfs_hash_hlist_ops { struct cfs_hash_ops { /** return hashed value from @key */ - unsigned (*hs_hash)(struct cfs_hash *hs, const void *key, - unsigned mask); + unsigned int (*hs_hash)(struct cfs_hash *hs, const void *key, + unsigned int mask); /** return key address of @hnode */ void * (*hs_key)(struct hlist_node *hnode); /** copy key from @hnode to @key */ @@ -461,7 +461,7 @@ struct cfs_hash_ops { } static inline unsigned -cfs_hash_id(struct cfs_hash *hs, const void *key, unsigned mask) +cfs_hash_id(struct cfs_hash *hs, const void *key, unsigned int mask) { return hs->hs_ops->hs_hash(hs, key, mask); } @@ -570,7 +570,7 @@ void cfs_hash_bd_get(struct cfs_hash *hs, const void *key, } static inline void -cfs_hash_bd_index_set(struct cfs_hash *hs, unsigned index, +cfs_hash_bd_index_set(struct cfs_hash *hs, unsigned int index, struct cfs_hash_bd *bd) { bd->bd_bucket = hs->hs_buckets[index >> hs->hs_bkt_bits]; @@ -677,10 +677,10 @@ struct hlist_node * /* Hash init/cleanup functions */ struct cfs_hash * -cfs_hash_create(char *name, unsigned cur_bits, unsigned max_bits, - unsigned bkt_bits, unsigned extra_bytes, - unsigned min_theta, unsigned max_theta, - struct cfs_hash_ops *ops, unsigned flags); +cfs_hash_create(char *name, unsigned int cur_bits, unsigned int max_bits, + unsigned int bkt_bits, unsigned int extra_bytes, + unsigned int min_theta, unsigned int max_theta, + struct cfs_hash_ops *ops, unsigned int flags); struct cfs_hash *cfs_hash_getref(struct cfs_hash *hs); void cfs_hash_putref(struct cfs_hash *hs); @@ -725,7 +725,7 @@ typedef int (*cfs_hash_for_each_cb_t)(struct cfs_hash *hs, cfs_hash_cond_del(struct cfs_hash *hs, cfs_hash_cond_opt_cb_t, void *data); void -cfs_hash_hlist_for_each(struct cfs_hash *hs, unsigned hindex, +cfs_hash_hlist_for_each(struct cfs_hash *hs, unsigned int hindex, cfs_hash_for_each_cb_t, void *data); int cfs_hash_is_empty(struct cfs_hash *hs); __u64 cfs_hash_size_get(struct cfs_hash *hs); @@ -813,9 +813,9 @@ static inline int __cfs_hash_theta(struct cfs_hash *hs) * Generic djb2 hash algorithm for character arrays. */ static inline unsigned -cfs_hash_djb2_hash(const void *key, size_t size, unsigned mask) +cfs_hash_djb2_hash(const void *key, size_t size, unsigned int mask) { - unsigned i, hash = 5381; + unsigned int i, hash = 5381; LASSERT(key != NULL); @@ -829,7 +829,7 @@ static inline int __cfs_hash_theta(struct cfs_hash *hs) * Generic u32 hash algorithm. */ static inline unsigned -cfs_hash_u32_hash(const __u32 key, unsigned mask) +cfs_hash_u32_hash(const __u32 key, unsigned int mask) { return ((key * CFS_GOLDEN_RATIO_PRIME_32) & mask); } @@ -838,9 +838,9 @@ static inline int __cfs_hash_theta(struct cfs_hash *hs) * Generic u64 hash algorithm. */ static inline unsigned -cfs_hash_u64_hash(const __u64 key, unsigned mask) +cfs_hash_u64_hash(const __u64 key, unsigned int mask) { - return ((unsigned)(key * CFS_GOLDEN_RATIO_PRIME_64) & mask); + return ((unsigned int)(key * CFS_GOLDEN_RATIO_PRIME_64) & mask); } /** iterate over all buckets in @bds (array of struct cfs_hash_bd) */ diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_string.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_string.h index 56341d9..f5a434d 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_string.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_string.h @@ -74,8 +74,8 @@ struct cfs_expr_list { char *cfs_trimwhite(char *str); int cfs_gettok(struct cfs_lstr *next, char delim, struct cfs_lstr *res); -int cfs_str2num_check(char *str, int nob, unsigned *num, - unsigned min, unsigned max); +int cfs_str2num_check(char *str, int nob, unsigned int *num, + unsigned int min, unsigned int max); int cfs_expr_list_match(__u32 value, struct cfs_expr_list *expr_list); int cfs_expr_list_print(char *buffer, int count, struct cfs_expr_list *expr_list); @@ -93,7 +93,7 @@ int cfs_expr_list_values(struct cfs_expr_list *expr_list, } void cfs_expr_list_free(struct cfs_expr_list *expr_list); -int cfs_expr_list_parse(char *str, int len, unsigned min, unsigned max, +int cfs_expr_list_parse(char *str, int len, unsigned int min, unsigned int max, struct cfs_expr_list **elpp); void cfs_expr_list_free_list(struct list_head *list); diff --git a/drivers/staging/lustre/include/linux/libcfs/linux/linux-cpu.h b/drivers/staging/lustre/include/linux/libcfs/linux/linux-cpu.h index f63cb47..dd0cd04 100644 --- a/drivers/staging/lustre/include/linux/libcfs/linux/linux-cpu.h +++ b/drivers/staging/lustre/include/linux/libcfs/linux/linux-cpu.h @@ -52,17 +52,17 @@ struct cfs_cpu_partition { /* nodes mask for this partition */ nodemask_t *cpt_nodemask; /* spread rotor for NUMA allocator */ - unsigned cpt_spread_rotor; + unsigned int cpt_spread_rotor; }; /** descriptor for CPU partitions */ struct cfs_cpt_table { /* version, reserved for hotplug */ - unsigned ctb_version; + unsigned int ctb_version; /* spread rotor for NUMA allocator */ - unsigned ctb_spread_rotor; + unsigned int ctb_spread_rotor; /* # of CPU partitions */ - unsigned ctb_nparts; + unsigned int ctb_nparts; /* partitions tables */ struct cfs_cpu_partition *ctb_parts; /* shadow HW CPU to CPU partition ID */ -- 1.7.1 From jsimmons at infradead.org Fri Nov 18 16:48:41 2016 From: jsimmons at infradead.org (James Simmons) Date: Fri, 18 Nov 2016 11:48:41 -0500 Subject: [lustre-devel] [PATCH 07/10] staging: lustre: libcfs: remove whitespace in libcfs_fail.h In-Reply-To: <1479487724-20386-1-git-send-email-jsimmons@infradead.org> References: <1479487724-20386-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479487724-20386-8-git-send-email-jsimmons@infradead.org> One last white space is still left in libcfs_fail.h. Lets remove it. Signed-off-by: James Simmons --- .../lustre/include/linux/libcfs/libcfs_fail.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h index 6f019a4..73cbb47 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h @@ -73,7 +73,7 @@ static inline bool CFS_FAIL_PRECHECK(__u32 id) { return cfs_fail_loc != 0 && ((cfs_fail_loc & CFS_FAIL_MASK_LOC) == (id & CFS_FAIL_MASK_LOC) || - (cfs_fail_loc & id & CFS_FAULT)); + (cfs_fail_loc & id & CFS_FAULT)); } static inline int cfs_fail_check_set(__u32 id, __u32 value, -- 1.7.1 From jsimmons at infradead.org Fri Nov 18 16:48:43 2016 From: jsimmons at infradead.org (James Simmons) Date: Fri, 18 Nov 2016 11:48:43 -0500 Subject: [lustre-devel] [PATCH 09/10] staging: lustre: libcfs: remove zero comparisons in headers In-Reply-To: <1479487724-20386-1-git-send-email-jsimmons@infradead.org> References: <1479487724-20386-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479487724-20386-10-git-send-email-jsimmons@infradead.org> Remove the zero comparisions in the libcfs headers. Signed-off-by: James Simmons --- .../lustre/include/linux/libcfs/libcfs_crypto.h | 2 +- .../lustre/include/linux/libcfs/libcfs_fail.h | 4 +- .../lustre/include/linux/libcfs/libcfs_hash.h | 32 ++++++++++---------- 3 files changed, 19 insertions(+), 19 deletions(-) diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_crypto.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_crypto.h index a0865e5..8f34c5d 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_crypto.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_crypto.h @@ -137,7 +137,7 @@ static inline unsigned char cfs_crypto_hash_alg(const char *algname) enum cfs_crypto_hash_alg hash_alg; for (hash_alg = 0; hash_alg < CFS_HASH_ALG_MAX; hash_alg++) - if (strcmp(hash_types[hash_alg].cht_name, algname) == 0) + if (!strcmp(hash_types[hash_alg].cht_name, algname)) return hash_alg; return CFS_HASH_ALG_UNKNOWN; diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h index 73cbb47..129047e 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h @@ -71,7 +71,7 @@ enum { static inline bool CFS_FAIL_PRECHECK(__u32 id) { - return cfs_fail_loc != 0 && + return cfs_fail_loc && ((cfs_fail_loc & CFS_FAIL_MASK_LOC) == (id & CFS_FAIL_MASK_LOC) || (cfs_fail_loc & id & CFS_FAULT)); } @@ -175,7 +175,7 @@ static inline void cfs_race(__u32 id) cfs_race_state = 0; CERROR("cfs_race id %x sleeping\n", id); rc = wait_event_interruptible(cfs_race_waitq, - cfs_race_state != 0); + !!cfs_race_state); CERROR("cfs_fail_race id %x awake, rc=%d\n", id, rc); } else { CERROR("cfs_fail_race id %x waking\n", id); diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h index d0ba3e0..6231f47 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h @@ -342,34 +342,34 @@ struct cfs_hash_ops { cfs_hash_with_no_lock(struct cfs_hash *hs) { /* caller will serialize all operations for this hash-table */ - return (hs->hs_flags & CFS_HASH_NO_LOCK) != 0; + return hs->hs_flags & CFS_HASH_NO_LOCK; } static inline int cfs_hash_with_no_bktlock(struct cfs_hash *hs) { /* no bucket lock, one single lock to protect the hash-table */ - return (hs->hs_flags & CFS_HASH_NO_BKTLOCK) != 0; + return hs->hs_flags & CFS_HASH_NO_BKTLOCK; } static inline int cfs_hash_with_rw_bktlock(struct cfs_hash *hs) { /* rwlock to protect hash bucket */ - return (hs->hs_flags & CFS_HASH_RW_BKTLOCK) != 0; + return hs->hs_flags & CFS_HASH_RW_BKTLOCK; } static inline int cfs_hash_with_spin_bktlock(struct cfs_hash *hs) { /* spinlock to protect hash bucket */ - return (hs->hs_flags & CFS_HASH_SPIN_BKTLOCK) != 0; + return hs->hs_flags & CFS_HASH_SPIN_BKTLOCK; } static inline int cfs_hash_with_add_tail(struct cfs_hash *hs) { - return (hs->hs_flags & CFS_HASH_ADD_TAIL) != 0; + return hs->hs_flags & CFS_HASH_ADD_TAIL; } static inline int @@ -380,55 +380,55 @@ struct cfs_hash_ops { * item can't be removed from hash unless it's * ZERO refcount */ - return (hs->hs_flags & CFS_HASH_NO_ITEMREF) != 0; + return hs->hs_flags & CFS_HASH_NO_ITEMREF; } static inline int cfs_hash_with_bigname(struct cfs_hash *hs) { - return (hs->hs_flags & CFS_HASH_BIGNAME) != 0; + return hs->hs_flags & CFS_HASH_BIGNAME; } static inline int cfs_hash_with_counter(struct cfs_hash *hs) { - return (hs->hs_flags & CFS_HASH_COUNTER) != 0; + return hs->hs_flags & CFS_HASH_COUNTER; } static inline int cfs_hash_with_rehash(struct cfs_hash *hs) { - return (hs->hs_flags & CFS_HASH_REHASH) != 0; + return hs->hs_flags & CFS_HASH_REHASH; } static inline int cfs_hash_with_rehash_key(struct cfs_hash *hs) { - return (hs->hs_flags & CFS_HASH_REHASH_KEY) != 0; + return hs->hs_flags & CFS_HASH_REHASH_KEY; } static inline int cfs_hash_with_shrink(struct cfs_hash *hs) { - return (hs->hs_flags & CFS_HASH_SHRINK) != 0; + return hs->hs_flags & CFS_HASH_SHRINK; } static inline int cfs_hash_with_assert_empty(struct cfs_hash *hs) { - return (hs->hs_flags & CFS_HASH_ASSERT_EMPTY) != 0; + return hs->hs_flags & CFS_HASH_ASSERT_EMPTY; } static inline int cfs_hash_with_depth(struct cfs_hash *hs) { - return (hs->hs_flags & CFS_HASH_DEPTH) != 0; + return hs->hs_flags & CFS_HASH_DEPTH; } static inline int cfs_hash_with_nblk_change(struct cfs_hash *hs) { - return (hs->hs_flags & CFS_HASH_NBLK_CHANGE) != 0; + return hs->hs_flags & CFS_HASH_NBLK_CHANGE; } static inline int @@ -442,14 +442,14 @@ struct cfs_hash_ops { cfs_hash_is_rehashing(struct cfs_hash *hs) { /* rehash is launched */ - return hs->hs_rehash_bits != 0; + return !!hs->hs_rehash_bits; } static inline int cfs_hash_is_iterating(struct cfs_hash *hs) { /* someone is calling cfs_hash_for_each_* */ - return hs->hs_iterating || hs->hs_iterators != 0; + return hs->hs_iterating || hs->hs_iterators; } static inline int -- 1.7.1 From jsimmons at infradead.org Fri Nov 18 16:48:44 2016 From: jsimmons at infradead.org (James Simmons) Date: Fri, 18 Nov 2016 11:48:44 -0500 Subject: [lustre-devel] [PATCH 10/10] staging: lustre: libcfs: use uXX instead of __uXX types in headers In-Reply-To: <1479487724-20386-1-git-send-email-jsimmons@infradead.org> References: <1479487724-20386-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479487724-20386-11-git-send-email-jsimmons@infradead.org> The types __[u|s]XX is only used for UAPI headers and userspace. Only keep these types for the libcfs headers that are UAPI headers. The rest convert to the standard uXX types. Signed-off-by: James Simmons --- .../staging/lustre/include/linux/libcfs/curproc.h | 2 +- .../lustre/include/linux/libcfs/libcfs_cpu.h | 2 +- .../lustre/include/linux/libcfs/libcfs_fail.h | 12 +++--- .../lustre/include/linux/libcfs/libcfs_hash.h | 46 ++++++++++---------- .../lustre/include/linux/libcfs/libcfs_string.h | 12 +++--- .../lustre/include/linux/libcfs/linux/linux-time.h | 8 ++-- 6 files changed, 41 insertions(+), 41 deletions(-) diff --git a/drivers/staging/lustre/include/linux/libcfs/curproc.h b/drivers/staging/lustre/include/linux/libcfs/curproc.h index 6469a67..1ea27c9 100644 --- a/drivers/staging/lustre/include/linux/libcfs/curproc.h +++ b/drivers/staging/lustre/include/linux/libcfs/curproc.h @@ -53,7 +53,7 @@ #define current_pid() (current->pid) #define current_comm() (current->comm) -typedef __u32 cfs_cap_t; +typedef u32 cfs_cap_t; #define CFS_CAP_CHOWN 0 #define CFS_CAP_DAC_OVERRIDE 1 diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h index 61bd951..6d8752a 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_cpu.h @@ -92,7 +92,7 @@ struct cfs_cpt_table { /* node mask */ nodemask_t ctb_nodemask; /* version */ - __u64 ctb_version; + u64 ctb_version; }; static inline cpumask_t * diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h index 129047e..fedb46d 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_fail.h @@ -39,8 +39,8 @@ extern wait_queue_head_t cfs_race_waitq; extern int cfs_race_state; -int __cfs_fail_check_set(__u32 id, __u32 value, int set); -int __cfs_fail_timeout_set(__u32 id, __u32 value, int ms, int set); +int __cfs_fail_check_set(u32 id, u32 value, int set); +int __cfs_fail_timeout_set(u32 id, u32 value, int ms, int set); enum { CFS_FAIL_LOC_NOSET = 0, @@ -69,14 +69,14 @@ enum { #define CFS_FAULT 0x02000000 /* match any CFS_FAULT_CHECK */ -static inline bool CFS_FAIL_PRECHECK(__u32 id) +static inline bool CFS_FAIL_PRECHECK(u32 id) { return cfs_fail_loc && ((cfs_fail_loc & CFS_FAIL_MASK_LOC) == (id & CFS_FAIL_MASK_LOC) || (cfs_fail_loc & id & CFS_FAULT)); } -static inline int cfs_fail_check_set(__u32 id, __u32 value, +static inline int cfs_fail_check_set(u32 id, u32 value, int set, int quiet) { int ret = 0; @@ -130,7 +130,7 @@ static inline int cfs_fail_check_set(__u32 id, __u32 value, #define CFS_FAIL_CHECK_RESET_QUIET(id, value) \ cfs_fail_check_set(id, value, CFS_FAIL_LOC_RESET, 1) -static inline int cfs_fail_timeout_set(__u32 id, __u32 value, int ms, int set) +static inline int cfs_fail_timeout_set(u32 id, u32 value, int ms, int set) { if (unlikely(CFS_FAIL_PRECHECK(id))) return __cfs_fail_timeout_set(id, value, ms, set); @@ -166,7 +166,7 @@ static inline int cfs_fail_timeout_set(__u32 id, __u32 value, int ms, int set) * sleep. The next thread that calls with the same fail_loc wakes up * the first and continues. */ -static inline void cfs_race(__u32 id) +static inline void cfs_race(u32 id) { if (CFS_FAIL_PRECHECK(id)) { if (unlikely(__cfs_fail_check_set(id, 0, CFS_FAIL_LOC_NOSET))) { diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h index 6231f47..0cc2fc4 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h @@ -89,8 +89,8 @@ */ struct cfs_hash_bucket { union cfs_hash_lock hsb_lock; /**< bucket lock */ - __u32 hsb_count; /**< current entries */ - __u32 hsb_version; /**< change version */ + u32 hsb_count; /**< current entries */ + u32 hsb_version; /**< change version */ unsigned int hsb_index; /**< index of bucket */ int hsb_depmax; /**< max depth on bucket */ long hsb_head[0]; /**< hash-head array */ @@ -156,7 +156,7 @@ enum cfs_hash_tag { */ CFS_HASH_NBLK_CHANGE = BIT(13), /** - * NB, we typed hs_flags as __u16, please change it + * NB, we typed hs_flags as u16, please change it * if you need to extend >=16 flags */ }; @@ -221,31 +221,31 @@ struct cfs_hash { /** total number of items on this hash-table */ atomic_t hs_count; /** hash flags, see cfs_hash_tag for detail */ - __u16 hs_flags; + u16 hs_flags; /** # of extra-bytes for bucket, for user saving extended attributes */ - __u16 hs_extra_bytes; + u16 hs_extra_bytes; /** wants to iterate */ - __u8 hs_iterating; + u8 hs_iterating; /** hash-table is dying */ - __u8 hs_exiting; + u8 hs_exiting; /** current hash bits */ - __u8 hs_cur_bits; + u8 hs_cur_bits; /** min hash bits */ - __u8 hs_min_bits; + u8 hs_min_bits; /** max hash bits */ - __u8 hs_max_bits; + u8 hs_max_bits; /** bits for rehash */ - __u8 hs_rehash_bits; + u8 hs_rehash_bits; /** bits for each bucket */ - __u8 hs_bkt_bits; + u8 hs_bkt_bits; /** resize min threshold */ - __u16 hs_min_theta; + u16 hs_min_theta; /** resize max threshold */ - __u16 hs_max_theta; + u16 hs_max_theta; /** resize count */ - __u32 hs_rehash_count; + u32 hs_rehash_count; /** # of iterators (caller of cfs_hash_for_each_*) */ - __u32 hs_iterators; + u32 hs_iterators; /** rehash workitem */ struct cfs_workitem hs_rehash_wi; /** refcount on this hash table */ @@ -584,14 +584,14 @@ void cfs_hash_bd_get(struct cfs_hash *hs, const void *key, cfs_hash_bkt_size(hs) - hs->hs_extra_bytes; } -static inline __u32 +static inline u32 cfs_hash_bd_version_get(struct cfs_hash_bd *bd) { /* need hold cfs_hash_bd_lock */ return bd->bd_bucket->hsb_version; } -static inline __u32 +static inline u32 cfs_hash_bd_count_get(struct cfs_hash_bd *bd) { /* need hold cfs_hash_bd_lock */ @@ -729,7 +729,7 @@ typedef int (*cfs_hash_for_each_cb_t)(struct cfs_hash *hs, cfs_hash_hlist_for_each(struct cfs_hash *hs, unsigned int hindex, cfs_hash_for_each_cb_t cb, void *data); int cfs_hash_is_empty(struct cfs_hash *hs); -__u64 cfs_hash_size_get(struct cfs_hash *hs); +u64 cfs_hash_size_get(struct cfs_hash *hs); /* * Rehash - Theta is calculated to be the average chained @@ -801,8 +801,8 @@ static inline int __cfs_hash_theta(struct cfs_hash *hs) __cfs_hash_set_theta(struct cfs_hash *hs, int min, int max) { LASSERT(min < max); - hs->hs_min_theta = (__u16)min; - hs->hs_max_theta = (__u16)max; + hs->hs_min_theta = (u16)min; + hs->hs_max_theta = (u16)max; } /* Generic debug formatting routines mainly for proc handler */ @@ -830,7 +830,7 @@ static inline int __cfs_hash_theta(struct cfs_hash *hs) * Generic u32 hash algorithm. */ static inline unsigned -cfs_hash_u32_hash(const __u32 key, unsigned int mask) +cfs_hash_u32_hash(const u32 key, unsigned int mask) { return ((key * CFS_GOLDEN_RATIO_PRIME_32) & mask); } @@ -839,7 +839,7 @@ static inline int __cfs_hash_theta(struct cfs_hash *hs) * Generic u64 hash algorithm. */ static inline unsigned -cfs_hash_u64_hash(const __u64 key, unsigned int mask) +cfs_hash_u64_hash(const u64 key, unsigned int mask) { return ((unsigned int)(key * CFS_GOLDEN_RATIO_PRIME_64) & mask); } diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_string.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_string.h index f5a434d..41795d9 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_string.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_string.h @@ -62,9 +62,9 @@ struct cfs_range_expr { * Link to cfs_expr_list::el_exprs. */ struct list_head re_link; - __u32 re_lo; - __u32 re_hi; - __u32 re_stride; + u32 re_lo; + u32 re_hi; + u32 re_stride; }; struct cfs_expr_list { @@ -76,13 +76,13 @@ struct cfs_expr_list { int cfs_gettok(struct cfs_lstr *next, char delim, struct cfs_lstr *res); int cfs_str2num_check(char *str, int nob, unsigned int *num, unsigned int min, unsigned int max); -int cfs_expr_list_match(__u32 value, struct cfs_expr_list *expr_list); +int cfs_expr_list_match(u32 value, struct cfs_expr_list *expr_list); int cfs_expr_list_print(char *buffer, int count, struct cfs_expr_list *expr_list); int cfs_expr_list_values(struct cfs_expr_list *expr_list, - int max, __u32 **values); + int max, u32 **values); static inline void -cfs_expr_list_values_free(__u32 *values, int num) +cfs_expr_list_values_free(u32 *values, int num) { /* * This array is allocated by LIBCFS_ALLOC(), so it shouldn't be freed diff --git a/drivers/staging/lustre/include/linux/libcfs/linux/linux-time.h b/drivers/staging/lustre/include/linux/libcfs/linux/linux-time.h index b646acd..709e1ce 100644 --- a/drivers/staging/lustre/include/linux/libcfs/linux/linux-time.h +++ b/drivers/staging/lustre/include/linux/libcfs/linux/linux-time.h @@ -76,23 +76,23 @@ static inline long cfs_duration_sec(long d) #define cfs_time_current_64 get_jiffies_64 -static inline __u64 cfs_time_add_64(__u64 t, __u64 d) +static inline u64 cfs_time_add_64(u64 t, u64 d) { return t + d; } -static inline __u64 cfs_time_shift_64(int seconds) +static inline u64 cfs_time_shift_64(int seconds) { return cfs_time_add_64(cfs_time_current_64(), cfs_time_seconds(seconds)); } -static inline int cfs_time_before_64(__u64 t1, __u64 t2) +static inline int cfs_time_before_64(u64 t1, u64 t2) { return (__s64)t2 - (__s64)t1 > 0; } -static inline int cfs_time_beforeq_64(__u64 t1, __u64 t2) +static inline int cfs_time_beforeq_64(u64 t1, u64 t2) { return (__s64)t2 - (__s64)t1 >= 0; } -- 1.7.1 From jsimmons at infradead.org Fri Nov 18 16:48:37 2016 From: jsimmons at infradead.org (James Simmons) Date: Fri, 18 Nov 2016 11:48:37 -0500 Subject: [lustre-devel] [PATCH 03/10] staging: lustre: libcfs: name parameters for function prototypes In-Reply-To: <1479487724-20386-1-git-send-email-jsimmons@infradead.org> References: <1479487724-20386-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479487724-20386-4-git-send-email-jsimmons@infradead.org> Give the parameters names for the function prototypes in the libcfs headers. Signed-off-by: James Simmons --- .../staging/lustre/include/linux/libcfs/libcfs.h | 4 ++-- .../lustre/include/linux/libcfs/libcfs_hash.h | 15 ++++++++------- .../lustre/include/linux/libcfs/libcfs_private.h | 4 ++-- .../lustre/include/linux/libcfs/libcfs_workitem.h | 4 ++-- 4 files changed, 14 insertions(+), 13 deletions(-) diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs.h b/drivers/staging/lustre/include/linux/libcfs/libcfs.h index 3b92d38..70eb08e 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs.h @@ -61,7 +61,7 @@ sigset_t cfs_block_allsigs(void); sigset_t cfs_block_sigs(unsigned long sigs); sigset_t cfs_block_sigsinv(unsigned long sigs); -void cfs_restore_sigs(sigset_t); +void cfs_restore_sigs(sigset_t sigset); void cfs_clear_sigpending(void); /* @@ -71,7 +71,7 @@ /* returns a random 32-bit integer */ unsigned int cfs_rand(void); /* seed the generator */ -void cfs_srand(unsigned int, unsigned int); +void cfs_srand(unsigned int seed1, unsigned int seed2); void cfs_get_random_bytes(void *buf, int size); #include "libcfs_debug.h" diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h index 28937d5..2f56707 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h @@ -708,25 +708,26 @@ typedef int (*cfs_hash_for_each_cb_t)(struct cfs_hash *hs, void * cfs_hash_lookup(struct cfs_hash *hs, const void *key); void -cfs_hash_for_each(struct cfs_hash *hs, cfs_hash_for_each_cb_t, void *data); +cfs_hash_for_each(struct cfs_hash *hs, cfs_hash_for_each_cb_t cb, void *data); void -cfs_hash_for_each_safe(struct cfs_hash *hs, cfs_hash_for_each_cb_t, void *data); +cfs_hash_for_each_safe(struct cfs_hash *hs, cfs_hash_for_each_cb_t cb, + void *data); int -cfs_hash_for_each_nolock(struct cfs_hash *hs, cfs_hash_for_each_cb_t, +cfs_hash_for_each_nolock(struct cfs_hash *hs, cfs_hash_for_each_cb_t cb, void *data, int start); int -cfs_hash_for_each_empty(struct cfs_hash *hs, cfs_hash_for_each_cb_t, +cfs_hash_for_each_empty(struct cfs_hash *hs, cfs_hash_for_each_cb_t cb, void *data); void cfs_hash_for_each_key(struct cfs_hash *hs, const void *key, - cfs_hash_for_each_cb_t, void *data); + cfs_hash_for_each_cb_t cb, void *data); typedef int (*cfs_hash_cond_opt_cb_t)(void *obj, void *data); void -cfs_hash_cond_del(struct cfs_hash *hs, cfs_hash_cond_opt_cb_t, void *data); +cfs_hash_cond_del(struct cfs_hash *hs, cfs_hash_cond_opt_cb_t cb, void *data); void cfs_hash_hlist_for_each(struct cfs_hash *hs, unsigned int hindex, - cfs_hash_for_each_cb_t, void *data); + cfs_hash_for_each_cb_t cb, void *data); int cfs_hash_is_empty(struct cfs_hash *hs); __u64 cfs_hash_size_get(struct cfs_hash *hs); diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h index 523e8f4..93bff1b 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h @@ -75,7 +75,7 @@ #define KLASSERT(e) LASSERT(e) -void __noreturn lbug_with_loc(struct libcfs_debug_msg_data *); +void __noreturn lbug_with_loc(struct libcfs_debug_msg_data *msg); #define LBUG() \ do { \ @@ -170,7 +170,7 @@ #endif void libcfs_run_upcall(char **argv); -void libcfs_run_lbug_upcall(struct libcfs_debug_msg_data *); +void libcfs_run_lbug_upcall(struct libcfs_debug_msg_data *msg); void libcfs_debug_dumplog(void); int libcfs_debug_init(unsigned long bufsize); int libcfs_debug_cleanup(void); diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_workitem.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_workitem.h index a7e1340..2accd9a 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_workitem.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_workitem.h @@ -62,9 +62,9 @@ struct cfs_wi_sched; -void cfs_wi_sched_destroy(struct cfs_wi_sched *); +void cfs_wi_sched_destroy(struct cfs_wi_sched *sched); int cfs_wi_sched_create(char *name, struct cfs_cpt_table *cptab, int cpt, - int nthrs, struct cfs_wi_sched **); + int nthrs, struct cfs_wi_sched **sched_pp); struct cfs_workitem; -- 1.7.1 From jsimmons at infradead.org Fri Nov 18 16:48:42 2016 From: jsimmons at infradead.org (James Simmons) Date: Fri, 18 Nov 2016 11:48:42 -0500 Subject: [lustre-devel] [PATCH 08/10] staging: lustre: libcfs: remove NULL comparisons in headers In-Reply-To: <1479487724-20386-1-git-send-email-jsimmons@infradead.org> References: <1479487724-20386-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479487724-20386-9-git-send-email-jsimmons@infradead.org> Remove the NULL comparisions in the libcfs headers. Signed-off-by: James Simmons --- .../lustre/include/linux/libcfs/libcfs_hash.h | 2 +- .../lustre/include/linux/libcfs/libcfs_private.h | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h index 85661b3..d0ba3e0 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_hash.h @@ -818,7 +818,7 @@ static inline int __cfs_hash_theta(struct cfs_hash *hs) { unsigned int i, hash = 5381; - LASSERT(key != NULL); + LASSERT(key); for (i = 0; i < size; i++) hash = hash * 33 + ((char *)key)[i]; diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h index 93bff1b..fc180b8 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h @@ -96,7 +96,7 @@ #define LIBCFS_ALLOC_POST(ptr, size) \ do { \ - if (unlikely((ptr) == NULL)) { \ + if (!unlikely((ptr))) { \ CERROR("LNET: out of memory at %s:%d (tried to alloc '" \ #ptr "' = %d)\n", __FILE__, __LINE__, (int)(size)); \ } else { \ @@ -147,7 +147,7 @@ #define LIBCFS_FREE(ptr, size) \ do { \ - if (unlikely((ptr) == NULL)) { \ + if (!unlikely((ptr))) { \ CERROR("LIBCFS: free NULL '" #ptr "' (%d bytes) at " \ "%s:%d\n", (int)(size), __FILE__, __LINE__); \ break; \ -- 1.7.1 From jsimmons at infradead.org Fri Nov 18 16:49:52 2016 From: jsimmons at infradead.org (James Simmons) Date: Fri, 18 Nov 2016 11:49:52 -0500 Subject: [lustre-devel] [PATCH v4] staging: lustre: llog: fix wrong offset in llog_process_thread() Message-ID: <1479487792-20489-1-git-send-email-jsimmons@infradead.org> From: Mikhail Pershin - llh_cat_idx may become bigger than llog bitmap size in llog_cat_set_first_idx() function - it is wrong to use previous cur_offset as new buffer offset, new offset should be calculated from value returned by llog_next_block(). - optimize llog_skip_over() to find llog entry offset by index for llog with fixed-size records. Signed-off-by: Mikhail Pershin Signed-off-by: Bob Glossman Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6714 Reviewed-on: http://review.whamcloud.com/15316 Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6163 Reviewed-on: http://review.whamcloud.com/18819 Reviewed-by: John L. Hammond Reviewed-by: James Simmons Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- ChangeLog: v1) Initial patch with umoddi issue v2) Included fix from patch LU-6163 that fixed umoddi problem v3) Remove no longer needed last_offset variable v4) Remove extra debugging bit that got included by mistake drivers/staging/lustre/lustre/obdclass/llog.c | 82 +++++++++++++++++------- 2 files changed, 59 insertions(+), 25 deletions(-) diff --git a/drivers/staging/lustre/lustre/obdclass/llog.c b/drivers/staging/lustre/lustre/obdclass/llog.c index 3bc1789..ae63047 100644 --- a/drivers/staging/lustre/lustre/obdclass/llog.c +++ b/drivers/staging/lustre/lustre/obdclass/llog.c @@ -217,8 +217,7 @@ static int llog_process_thread(void *arg) struct llog_log_hdr *llh = loghandle->lgh_hdr; struct llog_process_cat_data *cd = lpi->lpi_catdata; char *buf; - __u64 cur_offset; - __u64 last_offset; + u64 cur_offset, tmp_offset; int chunk_size; int rc = 0, index = 1, last_index; int saved_index = 0; @@ -229,6 +228,8 @@ static int llog_process_thread(void *arg) cur_offset = llh->llh_hdr.lrh_len; chunk_size = llh->llh_hdr.lrh_len; + /* expect chunk_size to be power of two */ + LASSERT(is_power_of_2(chunk_size)); buf = libcfs_kvzalloc(chunk_size, GFP_NOFS); if (!buf) { @@ -245,38 +246,50 @@ static int llog_process_thread(void *arg) else last_index = LLOG_HDR_BITMAP_SIZE(llh) - 1; - /* Record is not in this buffer. */ - if (index > last_index) - goto out; - while (rc == 0) { + unsigned int buf_offset = 0; struct llog_rec_hdr *rec; + bool partial_chunk; + off_t chunk_offset; /* skip records not set in bitmap */ while (index <= last_index && !ext2_test_bit(index, LLOG_HDR_BITMAP(llh))) ++index; - LASSERT(index <= last_index + 1); - if (index == last_index + 1) + if (index > last_index) break; -repeat: + CDEBUG(D_OTHER, "index: %d last_index %d\n", index, last_index); - +repeat: /* get the buf with our target record; avoid old garbage */ memset(buf, 0, chunk_size); - last_offset = cur_offset; rc = llog_next_block(lpi->lpi_env, loghandle, &saved_index, index, &cur_offset, buf, chunk_size); if (rc) goto out; + /* + * NB: after llog_next_block() call the cur_offset is the + * offset of the next block after read one. + * The absolute offset of the current chunk is calculated + * from cur_offset value and stored in chunk_offset variable. + */ + tmp_offset = cur_offset; + if (do_div(tmp_offset, chunk_size)) { + partial_chunk = true; + chunk_offset = cur_offset & ~(chunk_size - 1); + } else { + partial_chunk = false; + chunk_offset = cur_offset - chunk_size; + } + /* NB: when rec->lrh_len is accessed it is already swabbed * since it is used at the "end" of the loop and the rec * swabbing is done at the beginning of the loop. */ - for (rec = (struct llog_rec_hdr *)buf; + for (rec = (struct llog_rec_hdr *)(buf + buf_offset); (char *)rec < buf + chunk_size; rec = llog_rec_hdr_next(rec)) { CDEBUG(D_OTHER, "processing rec 0x%p type %#x\n", @@ -288,13 +301,28 @@ static int llog_process_thread(void *arg) CDEBUG(D_OTHER, "after swabbing, type=%#x idx=%d\n", rec->lrh_type, rec->lrh_index); - if (rec->lrh_index == 0) { - /* probably another rec just got added? */ - rc = 0; - if (index <= loghandle->lgh_last_idx) - goto repeat; - goto out; /* no more records */ + /* + * for partial chunk the end of it is zeroed, check + * for index 0 to distinguish it. + */ + if (partial_chunk && !rec->lrh_index) { + /* concurrent llog_add() might add new records + * while llog_processing, check this is not + * the case and re-read the current chunk + * otherwise. + */ + if (index > loghandle->lgh_last_idx) { + rc = 0; + goto out; + } + CDEBUG(D_OTHER, "Re-read last llog buffer for new records, index %u, last %u\n", + index, loghandle->lgh_last_idx); + /* save offset inside buffer for the re-read */ + buf_offset = (char *)rec - (char *)buf; + cur_offset = chunk_offset; + goto repeat; } + if (!rec->lrh_len || rec->lrh_len > chunk_size) { CWARN("invalid length %d in llog record for index %d/%d\n", rec->lrh_len, @@ -309,6 +337,14 @@ static int llog_process_thread(void *arg) continue; } + if (rec->lrh_index != index) { + CERROR("%s: Invalid record: index %u but expected %u\n", + loghandle->lgh_ctxt->loc_obd->obd_name, + rec->lrh_index, index); + rc = -ERANGE; + goto out; + } + CDEBUG(D_OTHER, "lrh_index: %d lrh_len: %d (%d remains)\n", rec->lrh_index, rec->lrh_len, @@ -316,7 +352,7 @@ static int llog_process_thread(void *arg) loghandle->lgh_cur_idx = rec->lrh_index; loghandle->lgh_cur_offset = (char *)rec - (char *)buf + - last_offset; + chunk_offset; /* if set, process the callback on this record */ if (ext2_test_bit(index, LLOG_HDR_BITMAP(llh))) { @@ -325,16 +361,14 @@ static int llog_process_thread(void *arg) last_called_index = index; if (rc) goto out; - } else { - CDEBUG(D_OTHER, "Skipped index %d\n", index); } - /* next record, still in buffer? */ - ++index; - if (index > last_index) { + /* exit if the last index is reached */ + if (index >= last_index) { rc = 0; goto out; } + index++; } } -- 1.7.1 From jsimmons at infradead.org Fri Nov 18 16:54:03 2016 From: jsimmons at infradead.org (James Simmons) Date: Fri, 18 Nov 2016 16:54:03 +0000 (GMT) Subject: [lustre-devel] [PATCH 32/35] staging: lustre: mount: fix lmd_parse() to handle commas in expr_list In-Reply-To: <20161114151246.GA29168@kroah.com> References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> <1478799065-24841-33-git-send-email-jsimmons@infradead.org> <20161114151246.GA29168@kroah.com> Message-ID: > > --- a/drivers/staging/lustre/lustre/obdclass/obd_mount.c > > +++ b/drivers/staging/lustre/lustre/obdclass/obd_mount.c > > @@ -871,6 +871,87 @@ static int lmd_parse_mgs(struct lustre_mount_data *lmd, char **ptr) > > return 0; > > } > > > > +/** > > + * Find the first comma delimiter from the specified \a buf and make \a *endh > > + * point to the string starting with the comma. The commas in expression list > > + * [...] will be skipped. > > + * > > + * \param[in] buf a comma-separated string > > + * \param[in] endh a pointer to a pointer that will point to the string > > + * starting with the comma > > Please drop this mess of \param, it's not needed and is not kernel-doc > format. Is it just the [in] mess that needs to be removed or does the doc style need to migrate to another format. Looking online doesn't reveal much and' I saw something about @arg format as well. I see both in tree. What is the right one to use and do clear docs on this format exist somewhere. From gregkh at linuxfoundation.org Fri Nov 18 17:14:26 2016 From: gregkh at linuxfoundation.org (Greg Kroah-Hartman) Date: Fri, 18 Nov 2016 18:14:26 +0100 Subject: [lustre-devel] [PATCH 32/35] staging: lustre: mount: fix lmd_parse() to handle commas in expr_list In-Reply-To: References: <1478799065-24841-1-git-send-email-jsimmons@infradead.org> <1478799065-24841-33-git-send-email-jsimmons@infradead.org> <20161114151246.GA29168@kroah.com> Message-ID: <20161118171426.GA24226@kroah.com> On Fri, Nov 18, 2016 at 04:54:03PM +0000, James Simmons wrote: > > > > --- a/drivers/staging/lustre/lustre/obdclass/obd_mount.c > > > +++ b/drivers/staging/lustre/lustre/obdclass/obd_mount.c > > > @@ -871,6 +871,87 @@ static int lmd_parse_mgs(struct lustre_mount_data *lmd, char **ptr) > > > return 0; > > > } > > > > > > +/** > > > + * Find the first comma delimiter from the specified \a buf and make \a *endh > > > + * point to the string starting with the comma. The commas in expression list > > > + * [...] will be skipped. > > > + * > > > + * \param[in] buf a comma-separated string > > > + * \param[in] endh a pointer to a pointer that will point to the string > > > + * starting with the comma > > > > Please drop this mess of \param, it's not needed and is not kernel-doc > > format. > > Is it just the [in] mess that needs to be removed or does the doc style > need to migrate to another format. Looking online doesn't reveal much and' > I saw something about @arg format as well. I see both in tree. What is the > right one to use and do clear docs on this format exist somewhere. @arg is the correct one, see the file Documentation/kernel-documentation.rst in the section, "How to format kernel-doc comments" for how to do this properly. thanks, greg k-h From green at linuxhacker.ru Mon Nov 21 05:46:48 2016 From: green at linuxhacker.ru (Oleg Drokin) Date: Mon, 21 Nov 2016 00:46:48 -0500 Subject: [lustre-devel] [PATCH] staging/lustre: Use proper number of bytes in copy_from_user Message-ID: <1479707208-1241688-1-git-send-email-green@linuxhacker.ru> From: Jian Yu This patch removes the usage of MAX_STRING_SIZE from copy_from_user() and just copies enough bytes to cover count passed in. Signed-off-by: Jian Yu Reviewed-on: http://review.whamcloud.com/23462 Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-8774 Reviewed-by: John L. Hammond Signed-off-by: Oleg Drokin --- drivers/staging/lustre/lustre/obdclass/lprocfs_status.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c index 8a2f02f3..db49992 100644 --- a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c +++ b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c @@ -400,10 +400,17 @@ int lprocfs_wr_uint(struct file *file, const char __user *buffer, char dummy[MAX_STRING_SIZE + 1], *end; unsigned long tmp; - dummy[MAX_STRING_SIZE] = '\0'; - if (copy_from_user(dummy, buffer, MAX_STRING_SIZE)) + if (count >= sizeof(dummy)) + return -EINVAL; + + if (count == 0) + return 0; + + if (copy_from_user(dummy, buffer, count)) return -EFAULT; + dummy[count] = '\0'; + tmp = simple_strtoul(dummy, &end, 0); if (dummy == end) return -EINVAL; -- 2.7.4 From gregkh at linuxfoundation.org Mon Nov 21 10:14:40 2016 From: gregkh at linuxfoundation.org (Greg Kroah-Hartman) Date: Mon, 21 Nov 2016 11:14:40 +0100 Subject: [lustre-devel] [PATCH] staging/lustre: Use proper number of bytes in copy_from_user In-Reply-To: <1479707208-1241688-1-git-send-email-green@linuxhacker.ru> References: <1479707208-1241688-1-git-send-email-green@linuxhacker.ru> Message-ID: <20161121101440.GA9749@kroah.com> On Mon, Nov 21, 2016 at 12:46:48AM -0500, Oleg Drokin wrote: > From: Jian Yu > > This patch removes the usage of MAX_STRING_SIZE from > copy_from_user() and just copies enough bytes to cover > count passed in. > > Signed-off-by: Jian Yu > Reviewed-on: http://review.whamcloud.com/23462 > Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-8774 > Reviewed-by: John L. Hammond > Signed-off-by: Oleg Drokin > --- > drivers/staging/lustre/lustre/obdclass/lprocfs_status.c | 11 +++++++++-- > 1 file changed, 9 insertions(+), 2 deletions(-) > > diff --git a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c > index 8a2f02f3..db49992 100644 > --- a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c > +++ b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c > @@ -400,10 +400,17 @@ int lprocfs_wr_uint(struct file *file, const char __user *buffer, > char dummy[MAX_STRING_SIZE + 1], *end; > unsigned long tmp; > > - dummy[MAX_STRING_SIZE] = '\0'; > - if (copy_from_user(dummy, buffer, MAX_STRING_SIZE)) > + if (count >= sizeof(dummy)) > + return -EINVAL; > + > + if (count == 0) > + return 0; > + > + if (copy_from_user(dummy, buffer, count)) > return -EFAULT; > > + dummy[count] = '\0'; > + You do know about simple_read_from_buffer(), right? I suggest using those simple_* functions where ever you are touching user buffers, like this code. thanks, greg k-h From wszhang159 at gmail.com Wed Nov 23 08:35:40 2016 From: wszhang159 at gmail.com (W. Zhang) Date: Wed, 23 Nov 2016 16:35:40 +0800 Subject: [lustre-devel] Failed to build a Lustre Storage Client on OpenVZ Kernel ! Is this a bug? In-Reply-To: References: Message-ID: Dear James and All, Attached is the file 'config.log'! Thanks very much for previous attention! Best Wishes! 2016-11-07 10:03 GMT+08:00 James Simmons : > > > Hello, > > > > I'm trying to build a Lustre client on OpenVZ kernel (version: > 042stab120.5, based on RHEL 6.8 kernel > > 2.6.32-642.6.1.el6), and some bugs occured: > > > > (1) When build version v2_8_59_0 in git tag repo of Lustre, I > encountered the following errors: > > " > > LD [M] /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/fld/ > fld.o > > CC [M] /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit > e/lloop.o > > In file included from /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit > e/lloop.c:109: > > /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit > e/llite_internal.h:1469: error: conflicting types for > > 'iov_iter_iovec' > > include/linux/fs.h:765: note: previous definition of 'iov_iter_iovec' > was here > > /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit > e/llite_internal.h: In function 'iov_iter_iovec': > > /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit > e/llite_internal.h:1472: error: 'const struct > > iov_iter' has no member named 'iov' > > /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit > e/llite_internal.h:1473: error: 'const struct > > iov_iter' has no member named 'iov' > > cc1: warnings being treated as errors > > /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit > e/llite_internal.h:1473: error: type defaults to 'int' > > in declaration of '_min2' > > ... > > " > > > > (2) When build old version v2_8_50_0 in git tag repo of Lustre, the > following errors display: > > " > > CC [M] /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit > e/vvp_lock.o > > CC [M] /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit > e/vvp_io.o > > /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit > e/vvp_io.c: In function 'vvp_mmap_locks': > > /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit > e/vvp_io.c:423: error: 'struct iov_iter' has no member > > named 'iov' > > /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit > e/vvp_io.c: In function 'vvp_io_advance': > > /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit > e/vvp_io.c:514: error: 'struct iov_iter' has no member > > named 'iov' > > ... > > " > > > > Files with full error context are attached. > > > > So, How can I fix the problem? Is this a bug which need to be fixed? > > Can you post your config.log. > > > > Thanks! Any suggestion is appreciated! > > > > Best Wishes! > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wszhang159 at gmail.com Wed Nov 23 09:05:25 2016 From: wszhang159 at gmail.com (W. Zhang) Date: Wed, 23 Nov 2016 17:05:25 +0800 Subject: [lustre-devel] Failed to build a Lustre Storage Client on OpenVZ Kernel ! Is this a bug? In-Reply-To: References: Message-ID: Dear James and All, Zipped file "*config.log*" is attached, file in the former is too big (beyond the limit of 150 KB). Thanks very much and sorry for the redundancy! Best Wishes! 2016-11-23 16:46 GMT+08:00 W. Zhang : > Sorry to All! > > I prepared the file '*config.log*' but forgot to attach it in the former! > (T-T) > > Here it is! > > Best Wishes! > > > > > > 2016-11-23 16:35 GMT+08:00 W. Zhang : > >> Dear James and All, >> >> Attached is the file 'config.log'! >> >> Thanks very much for previous attention! >> >> Best Wishes! >> >> 2016-11-07 10:03 GMT+08:00 James Simmons : >> >>> >>> > Hello, >>> > >>> > I'm trying to build a Lustre client on OpenVZ kernel (version: >>> 042stab120.5, based on RHEL 6.8 kernel >>> > 2.6.32-642.6.1.el6), and some bugs occured: >>> > >>> > (1) When build version v2_8_59_0 in git tag repo of Lustre, I >>> encountered the following errors: >>> > " >>> > LD [M] /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/fld/ >>> fld.o >>> > CC [M] /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit >>> e/lloop.o >>> > In file included from /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit >>> e/lloop.c:109: >>> > /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit >>> e/llite_internal.h:1469: error: conflicting types for >>> > 'iov_iter_iovec' >>> > include/linux/fs.h:765: note: previous definition of 'iov_iter_iovec' >>> was here >>> > /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit >>> e/llite_internal.h: In function 'iov_iter_iovec': >>> > /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit >>> e/llite_internal.h:1472: error: 'const struct >>> > iov_iter' has no member named 'iov' >>> > /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit >>> e/llite_internal.h:1473: error: 'const struct >>> > iov_iter' has no member named 'iov' >>> > cc1: warnings being treated as errors >>> > /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit >>> e/llite_internal.h:1473: error: type defaults to 'int' >>> > in declaration of '_min2' >>> > ... >>> > " >>> > >>> > (2) When build old version v2_8_50_0 in git tag repo of Lustre, the >>> following errors display: >>> > " >>> > CC [M] /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit >>> e/vvp_lock.o >>> > CC [M] /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit >>> e/vvp_io.o >>> > /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit >>> e/vvp_io.c: In function 'vvp_mmap_locks': >>> > /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit >>> e/vvp_io.c:423: error: 'struct iov_iter' has no member >>> > named 'iov' >>> > /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit >>> e/vvp_io.c: In function 'vvp_io_advance': >>> > /opt/src/lustre_testvz/lustre_new/lustre-r161012/lustre/llit >>> e/vvp_io.c:514: error: 'struct iov_iter' has no member >>> > named 'iov' >>> > ... >>> > " >>> > >>> > Files with full error context are attached. >>> > >>> > So, How can I fix the problem? Is this a bug which need to be fixed? >>> >>> Can you post your config.log. >>> >>> >>> > Thanks! Any suggestion is appreciated! >>> > >>> > Best Wishes! >>> > >>> > >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: config.log.zip Type: application/zip Size: 33503 bytes Desc: not available URL: From dan.carpenter at oracle.com Wed Nov 23 12:29:59 2016 From: dan.carpenter at oracle.com (Dan Carpenter) Date: Wed, 23 Nov 2016 15:29:59 +0300 Subject: [lustre-devel] [bug report] staging: add Lustre file system client support Message-ID: <20161123122959.GA17323@mwanda> Hi Lustre Devs, The patch d7e09d0397e8: "staging: add Lustre file system client support" from May 2, 2013, leads to the following static checker warning: drivers/staging/lustre/lnet/selftest/console.c:1336 lstcon_test_add() error: 'paramlen' from user is not capped properly The story here, is that "paramlen" is capped but only if "param" is non-NULL. This causes a problem. drivers/staging/lustre/lnet/selftest/console.c 1311 1312 LIBCFS_ALLOC(test, offsetof(struct lstcon_test, tes_param[paramlen])); We don't know that paramlen is non-NULL here. Because of integer overflows we could end up allocating less than intended. 1313 if (!test) { 1314 CERROR("Can't allocate test descriptor\n"); 1315 rc = -ENOMEM; 1316 1317 goto out; 1318 } 1319 1320 test->tes_hdr.tsb_id = batch->bat_hdr.tsb_id; Which will lead to memory corruption when we use "test". 1321 test->tes_batch = batch; 1322 test->tes_type = type; 1323 test->tes_oneside = 0; /* TODO */ 1324 test->tes_loop = loop; 1325 test->tes_concur = concur; 1326 test->tes_stop_onerr = 1; /* TODO */ 1327 test->tes_span = span; 1328 test->tes_dist = dist; 1329 test->tes_cliidx = 0; /* just used for creating RPC */ 1330 test->tes_src_grp = src_grp; 1331 test->tes_dst_grp = dst_grp; 1332 INIT_LIST_HEAD(&test->tes_trans_list); 1333 1334 if (param) { Smatch is not smart enough to trace the implication that "'param' is non-NULL, means that 'paramlen' has been verified" across a function boundary. Storing that sort of information would really increase the hardware requirements for running Smatch so it's not something I have planned currently. 1335 test->tes_paramlen = paramlen; 1336 memcpy(&test->tes_param[0], param, paramlen); 1337 } 1338 regards, dan carpenter From jsimmons at infradead.org Wed Nov 23 22:59:48 2016 From: jsimmons at infradead.org (James Simmons) Date: Wed, 23 Nov 2016 17:59:48 -0500 Subject: [lustre-devel] [PATCH 2/5] staging: lustre: ldlm: remove ldlm_side_t typedef usage from code In-Reply-To: <1479941991-30059-1-git-send-email-jsimmons@infradead.org> References: <1479941991-30059-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479941991-30059-3-git-send-email-jsimmons@infradead.org> From: Andreas Dilger Replace usage of ldlm_side_t with named enums to conform to upstream coding style. Signed-off-by: Andreas Dilger Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6142 Reviewed-on: http://review.whamcloud.com/15300 Reviewed-on: http://review.whamcloud.com/15301 Reviewed-by: James Simmons Reviewed-by: Bob Glossman Reviewed-by: frank zago Reviewed-by: Dmitry Eremin Reviewed-by: John L. Hammond Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- drivers/staging/lustre/lustre/include/lustre_dlm.h | 12 ++++++------ drivers/staging/lustre/lustre/ldlm/ldlm_internal.h | 17 +++++++++-------- drivers/staging/lustre/lustre/ldlm/ldlm_pool.c | 9 +++++---- drivers/staging/lustre/lustre/ldlm/ldlm_resource.c | 14 +++++++------- 4 files changed, 27 insertions(+), 25 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre_dlm.h b/drivers/staging/lustre/lustre/include/lustre_dlm.h index 02921bf..02c90b0 100644 --- a/drivers/staging/lustre/lustre/include/lustre_dlm.h +++ b/drivers/staging/lustre/lustre/include/lustre_dlm.h @@ -86,10 +86,10 @@ enum ldlm_error { * decisions about lack of conflicts or do any autonomous lock granting without * first speaking to a server. */ -typedef enum { +enum ldlm_side { LDLM_NAMESPACE_SERVER = 1 << 0, LDLM_NAMESPACE_CLIENT = 1 << 1 -} ldlm_side_t; +}; /** * The blocking callback is overloaded to perform two functions. These flags @@ -359,7 +359,7 @@ struct ldlm_namespace { struct obd_device *ns_obd; /** Flag indicating if namespace is on client instead of server */ - ldlm_side_t ns_client; + enum ldlm_side ns_client; /** Resource hash table for namespace. */ struct cfs_hash *ns_rs_hash; @@ -1187,7 +1187,7 @@ enum ldlm_mode ldlm_revalidate_lock_handle(const struct lustre_handle *lockh, /* resource.c */ struct ldlm_namespace * ldlm_namespace_new(struct obd_device *obd, char *name, - ldlm_side_t client, enum ldlm_appetite apt, + enum ldlm_side client, enum ldlm_appetite apt, enum ldlm_ns_type ns_type); int ldlm_namespace_cleanup(struct ldlm_namespace *ns, __u64 flags); void ldlm_namespace_get(struct ldlm_namespace *ns); @@ -1206,7 +1206,7 @@ void ldlm_resource_add_lock(struct ldlm_resource *res, struct ldlm_lock *lock); void ldlm_resource_unlink_lock(struct ldlm_lock *lock); void ldlm_res2desc(struct ldlm_resource *res, struct ldlm_resource_desc *desc); -void ldlm_dump_all_namespaces(ldlm_side_t client, int level); +void ldlm_dump_all_namespaces(enum ldlm_side client, int level); void ldlm_namespace_dump(int level, struct ldlm_namespace *); void ldlm_resource_dump(int level, struct ldlm_resource *); int ldlm_lock_change_resource(struct ldlm_namespace *, struct ldlm_lock *, @@ -1331,7 +1331,7 @@ static inline void check_res_locked(struct ldlm_resource *res) void ldlm_pools_fini(void); int ldlm_pool_init(struct ldlm_pool *pl, struct ldlm_namespace *ns, - int idx, ldlm_side_t client); + int idx, enum ldlm_side client); void ldlm_pool_fini(struct ldlm_pool *pl); void ldlm_pool_add(struct ldlm_pool *pl, struct ldlm_lock *lock); void ldlm_pool_del(struct ldlm_pool *pl, struct ldlm_lock *lock); diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h b/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h index 1a98a31..e2c3c6e 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h @@ -39,13 +39,13 @@ extern struct mutex ldlm_cli_namespace_lock; extern struct list_head ldlm_cli_active_namespace_list; -static inline int ldlm_namespace_nr_read(ldlm_side_t client) +static inline int ldlm_namespace_nr_read(enum ldlm_side client) { return client == LDLM_NAMESPACE_SERVER ? ldlm_srv_namespace_nr : ldlm_cli_namespace_nr; } -static inline void ldlm_namespace_nr_inc(ldlm_side_t client) +static inline void ldlm_namespace_nr_inc(enum ldlm_side client) { if (client == LDLM_NAMESPACE_SERVER) ldlm_srv_namespace_nr++; @@ -53,7 +53,7 @@ static inline void ldlm_namespace_nr_inc(ldlm_side_t client) ldlm_cli_namespace_nr++; } -static inline void ldlm_namespace_nr_dec(ldlm_side_t client) +static inline void ldlm_namespace_nr_dec(enum ldlm_side client) { if (client == LDLM_NAMESPACE_SERVER) ldlm_srv_namespace_nr--; @@ -61,13 +61,13 @@ static inline void ldlm_namespace_nr_dec(ldlm_side_t client) ldlm_cli_namespace_nr--; } -static inline struct list_head *ldlm_namespace_list(ldlm_side_t client) +static inline struct list_head *ldlm_namespace_list(enum ldlm_side client) { return client == LDLM_NAMESPACE_SERVER ? &ldlm_srv_namespace_list : &ldlm_cli_active_namespace_list; } -static inline struct mutex *ldlm_namespace_lock(ldlm_side_t client) +static inline struct mutex *ldlm_namespace_lock(enum ldlm_side client) { return client == LDLM_NAMESPACE_SERVER ? &ldlm_srv_namespace_lock : &ldlm_cli_namespace_lock; @@ -79,10 +79,11 @@ static inline int ldlm_ns_empty(struct ldlm_namespace *ns) return atomic_read(&ns->ns_bref) == 0; } -void ldlm_namespace_move_to_active_locked(struct ldlm_namespace *, ldlm_side_t); +void ldlm_namespace_move_to_active_locked(struct ldlm_namespace *, + enum ldlm_side); void ldlm_namespace_move_to_inactive_locked(struct ldlm_namespace *, - ldlm_side_t); -struct ldlm_namespace *ldlm_namespace_first_locked(ldlm_side_t); + enum ldlm_side); +struct ldlm_namespace *ldlm_namespace_first_locked(enum ldlm_side); /* ldlm_request.c */ /* Cancel lru flag, it indicates we cancel aged locks. */ diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c index b820309..252bddb 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c @@ -651,7 +651,7 @@ static void ldlm_pool_debugfs_fini(struct ldlm_pool *pl) } int ldlm_pool_init(struct ldlm_pool *pl, struct ldlm_namespace *ns, - int idx, ldlm_side_t client) + int idx, enum ldlm_side client) { int rc; @@ -788,7 +788,7 @@ static int ldlm_pool_granted(struct ldlm_pool *pl) * count locks from all namespaces (if possible). Returns number of * cached locks. */ -static unsigned long ldlm_pools_count(ldlm_side_t client, gfp_t gfp_mask) +static unsigned long ldlm_pools_count(enum ldlm_side client, gfp_t gfp_mask) { unsigned long total = 0; int nr_ns; @@ -837,7 +837,8 @@ static unsigned long ldlm_pools_count(ldlm_side_t client, gfp_t gfp_mask) return total; } -static unsigned long ldlm_pools_scan(ldlm_side_t client, int nr, gfp_t gfp_mask) +static unsigned long ldlm_pools_scan(enum ldlm_side client, int nr, + gfp_t gfp_mask) { unsigned long freed = 0; int tmp, nr_ns; @@ -895,7 +896,7 @@ static unsigned long ldlm_pools_cli_scan(struct shrinker *s, sc->gfp_mask); } -static int ldlm_pools_recalc(ldlm_side_t client) +static int ldlm_pools_recalc(enum ldlm_side client) { struct ldlm_namespace *ns; struct ldlm_namespace *ns_old = NULL; diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c b/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c index c452400..ca2cd4a 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c @@ -612,7 +612,7 @@ struct ldlm_ns_hash_def { /** Register \a ns in the list of namespaces */ static void ldlm_namespace_register(struct ldlm_namespace *ns, - ldlm_side_t client) + enum ldlm_side client) { mutex_lock(ldlm_namespace_lock(client)); LASSERT(list_empty(&ns->ns_list_chain)); @@ -625,7 +625,7 @@ static void ldlm_namespace_register(struct ldlm_namespace *ns, * Create and initialize new empty namespace. */ struct ldlm_namespace *ldlm_namespace_new(struct obd_device *obd, char *name, - ldlm_side_t client, + enum ldlm_side client, enum ldlm_appetite apt, enum ldlm_ns_type ns_type) { @@ -954,7 +954,7 @@ void ldlm_namespace_free_prior(struct ldlm_namespace *ns, /** Unregister \a ns from the list of namespaces. */ static void ldlm_namespace_unregister(struct ldlm_namespace *ns, - ldlm_side_t client) + enum ldlm_side client) { mutex_lock(ldlm_namespace_lock(client)); LASSERT(!list_empty(&ns->ns_list_chain)); @@ -1018,7 +1018,7 @@ void ldlm_namespace_put(struct ldlm_namespace *ns) /** Should be called with ldlm_namespace_lock(client) taken. */ void ldlm_namespace_move_to_active_locked(struct ldlm_namespace *ns, - ldlm_side_t client) + enum ldlm_side client) { LASSERT(!list_empty(&ns->ns_list_chain)); LASSERT(mutex_is_locked(ldlm_namespace_lock(client))); @@ -1027,7 +1027,7 @@ void ldlm_namespace_move_to_active_locked(struct ldlm_namespace *ns, /** Should be called with ldlm_namespace_lock(client) taken. */ void ldlm_namespace_move_to_inactive_locked(struct ldlm_namespace *ns, - ldlm_side_t client) + enum ldlm_side client) { LASSERT(!list_empty(&ns->ns_list_chain)); LASSERT(mutex_is_locked(ldlm_namespace_lock(client))); @@ -1035,7 +1035,7 @@ void ldlm_namespace_move_to_inactive_locked(struct ldlm_namespace *ns, } /** Should be called with ldlm_namespace_lock(client) taken. */ -struct ldlm_namespace *ldlm_namespace_first_locked(ldlm_side_t client) +struct ldlm_namespace *ldlm_namespace_first_locked(enum ldlm_side client) { LASSERT(mutex_is_locked(ldlm_namespace_lock(client))); LASSERT(!list_empty(ldlm_namespace_list(client))); @@ -1305,7 +1305,7 @@ void ldlm_res2desc(struct ldlm_resource *res, struct ldlm_resource_desc *desc) * Print information about all locks in all namespaces on this node to debug * log. */ -void ldlm_dump_all_namespaces(ldlm_side_t client, int level) +void ldlm_dump_all_namespaces(enum ldlm_side client, int level) { struct list_head *tmp; -- 1.7.1 From jsimmons at infradead.org Wed Nov 23 22:59:51 2016 From: jsimmons at infradead.org (James Simmons) Date: Wed, 23 Nov 2016 17:59:51 -0500 Subject: [lustre-devel] [PATCH 5/5] staging: lustre: ldlm: change "int" to proper enum type In-Reply-To: <1479941991-30059-1-git-send-email-jsimmons@infradead.org> References: <1479941991-30059-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479941991-30059-6-git-send-email-jsimmons@infradead.org> From: Andreas Dilger Fix users of flags that were using "int" instead of named enum. Rename some "flags" variables to distinguish between different flags. Signed-off-by: Andreas Dilger Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6142 Reviewed-on: http://review.whamcloud.com/15300 Reviewed-on: http://review.whamcloud.com/15301 Reviewed-by: James Simmons Reviewed-by: Bob Glossman Reviewed-by: frank zago Reviewed-by: Dmitry Eremin Reviewed-by: John L. Hammond Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- drivers/staging/lustre/lustre/include/lustre_dlm.h | 14 ++++++++------ drivers/staging/lustre/lustre/ldlm/ldlm_internal.h | 8 ++++---- drivers/staging/lustre/lustre/ldlm/ldlm_lock.c | 19 +++++++++++-------- 3 files changed, 23 insertions(+), 18 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre_dlm.h b/drivers/staging/lustre/lustre/include/lustre_dlm.h index 62304ba..dfef4fa 100644 --- a/drivers/staging/lustre/lustre/include/lustre_dlm.h +++ b/drivers/staging/lustre/lustre/include/lustre_dlm.h @@ -965,8 +965,8 @@ struct ldlm_ast_work { * Common ldlm_enqueue parameters */ struct ldlm_enqueue_info { - __u32 ei_type; /** Type of the lock being enqueued. */ - __u32 ei_mode; /** Mode of the lock being enqueued. */ + enum ldlm_type ei_type; /** Type of the lock being enqueued. */ + enum ldlm_mode ei_mode; /** Mode of the lock being enqueued. */ void *ei_cb_bl; /** blocking lock callback */ void *ei_cb_cp; /** lock completion callback */ void *ei_cb_gl; /** lock glimpse callback */ @@ -1166,10 +1166,12 @@ static inline int ldlm_res_lvbo_update(struct ldlm_resource *res, struct ldlm_lock *ldlm_lock_get(struct ldlm_lock *lock); void ldlm_lock_put(struct ldlm_lock *lock); void ldlm_lock2desc(struct ldlm_lock *lock, struct ldlm_lock_desc *desc); -void ldlm_lock_addref(const struct lustre_handle *lockh, __u32 mode); -int ldlm_lock_addref_try(const struct lustre_handle *lockh, __u32 mode); -void ldlm_lock_decref(const struct lustre_handle *lockh, __u32 mode); -void ldlm_lock_decref_and_cancel(const struct lustre_handle *lockh, __u32 mode); +void ldlm_lock_addref(const struct lustre_handle *lockh, enum ldlm_mode mode); +int ldlm_lock_addref_try(const struct lustre_handle *lockh, + enum ldlm_mode mode); +void ldlm_lock_decref(const struct lustre_handle *lockh, enum ldlm_mode mode); +void ldlm_lock_decref_and_cancel(const struct lustre_handle *lockh, + enum ldlm_mode mode); void ldlm_lock_fail_match_locked(struct ldlm_lock *lock); void ldlm_lock_allow_match(struct ldlm_lock *lock); void ldlm_lock_allow_match_locked(struct ldlm_lock *lock); diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h b/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h index 49f74db..5c02501 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h @@ -138,10 +138,10 @@ struct ldlm_lock * void *data, __u32 lvb_len, enum lvb_type lvb_type); enum ldlm_error ldlm_lock_enqueue(struct ldlm_namespace *, struct ldlm_lock **, void *cookie, __u64 *flags); -void ldlm_lock_addref_internal(struct ldlm_lock *, __u32 mode); -void ldlm_lock_addref_internal_nolock(struct ldlm_lock *, __u32 mode); -void ldlm_lock_decref_internal(struct ldlm_lock *, __u32 mode); -void ldlm_lock_decref_internal_nolock(struct ldlm_lock *, __u32 mode); +void ldlm_lock_addref_internal(struct ldlm_lock *, enum ldlm_mode mode); +void ldlm_lock_addref_internal_nolock(struct ldlm_lock *, enum ldlm_mode mode); +void ldlm_lock_decref_internal(struct ldlm_lock *, enum ldlm_mode mode); +void ldlm_lock_decref_internal_nolock(struct ldlm_lock *, enum ldlm_mode mode); int ldlm_run_ast_work(struct ldlm_namespace *ns, struct list_head *rpc_list, enum ldlm_desc_ast_t ast_type); int ldlm_lock_remove_from_lru_check(struct ldlm_lock *lock, time_t last_use); diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c index 15eeb2b..d03e6d4 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c @@ -643,7 +643,7 @@ static void ldlm_add_ast_work_item(struct ldlm_lock *lock, * r/w reference type is determined by \a mode * Calls ldlm_lock_addref_internal. */ -void ldlm_lock_addref(const struct lustre_handle *lockh, __u32 mode) +void ldlm_lock_addref(const struct lustre_handle *lockh, enum ldlm_mode mode) { struct ldlm_lock *lock; @@ -661,7 +661,8 @@ void ldlm_lock_addref(const struct lustre_handle *lockh, __u32 mode) * Removes lock from LRU if it is there. * Assumes the LDLM lock is already locked. */ -void ldlm_lock_addref_internal_nolock(struct ldlm_lock *lock, __u32 mode) +void ldlm_lock_addref_internal_nolock(struct ldlm_lock *lock, + enum ldlm_mode mode) { ldlm_lock_remove_from_lru(lock); if (mode & (LCK_NL | LCK_CR | LCK_PR)) { @@ -685,7 +686,7 @@ void ldlm_lock_addref_internal_nolock(struct ldlm_lock *lock, __u32 mode) * * \retval -EAGAIN lock is being canceled. */ -int ldlm_lock_addref_try(const struct lustre_handle *lockh, __u32 mode) +int ldlm_lock_addref_try(const struct lustre_handle *lockh, enum ldlm_mode mode) { struct ldlm_lock *lock; int result; @@ -711,7 +712,7 @@ int ldlm_lock_addref_try(const struct lustre_handle *lockh, __u32 mode) * Locks LDLM lock and calls ldlm_lock_addref_internal_nolock to do the work. * Only called for local locks. */ -void ldlm_lock_addref_internal(struct ldlm_lock *lock, __u32 mode) +void ldlm_lock_addref_internal(struct ldlm_lock *lock, enum ldlm_mode mode) { lock_res_and_lock(lock); ldlm_lock_addref_internal_nolock(lock, mode); @@ -725,7 +726,8 @@ void ldlm_lock_addref_internal(struct ldlm_lock *lock, __u32 mode) * Does NOT add lock to LRU if no r/w references left to accommodate flock locks * that cannot be placed in LRU. */ -void ldlm_lock_decref_internal_nolock(struct ldlm_lock *lock, __u32 mode) +void ldlm_lock_decref_internal_nolock(struct ldlm_lock *lock, + enum ldlm_mode mode) { LDLM_DEBUG(lock, "ldlm_lock_decref(%s)", ldlm_lockname[mode]); if (mode & (LCK_NL | LCK_CR | LCK_PR)) { @@ -751,7 +753,7 @@ void ldlm_lock_decref_internal_nolock(struct ldlm_lock *lock, __u32 mode) * on the namespace. * For blocked LDLM locks if r/w count drops to zero, blocking_ast is called. */ -void ldlm_lock_decref_internal(struct ldlm_lock *lock, __u32 mode) +void ldlm_lock_decref_internal(struct ldlm_lock *lock, enum ldlm_mode mode) { struct ldlm_namespace *ns; @@ -822,7 +824,7 @@ void ldlm_lock_decref_internal(struct ldlm_lock *lock, __u32 mode) /** * Decrease reader/writer refcount for LDLM lock with handle \a lockh */ -void ldlm_lock_decref(const struct lustre_handle *lockh, __u32 mode) +void ldlm_lock_decref(const struct lustre_handle *lockh, enum ldlm_mode mode) { struct ldlm_lock *lock = __ldlm_handle2lock(lockh, 0); @@ -837,7 +839,8 @@ void ldlm_lock_decref(const struct lustre_handle *lockh, __u32 mode) * \a lockh and mark it for subsequent cancellation once r/w refcount * drops to zero instead of putting into LRU. */ -void ldlm_lock_decref_and_cancel(const struct lustre_handle *lockh, __u32 mode) +void ldlm_lock_decref_and_cancel(const struct lustre_handle *lockh, + enum ldlm_mode mode) { struct ldlm_lock *lock = __ldlm_handle2lock(lockh, 0); -- 1.7.1 From jsimmons at infradead.org Wed Nov 23 22:59:46 2016 From: jsimmons at infradead.org (James Simmons) Date: Wed, 23 Nov 2016 17:59:46 -0500 Subject: [lustre-devel] [PATCH 0/5] staging: lustre: ldlm: remove ldlm typedef usage from code Message-ID: <1479941991-30059-1-git-send-email-jsimmons@infradead.org> Replace usage of ldlm_policy_data_t, ldlm_wire_policy_t, ldlm_side_t with named enums to conform to upstream coding style. Fix users of flags that were using "int" instead of named enum. Rename some "flags" variables to distinguish between different flags. Rename LDLM_CANCEL_* flags (used with enum ldlm_lru_flags) to LDLM_LRU_FLAGS_* to avoid confusion with enum ldlm_cancel_flags. Signed-off-by: Andreas Dilger Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6142 Reviewed-on: http://review.whamcloud.com/15300 Reviewed-on: http://review.whamcloud.com/15301 Reviewed-by: James Simmons Reviewed-by: Bob Glossman Reviewed-by: frank zago Reviewed-by: Dmitry Eremin Reviewed-by: John L. Hammond Reviewed-by: Oleg Drokin Signed-off-by: James Simmons Andreas Dilger (5): staging: lustre: ldlm: remove ldlm_policy_data_t typedef usage from code staging: lustre: ldlm: remove ldlm_side_t typedef usage from code staging: lustre: ldlm: remove ldlm_wire_policy_data_t typedef usage from code staging: lustre: ldlm: rename LDLM_CANCEL_* flags staging: lustre: ldlm: change "int" to proper enum type .../lustre/lustre/include/lustre/lustre_idl.h | 6 +- drivers/staging/lustre/lustre/include/lustre_dlm.h | 44 +++++----- drivers/staging/lustre/lustre/include/obd.h | 6 +- drivers/staging/lustre/lustre/include/obd_class.h | 6 +- drivers/staging/lustre/lustre/ldlm/ldlm_extent.c | 8 +- drivers/staging/lustre/lustre/ldlm/ldlm_flock.c | 8 +- .../staging/lustre/lustre/ldlm/ldlm_inodebits.c | 8 +- drivers/staging/lustre/lustre/ldlm/ldlm_internal.h | 85 ++++++++++---------- drivers/staging/lustre/lustre/ldlm/ldlm_lock.c | 33 ++++---- drivers/staging/lustre/lustre/ldlm/ldlm_plain.c | 8 +- drivers/staging/lustre/lustre/ldlm/ldlm_pool.c | 13 ++-- drivers/staging/lustre/lustre/ldlm/ldlm_request.c | 50 ++++++------ drivers/staging/lustre/lustre/ldlm/ldlm_resource.c | 22 +++--- drivers/staging/lustre/lustre/llite/file.c | 10 ++- .../staging/lustre/lustre/llite/llite_internal.h | 2 +- drivers/staging/lustre/lustre/llite/llite_mmap.c | 2 +- drivers/staging/lustre/lustre/llite/vvp_io.c | 2 +- drivers/staging/lustre/lustre/lmv/lmv_obd.c | 11 ++- drivers/staging/lustre/lustre/mdc/mdc_internal.h | 7 +- drivers/staging/lustre/lustre/mdc/mdc_locks.c | 26 +++--- drivers/staging/lustre/lustre/mdc/mdc_reint.c | 2 +- drivers/staging/lustre/lustre/mgc/mgc_request.c | 2 +- .../staging/lustre/lustre/osc/osc_cl_internal.h | 5 +- drivers/staging/lustre/lustre/osc/osc_internal.h | 4 +- drivers/staging/lustre/lustre/osc/osc_lock.c | 6 +- drivers/staging/lustre/lustre/osc/osc_object.c | 2 +- drivers/staging/lustre/lustre/osc/osc_page.c | 3 +- drivers/staging/lustre/lustre/osc/osc_request.c | 4 +- .../staging/lustre/lustre/ptlrpc/pack_generic.c | 2 +- 29 files changed, 200 insertions(+), 187 deletions(-) From jsimmons at infradead.org Wed Nov 23 22:59:47 2016 From: jsimmons at infradead.org (James Simmons) Date: Wed, 23 Nov 2016 17:59:47 -0500 Subject: [lustre-devel] [PATCH 1/5] staging: lustre: ldlm: remove ldlm_policy_data_t typedef usage from code In-Reply-To: <1479941991-30059-1-git-send-email-jsimmons@infradead.org> References: <1479941991-30059-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479941991-30059-2-git-send-email-jsimmons@infradead.org> From: Andreas Dilger Replace usage of ldlm_policy_data_t with named enums to conform to upstream coding style. Signed-off-by: Andreas Dilger Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6142 Reviewed-on: http://review.whamcloud.com/15300 Reviewed-on: http://review.whamcloud.com/15301 Reviewed-by: James Simmons Reviewed-by: Bob Glossman Reviewed-by: frank zago Reviewed-by: Dmitry Eremin Reviewed-by: John L. Hammond Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- drivers/staging/lustre/lustre/include/lustre_dlm.h | 16 ++++++------ drivers/staging/lustre/lustre/include/obd.h | 6 ++-- drivers/staging/lustre/lustre/include/obd_class.h | 6 ++-- drivers/staging/lustre/lustre/ldlm/ldlm_extent.c | 4 +- drivers/staging/lustre/lustre/ldlm/ldlm_flock.c | 4 +- .../staging/lustre/lustre/ldlm/ldlm_inodebits.c | 4 +- drivers/staging/lustre/lustre/ldlm/ldlm_internal.h | 20 +++++++------- drivers/staging/lustre/lustre/ldlm/ldlm_lock.c | 10 ++++---- drivers/staging/lustre/lustre/ldlm/ldlm_plain.c | 4 +- drivers/staging/lustre/lustre/ldlm/ldlm_request.c | 6 ++-- drivers/staging/lustre/lustre/llite/file.c | 10 ++++--- .../staging/lustre/lustre/llite/llite_internal.h | 2 +- drivers/staging/lustre/lustre/llite/llite_mmap.c | 2 +- drivers/staging/lustre/lustre/llite/vvp_io.c | 2 +- drivers/staging/lustre/lustre/lmv/lmv_obd.c | 11 ++++---- drivers/staging/lustre/lustre/mdc/mdc_internal.h | 7 +++-- drivers/staging/lustre/lustre/mdc/mdc_locks.c | 26 ++++++++++---------- drivers/staging/lustre/lustre/mdc/mdc_reint.c | 2 +- drivers/staging/lustre/lustre/mgc/mgc_request.c | 2 +- .../staging/lustre/lustre/osc/osc_cl_internal.h | 5 ++- drivers/staging/lustre/lustre/osc/osc_internal.h | 4 +- drivers/staging/lustre/lustre/osc/osc_lock.c | 6 ++-- drivers/staging/lustre/lustre/osc/osc_object.c | 2 +- drivers/staging/lustre/lustre/osc/osc_page.c | 3 +- drivers/staging/lustre/lustre/osc/osc_request.c | 4 +- 25 files changed, 87 insertions(+), 81 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre_dlm.h b/drivers/staging/lustre/lustre/include/lustre_dlm.h index d94237d..02921bf 100644 --- a/drivers/staging/lustre/lustre/include/lustre_dlm.h +++ b/drivers/staging/lustre/lustre/include/lustre_dlm.h @@ -553,15 +553,15 @@ struct ldlm_flock { __u32 pid; }; -typedef union { +union ldlm_policy_data { struct ldlm_extent l_extent; struct ldlm_flock l_flock; struct ldlm_inodebits l_inodebits; -} ldlm_policy_data_t; +}; void ldlm_convert_policy_to_local(struct obd_export *exp, enum ldlm_type type, const ldlm_wire_policy_data_t *wpolicy, - ldlm_policy_data_t *lpolicy); + union ldlm_policy_data *lpolicy); enum lvb_type { LVB_T_NONE = 0, @@ -690,7 +690,7 @@ struct ldlm_lock { * Representation of private data specific for a lock type. * Examples are: extent range for extent lock or bitmask for ibits locks */ - ldlm_policy_data_t l_policy_data; + union ldlm_policy_data l_policy_data; /** * Lock state flags. Protected by lr_lock. @@ -1175,7 +1175,7 @@ static inline int ldlm_res_lvbo_update(struct ldlm_resource *res, void ldlm_lock_allow_match_locked(struct ldlm_lock *lock); enum ldlm_mode ldlm_lock_match(struct ldlm_namespace *ns, __u64 flags, const struct ldlm_res_id *, - enum ldlm_type type, ldlm_policy_data_t *, + enum ldlm_type type, union ldlm_policy_data *, enum ldlm_mode mode, struct lustre_handle *, int unref); enum ldlm_mode ldlm_revalidate_lock_handle(const struct lustre_handle *lockh, @@ -1239,7 +1239,7 @@ int ldlm_lock_change_resource(struct ldlm_namespace *, struct ldlm_lock *, int ldlm_cli_enqueue(struct obd_export *exp, struct ptlrpc_request **reqp, struct ldlm_enqueue_info *einfo, const struct ldlm_res_id *res_id, - ldlm_policy_data_t const *policy, __u64 *flags, + union ldlm_policy_data const *policy, __u64 *flags, void *lvb, __u32 lvb_len, enum lvb_type lvb_type, struct lustre_handle *lockh, int async); int ldlm_prep_enqueue_req(struct obd_export *exp, @@ -1263,13 +1263,13 @@ int ldlm_cli_cancel_unused(struct ldlm_namespace *, const struct ldlm_res_id *, enum ldlm_cancel_flags flags, void *opaque); int ldlm_cli_cancel_unused_resource(struct ldlm_namespace *ns, const struct ldlm_res_id *res_id, - ldlm_policy_data_t *policy, + union ldlm_policy_data *policy, enum ldlm_mode mode, enum ldlm_cancel_flags flags, void *opaque); int ldlm_cancel_resource_local(struct ldlm_resource *res, struct list_head *cancels, - ldlm_policy_data_t *policy, + union ldlm_policy_data *policy, enum ldlm_mode mode, __u64 lock_flags, enum ldlm_cancel_flags cancel_flags, void *opaque); diff --git a/drivers/staging/lustre/lustre/include/obd.h b/drivers/staging/lustre/lustre/include/obd.h index 60efaaa..d4d2ffa 100644 --- a/drivers/staging/lustre/lustre/include/obd.h +++ b/drivers/staging/lustre/lustre/include/obd.h @@ -879,7 +879,7 @@ struct md_ops { const void *, size_t, umode_t, uid_t, gid_t, cfs_cap_t, __u64, struct ptlrpc_request **); int (*enqueue)(struct obd_export *, struct ldlm_enqueue_info *, - const ldlm_policy_data_t *, + const union ldlm_policy_data *, struct lookup_intent *, struct md_op_data *, struct lustre_handle *, __u64); int (*getattr)(struct obd_export *, struct md_op_data *, @@ -935,11 +935,11 @@ struct md_ops { enum ldlm_mode (*lock_match)(struct obd_export *, __u64, const struct lu_fid *, enum ldlm_type, - ldlm_policy_data_t *, enum ldlm_mode, + union ldlm_policy_data *, enum ldlm_mode, struct lustre_handle *); int (*cancel_unused)(struct obd_export *, const struct lu_fid *, - ldlm_policy_data_t *, enum ldlm_mode, + union ldlm_policy_data *, enum ldlm_mode, enum ldlm_cancel_flags flags, void *opaque); int (*get_fid_from_lsm)(struct obd_export *, diff --git a/drivers/staging/lustre/lustre/include/obd_class.h b/drivers/staging/lustre/lustre/include/obd_class.h index 9099b51..7ec2520 100644 --- a/drivers/staging/lustre/lustre/include/obd_class.h +++ b/drivers/staging/lustre/lustre/include/obd_class.h @@ -1208,7 +1208,7 @@ static inline int md_create(struct obd_export *exp, struct md_op_data *op_data, static inline int md_enqueue(struct obd_export *exp, struct ldlm_enqueue_info *einfo, - const ldlm_policy_data_t *policy, + const union ldlm_policy_data *policy, struct lookup_intent *it, struct md_op_data *op_data, struct lustre_handle *lockh, @@ -1407,7 +1407,7 @@ static inline int md_set_lock_data(struct obd_export *exp, static inline int md_cancel_unused(struct obd_export *exp, const struct lu_fid *fid, - ldlm_policy_data_t *policy, + union ldlm_policy_data *policy, enum ldlm_mode mode, enum ldlm_cancel_flags flags, void *opaque) @@ -1425,7 +1425,7 @@ static inline int md_cancel_unused(struct obd_export *exp, static inline enum ldlm_mode md_lock_match(struct obd_export *exp, __u64 flags, const struct lu_fid *fid, enum ldlm_type type, - ldlm_policy_data_t *policy, + union ldlm_policy_data *policy, enum ldlm_mode mode, struct lustre_handle *lockh) { diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c b/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c index a7b34e4..78cd70d 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c @@ -241,7 +241,7 @@ void ldlm_extent_unlink_lock(struct ldlm_lock *lock) } void ldlm_extent_policy_wire_to_local(const ldlm_wire_policy_data_t *wpolicy, - ldlm_policy_data_t *lpolicy) + union ldlm_policy_data *lpolicy) { memset(lpolicy, 0, sizeof(*lpolicy)); lpolicy->l_extent.start = wpolicy->l_extent.start; @@ -249,7 +249,7 @@ void ldlm_extent_policy_wire_to_local(const ldlm_wire_policy_data_t *wpolicy, lpolicy->l_extent.gid = wpolicy->l_extent.gid; } -void ldlm_extent_policy_local_to_wire(const ldlm_policy_data_t *lpolicy, +void ldlm_extent_policy_local_to_wire(const union ldlm_policy_data *lpolicy, ldlm_wire_policy_data_t *wpolicy) { memset(wpolicy, 0, sizeof(*wpolicy)); diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c b/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c index 98838e7..2683a7b 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c @@ -613,7 +613,7 @@ struct ldlm_flock_wait_data { EXPORT_SYMBOL(ldlm_flock_completion_ast); void ldlm_flock_policy_wire_to_local(const ldlm_wire_policy_data_t *wpolicy, - ldlm_policy_data_t *lpolicy) + union ldlm_policy_data *lpolicy) { memset(lpolicy, 0, sizeof(*lpolicy)); lpolicy->l_flock.start = wpolicy->l_flock.lfw_start; @@ -622,7 +622,7 @@ void ldlm_flock_policy_wire_to_local(const ldlm_wire_policy_data_t *wpolicy, lpolicy->l_flock.owner = wpolicy->l_flock.lfw_owner; } -void ldlm_flock_policy_local_to_wire(const ldlm_policy_data_t *lpolicy, +void ldlm_flock_policy_local_to_wire(const union ldlm_policy_data *lpolicy, ldlm_wire_policy_data_t *wpolicy) { memset(wpolicy, 0, sizeof(*wpolicy)); diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_inodebits.c b/drivers/staging/lustre/lustre/ldlm/ldlm_inodebits.c index 79f4e6f..11b45ae 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_inodebits.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_inodebits.c @@ -55,13 +55,13 @@ #include "ldlm_internal.h" void ldlm_ibits_policy_wire_to_local(const ldlm_wire_policy_data_t *wpolicy, - ldlm_policy_data_t *lpolicy) + union ldlm_policy_data *lpolicy) { memset(lpolicy, 0, sizeof(*lpolicy)); lpolicy->l_inodebits.bits = wpolicy->l_inodebits.bits; } -void ldlm_ibits_policy_local_to_wire(const ldlm_policy_data_t *lpolicy, +void ldlm_ibits_policy_local_to_wire(const union ldlm_policy_data *lpolicy, ldlm_wire_policy_data_t *wpolicy) { memset(wpolicy, 0, sizeof(*wpolicy)); diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h b/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h index 0099ff3..1a98a31 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h @@ -312,24 +312,24 @@ static inline int is_granted_or_cancelled(struct ldlm_lock *lock) } typedef void (*ldlm_policy_wire_to_local_t)(const ldlm_wire_policy_data_t *, - ldlm_policy_data_t *); + union ldlm_policy_data *); -typedef void (*ldlm_policy_local_to_wire_t)(const ldlm_policy_data_t *, +typedef void (*ldlm_policy_local_to_wire_t)(const union ldlm_policy_data *, ldlm_wire_policy_data_t *); void ldlm_plain_policy_wire_to_local(const ldlm_wire_policy_data_t *wpolicy, - ldlm_policy_data_t *lpolicy); -void ldlm_plain_policy_local_to_wire(const ldlm_policy_data_t *lpolicy, + union ldlm_policy_data *lpolicy); +void ldlm_plain_policy_local_to_wire(const union ldlm_policy_data *lpolicy, ldlm_wire_policy_data_t *wpolicy); void ldlm_ibits_policy_wire_to_local(const ldlm_wire_policy_data_t *wpolicy, - ldlm_policy_data_t *lpolicy); -void ldlm_ibits_policy_local_to_wire(const ldlm_policy_data_t *lpolicy, + union ldlm_policy_data *lpolicy); +void ldlm_ibits_policy_local_to_wire(const union ldlm_policy_data *lpolicy, ldlm_wire_policy_data_t *wpolicy); void ldlm_extent_policy_wire_to_local(const ldlm_wire_policy_data_t *wpolicy, - ldlm_policy_data_t *lpolicy); -void ldlm_extent_policy_local_to_wire(const ldlm_policy_data_t *lpolicy, + union ldlm_policy_data *lpolicy); +void ldlm_extent_policy_local_to_wire(const union ldlm_policy_data *lpolicy, ldlm_wire_policy_data_t *wpolicy); void ldlm_flock_policy_wire_to_local(const ldlm_wire_policy_data_t *wpolicy, - ldlm_policy_data_t *lpolicy); -void ldlm_flock_policy_local_to_wire(const ldlm_policy_data_t *lpolicy, + union ldlm_policy_data *lpolicy); +void ldlm_flock_policy_local_to_wire(const union ldlm_policy_data *lpolicy, ldlm_wire_policy_data_t *wpolicy); diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c index 17d532e..3dc0a06 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c @@ -81,7 +81,7 @@ * Converts lock policy from local format to on the wire lock_desc format */ static void ldlm_convert_policy_to_wire(enum ldlm_type type, - const ldlm_policy_data_t *lpolicy, + const union ldlm_policy_data *lpolicy, ldlm_wire_policy_data_t *wpolicy) { ldlm_policy_local_to_wire_t convert; @@ -96,7 +96,7 @@ static void ldlm_convert_policy_to_wire(enum ldlm_type type, */ void ldlm_convert_policy_to_local(struct obd_export *exp, enum ldlm_type type, const ldlm_wire_policy_data_t *wpolicy, - ldlm_policy_data_t *lpolicy) + union ldlm_policy_data *lpolicy) { ldlm_policy_wire_to_local_t convert; @@ -1049,7 +1049,7 @@ struct lock_match_data { struct ldlm_lock *lmd_old; struct ldlm_lock *lmd_lock; enum ldlm_mode *lmd_mode; - ldlm_policy_data_t *lmd_policy; + union ldlm_policy_data *lmd_policy; __u64 lmd_flags; int lmd_unref; }; @@ -1063,7 +1063,7 @@ struct lock_match_data { */ static int lock_matches(struct ldlm_lock *lock, struct lock_match_data *data) { - ldlm_policy_data_t *lpol = &lock->l_policy_data; + union ldlm_policy_data *lpol = &lock->l_policy_data; enum ldlm_mode match; if (lock == data->lmd_old) @@ -1280,7 +1280,7 @@ void ldlm_lock_allow_match(struct ldlm_lock *lock) enum ldlm_mode ldlm_lock_match(struct ldlm_namespace *ns, __u64 flags, const struct ldlm_res_id *res_id, enum ldlm_type type, - ldlm_policy_data_t *policy, + union ldlm_policy_data *policy, enum ldlm_mode mode, struct lustre_handle *lockh, int unref) { diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_plain.c b/drivers/staging/lustre/lustre/ldlm/ldlm_plain.c index 0aed39c..2603293 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_plain.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_plain.c @@ -55,12 +55,12 @@ #include "ldlm_internal.h" void ldlm_plain_policy_wire_to_local(const ldlm_wire_policy_data_t *wpolicy, - ldlm_policy_data_t *lpolicy) + union ldlm_policy_data *lpolicy) { /* No policy for plain locks */ } -void ldlm_plain_policy_local_to_wire(const ldlm_policy_data_t *lpolicy, +void ldlm_plain_policy_local_to_wire(const union ldlm_policy_data *lpolicy, ldlm_wire_policy_data_t *wpolicy) { /* No policy for plain locks */ diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c index 836789f..b772982 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c @@ -682,7 +682,7 @@ int ldlm_prep_enqueue_req(struct obd_export *exp, struct ptlrpc_request *req, int ldlm_cli_enqueue(struct obd_export *exp, struct ptlrpc_request **reqp, struct ldlm_enqueue_info *einfo, const struct ldlm_res_id *res_id, - ldlm_policy_data_t const *policy, __u64 *flags, + union ldlm_policy_data const *policy, __u64 *flags, void *lvb, __u32 lvb_len, enum lvb_type lvb_type, struct lustre_handle *lockh, int async) { @@ -1528,7 +1528,7 @@ int ldlm_cancel_lru(struct ldlm_namespace *ns, int nr, */ int ldlm_cancel_resource_local(struct ldlm_resource *res, struct list_head *cancels, - ldlm_policy_data_t *policy, + union ldlm_policy_data *policy, enum ldlm_mode mode, __u64 lock_flags, enum ldlm_cancel_flags cancel_flags, void *opaque) @@ -1645,7 +1645,7 @@ int ldlm_cli_cancel_list(struct list_head *cancels, int count, */ int ldlm_cli_cancel_unused_resource(struct ldlm_namespace *ns, const struct ldlm_res_id *res_id, - ldlm_policy_data_t *policy, + union ldlm_policy_data *policy, enum ldlm_mode mode, enum ldlm_cancel_flags flags, void *opaque) diff --git a/drivers/staging/lustre/lustre/llite/file.c b/drivers/staging/lustre/lustre/llite/file.c index 28c2501..6c2abb3 100644 --- a/drivers/staging/lustre/lustre/llite/file.c +++ b/drivers/staging/lustre/lustre/llite/file.c @@ -265,7 +265,9 @@ static int ll_md_close(struct obd_export *md_exp, struct inode *inode, int lockmode; __u64 flags = LDLM_FL_BLOCK_GRANTED | LDLM_FL_TEST_LOCK; struct lustre_handle lockh; - ldlm_policy_data_t policy = {.l_inodebits = {MDS_INODELOCK_OPEN} }; + union ldlm_policy_data policy = { + .l_inodebits = { MDS_INODELOCK_OPEN } + }; int rc = 0; /* clear group lock, if present */ @@ -2427,7 +2429,7 @@ int ll_fsync(struct file *file, loff_t start, loff_t end, int datasync) }; struct md_op_data *op_data; struct lustre_handle lockh = {0}; - ldlm_policy_data_t flock = { {0} }; + union ldlm_policy_data flock = { { 0 } }; int fl_type = file_lock->fl_type; __u64 flags = 0; int rc; @@ -2676,7 +2678,7 @@ int ll_have_md_lock(struct inode *inode, __u64 *bits, enum ldlm_mode l_req_mode) { struct lustre_handle lockh; - ldlm_policy_data_t policy; + union ldlm_policy_data policy; enum ldlm_mode mode = (l_req_mode == LCK_MINMODE) ? (LCK_CR | LCK_CW | LCK_PR | LCK_PW) : l_req_mode; struct lu_fid *fid; @@ -2717,7 +2719,7 @@ enum ldlm_mode ll_take_md_lock(struct inode *inode, __u64 bits, struct lustre_handle *lockh, __u64 flags, enum ldlm_mode mode) { - ldlm_policy_data_t policy = { .l_inodebits = {bits} }; + union ldlm_policy_data policy = { .l_inodebits = { bits } }; struct lu_fid *fid; fid = &ll_i2info(inode)->lli_fid; diff --git a/drivers/staging/lustre/lustre/llite/llite_internal.h b/drivers/staging/lustre/lustre/llite/llite_internal.h index c0513bf..50bb328 100644 --- a/drivers/staging/lustre/lustre/llite/llite_internal.h +++ b/drivers/staging/lustre/lustre/llite/llite_internal.h @@ -905,7 +905,7 @@ struct ll_thread_info { int ll_teardown_mmaps(struct address_space *mapping, __u64 first, __u64 last); int ll_file_mmap(struct file *file, struct vm_area_struct *vma); -void policy_from_vma(ldlm_policy_data_t *policy, struct vm_area_struct *vma, +void policy_from_vma(union ldlm_policy_data *policy, struct vm_area_struct *vma, unsigned long addr, size_t count); struct vm_area_struct *our_vma(struct mm_struct *mm, unsigned long addr, size_t count); diff --git a/drivers/staging/lustre/lustre/llite/llite_mmap.c b/drivers/staging/lustre/lustre/llite/llite_mmap.c index ad2a699..b56203b 100644 --- a/drivers/staging/lustre/lustre/llite/llite_mmap.c +++ b/drivers/staging/lustre/lustre/llite/llite_mmap.c @@ -47,7 +47,7 @@ static const struct vm_operations_struct ll_file_vm_ops; -void policy_from_vma(ldlm_policy_data_t *policy, +void policy_from_vma(union ldlm_policy_data *policy, struct vm_area_struct *vma, unsigned long addr, size_t count) { diff --git a/drivers/staging/lustre/lustre/llite/vvp_io.c b/drivers/staging/lustre/lustre/llite/vvp_io.c index 33b38bd..2bf3e47 100644 --- a/drivers/staging/lustre/lustre/llite/vvp_io.c +++ b/drivers/staging/lustre/lustre/llite/vvp_io.c @@ -370,7 +370,7 @@ static int vvp_mmap_locks(const struct lu_env *env, struct mm_struct *mm = current->mm; struct vm_area_struct *vma; struct cl_lock_descr *descr = &cti->vti_descr; - ldlm_policy_data_t policy; + union ldlm_policy_data policy; unsigned long addr; ssize_t count; int result = 0; diff --git a/drivers/staging/lustre/lustre/lmv/lmv_obd.c b/drivers/staging/lustre/lustre/lmv/lmv_obd.c index 0e94a58..f124f6c 100644 --- a/drivers/staging/lustre/lustre/lmv/lmv_obd.c +++ b/drivers/staging/lustre/lustre/lmv/lmv_obd.c @@ -1720,7 +1720,7 @@ static int lmv_create(struct obd_export *exp, struct md_op_data *op_data, static int lmv_enqueue(struct obd_export *exp, struct ldlm_enqueue_info *einfo, - const ldlm_policy_data_t *policy, + const union ldlm_policy_data *policy, struct lookup_intent *it, struct md_op_data *op_data, struct lustre_handle *lockh, __u64 extra_lock_flags) { @@ -1816,7 +1816,7 @@ static int lmv_early_cancel(struct obd_export *exp, struct lmv_tgt_desc *tgt, struct lu_fid *fid = md_op_data_fid(op_data, flag); struct obd_device *obd = exp->exp_obd; struct lmv_obd *lmv = &obd->u.lmv; - ldlm_policy_data_t policy = { {0} }; + union ldlm_policy_data policy = { { 0 } }; int rc = 0; if (!fid_is_sane(fid)) @@ -2890,8 +2890,9 @@ void lmv_free_memmd(struct lmv_stripe_md *lsm) EXPORT_SYMBOL(lmv_free_memmd); static int lmv_cancel_unused(struct obd_export *exp, const struct lu_fid *fid, - ldlm_policy_data_t *policy, enum ldlm_mode mode, - enum ldlm_cancel_flags flags, void *opaque) + union ldlm_policy_data *policy, + enum ldlm_mode mode, enum ldlm_cancel_flags flags, + void *opaque) { struct obd_device *obd = exp->exp_obd; struct lmv_obd *lmv = &obd->u.lmv; @@ -2933,7 +2934,7 @@ static int lmv_set_lock_data(struct obd_export *exp, static enum ldlm_mode lmv_lock_match(struct obd_export *exp, __u64 flags, const struct lu_fid *fid, enum ldlm_type type, - ldlm_policy_data_t *policy, + union ldlm_policy_data *policy, enum ldlm_mode mode, struct lustre_handle *lockh) { diff --git a/drivers/staging/lustre/lustre/mdc/mdc_internal.h b/drivers/staging/lustre/lustre/mdc/mdc_internal.h index d2af8e7..881c6a0 100644 --- a/drivers/staging/lustre/lustre/mdc/mdc_internal.h +++ b/drivers/staging/lustre/lustre/mdc/mdc_internal.h @@ -75,7 +75,7 @@ int mdc_intent_lock(struct obd_export *exp, __u64 extra_lock_flags); int mdc_enqueue(struct obd_export *exp, struct ldlm_enqueue_info *einfo, - const ldlm_policy_data_t *policy, + const union ldlm_policy_data *policy, struct lookup_intent *it, struct md_op_data *op_data, struct lustre_handle *lockh, __u64 extra_lock_flags); @@ -109,7 +109,7 @@ int mdc_setattr(struct obd_export *exp, struct md_op_data *op_data, int mdc_unlink(struct obd_export *exp, struct md_op_data *op_data, struct ptlrpc_request **request); int mdc_cancel_unused(struct obd_export *exp, const struct lu_fid *fid, - ldlm_policy_data_t *policy, enum ldlm_mode mode, + union ldlm_policy_data *policy, enum ldlm_mode mode, enum ldlm_cancel_flags flags, void *opaque); int mdc_revalidate_lock(struct obd_export *exp, struct lookup_intent *it, @@ -121,7 +121,8 @@ int mdc_intent_getattr_async(struct obd_export *exp, enum ldlm_mode mdc_lock_match(struct obd_export *exp, __u64 flags, const struct lu_fid *fid, enum ldlm_type type, - ldlm_policy_data_t *policy, enum ldlm_mode mode, + union ldlm_policy_data *policy, + enum ldlm_mode mode, struct lustre_handle *lockh); static inline int mdc_prep_elc_req(struct obd_export *exp, diff --git a/drivers/staging/lustre/lustre/mdc/mdc_locks.c b/drivers/staging/lustre/lustre/mdc/mdc_locks.c index 42a128f..6494c5b 100644 --- a/drivers/staging/lustre/lustre/mdc/mdc_locks.c +++ b/drivers/staging/lustre/lustre/mdc/mdc_locks.c @@ -131,7 +131,8 @@ int mdc_set_lock_data(struct obd_export *exp, const struct lustre_handle *lockh, enum ldlm_mode mdc_lock_match(struct obd_export *exp, __u64 flags, const struct lu_fid *fid, enum ldlm_type type, - ldlm_policy_data_t *policy, enum ldlm_mode mode, + union ldlm_policy_data *policy, + enum ldlm_mode mode, struct lustre_handle *lockh) { struct ldlm_res_id res_id; @@ -147,7 +148,7 @@ enum ldlm_mode mdc_lock_match(struct obd_export *exp, __u64 flags, int mdc_cancel_unused(struct obd_export *exp, const struct lu_fid *fid, - ldlm_policy_data_t *policy, + union ldlm_policy_data *policy, enum ldlm_mode mode, enum ldlm_cancel_flags flags, void *opaque) @@ -686,20 +687,20 @@ static int mdc_finish_enqueue(struct obd_export *exp, * we don't know in advance the file type. */ int mdc_enqueue(struct obd_export *exp, struct ldlm_enqueue_info *einfo, - const ldlm_policy_data_t *policy, + const union ldlm_policy_data *policy, struct lookup_intent *it, struct md_op_data *op_data, struct lustre_handle *lockh, u64 extra_lock_flags) { - static const ldlm_policy_data_t lookup_policy = { + static const union ldlm_policy_data lookup_policy = { .l_inodebits = { MDS_INODELOCK_LOOKUP } }; - static const ldlm_policy_data_t update_policy = { + static const union ldlm_policy_data update_policy = { .l_inodebits = { MDS_INODELOCK_UPDATE } }; - static const ldlm_policy_data_t layout_policy = { + static const union ldlm_policy_data layout_policy = { .l_inodebits = { MDS_INODELOCK_LAYOUT } }; - static const ldlm_policy_data_t getxattr_policy = { + static const union ldlm_policy_data getxattr_policy = { .l_inodebits = { MDS_INODELOCK_XATTR } }; struct obd_device *obddev = class_exp2obd(exp); @@ -925,7 +926,7 @@ static int mdc_finish_intent_lock(struct obd_export *exp, */ lock = ldlm_handle2lock(lockh); if (lock) { - ldlm_policy_data_t policy = lock->l_policy_data; + union ldlm_policy_data policy = lock->l_policy_data; LDLM_DEBUG(lock, "matching against this"); @@ -961,7 +962,7 @@ int mdc_revalidate_lock(struct obd_export *exp, struct lookup_intent *it, */ struct ldlm_res_id res_id; struct lustre_handle lockh; - ldlm_policy_data_t policy; + union ldlm_policy_data policy; enum ldlm_mode mode; if (it->it_lock_handle) { @@ -1163,10 +1164,9 @@ int mdc_intent_getattr_async(struct obd_export *exp, * for statahead currently. Consider CMD in future, such two bits * maybe managed by different MDS, should be adjusted then. */ - ldlm_policy_data_t policy = { - .l_inodebits = { MDS_INODELOCK_LOOKUP | - MDS_INODELOCK_UPDATE } - }; + union ldlm_policy_data policy = { + .l_inodebits = { MDS_INODELOCK_LOOKUP | MDS_INODELOCK_UPDATE } + }; int rc = 0; __u64 flags = LDLM_FL_HAS_INTENT; diff --git a/drivers/staging/lustre/lustre/mdc/mdc_reint.c b/drivers/staging/lustre/lustre/mdc/mdc_reint.c index b551c57..5119588 100644 --- a/drivers/staging/lustre/lustre/mdc/mdc_reint.c +++ b/drivers/staging/lustre/lustre/mdc/mdc_reint.c @@ -66,7 +66,7 @@ int mdc_resource_get_unused(struct obd_export *exp, const struct lu_fid *fid, __u64 bits) { struct ldlm_namespace *ns = exp->exp_obd->obd_namespace; - ldlm_policy_data_t policy = {}; + union ldlm_policy_data policy = {}; struct ldlm_res_id res_id; struct ldlm_resource *res; int count; diff --git a/drivers/staging/lustre/lustre/mgc/mgc_request.c b/drivers/staging/lustre/lustre/mgc/mgc_request.c index a2cd5dd..23600fb 100644 --- a/drivers/staging/lustre/lustre/mgc/mgc_request.c +++ b/drivers/staging/lustre/lustre/mgc/mgc_request.c @@ -886,7 +886,7 @@ static int mgc_set_mgs_param(struct obd_export *exp, /* Take a config lock so we can get cancel notifications */ static int mgc_enqueue(struct obd_export *exp, __u32 type, - ldlm_policy_data_t *policy, __u32 mode, + union ldlm_policy_data *policy, __u32 mode, __u64 *flags, void *bl_cb, void *cp_cb, void *gl_cb, void *data, __u32 lvb_len, void *lvb_swabber, struct lustre_handle *lockh) diff --git a/drivers/staging/lustre/lustre/osc/osc_cl_internal.h b/drivers/staging/lustre/lustre/osc/osc_cl_internal.h index c9470d1..bf05a2d 100644 --- a/drivers/staging/lustre/lustre/osc/osc_cl_internal.h +++ b/drivers/staging/lustre/lustre/osc/osc_cl_internal.h @@ -95,7 +95,7 @@ struct osc_session { #define OTI_PVEC_SIZE 256 struct osc_thread_info { struct ldlm_res_id oti_resname; - ldlm_policy_data_t oti_policy; + union ldlm_policy_data oti_policy; struct cl_lock_descr oti_descr; struct cl_attr oti_attr; struct lustre_handle oti_handle; @@ -394,7 +394,8 @@ struct lu_object *osc_object_alloc(const struct lu_env *env, int osc_page_init(const struct lu_env *env, struct cl_object *obj, struct cl_page *page, pgoff_t ind); -void osc_index2policy(ldlm_policy_data_t *policy, const struct cl_object *obj, +void osc_index2policy(union ldlm_policy_data *policy, + const struct cl_object *obj, pgoff_t start, pgoff_t end); int osc_lvb_print(const struct lu_env *env, void *cookie, lu_printer_t p, const struct ost_lvb *lvb); diff --git a/drivers/staging/lustre/lustre/osc/osc_internal.h b/drivers/staging/lustre/lustre/osc/osc_internal.h index 12d3b58..981b7e1 100644 --- a/drivers/staging/lustre/lustre/osc/osc_internal.h +++ b/drivers/staging/lustre/lustre/osc/osc_internal.h @@ -107,14 +107,14 @@ typedef int (*osc_enqueue_upcall_f)(void *cookie, struct lustre_handle *lockh, int rc); int osc_enqueue_base(struct obd_export *exp, struct ldlm_res_id *res_id, - __u64 *flags, ldlm_policy_data_t *policy, + __u64 *flags, union ldlm_policy_data *policy, struct ost_lvb *lvb, int kms_valid, osc_enqueue_upcall_f upcall, void *cookie, struct ldlm_enqueue_info *einfo, struct ptlrpc_request_set *rqset, int async, int agl); int osc_match_base(struct obd_export *exp, struct ldlm_res_id *res_id, - __u32 type, ldlm_policy_data_t *policy, __u32 mode, + __u32 type, union ldlm_policy_data *policy, __u32 mode, __u64 *flags, void *data, struct lustre_handle *lockh, int unref); diff --git a/drivers/staging/lustre/lustre/osc/osc_lock.c b/drivers/staging/lustre/lustre/osc/osc_lock.c index bfc8d38..130460d 100644 --- a/drivers/staging/lustre/lustre/osc/osc_lock.c +++ b/drivers/staging/lustre/lustre/osc/osc_lock.c @@ -145,7 +145,7 @@ static void osc_lock_fini(const struct lu_env *env, static void osc_lock_build_policy(const struct lu_env *env, const struct cl_lock *lock, - ldlm_policy_data_t *policy) + union ldlm_policy_data *policy) { const struct cl_lock_descr *d = &lock->cll_descr; @@ -914,7 +914,7 @@ static int osc_lock_enqueue(const struct lu_env *env, struct osc_lock *oscl = cl2osc_lock(slice); struct cl_lock *lock = slice->cls_lock; struct ldlm_res_id *resname = &info->oti_resname; - ldlm_policy_data_t *policy = &info->oti_policy; + union ldlm_policy_data *policy = &info->oti_policy; osc_enqueue_upcall_f upcall = osc_lock_upcall; void *cookie = oscl; bool async = false; @@ -1186,7 +1186,7 @@ struct ldlm_lock *osc_dlmlock_at_pgoff(const struct lu_env *env, { struct osc_thread_info *info = osc_env_info(env); struct ldlm_res_id *resname = &info->oti_resname; - ldlm_policy_data_t *policy = &info->oti_policy; + union ldlm_policy_data *policy = &info->oti_policy; struct lustre_handle lockh; struct ldlm_lock *lock = NULL; enum ldlm_mode mode; diff --git a/drivers/staging/lustre/lustre/osc/osc_object.c b/drivers/staging/lustre/lustre/osc/osc_object.c index ab1c11c..3592796 100644 --- a/drivers/staging/lustre/lustre/osc/osc_object.c +++ b/drivers/staging/lustre/lustre/osc/osc_object.c @@ -223,7 +223,7 @@ static int osc_object_fiemap(const struct lu_env *env, struct cl_object *obj, struct fiemap *fiemap, size_t *buflen) { struct obd_export *exp = osc_export(cl2osc(obj)); - ldlm_policy_data_t policy; + union ldlm_policy_data policy; struct ptlrpc_request *req; struct lustre_handle lockh; struct ldlm_res_id resid; diff --git a/drivers/staging/lustre/lustre/osc/osc_page.c b/drivers/staging/lustre/lustre/osc/osc_page.c index 93248d1..7a1f102 100644 --- a/drivers/staging/lustre/lustre/osc/osc_page.c +++ b/drivers/staging/lustre/lustre/osc/osc_page.c @@ -109,7 +109,8 @@ int osc_page_cache_add(const struct lu_env *env, return result; } -void osc_index2policy(ldlm_policy_data_t *policy, const struct cl_object *obj, +void osc_index2policy(union ldlm_policy_data *policy, + const struct cl_object *obj, pgoff_t start, pgoff_t end) { memset(policy, 0, sizeof(*policy)); diff --git a/drivers/staging/lustre/lustre/osc/osc_request.c b/drivers/staging/lustre/lustre/osc/osc_request.c index 7f19cfa..4d4d3eb 100644 --- a/drivers/staging/lustre/lustre/osc/osc_request.c +++ b/drivers/staging/lustre/lustre/osc/osc_request.c @@ -1953,7 +1953,7 @@ static int osc_enqueue_interpret(const struct lu_env *env, * release locks just after they are obtained. */ int osc_enqueue_base(struct obd_export *exp, struct ldlm_res_id *res_id, - __u64 *flags, ldlm_policy_data_t *policy, + __u64 *flags, union ldlm_policy_data *policy, struct ost_lvb *lvb, int kms_valid, osc_enqueue_upcall_f upcall, void *cookie, struct ldlm_enqueue_info *einfo, @@ -2104,7 +2104,7 @@ int osc_enqueue_base(struct obd_export *exp, struct ldlm_res_id *res_id, } int osc_match_base(struct obd_export *exp, struct ldlm_res_id *res_id, - __u32 type, ldlm_policy_data_t *policy, __u32 mode, + __u32 type, union ldlm_policy_data *policy, __u32 mode, __u64 *flags, void *data, struct lustre_handle *lockh, int unref) { -- 1.7.1 From jsimmons at infradead.org Wed Nov 23 22:59:50 2016 From: jsimmons at infradead.org (James Simmons) Date: Wed, 23 Nov 2016 17:59:50 -0500 Subject: [lustre-devel] [PATCH 4/5] staging: lustre: ldlm: rename LDLM_CANCEL_* flags In-Reply-To: <1479941991-30059-1-git-send-email-jsimmons@infradead.org> References: <1479941991-30059-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479941991-30059-5-git-send-email-jsimmons@infradead.org> From: Andreas Dilger Rename LDLM_CANCEL_* flags (used with enum ldlm_lru_flags) to LDLM_LRU_FLAGS_* to avoid confusion with enum ldlm_cancel_flags. Signed-off-by: Andreas Dilger Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6142 Reviewed-on: http://review.whamcloud.com/15300 Reviewed-on: http://review.whamcloud.com/15301 Reviewed-by: James Simmons Reviewed-by: Bob Glossman Reviewed-by: frank zago Reviewed-by: Dmitry Eremin Reviewed-by: John L. Hammond Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- drivers/staging/lustre/lustre/ldlm/ldlm_internal.h | 16 ++++---- drivers/staging/lustre/lustre/ldlm/ldlm_pool.c | 4 +- drivers/staging/lustre/lustre/ldlm/ldlm_request.c | 44 ++++++++++---------- drivers/staging/lustre/lustre/ldlm/ldlm_resource.c | 8 ++-- 4 files changed, 36 insertions(+), 36 deletions(-) diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h b/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h index 9085ee5..49f74db 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h @@ -88,14 +88,14 @@ void ldlm_namespace_move_to_inactive_locked(struct ldlm_namespace *, /* ldlm_request.c */ /* Cancel lru flag, it indicates we cancel aged locks. */ enum { - LDLM_CANCEL_AGED = 1 << 0, /* Cancel aged locks (non lru resize). */ - LDLM_CANCEL_PASSED = 1 << 1, /* Cancel passed number of locks. */ - LDLM_CANCEL_SHRINK = 1 << 2, /* Cancel locks from shrinker. */ - LDLM_CANCEL_LRUR = 1 << 3, /* Cancel locks from lru resize. */ - LDLM_CANCEL_NO_WAIT = 1 << 4, /* Cancel locks w/o blocking (neither - * sending nor waiting for any rpcs) - */ - LDLM_CANCEL_LRUR_NO_WAIT = 1 << 5, /* LRUR + NO_WAIT */ + LDLM_LRU_FLAG_AGED = BIT(0), /* Cancel aged locks (non lru resize). */ + LDLM_LRU_FLAG_PASSED = BIT(1), /* Cancel passed number of locks. */ + LDLM_LRU_FLAG_SHRINK = BIT(2), /* Cancel locks from shrinker. */ + LDLM_LRU_FLAG_LRUR = BIT(3), /* Cancel locks from lru resize. */ + LDLM_LRU_FLAG_NO_WAIT = BIT(4), /* Cancel locks w/o blocking (neither + * sending nor waiting for any rpcs) + */ + LDLM_LRU_FLAG_LRUR_NO_WAIT = BIT(5), /* LRUR + NO_WAIT */ }; int ldlm_cancel_lru(struct ldlm_namespace *ns, int nr, diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c index 252bddb..8dfb3c8 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_pool.c @@ -293,7 +293,7 @@ static int ldlm_cli_pool_recalc(struct ldlm_pool *pl) * take into account pl->pl_recalc_time here. */ ret = ldlm_cancel_lru(container_of(pl, struct ldlm_namespace, ns_pool), - 0, LCF_ASYNC, LDLM_CANCEL_LRUR); + 0, LCF_ASYNC, LDLM_LRU_FLAG_LRUR); out: spin_lock(&pl->pl_lock); @@ -339,7 +339,7 @@ static int ldlm_cli_pool_shrink(struct ldlm_pool *pl, if (nr == 0) return (unused / 100) * sysctl_vfs_cache_pressure; else - return ldlm_cancel_lru(ns, nr, LCF_ASYNC, LDLM_CANCEL_SHRINK); + return ldlm_cancel_lru(ns, nr, LCF_ASYNC, LDLM_LRU_FLAG_SHRINK); } static const struct ldlm_pool_ops ldlm_cli_pool_ops = { diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c index b772982..c1f8693 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_request.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_request.c @@ -593,7 +593,7 @@ int ldlm_prep_elc_req(struct obd_export *exp, struct ptlrpc_request *req, avail = ldlm_capsule_handles_avail(pill, RCL_CLIENT, canceloff); flags = ns_connect_lru_resize(ns) ? - LDLM_CANCEL_LRUR_NO_WAIT : LDLM_CANCEL_AGED; + LDLM_LRU_FLAG_LRUR_NO_WAIT : LDLM_LRU_FLAG_AGED; to_free = !ns_connect_lru_resize(ns) && opc == LDLM_ENQUEUE ? 1 : 0; @@ -1057,7 +1057,7 @@ int ldlm_cli_cancel(const struct lustre_handle *lockh, ns = ldlm_lock_to_ns(lock); flags = ns_connect_lru_resize(ns) ? - LDLM_CANCEL_LRUR : LDLM_CANCEL_AGED; + LDLM_LRU_FLAG_LRUR : LDLM_LRU_FLAG_AGED; count += ldlm_cancel_lru_local(ns, &cancels, 0, avail - 1, LCF_BL_AST, flags); } @@ -1284,21 +1284,21 @@ typedef enum ldlm_policy_res (*ldlm_cancel_lru_policy_t)( static ldlm_cancel_lru_policy_t ldlm_cancel_lru_policy(struct ldlm_namespace *ns, int flags) { - if (flags & LDLM_CANCEL_NO_WAIT) + if (flags & LDLM_LRU_FLAG_NO_WAIT) return ldlm_cancel_no_wait_policy; if (ns_connect_lru_resize(ns)) { - if (flags & LDLM_CANCEL_SHRINK) + if (flags & LDLM_LRU_FLAG_SHRINK) /* We kill passed number of old locks. */ return ldlm_cancel_passed_policy; - else if (flags & LDLM_CANCEL_LRUR) + else if (flags & LDLM_LRU_FLAG_LRUR) return ldlm_cancel_lrur_policy; - else if (flags & LDLM_CANCEL_PASSED) + else if (flags & LDLM_LRU_FLAG_PASSED) return ldlm_cancel_passed_policy; - else if (flags & LDLM_CANCEL_LRUR_NO_WAIT) + else if (flags & LDLM_LRU_FLAG_LRUR_NO_WAIT) return ldlm_cancel_lrur_no_wait_policy; } else { - if (flags & LDLM_CANCEL_AGED) + if (flags & LDLM_LRU_FLAG_AGED) return ldlm_cancel_aged_policy; } @@ -1322,21 +1322,21 @@ typedef enum ldlm_policy_res (*ldlm_cancel_lru_policy_t)( * * Calling policies for enabled LRU resize: * ---------------------------------------- - * flags & LDLM_CANCEL_LRUR - use LRU resize policy (SLV from server) to - * cancel not more than \a count locks; + * flags & LDLM_LRU_FLAG_LRUR - use LRU resize policy (SLV from server) to + * cancel not more than \a count locks; * - * flags & LDLM_CANCEL_PASSED - cancel \a count number of old locks (located at - * the beginning of LRU list); + * flags & LDLM_LRU_FLAG_PASSED - cancel \a count number of old locks (located at + * the beginning of LRU list); * - * flags & LDLM_CANCEL_SHRINK - cancel not more than \a count locks according to - * memory pressure policy function; + * flags & LDLM_LRU_FLAG_SHRINK - cancel not more than \a count locks according to + * memory pressure policy function; * - * flags & LDLM_CANCEL_AGED - cancel \a count locks according to "aged policy". + * flags & LDLM_LRU_FLAG_AGED - cancel \a count locks according to "aged policy". * - * flags & LDLM_CANCEL_NO_WAIT - cancel as many unused locks as possible - * (typically before replaying locks) w/o - * sending any RPCs or waiting for any - * outstanding RPC to complete. + * flags & LDLM_LRU_FLAG_NO_WAIT - cancel as many unused locks as possible + * (typically before replaying locks) w/o + * sending any RPCs or waiting for any + * outstanding RPC to complete. */ static int ldlm_prepare_lru_list(struct ldlm_namespace *ns, struct list_head *cancels, int count, int max, @@ -1345,7 +1345,7 @@ static int ldlm_prepare_lru_list(struct ldlm_namespace *ns, ldlm_cancel_lru_policy_t pf; struct ldlm_lock *lock, *next; int added = 0, unused, remained; - int no_wait = flags & (LDLM_CANCEL_NO_WAIT | LDLM_CANCEL_LRUR_NO_WAIT); + int no_wait = flags & (LDLM_LRU_FLAG_NO_WAIT | LDLM_LRU_FLAG_LRUR_NO_WAIT); spin_lock(&ns->ns_lock); unused = ns->ns_nr_unused; @@ -2000,11 +2000,11 @@ static void ldlm_cancel_unused_locks_for_replay(struct ldlm_namespace *ns) ldlm_ns_name(ns), ns->ns_nr_unused); /* We don't need to care whether or not LRU resize is enabled - * because the LDLM_CANCEL_NO_WAIT policy doesn't use the + * because the LDLM_LRU_FLAG_NO_WAIT policy doesn't use the * count parameter */ canceled = ldlm_cancel_lru_local(ns, &cancels, ns->ns_nr_unused, 0, - LCF_LOCAL, LDLM_CANCEL_NO_WAIT); + LCF_LOCAL, LDLM_LRU_FLAG_NO_WAIT); CDEBUG(D_DLMTRACE, "Canceled %d unused locks from namespace %s\n", canceled, ldlm_ns_name(ns)); diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c b/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c index ca2cd4a..1095331 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_resource.c @@ -226,7 +226,7 @@ static ssize_t lru_size_store(struct kobject *kobj, struct attribute *attr, /* Try to cancel all @ns_nr_unused locks. */ canceled = ldlm_cancel_lru(ns, unused, 0, - LDLM_CANCEL_PASSED); + LDLM_LRU_FLAG_PASSED); if (canceled < unused) { CDEBUG(D_DLMTRACE, "not all requested locks are canceled, requested: %d, canceled: %d\n", @@ -237,7 +237,7 @@ static ssize_t lru_size_store(struct kobject *kobj, struct attribute *attr, } else { tmp = ns->ns_max_unused; ns->ns_max_unused = 0; - ldlm_cancel_lru(ns, 0, 0, LDLM_CANCEL_PASSED); + ldlm_cancel_lru(ns, 0, 0, LDLM_LRU_FLAG_PASSED); ns->ns_max_unused = tmp; } return count; @@ -262,7 +262,7 @@ static ssize_t lru_size_store(struct kobject *kobj, struct attribute *attr, "changing namespace %s unused locks from %u to %u\n", ldlm_ns_name(ns), ns->ns_nr_unused, (unsigned int)tmp); - ldlm_cancel_lru(ns, tmp, LCF_ASYNC, LDLM_CANCEL_PASSED); + ldlm_cancel_lru(ns, tmp, LCF_ASYNC, LDLM_LRU_FLAG_PASSED); if (!lru_resize) { CDEBUG(D_DLMTRACE, @@ -276,7 +276,7 @@ static ssize_t lru_size_store(struct kobject *kobj, struct attribute *attr, ldlm_ns_name(ns), ns->ns_max_unused, (unsigned int)tmp); ns->ns_max_unused = (unsigned int)tmp; - ldlm_cancel_lru(ns, 0, LCF_ASYNC, LDLM_CANCEL_PASSED); + ldlm_cancel_lru(ns, 0, LCF_ASYNC, LDLM_LRU_FLAG_PASSED); /* Make sure that LRU resize was originally supported before * turning it on here. -- 1.7.1 From jsimmons at infradead.org Wed Nov 23 22:59:49 2016 From: jsimmons at infradead.org (James Simmons) Date: Wed, 23 Nov 2016 17:59:49 -0500 Subject: [lustre-devel] [PATCH 3/5] staging: lustre: ldlm: remove ldlm_wire_policy_data_t typedef usage from code In-Reply-To: <1479941991-30059-1-git-send-email-jsimmons@infradead.org> References: <1479941991-30059-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479941991-30059-4-git-send-email-jsimmons@infradead.org> From: Andreas Dilger Replace usage of ldlm_wire_policy_data_t with named enums to conform to upstream coding style. Signed-off-by: Andreas Dilger Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6142 Reviewed-on: http://review.whamcloud.com/15300 Reviewed-on: http://review.whamcloud.com/15301 Reviewed-by: James Simmons Reviewed-by: Bob Glossman Reviewed-by: frank zago Reviewed-by: Dmitry Eremin Reviewed-by: John L. Hammond Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- .../lustre/lustre/include/lustre/lustre_idl.h | 6 +++--- drivers/staging/lustre/lustre/include/lustre_dlm.h | 2 +- drivers/staging/lustre/lustre/ldlm/ldlm_extent.c | 4 ++-- drivers/staging/lustre/lustre/ldlm/ldlm_flock.c | 4 ++-- .../staging/lustre/lustre/ldlm/ldlm_inodebits.c | 4 ++-- drivers/staging/lustre/lustre/ldlm/ldlm_internal.h | 20 ++++++++++---------- drivers/staging/lustre/lustre/ldlm/ldlm_lock.c | 4 ++-- drivers/staging/lustre/lustre/ldlm/ldlm_plain.c | 4 ++-- .../staging/lustre/lustre/ptlrpc/pack_generic.c | 2 +- 9 files changed, 25 insertions(+), 25 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h index 5c1fb6c..6a4a7d8 100644 --- a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h +++ b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h @@ -2623,11 +2623,11 @@ struct ldlm_flock_wire { * on the resource type. */ -typedef union { +union ldlm_wire_policy_data { struct ldlm_extent l_extent; struct ldlm_flock_wire l_flock; struct ldlm_inodebits l_inodebits; -} ldlm_wire_policy_data_t; +}; union ldlm_gl_desc { struct ldlm_gl_lquota_desc lquota_desc; @@ -2669,7 +2669,7 @@ struct ldlm_lock_desc { struct ldlm_resource_desc l_resource; enum ldlm_mode l_req_mode; enum ldlm_mode l_granted_mode; - ldlm_wire_policy_data_t l_policy_data; + union ldlm_wire_policy_data l_policy_data; }; #define LDLM_LOCKREQ_HANDLES 2 diff --git a/drivers/staging/lustre/lustre/include/lustre_dlm.h b/drivers/staging/lustre/lustre/include/lustre_dlm.h index 02c90b0..62304ba 100644 --- a/drivers/staging/lustre/lustre/include/lustre_dlm.h +++ b/drivers/staging/lustre/lustre/include/lustre_dlm.h @@ -560,7 +560,7 @@ struct ldlm_flock { }; void ldlm_convert_policy_to_local(struct obd_export *exp, enum ldlm_type type, - const ldlm_wire_policy_data_t *wpolicy, + const union ldlm_wire_policy_data *wpolicy, union ldlm_policy_data *lpolicy); enum lvb_type { diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c b/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c index 78cd70d..32b73ee 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c @@ -240,7 +240,7 @@ void ldlm_extent_unlink_lock(struct ldlm_lock *lock) } } -void ldlm_extent_policy_wire_to_local(const ldlm_wire_policy_data_t *wpolicy, +void ldlm_extent_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy, union ldlm_policy_data *lpolicy) { memset(lpolicy, 0, sizeof(*lpolicy)); @@ -250,7 +250,7 @@ void ldlm_extent_policy_wire_to_local(const ldlm_wire_policy_data_t *wpolicy, } void ldlm_extent_policy_local_to_wire(const union ldlm_policy_data *lpolicy, - ldlm_wire_policy_data_t *wpolicy) + union ldlm_wire_policy_data *wpolicy) { memset(wpolicy, 0, sizeof(*wpolicy)); wpolicy->l_extent.start = lpolicy->l_extent.start; diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c b/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c index 2683a7b..7221607 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c @@ -612,7 +612,7 @@ struct ldlm_flock_wait_data { } EXPORT_SYMBOL(ldlm_flock_completion_ast); -void ldlm_flock_policy_wire_to_local(const ldlm_wire_policy_data_t *wpolicy, +void ldlm_flock_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy, union ldlm_policy_data *lpolicy) { memset(lpolicy, 0, sizeof(*lpolicy)); @@ -623,7 +623,7 @@ void ldlm_flock_policy_wire_to_local(const ldlm_wire_policy_data_t *wpolicy, } void ldlm_flock_policy_local_to_wire(const union ldlm_policy_data *lpolicy, - ldlm_wire_policy_data_t *wpolicy) + union ldlm_wire_policy_data *wpolicy) { memset(wpolicy, 0, sizeof(*wpolicy)); wpolicy->l_flock.lfw_start = lpolicy->l_flock.start; diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_inodebits.c b/drivers/staging/lustre/lustre/ldlm/ldlm_inodebits.c index 11b45ae..8e1709d 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_inodebits.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_inodebits.c @@ -54,7 +54,7 @@ #include "../include/lustre_lib.h" #include "ldlm_internal.h" -void ldlm_ibits_policy_wire_to_local(const ldlm_wire_policy_data_t *wpolicy, +void ldlm_ibits_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy, union ldlm_policy_data *lpolicy) { memset(lpolicy, 0, sizeof(*lpolicy)); @@ -62,7 +62,7 @@ void ldlm_ibits_policy_wire_to_local(const ldlm_wire_policy_data_t *wpolicy, } void ldlm_ibits_policy_local_to_wire(const union ldlm_policy_data *lpolicy, - ldlm_wire_policy_data_t *wpolicy) + union ldlm_wire_policy_data *wpolicy) { memset(wpolicy, 0, sizeof(*wpolicy)); wpolicy->l_inodebits.bits = lpolicy->l_inodebits.bits; diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h b/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h index e2c3c6e..9085ee5 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_internal.h @@ -312,25 +312,25 @@ static inline int is_granted_or_cancelled(struct ldlm_lock *lock) return ret; } -typedef void (*ldlm_policy_wire_to_local_t)(const ldlm_wire_policy_data_t *, +typedef void (*ldlm_policy_wire_to_local_t)(const union ldlm_wire_policy_data *, union ldlm_policy_data *); typedef void (*ldlm_policy_local_to_wire_t)(const union ldlm_policy_data *, - ldlm_wire_policy_data_t *); + union ldlm_wire_policy_data *); -void ldlm_plain_policy_wire_to_local(const ldlm_wire_policy_data_t *wpolicy, +void ldlm_plain_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy, union ldlm_policy_data *lpolicy); void ldlm_plain_policy_local_to_wire(const union ldlm_policy_data *lpolicy, - ldlm_wire_policy_data_t *wpolicy); -void ldlm_ibits_policy_wire_to_local(const ldlm_wire_policy_data_t *wpolicy, + union ldlm_wire_policy_data *wpolicy); +void ldlm_ibits_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy, union ldlm_policy_data *lpolicy); void ldlm_ibits_policy_local_to_wire(const union ldlm_policy_data *lpolicy, - ldlm_wire_policy_data_t *wpolicy); -void ldlm_extent_policy_wire_to_local(const ldlm_wire_policy_data_t *wpolicy, + union ldlm_wire_policy_data *wpolicy); +void ldlm_extent_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy, union ldlm_policy_data *lpolicy); void ldlm_extent_policy_local_to_wire(const union ldlm_policy_data *lpolicy, - ldlm_wire_policy_data_t *wpolicy); -void ldlm_flock_policy_wire_to_local(const ldlm_wire_policy_data_t *wpolicy, + union ldlm_wire_policy_data *wpolicy); +void ldlm_flock_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy, union ldlm_policy_data *lpolicy); void ldlm_flock_policy_local_to_wire(const union ldlm_policy_data *lpolicy, - ldlm_wire_policy_data_t *wpolicy); + union ldlm_wire_policy_data *wpolicy); diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c index 3dc0a06..15eeb2b 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c @@ -82,7 +82,7 @@ */ static void ldlm_convert_policy_to_wire(enum ldlm_type type, const union ldlm_policy_data *lpolicy, - ldlm_wire_policy_data_t *wpolicy) + union ldlm_wire_policy_data *wpolicy) { ldlm_policy_local_to_wire_t convert; @@ -95,7 +95,7 @@ static void ldlm_convert_policy_to_wire(enum ldlm_type type, * Converts lock policy from on the wire lock_desc format to local format */ void ldlm_convert_policy_to_local(struct obd_export *exp, enum ldlm_type type, - const ldlm_wire_policy_data_t *wpolicy, + const union ldlm_wire_policy_data *wpolicy, union ldlm_policy_data *lpolicy) { ldlm_policy_wire_to_local_t convert; diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_plain.c b/drivers/staging/lustre/lustre/ldlm/ldlm_plain.c index 2603293..862ea0a 100644 --- a/drivers/staging/lustre/lustre/ldlm/ldlm_plain.c +++ b/drivers/staging/lustre/lustre/ldlm/ldlm_plain.c @@ -54,14 +54,14 @@ #include "ldlm_internal.h" -void ldlm_plain_policy_wire_to_local(const ldlm_wire_policy_data_t *wpolicy, +void ldlm_plain_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy, union ldlm_policy_data *lpolicy) { /* No policy for plain locks */ } void ldlm_plain_policy_local_to_wire(const union ldlm_policy_data *lpolicy, - ldlm_wire_policy_data_t *wpolicy) + union ldlm_wire_policy_data *wpolicy) { /* No policy for plain locks */ } diff --git a/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c b/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c index 39c4e18..2c9debb 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c +++ b/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c @@ -2012,7 +2012,7 @@ static void lustre_swab_ldlm_res_id(struct ldlm_res_id *id) __swab64s(&id->name[i]); } -static void lustre_swab_ldlm_policy_data(ldlm_wire_policy_data_t *d) +static void lustre_swab_ldlm_policy_data(union ldlm_wire_policy_data *d) { /* the lock data is a union and the first two fields are always an * extent so it's ok to process an LDLM_EXTENT and LDLM_FLOCK lock -- 1.7.1 From jsimmons at infradead.org Wed Nov 23 23:01:16 2016 From: jsimmons at infradead.org (James Simmons) Date: Wed, 23 Nov 2016 18:01:16 -0500 Subject: [lustre-devel] [PATCH 0/4] staging: lustre: obd: reserve connection flags Message-ID: <1479942080-30166-1-git-send-email-jsimmons@infradead.org> Reserve several OBD_CONNECT_* flags for future use. Fan Yong (4): staging: lustre: obd: reserve connection flag OBD_CONNECT_SUBTREE staging: lustre: obd: reserve connection flag OBD_CONNECT_LOCK_AHEAD staging: lustre: obd: reserve connection flag OBD_CONNECT_OBDOPACK staging: lustre: obd: reserve connection flag OBD_CONNECT_FLAGS2 .../lustre/lustre/include/lustre/lustre_idl.h | 6 +++++- .../lustre/lustre/obdclass/lprocfs_status.c | 5 ++++- .../staging/lustre/lustre/ptlrpc/pack_generic.c | 3 ++- drivers/staging/lustre/lustre/ptlrpc/wiretest.c | 16 ++++++++++++---- 4 files changed, 23 insertions(+), 7 deletions(-) From jsimmons at infradead.org Wed Nov 23 23:01:20 2016 From: jsimmons at infradead.org (James Simmons) Date: Wed, 23 Nov 2016 18:01:20 -0500 Subject: [lustre-devel] [PATCH 4/4] staging: lustre: obd: reserve connection flag OBD_CONNECT_FLAGS2 In-Reply-To: <1479942080-30166-1-git-send-email-jsimmons@infradead.org> References: <1479942080-30166-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479942080-30166-5-git-send-email-jsimmons@infradead.org> From: Fan Yong This is a feature for the client and server to use obd_connect_flags2 to communicate future feature flags. The client should set this flag whenever any flags in that field are requested, and the server should mask unsupported features from this field (assuming it understands OBD_CONNECT_FLAGS2). When checking if an OBD_CONNECT2_xxxx feature is supported, the client/server needs to firstly check if OBD_CONNECT_FLAGS2 is supported, since this field is also beyond the end of the old obd_connect_data. Land the connection flags to upstream client earlier for reserving the slot to avoid potential conflict with others. Signed-off-by: Fan Yong Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-7543 Reviewed-on: http://review.whamcloud.com/17647 Reviewed-by: Andreas Dilger Reviewed-by: Patrick Farrell Signed-off-by: James Simmons --- .../lustre/lustre/include/lustre/lustre_idl.h | 3 ++- .../lustre/lustre/obdclass/lprocfs_status.c | 1 + .../staging/lustre/lustre/ptlrpc/pack_generic.c | 3 ++- drivers/staging/lustre/lustre/ptlrpc/wiretest.c | 10 ++++++---- 4 files changed, 11 insertions(+), 6 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h index a4020d2..5f42c54 100644 --- a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h +++ b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h @@ -1194,6 +1194,7 @@ struct ptlrpc_body_v2 { /** bulk matchbits is sent within ptlrpc_body */ #define OBD_CONNECT_BULK_MBITS 0x2000000000000000ULL #define OBD_CONNECT_OBDOPACK 0x4000000000000000ULL /* compact OUT obdo */ +#define OBD_CONNECT_FLAGS2 0x8000000000000000ULL /* second flags word */ /* XXX README XXX: * Please DO NOT add flag values here before first ensuring that this same @@ -1249,7 +1250,7 @@ struct obd_connect_data { __u16 ocd_maxmodrpcs; /* Maximum modify RPCs in parallel */ __u16 padding0; /* added 2.1.0. also fix lustre_swab_connect */ __u32 padding1; /* added 2.1.0. also fix lustre_swab_connect */ - __u64 padding2; /* added 2.1.0. also fix lustre_swab_connect */ + __u64 ocd_connect_flags2; __u64 padding3; /* added 2.1.0. also fix lustre_swab_connect */ __u64 padding4; /* added 2.1.0. also fix lustre_swab_connect */ __u64 padding5; /* added 2.1.0. also fix lustre_swab_connect */ diff --git a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c index 85b505e..2f2e8fc 100644 --- a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c +++ b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c @@ -106,6 +106,7 @@ "lock_ahead", "bulk_mbits", "compact_obdo", + "second_flags", NULL }; diff --git a/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c b/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c index 39c4e18..30c5e91 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c +++ b/drivers/staging/lustre/lustre/ptlrpc/pack_generic.c @@ -1566,7 +1566,8 @@ void lustre_swab_connect(struct obd_connect_data *ocd) __swab16s(&ocd->ocd_maxmodrpcs); CLASSERT(offsetof(typeof(*ocd), padding0)); CLASSERT(offsetof(typeof(*ocd), padding1) != 0); - CLASSERT(offsetof(typeof(*ocd), padding2) != 0); + if (ocd->ocd_connect_flags & OBD_CONNECT_FLAGS2) + __swab64s(&ocd->ocd_connect_flags2); CLASSERT(offsetof(typeof(*ocd), padding3) != 0); CLASSERT(offsetof(typeof(*ocd), padding4) != 0); CLASSERT(offsetof(typeof(*ocd), padding5) != 0); diff --git a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c index 04e9471..f50f487 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c +++ b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c @@ -935,10 +935,10 @@ void lustre_assert_wire_constants(void) (long long)(int)offsetof(struct obd_connect_data, padding1)); LASSERTF((int)sizeof(((struct obd_connect_data *)0)->padding1) == 4, "found %lld\n", (long long)(int)sizeof(((struct obd_connect_data *)0)->padding1)); - LASSERTF((int)offsetof(struct obd_connect_data, padding2) == 80, "found %lld\n", - (long long)(int)offsetof(struct obd_connect_data, padding2)); - LASSERTF((int)sizeof(((struct obd_connect_data *)0)->padding2) == 8, "found %lld\n", - (long long)(int)sizeof(((struct obd_connect_data *)0)->padding2)); + LASSERTF((int)offsetof(struct obd_connect_data, ocd_connect_flags2) == 80, "found %lld\n", + (long long)(int)offsetof(struct obd_connect_data, ocd_connect_flags2)); + LASSERTF((int)sizeof(((struct obd_connect_data *)0)->ocd_connect_flags2) == 8, "found %lld\n", + (long long)(int)sizeof(((struct obd_connect_data *)0)->ocd_connect_flags2)); LASSERTF((int)offsetof(struct obd_connect_data, padding3) == 88, "found %lld\n", (long long)(int)offsetof(struct obd_connect_data, padding3)); LASSERTF((int)sizeof(((struct obd_connect_data *)0)->padding3) == 8, "found %lld\n", @@ -1111,6 +1111,8 @@ void lustre_assert_wire_constants(void) OBD_CONNECT_LOCK_AHEAD); LASSERTF(OBD_CONNECT_OBDOPACK == 0x4000000000000000ULL, "found 0x%.16llxULL\n", OBD_CONNECT_OBDOPACK); + LASSERTF(OBD_CONNECT_FLAGS2 == 0x8000000000000000ULL, "found 0x%.16llxULL\n", + OBD_CONNECT_FLAGS2); LASSERTF(OBD_CKSUM_CRC32 == 0x00000001UL, "found 0x%.8xUL\n", (unsigned)OBD_CKSUM_CRC32); LASSERTF(OBD_CKSUM_ADLER == 0x00000002UL, "found 0x%.8xUL\n", -- 1.7.1 From jsimmons at infradead.org Wed Nov 23 23:01:18 2016 From: jsimmons at infradead.org (James Simmons) Date: Wed, 23 Nov 2016 18:01:18 -0500 Subject: [lustre-devel] [PATCH 2/4] staging: lustre: obd: reserve connection flag OBD_CONNECT_LOCK_AHEAD In-Reply-To: <1479942080-30166-1-git-send-email-jsimmons@infradead.org> References: <1479942080-30166-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479942080-30166-3-git-send-email-jsimmons@infradead.org> From: Fan Yong The connection flag OBD_CONNECT_LOCK_AHEAD will be used for the following the patch: LU-6917 LDLM lock ahead http://review.whamcloud.com/13564 Land the connection flags to upstream client earlier for reserving the slot to avoid potential conflict with others. Signed-off-by: Fan Yong Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-7543 Reviewed-on: http://review.whamcloud.com/17645 Reviewed-by: Andreas Dilger Reviewed-by: Patrick Farrell Signed-off-by: James Simmons --- .../lustre/lustre/include/lustre/lustre_idl.h | 1 + .../lustre/lustre/obdclass/lprocfs_status.c | 1 + drivers/staging/lustre/lustre/ptlrpc/wiretest.c | 2 ++ 3 files changed, 4 insertions(+), 0 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h index eb0dce6..1079d0a 100644 --- a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h +++ b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h @@ -1190,6 +1190,7 @@ struct ptlrpc_body_v2 { */ #define OBD_CONNECT_DIR_STRIPE 0x400000000000000ULL/* striped DNE dir */ #define OBD_CONNECT_SUBTREE 0x800000000000000ULL /* fileset mount */ +#define OBD_CONNECT_LOCK_AHEAD 0x1000000000000000ULL /* lock ahead */ /** bulk matchbits is sent within ptlrpc_body */ #define OBD_CONNECT_BULK_MBITS 0x2000000000000000ULL diff --git a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c index f78928f..5e341e3 100644 --- a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c +++ b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c @@ -103,6 +103,7 @@ "multi_mod_rpcs", "dir_stripe", "subtree", + "lock_ahead", "bulk_mbits", "unknown", NULL diff --git a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c index 5619f11..afa79d5 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c +++ b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c @@ -1107,6 +1107,8 @@ void lustre_assert_wire_constants(void) OBD_CONNECT_DIR_STRIPE); LASSERTF(OBD_CONNECT_SUBTREE == 0x800000000000000ULL, "found 0x%.16llxULL\n", OBD_CONNECT_SUBTREE); + LASSERTF(OBD_CONNECT_LOCK_AHEAD == 0x1000000000000000ULL, "found 0x%.16llxULL\n", + OBD_CONNECT_LOCK_AHEAD); LASSERTF(OBD_CKSUM_CRC32 == 0x00000001UL, "found 0x%.8xUL\n", (unsigned)OBD_CKSUM_CRC32); LASSERTF(OBD_CKSUM_ADLER == 0x00000002UL, "found 0x%.8xUL\n", -- 1.7.1 From jsimmons at infradead.org Wed Nov 23 23:01:17 2016 From: jsimmons at infradead.org (James Simmons) Date: Wed, 23 Nov 2016 18:01:17 -0500 Subject: [lustre-devel] [PATCH 1/4] staging: lustre: obd: reserve connection flag OBD_CONNECT_SUBTREE In-Reply-To: <1479942080-30166-1-git-send-email-jsimmons@infradead.org> References: <1479942080-30166-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479942080-30166-2-git-send-email-jsimmons@infradead.org> From: Fan Yong The connection flag OBD_CONNECT_SUBTREE will be used for the following the patch: LU-28 mounting of filesystem from MDS http://review.whamcloud.com/5007 Land the connection flags to master earlier for reserving the slot to avoid potential conflict with others. Signed-off-by: Fan Yong Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-7543 Reviewed-on: http://review.whamcloud.com/17644 Reviewed-by: Andreas Dilger Reviewed-by: Patrick Farrell Signed-off-by: James Simmons --- .../lustre/lustre/include/lustre/lustre_idl.h | 1 + .../lustre/lustre/obdclass/lprocfs_status.c | 1 + drivers/staging/lustre/lustre/ptlrpc/wiretest.c | 2 ++ 3 files changed, 4 insertions(+), 0 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h index 5c1fb6c..eb0dce6 100644 --- a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h +++ b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h @@ -1189,6 +1189,7 @@ struct ptlrpc_body_v2 { * RPCs in parallel */ #define OBD_CONNECT_DIR_STRIPE 0x400000000000000ULL/* striped DNE dir */ +#define OBD_CONNECT_SUBTREE 0x800000000000000ULL /* fileset mount */ /** bulk matchbits is sent within ptlrpc_body */ #define OBD_CONNECT_BULK_MBITS 0x2000000000000000ULL diff --git a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c index db49992..f78928f 100644 --- a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c +++ b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c @@ -102,6 +102,7 @@ "unlink_close", "multi_mod_rpcs", "dir_stripe", + "subtree", "bulk_mbits", "unknown", NULL diff --git a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c index b239563..5619f11 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c +++ b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c @@ -1105,6 +1105,8 @@ void lustre_assert_wire_constants(void) OBD_CONNECT_MULTIMODRPCS); LASSERTF(OBD_CONNECT_DIR_STRIPE == 0x400000000000000ULL, "found 0x%.16llxULL\n", OBD_CONNECT_DIR_STRIPE); + LASSERTF(OBD_CONNECT_SUBTREE == 0x800000000000000ULL, "found 0x%.16llxULL\n", + OBD_CONNECT_SUBTREE); LASSERTF(OBD_CKSUM_CRC32 == 0x00000001UL, "found 0x%.8xUL\n", (unsigned)OBD_CKSUM_CRC32); LASSERTF(OBD_CKSUM_ADLER == 0x00000002UL, "found 0x%.8xUL\n", -- 1.7.1 From jsimmons at infradead.org Wed Nov 23 23:01:19 2016 From: jsimmons at infradead.org (James Simmons) Date: Wed, 23 Nov 2016 18:01:19 -0500 Subject: [lustre-devel] [PATCH 3/4] staging: lustre: obd: reserve connection flag OBD_CONNECT_OBDOPACK In-Reply-To: <1479942080-30166-1-git-send-email-jsimmons@infradead.org> References: <1479942080-30166-1-git-send-email-jsimmons@infradead.org> Message-ID: <1479942080-30166-4-git-send-email-jsimmons@infradead.org> From: Fan Yong The connection flag OBD_CONNECT_OBDOPACK will be used for the following the patch: LU-4215 optimize OUT protocol http://review.whamcloud.com/15336 Land the connection flags to upstream client earlier for reserving the slot to avoid potential conflict with others. Signed-off-by: Fan Yong Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-7543 Reviewed-on: http://review.whamcloud.com/17646 Reviewed-by: Andreas Dilger Reviewed-by: Patrick Farrell Signed-off-by: James Simmons --- .../lustre/lustre/include/lustre/lustre_idl.h | 1 + .../lustre/lustre/obdclass/lprocfs_status.c | 2 +- drivers/staging/lustre/lustre/ptlrpc/wiretest.c | 2 ++ 3 files changed, 4 insertions(+), 1 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h index 1079d0a..a4020d2 100644 --- a/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h +++ b/drivers/staging/lustre/lustre/include/lustre/lustre_idl.h @@ -1193,6 +1193,7 @@ struct ptlrpc_body_v2 { #define OBD_CONNECT_LOCK_AHEAD 0x1000000000000000ULL /* lock ahead */ /** bulk matchbits is sent within ptlrpc_body */ #define OBD_CONNECT_BULK_MBITS 0x2000000000000000ULL +#define OBD_CONNECT_OBDOPACK 0x4000000000000000ULL /* compact OUT obdo */ /* XXX README XXX: * Please DO NOT add flag values here before first ensuring that this same diff --git a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c index 5e341e3..85b505e 100644 --- a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c +++ b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c @@ -105,7 +105,7 @@ "subtree", "lock_ahead", "bulk_mbits", - "unknown", + "compact_obdo", NULL }; diff --git a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c index afa79d5..04e9471 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/wiretest.c +++ b/drivers/staging/lustre/lustre/ptlrpc/wiretest.c @@ -1109,6 +1109,8 @@ void lustre_assert_wire_constants(void) OBD_CONNECT_SUBTREE); LASSERTF(OBD_CONNECT_LOCK_AHEAD == 0x1000000000000000ULL, "found 0x%.16llxULL\n", OBD_CONNECT_LOCK_AHEAD); + LASSERTF(OBD_CONNECT_OBDOPACK == 0x4000000000000000ULL, "found 0x%.16llxULL\n", + OBD_CONNECT_OBDOPACK); LASSERTF(OBD_CKSUM_CRC32 == 0x00000001UL, "found 0x%.8xUL\n", (unsigned)OBD_CKSUM_CRC32); LASSERTF(OBD_CKSUM_ADLER == 0x00000002UL, "found 0x%.8xUL\n", -- 1.7.1 From jsimmons at infradead.org Wed Nov 23 23:01:45 2016 From: jsimmons at infradead.org (James Simmons) Date: Wed, 23 Nov 2016 18:01:45 -0500 Subject: [lustre-devel] [PATCH v2] staging: lustre: osc: Performance tune for LRU Message-ID: <1479942105-30220-1-git-send-email-jsimmons@infradead.org> From: Jinshan Xiong Early launch page LRU work in osc_io_rw_iter_init(); Change the page LRU shrinking policy by OSC attributes; Delete the contented lock osc_object::oo_seatbelt Signed-off-by: Jinshan Xiong Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-5108 Reviewed-on: http://review.whamcloud.com/10458 Reviewed-by: Bobi Jam Reviewed-by: Fan Yong Reviewed-by: Oleg Drokin Signed-off-by: James Simmons --- ChangeLog: v1) Initial patch v2) Fix up time64_t handling to work on 32 bit platforms drivers/staging/lustre/lustre/include/obd.h | 35 ++++- drivers/staging/lustre/lustre/osc/lproc_osc.c | 6 +- drivers/staging/lustre/lustre/osc/osc_cache.c | 34 ++--- .../staging/lustre/lustre/osc/osc_cl_internal.h | 19 --- drivers/staging/lustre/lustre/osc/osc_internal.h | 7 +- drivers/staging/lustre/lustre/osc/osc_io.c | 13 ++- drivers/staging/lustre/lustre/osc/osc_object.c | 9 -- drivers/staging/lustre/lustre/osc/osc_page.c | 148 ++++++++++++-------- drivers/staging/lustre/lustre/osc/osc_quota.c | 4 +- drivers/staging/lustre/lustre/osc/osc_request.c | 17 +-- 10 files changed, 162 insertions(+), 130 deletions(-) diff --git a/drivers/staging/lustre/lustre/include/obd.h b/drivers/staging/lustre/lustre/include/obd.h index 60efaaa..9803403 100644 --- a/drivers/staging/lustre/lustre/include/obd.h +++ b/drivers/staging/lustre/lustre/include/obd.h @@ -247,15 +247,42 @@ struct client_obd { struct obd_histogram cl_read_offset_hist; struct obd_histogram cl_write_offset_hist; - /* lru for osc caching pages */ + /* LRU for osc caching pages */ struct cl_client_cache *cl_cache; - struct list_head cl_lru_osc; /* member of cl_cache->ccc_lru */ + /** member of cl_cache->ccc_lru */ + struct list_head cl_lru_osc; + /** # of available LRU slots left in the per-OSC cache. + * Available LRU slots are shared by all OSCs of the same file system, + * therefore this is a pointer to cl_client_cache::ccc_lru_left. + */ atomic_long_t *cl_lru_left; + /** # of busy LRU pages. A page is considered busy if it's in writeback + * queue, or in transfer. Busy pages can't be discarded so they are not + * in LRU cache. + */ atomic_long_t cl_lru_busy; + /** # of LRU pages in the cache for this client_obd */ atomic_long_t cl_lru_in_list; + /** # of threads are shrinking LRU cache. To avoid contention, it's not + * allowed to have multiple threads shrinking LRU cache. + */ atomic_t cl_lru_shrinkers; - struct list_head cl_lru_list; /* lru page list */ - spinlock_t cl_lru_list_lock; /* page list protector */ + /** The time when this LRU cache was last used. */ + time64_t cl_lru_last_used; + /** stats: how many reclaims have happened for this client_obd. + * reclaim and shrink - shrink is async, voluntarily rebalancing; + * reclaim is sync, initiated by IO thread when the LRU slots are + * in shortage. + */ + u64 cl_lru_reclaim; + /** List of LRU pages for this client_obd */ + struct list_head cl_lru_list; + /** Lock for LRU page list */ + spinlock_t cl_lru_list_lock; + /** # of unstable pages in this client_obd. + * An unstable page is a page state that WRITE RPC has finished but + * the transaction has NOT yet committed. + */ atomic_long_t cl_unstable_count; /* number of in flight destroy rpcs is limited to max_rpcs_in_flight */ diff --git a/drivers/staging/lustre/lustre/osc/lproc_osc.c b/drivers/staging/lustre/lustre/osc/lproc_osc.c index 4f56a49..575b296 100644 --- a/drivers/staging/lustre/lustre/osc/lproc_osc.c +++ b/drivers/staging/lustre/lustre/osc/lproc_osc.c @@ -183,10 +183,12 @@ static int osc_cached_mb_seq_show(struct seq_file *m, void *v) seq_printf(m, "used_mb: %ld\n" - "busy_cnt: %ld\n", + "busy_cnt: %ld\n" + "reclaim: %llu\n", (atomic_long_read(&cli->cl_lru_in_list) + atomic_long_read(&cli->cl_lru_busy)) >> shift, - atomic_long_read(&cli->cl_lru_busy)); + atomic_long_read(&cli->cl_lru_busy), + cli->cl_lru_reclaim); return 0; } diff --git a/drivers/staging/lustre/lustre/osc/osc_cache.c b/drivers/staging/lustre/lustre/osc/osc_cache.c index 0b5f4b2..b0f030c 100644 --- a/drivers/staging/lustre/lustre/osc/osc_cache.c +++ b/drivers/staging/lustre/lustre/osc/osc_cache.c @@ -961,7 +961,7 @@ static int osc_extent_wait(const struct lu_env *env, struct osc_extent *ext, if (rc == -ETIMEDOUT) { OSC_EXTENT_DUMP(D_ERROR, ext, "%s: wait ext to %u timedout, recovery in progress?\n", - osc_export(obj)->exp_obd->obd_name, state); + cli_name(osc_cli(obj)), state); lwi = LWI_INTR(NULL, NULL); rc = l_wait_event(ext->oe_waitq, extent_wait_cb(ext, state), @@ -1329,7 +1329,6 @@ static int osc_completion(const struct lu_env *env, struct osc_async_page *oap, { struct osc_page *opg = oap2osc_page(oap); struct cl_page *page = oap2cl_page(oap); - struct osc_object *obj = cl2osc(opg->ops_cl.cpl_obj); enum cl_req_type crt; int srvlock; @@ -1344,13 +1343,6 @@ static int osc_completion(const struct lu_env *env, struct osc_async_page *oap, /* Clear opg->ops_transfer_pinned before VM lock is released. */ opg->ops_transfer_pinned = 0; - spin_lock(&obj->oo_seatbelt); - LASSERT(opg->ops_submitter); - LASSERT(!list_empty(&opg->ops_inflight)); - list_del_init(&opg->ops_inflight); - opg->ops_submitter = NULL; - spin_unlock(&obj->oo_seatbelt); - opg->ops_submit_time = 0; srvlock = oap->oap_brw_flags & OBD_BRW_SRVLOCK; @@ -1381,10 +1373,10 @@ static int osc_completion(const struct lu_env *env, struct osc_async_page *oap, #define OSC_DUMP_GRANT(lvl, cli, fmt, args...) do { \ struct client_obd *__tmp = (cli); \ - CDEBUG(lvl, "%s: grant { dirty: %ld/%ld dirty_pages: %ld/%lu " \ + CDEBUG(lvl, "%s: grant { dirty: %lu/%lu dirty_pages: %ld/%lu " \ "dropped: %ld avail: %ld, reserved: %ld, flight: %d }" \ "lru {in list: %ld, left: %ld, waiters: %d }" fmt "\n", \ - __tmp->cl_import->imp_obd->obd_name, \ + cli_name(__tmp), \ __tmp->cl_dirty_pages, __tmp->cl_dirty_max_pages, \ atomic_long_read(&obd_dirty_pages), obd_max_dirty_pages, \ __tmp->cl_lost_grant, __tmp->cl_avail_grant, \ @@ -1622,7 +1614,7 @@ static int osc_enter_cache(const struct lu_env *env, struct client_obd *cli, osc_io_unplug_async(env, cli, NULL); CDEBUG(D_CACHE, "%s: sleeping for cache space @ %p for %p\n", - cli->cl_import->imp_obd->obd_name, &ocw, oap); + cli_name(cli), &ocw, oap); rc = l_wait_event(ocw.ocw_waitq, ocw_granted(cli, &ocw), &lwi); @@ -1666,7 +1658,7 @@ static int osc_enter_cache(const struct lu_env *env, struct client_obd *cli, break; default: CDEBUG(D_CACHE, "%s: event for cache space @ %p never arrived due to %d, fall back to sync i/o\n", - cli->cl_import->imp_obd->obd_name, &ocw, rc); + cli_name(cli), &ocw, rc); break; } out: @@ -2246,14 +2238,9 @@ static int osc_io_unplug0(const struct lu_env *env, struct client_obd *cli, return 0; if (!async) { - /* disable osc_lru_shrink() temporarily to avoid - * potential stack overrun problem. LU-2859 - */ - atomic_inc(&cli->cl_lru_shrinkers); spin_lock(&cli->cl_loi_list_lock); osc_check_rpcs(env, cli); spin_unlock(&cli->cl_loi_list_lock); - atomic_dec(&cli->cl_lru_shrinkers); } else { CDEBUG(D_CACHE, "Queue writeback work for client %p.\n", cli); LASSERT(cli->cl_writeback_work); @@ -2475,7 +2462,6 @@ int osc_teardown_async_page(const struct lu_env *env, struct osc_object *obj, struct osc_page *ops) { struct osc_async_page *oap = &ops->ops_oap; - struct osc_extent *ext = NULL; int rc = 0; LASSERT(oap->oap_magic == OAP_MAGIC); @@ -2483,12 +2469,15 @@ int osc_teardown_async_page(const struct lu_env *env, CDEBUG(D_INFO, "teardown oap %p page %p at index %lu.\n", oap, ops, osc_index(oap2osc(oap))); - osc_object_lock(obj); if (!list_empty(&oap->oap_rpc_item)) { CDEBUG(D_CACHE, "oap %p is not in cache.\n", oap); rc = -EBUSY; } else if (!list_empty(&oap->oap_pending_item)) { + struct osc_extent *ext = NULL; + + osc_object_lock(obj); ext = osc_extent_lookup(obj, osc_index(oap2osc(oap))); + osc_object_unlock(obj); /* only truncated pages are allowed to be taken out. * See osc_extent_truncate() and osc_cache_truncate_start() * for details. @@ -2498,10 +2487,9 @@ int osc_teardown_async_page(const struct lu_env *env, osc_index(oap2osc(oap))); rc = -EBUSY; } + if (ext) + osc_extent_put(env, ext); } - osc_object_unlock(obj); - if (ext) - osc_extent_put(env, ext); return rc; } diff --git a/drivers/staging/lustre/lustre/osc/osc_cl_internal.h b/drivers/staging/lustre/lustre/osc/osc_cl_internal.h index c9470d1..e4a214d 100644 --- a/drivers/staging/lustre/lustre/osc/osc_cl_internal.h +++ b/drivers/staging/lustre/lustre/osc/osc_cl_internal.h @@ -120,16 +120,6 @@ struct osc_object { int oo_contended; unsigned long oo_contention_time; /** - * List of pages in transfer. - */ - struct list_head oo_inflight[CRT_NR]; - /** - * Lock, protecting osc_page::ops_inflight, because a seat-belt is - * locked during take-off and landing. - */ - spinlock_t oo_seatbelt; - - /** * used by the osc to keep track of what objects to build into rpcs. * Protected by client_obd->cli_loi_list_lock. */ @@ -357,15 +347,6 @@ struct osc_page { */ struct list_head ops_lru; /** - * Linkage into a per-osc_object list of pages in flight. For - * debugging. - */ - struct list_head ops_inflight; - /** - * Thread that submitted this page for transfer. For debugging. - */ - struct task_struct *ops_submitter; - /** * Submit time - the time when the page is starting RPC. For debugging. */ unsigned long ops_submit_time; diff --git a/drivers/staging/lustre/lustre/osc/osc_internal.h b/drivers/staging/lustre/lustre/osc/osc_internal.h index 12d3b58..0e181e2 100644 --- a/drivers/staging/lustre/lustre/osc/osc_internal.h +++ b/drivers/staging/lustre/lustre/osc/osc_internal.h @@ -133,7 +133,7 @@ int osc_build_rpc(const struct lu_env *env, struct client_obd *cli, struct list_head *ext_list, int cmd); long osc_lru_shrink(const struct lu_env *env, struct client_obd *cli, long target, bool force); -long osc_lru_reclaim(struct client_obd *cli); +long osc_lru_reclaim(struct client_obd *cli, unsigned long npages); unsigned long osc_ldlm_weigh_ast(struct ldlm_lock *dlmlock); @@ -155,6 +155,11 @@ static inline unsigned long rpcs_in_flight(struct client_obd *cli) return cli->cl_r_in_flight + cli->cl_w_in_flight; } +static inline char *cli_name(struct client_obd *cli) +{ + return cli->cl_import->imp_obd->obd_name; +} + struct osc_device { struct cl_device od_cl; struct obd_export *od_exp; diff --git a/drivers/staging/lustre/lustre/osc/osc_io.c b/drivers/staging/lustre/lustre/osc/osc_io.c index 97b012d..439ba64 100644 --- a/drivers/staging/lustre/lustre/osc/osc_io.c +++ b/drivers/staging/lustre/lustre/osc/osc_io.c @@ -352,7 +352,7 @@ static int osc_io_rw_iter_init(const struct lu_env *env, npages = max_pages; c = atomic_long_read(cli->cl_lru_left); - if (c < npages && osc_lru_reclaim(cli) > 0) + if (c < npages && osc_lru_reclaim(cli, npages) > 0) c = atomic_long_read(cli->cl_lru_left); while (c >= npages) { if (c == atomic_long_cmpxchg(cli->cl_lru_left, c, c - npages)) { @@ -361,6 +361,17 @@ static int osc_io_rw_iter_init(const struct lu_env *env, } c = atomic_long_read(cli->cl_lru_left); } + if (atomic_long_read(cli->cl_lru_left) < max_pages) { + /* + * If there aren't enough pages in the per-OSC LRU then + * wake up the LRU thread to try and clear out space, so + * we don't block if pages are being dirtied quickly. + */ + CDEBUG(D_CACHE, "%s: queue LRU, left: %lu/%ld.\n", + cli_name(cli), atomic_long_read(cli->cl_lru_left), + max_pages); + (void)ptlrpcd_queue_work(cli->cl_lru_work); + } return 0; } diff --git a/drivers/staging/lustre/lustre/osc/osc_object.c b/drivers/staging/lustre/lustre/osc/osc_object.c index ab1c11c..ca1f8cd 100644 --- a/drivers/staging/lustre/lustre/osc/osc_object.c +++ b/drivers/staging/lustre/lustre/osc/osc_object.c @@ -71,13 +71,8 @@ static int osc_object_init(const struct lu_env *env, struct lu_object *obj, { struct osc_object *osc = lu2osc(obj); const struct cl_object_conf *cconf = lu2cl_conf(conf); - int i; osc->oo_oinfo = cconf->u.coc_oinfo; - spin_lock_init(&osc->oo_seatbelt); - for (i = 0; i < CRT_NR; ++i) - INIT_LIST_HEAD(&osc->oo_inflight[i]); - INIT_LIST_HEAD(&osc->oo_ready_item); INIT_LIST_HEAD(&osc->oo_hp_ready_item); INIT_LIST_HEAD(&osc->oo_write_item); @@ -103,10 +98,6 @@ static int osc_object_init(const struct lu_env *env, struct lu_object *obj, static void osc_object_free(const struct lu_env *env, struct lu_object *obj) { struct osc_object *osc = lu2osc(obj); - int i; - - for (i = 0; i < CRT_NR; ++i) - LASSERT(list_empty(&osc->oo_inflight[i])); LASSERT(list_empty(&osc->oo_ready_item)); LASSERT(list_empty(&osc->oo_hp_ready_item)); diff --git a/drivers/staging/lustre/lustre/osc/osc_page.c b/drivers/staging/lustre/lustre/osc/osc_page.c index 93248d1..6f4751b 100644 --- a/drivers/staging/lustre/lustre/osc/osc_page.c +++ b/drivers/staging/lustre/lustre/osc/osc_page.c @@ -37,6 +37,7 @@ #define DEBUG_SUBSYSTEM S_OSC +#include #include "osc_cl_internal.h" static void osc_lru_del(struct client_obd *cli, struct osc_page *opg); @@ -86,11 +87,6 @@ static void osc_page_transfer_add(const struct lu_env *env, struct osc_object *obj = cl2osc(opg->ops_cl.cpl_obj); osc_lru_use(osc_cli(obj), opg); - - spin_lock(&obj->oo_seatbelt); - list_add(&opg->ops_inflight, &obj->oo_inflight[crt]); - opg->ops_submitter = current; - spin_unlock(&obj->oo_seatbelt); } int osc_page_cache_add(const struct lu_env *env, @@ -139,7 +135,7 @@ static int osc_page_print(const struct lu_env *env, struct osc_object *obj = cl2osc(slice->cpl_obj); struct client_obd *cli = &osc_export(obj)->exp_obd->u.cli; - return (*printer)(env, cookie, LUSTRE_OSC_NAME "-page@%p %lu: 1< %#x %d %u %s %s > 2< %llu %u %u %#x %#x | %p %p %p > 3< %s %p %d %lu %d > 4< %d %d %d %lu %s | %s %s %s %s > 5< %s %s %s %s | %d %s | %d %s %s>\n", + return (*printer)(env, cookie, LUSTRE_OSC_NAME "-page@%p %lu: 1< %#x %d %u %s %s > 2< %llu %u %u %#x %#x | %p %p %p > 3< %d %lu %d > 4< %d %d %d %lu %s | %s %s %s %s > 5< %s %s %s %s | %d %s | %d %s %s>\n", opg, osc_index(opg), /* 1 */ oap->oap_magic, oap->oap_cmd, @@ -151,8 +147,7 @@ static int osc_page_print(const struct lu_env *env, oap->oap_async_flags, oap->oap_brw_flags, oap->oap_request, oap->oap_cli, obj, /* 3 */ - osc_list(&opg->ops_inflight), - opg->ops_submitter, opg->ops_transfer_pinned, + opg->ops_transfer_pinned, osc_submit_duration(opg), opg->ops_srvlock, /* 4 */ cli->cl_r_in_flight, cli->cl_w_in_flight, @@ -191,14 +186,6 @@ static void osc_page_delete(const struct lu_env *env, LASSERT(0); } - spin_lock(&obj->oo_seatbelt); - if (opg->ops_submitter) { - LASSERT(!list_empty(&opg->ops_inflight)); - list_del_init(&opg->ops_inflight); - opg->ops_submitter = NULL; - } - spin_unlock(&obj->oo_seatbelt); - osc_lru_del(osc_cli(obj), opg); if (slice->cpl_page->cp_type == CPT_CACHEABLE) { @@ -281,10 +268,6 @@ int osc_page_init(const struct lu_env *env, struct cl_object *obj, cl_page_slice_add(page, &opg->ops_cl, obj, index, &osc_page_ops); } - /* ops_inflight and ops_lru are the same field, but it doesn't - * hurt to initialize it twice :-) - */ - INIT_LIST_HEAD(&opg->ops_inflight); INIT_LIST_HEAD(&opg->ops_lru); /* reserve an LRU space for this page */ @@ -342,16 +325,27 @@ void osc_page_submit(const struct lu_env *env, struct osc_page *opg, * OSC to free slots voluntarily to maintain a reasonable number of free slots * at any time. */ - static DECLARE_WAIT_QUEUE_HEAD(osc_lru_waitq); -/* LRU pages are freed in batch mode. OSC should at least free this - * number of pages to avoid running out of LRU budget, and.. + +/** + * LRU pages are freed in batch mode. OSC should at least free this + * number of pages to avoid running out of LRU slots. */ -static const int lru_shrink_min = 2 << (20 - PAGE_SHIFT); /* 2M */ -/* free this number at most otherwise it will take too long time to finish. */ -static const int lru_shrink_max = 8 << (20 - PAGE_SHIFT); /* 8M */ +static inline int lru_shrink_min(struct client_obd *cli) +{ + return cli->cl_max_pages_per_rpc * 2; +} + +/** + * free this number at most otherwise it will take too long time to finish. + */ +static inline int lru_shrink_max(struct client_obd *cli) +{ + return cli->cl_max_pages_per_rpc * cli->cl_max_rpcs_in_flight; +} -/* Check if we can free LRU slots from this OSC. If there exists LRU waiters, +/** + * Check if we can free LRU slots from this OSC. If there exists LRU waiters, * we should free slots aggressively. In this way, slots are freed in a steady * step to maintain fairness among OSCs. * @@ -368,13 +362,20 @@ static int osc_cache_too_much(struct client_obd *cli) /* if it's going to run out LRU slots, we should free some, but not * too much to maintain fairness among OSCs. */ - if (atomic_long_read(cli->cl_lru_left) < cache->ccc_lru_max >> 4) { + if (atomic_long_read(cli->cl_lru_left) < cache->ccc_lru_max >> 2) { if (pages >= budget) - return lru_shrink_max; + return lru_shrink_max(cli); else if (pages >= budget / 2) - return lru_shrink_min; - } else if (pages >= budget * 2) { - return lru_shrink_min; + return lru_shrink_min(cli); + } else { + time64_t duration = ktime_get_real_seconds(); + + /* knock out pages by duration of no IO activity */ + duration =- cli->cl_lru_last_used; + duration >>= 6; /* approximately 1 minute */ + if (duration > 0 && + pages >= div64_s64((s64)budget, duration)) + return lru_shrink_min(cli); } return 0; } @@ -382,11 +383,21 @@ static int osc_cache_too_much(struct client_obd *cli) int lru_queue_work(const struct lu_env *env, void *data) { struct client_obd *cli = data; + int count; - CDEBUG(D_CACHE, "Run LRU work for client obd %p.\n", cli); + CDEBUG(D_CACHE, "%s: run LRU work for client obd\n", cli_name(cli)); - if (osc_cache_too_much(cli)) - osc_lru_shrink(env, cli, lru_shrink_max, true); + count = osc_cache_too_much(cli); + if (count > 0) { + int rc = osc_lru_shrink(env, cli, count, false); + + CDEBUG(D_CACHE, "%s: shrank %d/%d pages from client obd\n", + cli_name(cli), rc, count); + if (rc >= count) { + CDEBUG(D_CACHE, "%s: queue again\n", cli_name(cli)); + ptlrpcd_queue_work(cli->cl_lru_work); + } + } return 0; } @@ -413,10 +424,10 @@ void osc_lru_add_batch(struct client_obd *cli, struct list_head *plist) list_splice_tail(&lru, &cli->cl_lru_list); atomic_long_sub(npages, &cli->cl_lru_busy); atomic_long_add(npages, &cli->cl_lru_in_list); + cli->cl_lru_last_used = ktime_get_real_seconds(); spin_unlock(&cli->cl_lru_list_lock); - /* XXX: May set force to be true for better performance */ - if (osc_cache_too_much(cli)) + if (waitqueue_active(&osc_lru_waitq)) (void)ptlrpcd_queue_work(cli->cl_lru_work); } } @@ -449,8 +460,10 @@ static void osc_lru_del(struct client_obd *cli, struct osc_page *opg) * this osc occupies too many LRU pages and kernel is * stealing one of them. */ - if (!memory_pressure_get()) + if (osc_cache_too_much(cli)) { + CDEBUG(D_CACHE, "%s: queue LRU workn", cli_name(cli)); (void)ptlrpcd_queue_work(cli->cl_lru_work); + } wake_up(&osc_lru_waitq); } else { LASSERT(list_empty(&opg->ops_lru)); @@ -482,6 +495,7 @@ static void discard_pagevec(const struct lu_env *env, struct cl_io *io, struct cl_page *page = pvec[i]; LASSERT(cl_page_is_owned(page, io)); + cl_page_delete(env, page); cl_page_discard(env, io, page); cl_page_disown(env, io, page); cl_page_put(env, page); @@ -532,6 +546,8 @@ long osc_lru_shrink(const struct lu_env *env, struct client_obd *cli, if (atomic_long_read(&cli->cl_lru_in_list) == 0 || target <= 0) return 0; + CDEBUG(D_CACHE, "%s: shrinkers: %d, force: %d\n", + cli_name(cli), atomic_read(&cli->cl_lru_shrinkers), force); if (!force) { if (atomic_read(&cli->cl_lru_shrinkers) > 0) return -EBUSY; @@ -548,11 +564,16 @@ long osc_lru_shrink(const struct lu_env *env, struct client_obd *cli, io = &osc_env_info(env)->oti_io; spin_lock(&cli->cl_lru_list_lock); + if (force) + cli->cl_lru_reclaim++; maxscan = min(target << 1, atomic_long_read(&cli->cl_lru_in_list)); list_for_each_entry_safe(opg, temp, &cli->cl_lru_list, ops_lru) { struct cl_page *page; bool will_free = false; + if (!force && atomic_read(&cli->cl_lru_shrinkers) > 1) + break; + if (--maxscan < 0) break; @@ -642,7 +663,13 @@ long osc_lru_shrink(const struct lu_env *env, struct client_obd *cli, return count > 0 ? count : rc; } -long osc_lru_reclaim(struct client_obd *cli) +/** + * Reclaim LRU pages by an IO thread. The caller wants to reclaim at least + * \@npages of LRU slots. For performance consideration, it's better to drop + * LRU pages in batch. Therefore, the actual number is adjusted at least + * max_pages_per_rpc. + */ +long osc_lru_reclaim(struct client_obd *cli, unsigned long npages) { struct lu_env *env; struct cl_client_cache *cache = cli->cl_cache; @@ -656,20 +683,23 @@ long osc_lru_reclaim(struct client_obd *cli) if (IS_ERR(env)) return 0; - rc = osc_lru_shrink(env, cli, osc_cache_too_much(cli), false); - if (rc != 0) { - if (rc == -EBUSY) - rc = 0; - - CDEBUG(D_CACHE, "%s: Free %ld pages from own LRU: %p.\n", - cli->cl_import->imp_obd->obd_name, rc, cli); + npages = max_t(int, npages, cli->cl_max_pages_per_rpc); + CDEBUG(D_CACHE, "%s: start to reclaim %ld pages from LRU\n", + cli_name(cli), npages); + rc = osc_lru_shrink(env, cli, npages, true); + if (rc >= npages) { + CDEBUG(D_CACHE, "%s: reclaimed %ld/%ld pages from LRU\n", + cli_name(cli), rc, npages); + if (osc_cache_too_much(cli) > 0) + ptlrpcd_queue_work(cli->cl_lru_work); goto out; + } else if (rc > 0) { + npages -= rc; } - CDEBUG(D_CACHE, "%s: cli %p no free slots, pages: %ld, busy: %ld.\n", - cli->cl_import->imp_obd->obd_name, cli, - atomic_long_read(&cli->cl_lru_in_list), - atomic_long_read(&cli->cl_lru_busy)); + CDEBUG(D_CACHE, "%s: cli %p no free slots, pages: %ld/%ld, want: %ld\n", + cli_name(cli), cli, atomic_long_read(&cli->cl_lru_in_list), + atomic_long_read(&cli->cl_lru_busy), npages); /* Reclaim LRU slots from other client_obd as it can't free enough * from its own. This should rarely happen. @@ -686,7 +716,7 @@ long osc_lru_reclaim(struct client_obd *cli) cl_lru_osc); CDEBUG(D_CACHE, "%s: cli %p LRU pages: %ld, busy: %ld.\n", - cli->cl_import->imp_obd->obd_name, cli, + cli_name(cli), cli, atomic_long_read(&cli->cl_lru_in_list), atomic_long_read(&cli->cl_lru_busy)); @@ -694,11 +724,12 @@ long osc_lru_reclaim(struct client_obd *cli) if (osc_cache_too_much(cli) > 0) { spin_unlock(&cache->ccc_lru_lock); - rc = osc_lru_shrink(env, cli, osc_cache_too_much(cli), - true); + rc = osc_lru_shrink(env, cli, npages, true); spin_lock(&cache->ccc_lru_lock); - if (rc != 0) + if (rc >= npages) break; + if (rc > 0) + npages -= rc; } } spin_unlock(&cache->ccc_lru_lock); @@ -706,7 +737,7 @@ long osc_lru_reclaim(struct client_obd *cli) out: cl_env_put(env, &refcheck); CDEBUG(D_CACHE, "%s: cli %p freed %ld pages.\n", - cli->cl_import->imp_obd->obd_name, cli, rc); + cli_name(cli), cli, rc); return rc; } @@ -736,7 +767,7 @@ static int osc_lru_reserve(const struct lu_env *env, struct osc_object *obj, LASSERT(atomic_long_read(cli->cl_lru_left) >= 0); while (!atomic_long_add_unless(cli->cl_lru_left, -1, 0)) { /* run out of LRU spaces, try to drop some by itself */ - rc = osc_lru_reclaim(cli); + rc = osc_lru_reclaim(cli, 1); if (rc < 0) break; if (rc > 0) @@ -839,7 +870,7 @@ void osc_dec_unstable_pages(struct ptlrpc_request *req) if (!unstable_count) wake_up_all(&cli->cl_cache->ccc_unstable_waitq); - if (osc_cache_too_much(cli)) + if (waitqueue_active(&osc_lru_waitq)) (void)ptlrpcd_queue_work(cli->cl_lru_work); } @@ -895,8 +926,7 @@ bool osc_over_unstable_soft_limit(struct client_obd *cli) CDEBUG(D_CACHE, "%s: cli: %p unstable pages: %lu, osc unstable pages: %lu\n", - cli->cl_import->imp_obd->obd_name, cli, - unstable_nr, osc_unstable_count); + cli_name(cli), cli, unstable_nr, osc_unstable_count); /* * If the LRU slots are in shortage - 25% remaining AND this OSC diff --git a/drivers/staging/lustre/lustre/osc/osc_quota.c b/drivers/staging/lustre/lustre/osc/osc_quota.c index acdd91a..4bb3b45 100644 --- a/drivers/staging/lustre/lustre/osc/osc_quota.c +++ b/drivers/staging/lustre/lustre/osc/osc_quota.c @@ -106,7 +106,7 @@ int osc_quota_setdq(struct client_obd *cli, const unsigned int qid[], } CDEBUG(D_QUOTA, "%s: setdq to insert for %s %d (%d)\n", - cli->cl_import->imp_obd->obd_name, + cli_name(cli), type == USRQUOTA ? "user" : "group", qid[type], rc); } else { @@ -122,7 +122,7 @@ int osc_quota_setdq(struct client_obd *cli, const unsigned int qid[], kmem_cache_free(osc_quota_kmem, oqi); CDEBUG(D_QUOTA, "%s: setdq to remove for %s %d (%p)\n", - cli->cl_import->imp_obd->obd_name, + cli_name(cli), type == USRQUOTA ? "user" : "group", qid[type], oqi); } diff --git a/drivers/staging/lustre/lustre/osc/osc_request.c b/drivers/staging/lustre/lustre/osc/osc_request.c index 7f19cfa..3d5cae9 100644 --- a/drivers/staging/lustre/lustre/osc/osc_request.c +++ b/drivers/staging/lustre/lustre/osc/osc_request.c @@ -583,14 +583,13 @@ static void osc_announce_cached(struct client_obd *cli, struct obdo *oa, oa->o_undirty = 0; } else if (unlikely(atomic_long_read(&obd_dirty_pages) - atomic_long_read(&obd_dirty_transit_pages) > - (obd_max_dirty_pages + 1))) { + (long)(obd_max_dirty_pages + 1))) { /* The atomic_read() allowing the atomic_inc() are * not covered by a lock thus they may safely race and trip * this CERROR() unless we add in a small fudge factor (+1). */ - CERROR("%s: dirty %ld + %ld > system dirty_max %lu\n", - cli->cl_import->imp_obd->obd_name, - atomic_long_read(&obd_dirty_pages), + CERROR("%s: dirty %ld + %ld > system dirty_max %ld\n", + cli_name(cli), atomic_long_read(&obd_dirty_pages), atomic_long_read(&obd_dirty_transit_pages), obd_max_dirty_pages); oa->o_undirty = 0; @@ -785,12 +784,10 @@ static int osc_add_shrink_grant(struct client_obd *client) osc_grant_shrink_grant_cb, NULL, &client->cl_grant_shrink_list); if (rc) { - CERROR("add grant client %s error %d\n", - client->cl_import->imp_obd->obd_name, rc); + CERROR("add grant client %s error %d\n", cli_name(client), rc); return rc; } - CDEBUG(D_CACHE, "add grant client %s\n", - client->cl_import->imp_obd->obd_name); + CDEBUG(D_CACHE, "add grant client %s\n", cli_name(client)); osc_update_next_shrink(client); return 0; } @@ -824,8 +821,8 @@ static void osc_init_grant(struct client_obd *cli, struct obd_connect_data *ocd) spin_unlock(&cli->cl_loi_list_lock); CDEBUG(D_CACHE, "%s, setting cl_avail_grant: %ld cl_lost_grant: %ld chunk bits: %d\n", - cli->cl_import->imp_obd->obd_name, - cli->cl_avail_grant, cli->cl_lost_grant, cli->cl_chunkbits); + cli_name(cli), cli->cl_avail_grant, cli->cl_lost_grant, + cli->cl_chunkbits); if (ocd->ocd_connect_flags & OBD_CONNECT_GRANT_SHRINK && list_empty(&cli->cl_grant_shrink_list)) -- 1.7.1 From dan.carpenter at oracle.com Thu Nov 24 11:10:39 2016 From: dan.carpenter at oracle.com (Dan Carpenter) Date: Thu, 24 Nov 2016 14:10:39 +0300 Subject: [lustre-devel] [patch] staging: lustre: lnet: memory corruption in selftest Message-ID: <20161124110934.GG17225@mwanda> We want sizeof(struct lstcon_node) but instead we're getting the sizeof a pointer. Fixes: 8d78f0f2ba76 ("staging: lustre: lnet: cleanup some of the > 80 line issues") Signed-off-by: Dan Carpenter diff --git a/drivers/staging/lustre/lnet/selftest/console.c b/drivers/staging/lustre/lnet/selftest/console.c index f2d11fa..f750b9c 100644 --- a/drivers/staging/lustre/lnet/selftest/console.c +++ b/drivers/staging/lustre/lnet/selftest/console.c @@ -86,7 +86,7 @@ lstcon_node_find(lnet_process_id_t id, struct lstcon_node **ndpp, int create) if (!create) return -ENOENT; - LIBCFS_ALLOC(*ndpp, sizeof(*ndpp) + sizeof(*ndl)); + LIBCFS_ALLOC(*ndpp, sizeof(**ndpp) + sizeof(*ndl)); if (!*ndpp) return -ENOMEM; From dan.carpenter at oracle.com Thu Nov 24 11:12:04 2016 From: dan.carpenter at oracle.com (Dan Carpenter) Date: Thu, 24 Nov 2016 14:12:04 +0300 Subject: [lustre-devel] [patch] staging: lustre/ptlrpc: small leak on allocation failure Message-ID: <20161124111204.GH17225@mwanda> We should free "desc" before returning NULL. Signed-off-by: Dan Carpenter diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c index ac959ef..8047413 100644 --- a/drivers/staging/lustre/lustre/ptlrpc/client.c +++ b/drivers/staging/lustre/lustre/ptlrpc/client.c @@ -128,12 +128,12 @@ struct ptlrpc_bulk_desc *ptlrpc_new_bulk(unsigned int nfrags, GET_KIOV(desc) = kcalloc(nfrags, sizeof(*GET_KIOV(desc)), GFP_NOFS); if (!GET_KIOV(desc)) - goto out; + goto free_desc; } else { GET_KVEC(desc) = kcalloc(nfrags, sizeof(*GET_KVEC(desc)), GFP_NOFS); if (!GET_KVEC(desc)) - goto out; + goto free_desc; } spin_lock_init(&desc->bd_lock); @@ -154,7 +154,8 @@ struct ptlrpc_bulk_desc *ptlrpc_new_bulk(unsigned int nfrags, LNetInvalidateHandle(&desc->bd_mds[i]); return desc; -out: +free_desc: + kfree(desc); return NULL; } From dan.carpenter at oracle.com Fri Nov 25 12:17:37 2016 From: dan.carpenter at oracle.com (Dan Carpenter) Date: Fri, 25 Nov 2016 15:17:37 +0300 Subject: [lustre-devel] [PATCH 08/10] staging: lustre: libcfs: remove NULL comparisons in headers In-Reply-To: <1479487724-20386-9-git-send-email-jsimmons@infradead.org> References: <1479487724-20386-1-git-send-email-jsimmons@infradead.org> <1479487724-20386-9-git-send-email-jsimmons@infradead.org> Message-ID: <20161125120841.GC6266@mwanda> On Fri, Nov 18, 2016 at 11:48:42AM -0500, James Simmons wrote: > diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h > index 93bff1b..fc180b8 100644 > --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h > +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h > @@ -96,7 +96,7 @@ > > #define LIBCFS_ALLOC_POST(ptr, size) \ > do { \ > - if (unlikely((ptr) == NULL)) { \ > + if (!unlikely((ptr))) { \ Argh.... No. You did this with a Perl script? This should be: if (unlikely(!ptr)) { ... > CERROR("LNET: out of memory at %s:%d (tried to alloc '" \ > #ptr "' = %d)\n", __FILE__, __LINE__, (int)(size)); \ > } else { \ > @@ -147,7 +147,7 @@ > > #define LIBCFS_FREE(ptr, size) \ > do { \ > - if (unlikely((ptr) == NULL)) { \ > + if (!unlikely((ptr))) { \ Same. Otherwise the unlikely hint is reversed from what we want. regards, dan carpenter From dan.carpenter at oracle.com Fri Nov 25 22:12:41 2016 From: dan.carpenter at oracle.com (Dan Carpenter) Date: Sat, 26 Nov 2016 01:12:41 +0300 Subject: [lustre-devel] [PATCH v2] staging: lustre: osc: Performance tune for LRU In-Reply-To: <1479942105-30220-1-git-send-email-jsimmons@infradead.org> References: <1479942105-30220-1-git-send-email-jsimmons@infradead.org> Message-ID: <20161125221241.GN6266@mwanda> On Wed, Nov 23, 2016 at 06:01:45PM -0500, James Simmons wrote: > From: Jinshan Xiong > > Early launch page LRU work in osc_io_rw_iter_init(); > Change the page LRU shrinking policy by OSC attributes; > Delete the contented lock osc_object::oo_seatbelt > The cli_name() stuff should be in a separate patch. It's hard to review this stuff (or possibly I just found the first deal breaker and gave up). Anyway, please review this again and break it up into separate patches. regards, dan carpenter PS: I glanced at the patch again and the very first line is changing "lru" to "LRU" in a comment. We'll get a million performance increases from that I bet. Don't mix random white space changes into it... From andreas.dilger at intel.com Fri Nov 25 23:16:48 2016 From: andreas.dilger at intel.com (Dilger, Andreas) Date: Fri, 25 Nov 2016 23:16:48 +0000 Subject: [lustre-devel] [patch] staging: lustre/ptlrpc: small leak on allocation failure In-Reply-To: <20161124111204.GH17225@mwanda> References: <20161124111204.GH17225@mwanda> Message-ID: On Nov 24, 2016, at 04:12, Dan Carpenter wrote: > > We should free "desc" before returning NULL. > > Signed-off-by: Dan Carpenter Reviewed-by: Andreas Dilger > diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c > index ac959ef..8047413 100644 > --- a/drivers/staging/lustre/lustre/ptlrpc/client.c > +++ b/drivers/staging/lustre/lustre/ptlrpc/client.c > @@ -128,12 +128,12 @@ struct ptlrpc_bulk_desc *ptlrpc_new_bulk(unsigned int nfrags, > GET_KIOV(desc) = kcalloc(nfrags, sizeof(*GET_KIOV(desc)), > GFP_NOFS); > if (!GET_KIOV(desc)) > - goto out; > + goto free_desc; > } else { > GET_KVEC(desc) = kcalloc(nfrags, sizeof(*GET_KVEC(desc)), > GFP_NOFS); > if (!GET_KVEC(desc)) > - goto out; > + goto free_desc; > } > > spin_lock_init(&desc->bd_lock); > @@ -154,7 +154,8 @@ struct ptlrpc_bulk_desc *ptlrpc_new_bulk(unsigned int nfrags, > LNetInvalidateHandle(&desc->bd_mds[i]); > > return desc; > -out: > +free_desc: > + kfree(desc); > return NULL; > } > > _______________________________________________ > lustre-devel mailing list > lustre-devel at lists.lustre.org > http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org From andreas.dilger at intel.com Fri Nov 25 23:49:16 2016 From: andreas.dilger at intel.com (Dilger, Andreas) Date: Fri, 25 Nov 2016 23:49:16 +0000 Subject: [lustre-devel] [PATCH 09/10] staging: lustre: libcfs: remove zero comparisons in headers In-Reply-To: <1479487724-20386-10-git-send-email-jsimmons@infradead.org> References: <1479487724-20386-1-git-send-email-jsimmons@infradead.org> <1479487724-20386-10-git-send-email-jsimmons@infradead.org> Message-ID: <0DBDC329-9FB2-4749-9EC3-5B81901BA6D5@intel.com> On Nov 18, 2016, at 09:48, James Simmons wrote: > > Remove the zero comparisions in the libcfs headers. > > Signed-off-by: James Simmons > --- > .../lustre/include/linux/libcfs/libcfs_crypto.h | 2 +- > .../lustre/include/linux/libcfs/libcfs_fail.h | 4 +- > .../lustre/include/linux/libcfs/libcfs_hash.h | 32 ++++++++++---------- > 3 files changed, 19 insertions(+), 19 deletions(-) > > diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_crypto.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_crypto.h > index a0865e5..8f34c5d 100644 > --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_crypto.h > +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_crypto.h > @@ -137,7 +137,7 @@ static inline unsigned char cfs_crypto_hash_alg(const char *algname) > enum cfs_crypto_hash_alg hash_alg; > > for (hash_alg = 0; hash_alg < CFS_HASH_ALG_MAX; hash_alg++) > - if (strcmp(hash_types[hash_alg].cht_name, algname) == 0) > + if (!strcmp(hash_types[hash_alg].cht_name, algname)) I'd really rather keep the "== 0" comparison for strcmp(), because IMHO !strcmp() reads like "the strings do not compare the same" and is confusing. > static inline int > cfs_hash_is_iterating(struct cfs_hash *hs) > { > /* someone is calling cfs_hash_for_each_* */ > - return hs->hs_iterating || hs->hs_iterators != 0; > + return hs->hs_iterating || hs->hs_iterators; Likewise, I'd rather keep comparisons == 0 or != 0 for cases where the variable is an actual number instead of just a return code or a pointer. I don't think we need to be totally dogmatic about removing every comparison with 0 in all of the code, as that can reduce readability in some cases. I know this patch has already landed, and it isn't the end of the world either way, but just wanted to forestall similar changes being made through the rest of the code. Cheers, Andreas From andreas.dilger at intel.com Sat Nov 26 06:22:49 2016 From: andreas.dilger at intel.com (Dilger, Andreas) Date: Sat, 26 Nov 2016 06:22:49 +0000 Subject: [lustre-devel] [patch] staging: lustre: lnet: memory corruption in selftest In-Reply-To: <20161124110934.GG17225@mwanda> References: <20161124110934.GG17225@mwanda> Message-ID: On Nov 24, 2016, at 04:10, Dan Carpenter wrote: > > We want sizeof(struct lstcon_node) but instead we're getting the sizeof > a pointer. > > Fixes: 8d78f0f2ba76 ("staging: lustre: lnet: cleanup some of the > 80 line issues") > Signed-off-by: Dan Carpenter Reviewed-by: Andreas Dilger > > diff --git a/drivers/staging/lustre/lnet/selftest/console.c b/drivers/staging/lustre/lnet/selftest/console.c > index f2d11fa..f750b9c 100644 > --- a/drivers/staging/lustre/lnet/selftest/console.c > +++ b/drivers/staging/lustre/lnet/selftest/console.c > @@ -86,7 +86,7 @@ lstcon_node_find(lnet_process_id_t id, struct lstcon_node **ndpp, int create) > if (!create) > return -ENOENT; > > - LIBCFS_ALLOC(*ndpp, sizeof(*ndpp) + sizeof(*ndl)); > + LIBCFS_ALLOC(*ndpp, sizeof(**ndpp) + sizeof(*ndl)); > if (!*ndpp) > return -ENOMEM; > From trent.geerdes at intel.com Mon Nov 28 17:31:29 2016 From: trent.geerdes at intel.com (Geerdes, Trent) Date: Mon, 28 Nov 2016 17:31:29 +0000 Subject: [lustre-devel] Intel HPDD Maintenance Notification Message-ID: Intel High Performance Data Division Maintenance Notification WHAT and WHEN: On Thursday Dec 1, 2016 8:00am - 1:00pm Mountain time the following sites will be unavailable. https://jira.hpdd.intel.com https://wiki.hpdd.intel.com http://review.whamcloud.com http://git.whamcloud.com http://downloads.hpdd.intel.com Jenkins (https://build.hpdd.intel.com) will be available but jobs will be paused during the maintenance window. WHY: The applications are being moved to new servers with more resources. The applications are being upgraded to newer versions. QUESTIONS: For questions or concerns contact Trent Geerdes e-mail: trent.geerdes at intel.com From jsimmons at infradead.org Mon Nov 28 22:01:37 2016 From: jsimmons at infradead.org (James Simmons) Date: Mon, 28 Nov 2016 22:01:37 +0000 (GMT) Subject: [lustre-devel] Failed to build a Lustre Storage Client on OpenVZ Kernel ! Is this a bug? In-Reply-To: References: Message-ID: > Dear James and All, > > Zipped file "config.log" is attached, file in the former is too big (beyond the limit of 150 KB). > > Thanks very much and sorry for the redundancy! > > Best Wishes! I was on vacation last week so sorry for the delay. I think I know what the problem is. Can you try the attached patch and see if it works. -------------- next part -------------- diff --git a/lustre/autoconf/lustre-core.m4 b/lustre/autoconf/lustre-core.m4 index 0578325..aac8dbd 100644 --- a/lustre/autoconf/lustre-core.m4 +++ b/lustre/autoconf/lustre-core.m4 @@ -1717,6 +1717,26 @@ vfs_rename_6args, [ ]) # LC_VFS_RENAME_6ARGS # +# LC_IOV_ITER_IOVEC +# +# 3.16 introduced iov_iter_iovec +# +AC_DEFUN([LC_IOV_ITER_IOVEC], [ +LB_CHECK_COMPILE([if Linux kernel has 'iov_iter_iovec'], +iov_iter_iovec, [ + #include + #include +],[ + struct iovec vec; + + vec = iov_iter_iovec(NULL); +], [ + AC_DEFINE(HAVE_IOV_ITER_IOVEC, 1, + [kernel has iov_iter_iovec]) +]) +]) # LC_IOV_ITER_IOVEC + +# # LC_DIRECTIO_USE_ITER # # 3.16 kernel changes direct IO to use iov_iter @@ -2356,6 +2376,7 @@ AC_DEFUN([LC_PROG_LINUX], [ LC_HAVE_TRUNCATE_IPAGES_FINAL # 3.15 + LC_IOV_ITER_IOVEC LC_VFS_RENAME_6ARGS # 3.16 diff --git a/lustre/llite/llite_internal.h b/lustre/llite/llite_internal.h index 3f7dfed..0e1252b 100644 --- a/lustre/llite/llite_internal.h +++ b/lustre/llite/llite_internal.h @@ -1463,6 +1463,7 @@ static inline void iov_iter_reexpand(struct iov_iter *i, size_t count) i->count = count; } +#ifndef HAVE_IOV_ITER_IOVEC static inline struct iovec iov_iter_iovec(const struct iov_iter *iter) { return (struct iovec) { @@ -1471,6 +1472,7 @@ static inline struct iovec iov_iter_iovec(const struct iov_iter *iter) iter->iov->iov_len - iter->iov_offset), }; } +#endif #define iov_for_each(iov, iter, start) \ for (iter = (start); \ From jsimmons at infradead.org Wed Nov 30 01:33:33 2016 From: jsimmons at infradead.org (James Simmons) Date: Tue, 29 Nov 2016 20:33:33 -0500 Subject: [lustre-devel] [PATCH] staging: lustre: libcfs: repair improper unlikely test Message-ID: <1480469613-7866-1-git-send-email-jsimmons@infradead.org> The scripts to replace NULL test got confused with the macro parenthesis so the unlikely test in libcfs_private.h ended up incorrect. This fixes this error. Reported-by: Dan Carpenter Signed-off-by: James Simmons --- .../lustre/include/linux/libcfs/libcfs_private.h | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h index fc180b8..41a651f 100644 --- a/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h +++ b/drivers/staging/lustre/include/linux/libcfs/libcfs_private.h @@ -96,7 +96,7 @@ #define LIBCFS_ALLOC_POST(ptr, size) \ do { \ - if (!unlikely((ptr))) { \ + if (unlikely(!(ptr))) { \ CERROR("LNET: out of memory at %s:%d (tried to alloc '" \ #ptr "' = %d)\n", __FILE__, __LINE__, (int)(size)); \ } else { \ @@ -147,7 +147,7 @@ #define LIBCFS_FREE(ptr, size) \ do { \ - if (!unlikely((ptr))) { \ + if (unlikely(!(ptr))) { \ CERROR("LIBCFS: free NULL '" #ptr "' (%d bytes) at " \ "%s:%d\n", (int)(size), __FILE__, __LINE__); \ break; \ -- 1.7.1 From andrew.kanner at gmail.com Thu Nov 3 22:51:53 2016 From: andrew.kanner at gmail.com (Andrew Kanner) Date: Thu, 03 Nov 2016 22:51:53 -0000 Subject: [lustre-devel] [PATCH v2] staging: lustre: fixed shadowed variable in socklnd_cb.c Message-ID: <1478213386-13853-1-git-send-email-andrew.kanner@gmail.com> Removed redundant declaration of variable 'tx' in local scope Fixed: sparse warning: socklnd_cb.c:2476:41: warning: symbol 'tx' shadows an earlier one socklnd_cb.c:2435:25: originally declared here Signed-off-by: Andrew Kanner --- drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c index c1c6f60..f31f4a1 100644 --- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c @@ -2473,8 +2473,8 @@ ksocknal_check_peer_timeouts(int idx) * holding only shared lock */ if (!list_empty(&peer->ksnp_tx_queue)) { - struct ksock_tx *tx = list_entry(peer->ksnp_tx_queue.next, - struct ksock_tx, tx_list); + tx = list_entry(peer->ksnp_tx_queue.next, + struct ksock_tx, tx_list); if (cfs_time_aftereq(cfs_time_current(), tx->tx_deadline)) { -- 2.1.4 From quwenruo at cn.fujitsu.com Thu Nov 10 02:29:36 2016 From: quwenruo at cn.fujitsu.com (Qu Wenruo) Date: Thu, 10 Nov 2016 02:29:36 -0000 Subject: [lustre-devel] [PATCH] f2fs: support multiple devices In-Reply-To: <0D1876A8-BB77-4C1A-BE4F-B4A0E81DD4EA@dilger.ca> References: <20161109205653.70061-1-jaegeuk@kernel.org> <0D1876A8-BB77-4C1A-BE4F-B4A0E81DD4EA@dilger.ca> Message-ID: <4ec4d8f2-da23-762d-ba81-12e76ed09793@cn.fujitsu.com> At 11/10/2016 06:57 AM, Andreas Dilger wrote: > On Nov 9, 2016, at 1:56 PM, Jaegeuk Kim wrote: >> >> This patch implements multiple devices support for f2fs. >> Given multiple devices by mkfs.f2fs, f2fs shows them entirely as one big >> volume under one f2fs instance. >> >> Internal block management is very simple, but we will modify block >> allocation and background GC policy to boost IO speed by exploiting them >> accoording to each device speed. > > How will you integrate this into FIEMAP, since it is now possible if a > file is split across multiple devices then it will return ambiguous block > numbers for a file. I've been meaning to merge the FIEMAP handling in > Lustre to support multiple devices in a single filesystem, so that this > can be detected in userspace. > > struct ll_fiemap_extent { > __u64 fe_logical; /* logical offset in bytes for the start of > * the extent from the beginning of the file > */ > __u64 fe_physical; /* physical offset in bytes for the start > * of the extent from the beginning of the disk > */ > __u64 fe_length; /* length in bytes for this extent */ > __u64 fe_reserved64[2]; > __u32 fe_flags; /* FIEMAP_EXTENT_* flags for this extent */ > __u32 fe_device; /* device number for this extent */ > __u32 fe_reserved[2]; > }; Btrfs introduce a new layer for multi-device (even for single device). So fiemap returned by btrfs is never real device bytenr, but logical address in btrfs logical address space. Much like traditional soft RAID. > > This adds the 32-bit "fe_device" field, which would optionally be filled > in by the filesystem (zero otherwise). It would return the kernel device > number (i.e. st_dev), or for network filesystem (with FIEMAP_EXTENT_NET > set) this could just return an integer device number since the device > number is meaningless (and may conflict) on a remote system. > > Since AFAIK Btrfs also has multiple device support there are an increasing > number of places where this would be useful. AFAIK, btrfs multi-device is here due to scrub with its data/meta csum. Unlike device-mapper based multi-device, btrfs has csum so it can detect which mirror is correct. This makes btrfs scrub a little better than soft raid. For example, for RAID1 if two mirror differs from each other, btrfs can find the correct one and rewrite it into the other mirror. And further more, btrfs supports snapshot and is faster than device-mapper based snapshot(LVM). This makes it a little more worthy to implement multi-device support in btrfs. But for f2fs, no data csum, no snapshot. I don't really see the point to use so many codes to implement it, especially we can use mdadm or LVM to implement it. Not to mention btrfs multi-device support still has quite a lot of bugs, like scrub can corrupt correct data stripes. Personally speaking, I am not a fan of btrfs multi-device management, despite the above advantage. As the complexity is really not worthy. (So I think XFS with LVM is much better than Btrfs considering the stability) Thanks, Qu > > Cheers, Andreas > >> >> Signed-off-by: Jaegeuk Kim >> --- >> fs/f2fs/data.c | 55 ++++++++++++++++--- >> fs/f2fs/f2fs.h | 29 ++++++++-- >> fs/f2fs/segment.c | 119 +++++++++++++++++++++++++++++------------ >> fs/f2fs/super.c | 138 ++++++++++++++++++++++++++++++++++++++---------- >> include/linux/f2fs_fs.h | 10 +++- >> 5 files changed, 277 insertions(+), 74 deletions(-) >> >> diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c >> index 47ded0c..e2be24e 100644 >> --- a/fs/f2fs/data.c >> +++ b/fs/f2fs/data.c >> @@ -88,6 +88,46 @@ static void f2fs_write_end_io(struct bio *bio) >> } >> >> /* >> + * Return true, if pre_bio's bdev is same as its target device. >> + */ >> +struct block_device *f2fs_target_device(struct f2fs_sb_info *sbi, >> + block_t blk_addr, struct bio *bio) >> +{ >> + struct block_device *bdev = sbi->sb->s_bdev; >> + int i; >> + >> + for (i = 0; i < sbi->s_ndevs; i++) { >> + if (FDEV(i).start_blk <= blk_addr && >> + FDEV(i).end_blk >= blk_addr) { >> + blk_addr -= FDEV(i).start_blk; >> + bdev = FDEV(i).bdev; >> + break; >> + } >> + } >> + if (bio) { >> + bio->bi_bdev = bdev; >> + bio->bi_iter.bi_sector = SECTOR_FROM_BLOCK(blk_addr); >> + } >> + return bdev; >> +} >> + >> +int f2fs_target_device_index(struct f2fs_sb_info *sbi, block_t blkaddr) >> +{ >> + int i; >> + >> + for (i = 0; i < sbi->s_ndevs; i++) >> + if (FDEV(i).start_blk <= blkaddr && FDEV(i).end_blk >= blkaddr) >> + return i; >> + return 0; >> +} >> + >> +static bool __same_bdev(struct f2fs_sb_info *sbi, >> + block_t blk_addr, struct bio *bio) >> +{ >> + return f2fs_target_device(sbi, blk_addr, NULL) == bio->bi_bdev; >> +} >> + >> +/* >> * Low-level block read/write IO operations. >> */ >> static struct bio *__bio_alloc(struct f2fs_sb_info *sbi, block_t blk_addr, >> @@ -97,8 +137,7 @@ static struct bio *__bio_alloc(struct f2fs_sb_info *sbi, block_t blk_addr, >> >> bio = f2fs_bio_alloc(npages); >> >> - bio->bi_bdev = sbi->sb->s_bdev; >> - bio->bi_iter.bi_sector = SECTOR_FROM_BLOCK(blk_addr); >> + f2fs_target_device(sbi, blk_addr, bio); >> bio->bi_end_io = is_read ? f2fs_read_end_io : f2fs_write_end_io; >> bio->bi_private = is_read ? NULL : sbi; >> >> @@ -273,7 +312,8 @@ void f2fs_submit_page_mbio(struct f2fs_io_info *fio) >> down_write(&io->io_rwsem); >> >> if (io->bio && (io->last_block_in_bio != fio->new_blkaddr - 1 || >> - (io->fio.op != fio->op || io->fio.op_flags != fio->op_flags))) >> + (io->fio.op != fio->op || io->fio.op_flags != fio->op_flags) || >> + !__same_bdev(sbi, fio->new_blkaddr, io->bio))) >> __submit_merged_bio(io); >> alloc_new: >> if (io->bio == NULL) { >> @@ -965,7 +1005,6 @@ static struct bio *f2fs_grab_bio(struct inode *inode, block_t blkaddr, >> { >> struct f2fs_sb_info *sbi = F2FS_I_SB(inode); >> struct fscrypt_ctx *ctx = NULL; >> - struct block_device *bdev = sbi->sb->s_bdev; >> struct bio *bio; >> >> if (f2fs_encrypted_inode(inode) && S_ISREG(inode->i_mode)) { >> @@ -983,8 +1022,7 @@ static struct bio *f2fs_grab_bio(struct inode *inode, block_t blkaddr, >> fscrypt_release_ctx(ctx); >> return ERR_PTR(-ENOMEM); >> } >> - bio->bi_bdev = bdev; >> - bio->bi_iter.bi_sector = SECTOR_FROM_BLOCK(blkaddr); >> + f2fs_target_device(sbi, blkaddr, bio); >> bio->bi_end_io = f2fs_read_end_io; >> bio->bi_private = ctx; >> >> @@ -1079,7 +1117,8 @@ static int f2fs_mpage_readpages(struct address_space *mapping, >> * This page will go to BIO. Do we need to send this >> * BIO off first? >> */ >> - if (bio && (last_block_in_bio != block_nr - 1)) { >> + if (bio && (last_block_in_bio != block_nr - 1 || >> + !__same_bdev(F2FS_I_SB(inode), block_nr, bio))) { >> submit_and_realloc: >> __submit_bio(F2FS_I_SB(inode), bio, DATA); >> bio = NULL; >> @@ -1738,6 +1777,8 @@ static ssize_t f2fs_direct_IO(struct kiocb *iocb, struct iov_iter *iter) >> return 0; >> if (test_opt(F2FS_I_SB(inode), LFS)) >> return 0; >> + if (F2FS_I_SB(inode)->s_ndevs) >> + return 0; >> >> trace_f2fs_direct_IO_enter(inode, offset, count, rw); >> >> diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h >> index 9650514..1737c45 100644 >> --- a/fs/f2fs/f2fs.h >> +++ b/fs/f2fs/f2fs.h >> @@ -730,6 +730,20 @@ struct f2fs_bio_info { >> struct rw_semaphore io_rwsem; /* blocking op for bio */ >> }; >> >> +#define FDEV(i) (sbi->devs[i]) >> +#define RDEV(i) (raw_super->devs[i]) >> +struct f2fs_dev_info { >> + struct block_device *bdev; >> + char path[MAX_PATH_LEN]; >> + unsigned int total_segments; >> + block_t start_blk; >> + block_t end_blk; >> +#ifdef CONFIG_BLK_DEV_ZONED >> + unsigned int nr_blkz; /* Total number of zones */ >> + u8 *blkz_type; /* Array of zones type */ >> +#endif >> +}; >> + >> enum inode_type { >> DIR_INODE, /* for dirty dir inode */ >> FILE_INODE, /* for dirty regular/symlink inode */ >> @@ -778,10 +792,8 @@ struct f2fs_sb_info { >> #endif >> >> #ifdef CONFIG_BLK_DEV_ZONED >> - unsigned int nr_blkz; /* Total number of zones */ >> unsigned int blocks_per_blkz; /* F2FS blocks per zone */ >> unsigned int log_blocks_per_blkz; /* log2 F2FS blocks per zone */ >> - u8 *blkz_type; /* Array of zones type */ >> #endif >> >> /* for node-related operations */ >> @@ -897,6 +909,8 @@ struct f2fs_sb_info { >> >> /* For shrinker support */ >> struct list_head s_list; >> + int s_ndevs; /* number of devices */ >> + struct f2fs_dev_info *devs; /* for device list */ >> struct mutex umount_mutex; >> unsigned int shrinker_run_no; >> >> @@ -2159,6 +2173,9 @@ void f2fs_submit_merged_bio_cond(struct f2fs_sb_info *, struct inode *, >> void f2fs_flush_merged_bios(struct f2fs_sb_info *); >> int f2fs_submit_page_bio(struct f2fs_io_info *); >> void f2fs_submit_page_mbio(struct f2fs_io_info *); >> +struct block_device *f2fs_target_device(struct f2fs_sb_info *, >> + block_t, struct bio *); >> +int f2fs_target_device_index(struct f2fs_sb_info *, block_t); >> void set_data_blkaddr(struct dnode_of_data *); >> void f2fs_update_data_blkaddr(struct dnode_of_data *, block_t); >> int reserve_new_blocks(struct dnode_of_data *, blkcnt_t); >> @@ -2446,11 +2463,15 @@ static inline int f2fs_sb_mounted_blkzoned(struct super_block *sb) >> >> #ifdef CONFIG_BLK_DEV_ZONED >> static inline int get_blkz_type(struct f2fs_sb_info *sbi, >> - block_t blkaddr) >> + struct block_device *bdev, block_t blkaddr) >> { >> unsigned int zno = blkaddr >> sbi->log_blocks_per_blkz; >> + int i; >> >> - return sbi->blkz_type[zno]; >> + for (i = 0; i < sbi->s_ndevs; i++) >> + if (FDEV(i).bdev == bdev) >> + return FDEV(i).blkz_type[zno]; >> + return -EINVAL; >> } >> #endif >> >> diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c >> index 7fb7dd3..ef727d1 100644 >> --- a/fs/f2fs/segment.c >> +++ b/fs/f2fs/segment.c >> @@ -403,6 +403,33 @@ void f2fs_balance_fs_bg(struct f2fs_sb_info *sbi) >> } >> } >> >> +static int __submit_flush_wait(struct block_device *bdev) >> +{ >> + struct bio *bio = f2fs_bio_alloc(0); >> + int ret; >> + >> + bio_set_op_attrs(bio, REQ_OP_WRITE, WRITE_FLUSH); >> + bio->bi_bdev = bdev; >> + ret = submit_bio_wait(bio); >> + bio_put(bio); >> + return ret; >> +} >> + >> +static int submit_flush_wait(struct f2fs_sb_info *sbi) >> +{ >> + int ret = __submit_flush_wait(sbi->sb->s_bdev); >> + int i; >> + >> + if (sbi->s_ndevs && !ret) { >> + for (i = 1; i < sbi->s_ndevs; i++) { >> + ret = __submit_flush_wait(FDEV(i).bdev); >> + if (ret) >> + break; >> + } >> + } >> + return ret; >> +} >> + >> static int issue_flush_thread(void *data) >> { >> struct f2fs_sb_info *sbi = data; >> @@ -413,25 +440,18 @@ static int issue_flush_thread(void *data) >> return 0; >> >> if (!llist_empty(&fcc->issue_list)) { >> - struct bio *bio; >> struct flush_cmd *cmd, *next; >> int ret; >> >> - bio = f2fs_bio_alloc(0); >> - >> fcc->dispatch_list = llist_del_all(&fcc->issue_list); >> fcc->dispatch_list = llist_reverse_order(fcc->dispatch_list); >> >> - bio->bi_bdev = sbi->sb->s_bdev; >> - bio_set_op_attrs(bio, REQ_OP_WRITE, WRITE_FLUSH); >> - ret = submit_bio_wait(bio); >> - >> + ret = submit_flush_wait(sbi); >> llist_for_each_entry_safe(cmd, next, >> fcc->dispatch_list, llnode) { >> cmd->ret = ret; >> complete(&cmd->wait); >> } >> - bio_put(bio); >> fcc->dispatch_list = NULL; >> } >> >> @@ -452,15 +472,11 @@ int f2fs_issue_flush(struct f2fs_sb_info *sbi) >> return 0; >> >> if (!test_opt(sbi, FLUSH_MERGE) || !atomic_read(&fcc->submit_flush)) { >> - struct bio *bio = f2fs_bio_alloc(0); >> int ret; >> >> atomic_inc(&fcc->submit_flush); >> - bio->bi_bdev = sbi->sb->s_bdev; >> - bio_set_op_attrs(bio, REQ_OP_WRITE, WRITE_FLUSH); >> - ret = submit_bio_wait(bio); >> + ret = submit_flush_wait(sbi); >> atomic_dec(&fcc->submit_flush); >> - bio_put(bio); >> return ret; >> } >> >> @@ -637,14 +653,18 @@ static void f2fs_submit_bio_wait_endio(struct bio *bio) >> >> /* this function is copied from blkdev_issue_discard from block/blk-lib.c */ >> static int __f2fs_issue_discard_async(struct f2fs_sb_info *sbi, >> - block_t blkstart, block_t blklen) >> + struct block_device *bdev, block_t blkstart, block_t blklen) >> { >> - struct block_device *bdev = sbi->sb->s_bdev; >> struct bio *bio = NULL; >> int err; >> >> trace_f2fs_issue_discard(sbi->sb, blkstart, blklen); >> >> + if (sbi->s_ndevs) { >> + int devi = f2fs_target_device_index(sbi, blkstart); >> + >> + blkstart -= FDEV(devi).start_blk; >> + } >> err = __blkdev_issue_discard(bdev, >> SECTOR_FROM_BLOCK(blkstart), >> SECTOR_FROM_BLOCK(blklen), >> @@ -662,18 +682,24 @@ static int __f2fs_issue_discard_async(struct f2fs_sb_info *sbi, >> } >> >> #ifdef CONFIG_BLK_DEV_ZONED >> -static int f2fs_issue_discard_zone(struct f2fs_sb_info *sbi, >> - block_t blkstart, block_t blklen) >> +static int __f2fs_issue_discard_zone(struct f2fs_sb_info *sbi, >> + struct block_device *bdev, block_t blkstart, block_t blklen) >> { >> - sector_t sector = SECTOR_FROM_BLOCK(blkstart); >> sector_t nr_sects = SECTOR_FROM_BLOCK(blklen); >> - struct block_device *bdev = sbi->sb->s_bdev; >> + sector_t sector; >> + int devi = 0; >> >> - if (nr_sects != bdev_zone_size(bdev)) { >> + if (sbi->s_ndevs) { >> + devi = f2fs_target_device_index(sbi, blkstart); >> + blkstart -= FDEV(devi).start_blk; >> + } >> + sector = SECTOR_FROM_BLOCK(blkstart); >> + >> + if (sector % bdev_zone_size(bdev) || nr_sects != bdev_zone_size(bdev)) { >> f2fs_msg(sbi->sb, KERN_INFO, >> - "Unaligned discard attempted (sector %llu + %llu)", >> - (unsigned long long)sector, >> - (unsigned long long)nr_sects); >> + "(%d) %s: Unaligned discard attempted (block %x + %x)", >> + devi, sbi->s_ndevs ? FDEV(devi).path: "", >> + blkstart, blklen); >> return -EIO; >> } >> >> @@ -682,14 +708,12 @@ static int f2fs_issue_discard_zone(struct f2fs_sb_info *sbi, >> * use regular discard if the drive supports it. For sequential >> * zones, reset the zone write pointer. >> */ >> - switch (get_blkz_type(sbi, blkstart)) { >> + switch (get_blkz_type(sbi, bdev, blkstart)) { >> >> case BLK_ZONE_TYPE_CONVENTIONAL: >> if (!blk_queue_discard(bdev_get_queue(bdev))) >> return 0; >> - return __f2fs_issue_discard_async(sbi, blkstart, >> - blklen); >> - >> + return __f2fs_issue_discard_async(sbi, bdev, blkstart, blklen); >> case BLK_ZONE_TYPE_SEQWRITE_REQ: >> case BLK_ZONE_TYPE_SEQWRITE_PREF: >> trace_f2fs_issue_reset_zone(sbi->sb, blkstart); >> @@ -702,14 +726,45 @@ static int f2fs_issue_discard_zone(struct f2fs_sb_info *sbi, >> } >> #endif >> >> +static int __issue_discard_async(struct f2fs_sb_info *sbi, >> + struct block_device *bdev, block_t blkstart, block_t blklen) >> +{ >> +#ifdef CONFIG_BLK_DEV_ZONED >> + if (f2fs_sb_mounted_blkzoned(sbi->sb) && >> + bdev_zoned_model(bdev) != BLK_ZONED_NONE) >> + return __f2fs_issue_discard_zone(sbi, bdev, blkstart, blklen); >> +#endif >> + return __f2fs_issue_discard_async(sbi, bdev, blkstart, blklen); >> +} >> + >> static int f2fs_issue_discard(struct f2fs_sb_info *sbi, >> block_t blkstart, block_t blklen) >> { >> + sector_t start = blkstart, len = 0; >> + struct block_device *bdev; >> struct seg_entry *se; >> unsigned int offset; >> block_t i; >> + int err = 0; >> + >> + bdev = f2fs_target_device(sbi, blkstart, NULL); >> + >> + for (i = blkstart; i < blkstart + blklen; i++, len++) { >> + if (i != start) { >> + struct block_device *bdev2 = >> + f2fs_target_device(sbi, i, NULL); >> + >> + if (bdev2 != bdev) { >> + err = __issue_discard_async(sbi, bdev, >> + start, len); >> + if (err) >> + return err; >> + bdev = bdev2; >> + start = i; >> + len = 0; >> + } >> + } >> >> - for (i = blkstart; i < blkstart + blklen; i++) { >> se = get_seg_entry(sbi, GET_SEGNO(sbi, i)); >> offset = GET_BLKOFF_FROM_SEG0(sbi, i); >> >> @@ -717,11 +772,9 @@ static int f2fs_issue_discard(struct f2fs_sb_info *sbi, >> sbi->discard_blks--; >> } >> >> -#ifdef CONFIG_BLK_DEV_ZONED >> - if (f2fs_sb_mounted_blkzoned(sbi->sb)) >> - return f2fs_issue_discard_zone(sbi, blkstart, blklen); >> -#endif >> - return __f2fs_issue_discard_async(sbi, blkstart, blklen); >> + if (len) >> + err = __issue_discard_async(sbi, bdev, start, len); >> + return err; >> } >> >> static void __add_discard_entry(struct f2fs_sb_info *sbi, >> diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c >> index eca9aea..4ccbb86 100644 >> --- a/fs/f2fs/super.c >> +++ b/fs/f2fs/super.c >> @@ -713,6 +713,19 @@ static void destroy_percpu_info(struct f2fs_sb_info *sbi) >> percpu_counter_destroy(&sbi->total_valid_inode_count); >> } >> >> +static void destroy_device_list(struct f2fs_sb_info *sbi) >> +{ >> + int i; >> + >> + for (i = 0; i < sbi->s_ndevs; i++) { >> + blkdev_put(FDEV(i).bdev, FMODE_EXCL); >> +#ifdef CONFIG_BLK_DEV_ZONED >> + kfree(FDEV(i).blkz_type); >> +#endif >> + } >> + kfree(sbi->devs); >> +} >> + >> static void f2fs_put_super(struct super_block *sb) >> { >> struct f2fs_sb_info *sbi = F2FS_SB(sb); >> @@ -773,6 +786,8 @@ static void f2fs_put_super(struct super_block *sb) >> crypto_free_shash(sbi->s_chksum_driver); >> kfree(sbi->raw_super); >> >> + destroy_device_list(sbi); >> + >> destroy_percpu_info(sbi); >> kfree(sbi); >> } >> @@ -1516,9 +1531,9 @@ static int init_percpu_info(struct f2fs_sb_info *sbi) >> } >> >> #ifdef CONFIG_BLK_DEV_ZONED >> -static int init_blkz_info(struct f2fs_sb_info *sbi) >> +static int init_blkz_info(struct f2fs_sb_info *sbi, int devi) >> { >> - struct block_device *bdev = sbi->sb->s_bdev; >> + struct block_device *bdev = FDEV(devi).bdev; >> sector_t nr_sectors = bdev->bd_part->nr_sects; >> sector_t sector = 0; >> struct blk_zone *zones; >> @@ -1529,15 +1544,21 @@ static int init_blkz_info(struct f2fs_sb_info *sbi) >> if (!f2fs_sb_mounted_blkzoned(sbi->sb)) >> return 0; >> >> + if (sbi->blocks_per_blkz && sbi->blocks_per_blkz != >> + SECTOR_TO_BLOCK(bdev_zone_size(bdev))) >> + return -EINVAL; >> sbi->blocks_per_blkz = SECTOR_TO_BLOCK(bdev_zone_size(bdev)); >> + if (sbi->log_blocks_per_blkz && sbi->log_blocks_per_blkz != >> + __ilog2_u32(sbi->blocks_per_blkz)) >> + return -EINVAL; >> sbi->log_blocks_per_blkz = __ilog2_u32(sbi->blocks_per_blkz); >> - sbi->nr_blkz = SECTOR_TO_BLOCK(nr_sectors) >> >> - sbi->log_blocks_per_blkz; >> + FDEV(devi).nr_blkz = SECTOR_TO_BLOCK(nr_sectors) >> >> + sbi->log_blocks_per_blkz; >> if (nr_sectors & (bdev_zone_size(bdev) - 1)) >> - sbi->nr_blkz++; >> + FDEV(devi).nr_blkz++; >> >> - sbi->blkz_type = kmalloc(sbi->nr_blkz, GFP_KERNEL); >> - if (!sbi->blkz_type) >> + FDEV(devi).blkz_type = kmalloc(FDEV(devi).nr_blkz, GFP_KERNEL); >> + if (!FDEV(devi).blkz_type) >> return -ENOMEM; >> >> #define F2FS_REPORT_NR_ZONES 4096 >> @@ -1562,7 +1583,7 @@ static int init_blkz_info(struct f2fs_sb_info *sbi) >> } >> >> for (i = 0; i < nr_zones; i++) { >> - sbi->blkz_type[n] = zones[i].type; >> + FDEV(devi).blkz_type[n] = zones[i].type; >> sector += zones[i].len; >> n++; >> } >> @@ -1666,6 +1687,77 @@ int f2fs_commit_super(struct f2fs_sb_info *sbi, bool recover) >> return err; >> } >> >> +static int f2fs_scan_devices(struct f2fs_sb_info *sbi) >> +{ >> + struct f2fs_super_block *raw_super = F2FS_RAW_SUPER(sbi); >> + int i; >> + >> + for (i = 0; i < MAX_DEVICES; i++) { >> + if (!RDEV(i).path[0]) >> + return 0; >> + >> + if (i == 0) { >> + sbi->devs = kzalloc(sizeof(struct f2fs_dev_info) * >> + MAX_DEVICES, GFP_KERNEL); >> + if (!sbi->devs) >> + return -ENOMEM; >> + } >> + >> + memcpy(FDEV(i).path, RDEV(i).path, MAX_PATH_LEN); >> + FDEV(i).total_segments = le32_to_cpu(RDEV(i).total_segments); >> + if (i == 0) { >> + FDEV(i).start_blk = 0; >> + FDEV(i).end_blk = FDEV(i).start_blk + >> + (FDEV(i).total_segments << >> + sbi->log_blocks_per_seg) - 1 + >> + le32_to_cpu(raw_super->segment0_blkaddr); >> + } else { >> + FDEV(i).start_blk = FDEV(i - 1).end_blk + 1; >> + FDEV(i).end_blk = FDEV(i).start_blk + >> + (FDEV(i).total_segments << >> + sbi->log_blocks_per_seg) - 1; >> + } >> + >> + FDEV(i).bdev = blkdev_get_by_path(FDEV(i).path, >> + sbi->sb->s_mode, sbi->sb->s_type); >> + if (IS_ERR(FDEV(i).bdev)) >> + return PTR_ERR(FDEV(i).bdev); >> + >> + /* to release errored devices */ >> + sbi->s_ndevs = i + 1; >> + >> +#ifdef CONFIG_BLK_DEV_ZONED >> + if (bdev_zoned_model(FDEV(i).bdev) == BLK_ZONED_HM && >> + !f2fs_sb_mounted_blkzoned(sbi->sb)) { >> + f2fs_msg(sbi->sb, KERN_ERR, >> + "Zoned block device feature not enabled\n"); >> + return -EINVAL; >> + } >> + if (bdev_zoned_model(FDEV(i).bdev) != BLK_ZONED_NONE) { >> + if (init_blkz_info(sbi, i)) { >> + f2fs_msg(sbi->sb, KERN_ERR, >> + "Failed to initialize F2FS blkzone information"); >> + return -EINVAL; >> + } >> + f2fs_msg(sbi->sb, KERN_INFO, >> + "Mount Device [%2d]: %20s, %8u, %8x - %8x (zone: %s)", >> + i, FDEV(i).path, >> + FDEV(i).total_segments, >> + FDEV(i).start_blk, FDEV(i).end_blk, >> + bdev_zoned_model(FDEV(i).bdev) == BLK_ZONED_HA ? >> + "Host-aware" : "Host-managed"); >> + continue; >> + } >> +#endif >> + f2fs_msg(sbi->sb, KERN_INFO, >> + "Mount Device [%2d]: %20s, %8u, %8x - %8x", >> + i, FDEV(i).path, >> + FDEV(i).total_segments, >> + FDEV(i).start_blk, FDEV(i).end_blk); >> + } >> + return 0; >> +} >> + >> static int f2fs_fill_super(struct super_block *sb, void *data, int silent) >> { >> struct f2fs_sb_info *sbi; >> @@ -1724,15 +1816,7 @@ static int f2fs_fill_super(struct super_block *sb, void *data, int silent) >> "Zoned block device support is not enabled\n"); >> goto free_sb_buf; >> } >> -#else >> - if (bdev_zoned_model(sb->s_bdev) == BLK_ZONED_HM && >> - !f2fs_sb_mounted_blkzoned(sb)) { >> - f2fs_msg(sb, KERN_ERR, >> - "Zoned block device feature not enabled\n"); >> - goto free_sb_buf; >> - } >> #endif >> - >> default_options(sbi); >> /* parse mount options */ >> options = kstrdup((const char *)data, GFP_KERNEL); >> @@ -1802,6 +1886,13 @@ static int f2fs_fill_super(struct super_block *sb, void *data, int silent) >> goto free_meta_inode; >> } >> >> + /* Initialize device list */ >> + err = f2fs_scan_devices(sbi); >> + if (err) { >> + f2fs_msg(sb, KERN_ERR, "Failed to find devices"); >> + goto free_devices; >> + } >> + >> sbi->total_valid_node_count = >> le32_to_cpu(sbi->ckpt->valid_node_count); >> percpu_counter_set(&sbi->total_valid_inode_count, >> @@ -1820,15 +1911,6 @@ static int f2fs_fill_super(struct super_block *sb, void *data, int silent) >> >> init_ino_entry_info(sbi); >> >> -#ifdef CONFIG_BLK_DEV_ZONED >> - err = init_blkz_info(sbi); >> - if (err) { >> - f2fs_msg(sb, KERN_ERR, >> - "Failed to initialize F2FS blkzone information"); >> - goto free_blkz; >> - } >> -#endif >> - >> /* setup f2fs internal modules */ >> err = build_segment_manager(sbi); >> if (err) { >> @@ -2007,10 +2089,8 @@ static int f2fs_fill_super(struct super_block *sb, void *data, int silent) >> destroy_node_manager(sbi); >> free_sm: >> destroy_segment_manager(sbi); >> -#ifdef CONFIG_BLK_DEV_ZONED >> -free_blkz: >> - kfree(sbi->blkz_type); >> -#endif >> +free_devices: >> + destroy_device_list(sbi); >> kfree(sbi->ckpt); >> free_meta_inode: >> make_bad_inode(sbi->meta_inode); >> diff --git a/include/linux/f2fs_fs.h b/include/linux/f2fs_fs.h >> index 422630b..cea41a1 100644 >> --- a/include/linux/f2fs_fs.h >> +++ b/include/linux/f2fs_fs.h >> @@ -52,10 +52,17 @@ >> >> #define VERSION_LEN 256 >> #define MAX_VOLUME_NAME 512 >> +#define MAX_PATH_LEN 64 >> +#define MAX_DEVICES 8 >> >> /* >> * For superblock >> */ >> +struct f2fs_device { >> + __u8 path[MAX_PATH_LEN]; >> + __le32 total_segments; >> +} __packed; >> + >> struct f2fs_super_block { >> __le32 magic; /* Magic Number */ >> __le16 major_ver; /* Major Version */ >> @@ -94,7 +101,8 @@ struct f2fs_super_block { >> __le32 feature; /* defined features */ >> __u8 encryption_level; /* versioning level for encryption */ >> __u8 encrypt_pw_salt[16]; /* Salt used for string2key algorithm */ >> - __u8 reserved[871]; /* valid reserved region */ >> + struct f2fs_device devs[MAX_DEVICES]; /* device list */ >> + __u8 reserved[327]; /* valid reserved region */ >> } __packed; >> >> /* >> -- >> 2.8.3 > > Cheers, Andreas > From colin.king at canonical.com Thu Nov 10 14:19:21 2016 From: colin.king at canonical.com (Colin King) Date: Thu, 10 Nov 2016 14:19:21 -0000 Subject: [lustre-devel] [PATCH] staging: lustre: lov: Fix signed wrap around when decrementing index 'i' Message-ID: <20161110141910.25095-1-colin.king@canonical.com> From: Colin Ian King Change predecrement compare to post decrement compare to avoid an unsigned integer wrap-around comparisomn when decrementing in the while loop. Issue found with static analysis with CoverityScan, CID 1375917 Signed-off-by: Colin Ian King --- drivers/staging/lustre/lustre/lov/lov_ea.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/staging/lustre/lustre/lov/lov_ea.c b/drivers/staging/lustre/lustre/lov/lov_ea.c index 53db170..399298c 100644 --- a/drivers/staging/lustre/lustre/lov/lov_ea.c +++ b/drivers/staging/lustre/lustre/lov/lov_ea.c @@ -102,7 +102,7 @@ struct lov_stripe_md *lsm_alloc_plain(u16 stripe_count) return lsm; err: - while (--i >= 0) + while (i-- > 0) kmem_cache_free(lov_oinfo_slab, lsm->lsm_oinfo[i]); kvfree(lsm); return NULL; -- 2.10.2 From nicholasjhanley at gmail.com Sat Nov 5 00:03:18 2016 From: nicholasjhanley at gmail.com (Nicholas Hanley) Date: Sat, 05 Nov 2016 00:03:18 -0000 Subject: [lustre-devel] [PATCH] staging: lustre: o2iblnd: replace space indentation with tabs Message-ID: <20161105000256.11714-1-nicholasjhanley@gmail.com> This patch fixes all CODE_INDENT checkpatch errors in o2iblnd. Signed-off-by: Nicholas Hanley --- .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c | 2 +- .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c | 22 +++++++++++----------- 2 files changed, 12 insertions(+), 12 deletions(-) diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c index 9e88021..13235b0 100644 --- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c +++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c @@ -1637,7 +1637,7 @@ int kiblnd_fmr_pool_map(struct kib_fmr_poolset *fps, struct kib_tx *tx, { __u64 *pages = tx->tx_pages; bool is_rx = (rd != tx->tx_rd); - bool tx_pages_mapped = 0; + bool tx_pages_mapped = 0; struct kib_fmr_pool *fpo; int npages = 0; __u64 version; diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c index b27de88..92692a2 100644 --- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c +++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c @@ -1912,12 +1912,12 @@ kiblnd_close_conn_locked(struct kib_conn *conn, int error) libcfs_nid2str(peer->ibp_nid)); } else { CNETERR("Closing conn to %s: error %d%s%s%s%s%s\n", - libcfs_nid2str(peer->ibp_nid), error, - list_empty(&conn->ibc_tx_queue) ? "" : "(sending)", - list_empty(&conn->ibc_tx_noops) ? "" : "(sending_noops)", - list_empty(&conn->ibc_tx_queue_rsrvd) ? "" : "(sending_rsrvd)", - list_empty(&conn->ibc_tx_queue_nocred) ? "" : "(sending_nocred)", - list_empty(&conn->ibc_active_txs) ? "" : "(waiting)"); + libcfs_nid2str(peer->ibp_nid), error, + list_empty(&conn->ibc_tx_queue) ? "" : "(sending)", + list_empty(&conn->ibc_tx_noops) ? "" : "(sending_noops)", + list_empty(&conn->ibc_tx_queue_rsrvd) ? "" : "(sending_rsrvd)", + list_empty(&conn->ibc_tx_queue_nocred) ? "" : "(sending_nocred)", + list_empty(&conn->ibc_active_txs) ? "" : "(waiting)"); } dev = ((struct kib_net *)peer->ibp_ni->ni_data)->ibn_dev; @@ -2643,7 +2643,7 @@ kiblnd_check_reconnect(struct kib_conn *conn, int version, if (incarnation) peer->ibp_incarnation = incarnation; out: - write_unlock_irqrestore(glock, flags); + write_unlock_irqrestore(glock, flags); CNETERR("%s: %s (%s), %x, %x, msg_size: %d, queue_depth: %d/%d, max_frags: %d/%d\n", libcfs_nid2str(peer->ibp_nid), @@ -2651,7 +2651,7 @@ kiblnd_check_reconnect(struct kib_conn *conn, int version, reason, IBLND_MSG_VERSION, version, msg_size, conn->ibc_queue_depth, queue_dep, conn->ibc_max_frags, frag_num); - /** + /** * if conn::ibc_reconnect is TRUE, connd will reconnect to the peer * while destroying the zombie */ @@ -2976,7 +2976,7 @@ kiblnd_cm_callback(struct rdma_cm_id *cmid, struct rdma_cm_event *event) case RDMA_CM_EVENT_ADDR_ERROR: peer = (struct kib_peer *)cmid->context; CNETERR("%s: ADDR ERROR %d\n", - libcfs_nid2str(peer->ibp_nid), event->status); + libcfs_nid2str(peer->ibp_nid), event->status); kiblnd_peer_connect_failed(peer, 1, -EHOSTUNREACH); kiblnd_peer_decref(peer); return -EHOSTUNREACH; /* rc destroys cmid */ @@ -3021,7 +3021,7 @@ kiblnd_cm_callback(struct rdma_cm_id *cmid, struct rdma_cm_event *event) return kiblnd_active_connect(cmid); CNETERR("Can't resolve route for %s: %d\n", - libcfs_nid2str(peer->ibp_nid), event->status); + libcfs_nid2str(peer->ibp_nid), event->status); kiblnd_peer_connect_failed(peer, 1, event->status); kiblnd_peer_decref(peer); return event->status; /* rc destroys cmid */ @@ -3031,7 +3031,7 @@ kiblnd_cm_callback(struct rdma_cm_id *cmid, struct rdma_cm_event *event) LASSERT(conn->ibc_state == IBLND_CONN_ACTIVE_CONNECT || conn->ibc_state == IBLND_CONN_PASSIVE_WAIT); CNETERR("%s: UNREACHABLE %d\n", - libcfs_nid2str(conn->ibc_peer->ibp_nid), event->status); + libcfs_nid2str(conn->ibc_peer->ibp_nid), event->status); kiblnd_connreq_done(conn, -ENETDOWN); kiblnd_conn_decref(conn); return 0; -- 2.10.1 From andrew.kanner at gmail.com Thu Nov 3 22:22:20 2016 From: andrew.kanner at gmail.com (Andrew Kanner) Date: Thu, 03 Nov 2016 22:22:20 -0000 Subject: [lustre-devel] [PATCH] staging: lustre: fixed shadowed variable in socklnd_cb.c In-Reply-To: References: <1478210059-6091-1-git-send-email-andrew.kanner@gmail.com> Message-ID: 2016-11-04 1:12 GMT+03:00 Dilger, Andreas : > On Nov 3, 2016, at 15:54, Andrew Kanner wrote: >> >> Changed variable 'tx' name in local scope >> Fixed: sparse warning: >> socklnd_cb.c:2476:41: warning: symbol 'tx' shadows an earlier one >> socklnd_cb.c:2435:25: originally declared here > > Looking at this more closely (or from a greater distance, hard to say), > the outer-scope "tx" is used only after this inner-scope "tx", so in > fact there is no benefit to having the inner-scope declaration at all. > > Removing it may save a stack variable (depending on how the compiler > optimizes), and shouldn't affect functionality. > > Cheers, Andreas I see, I'll correct it and resend as PATCH v2. Thank you. From christophe.jaillet at wanadoo.fr Sun Nov 6 17:11:28 2016 From: christophe.jaillet at wanadoo.fr (Christophe JAILLET) Date: Sun, 06 Nov 2016 17:11:28 -0000 Subject: [lustre-devel] [PATCH 2/2] staging: lustre: obdclass: Add handling of error returned by lustre_cfg_new Message-ID: <20161106171123.24929-1-christophe.jaillet@wanadoo.fr> 'lustre_cfg_new()' can return ERR_PTR(-ENOMEM). Handle these errors and propagate the error code to the callers. Error handling has been rearranged in 'lustre_process_log()' with the addition of a label in order to free some resources. Signed-off-by: Christophe JAILLET --- drivers/staging/lustre/lustre/obdclass/obd_mount.c | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/drivers/staging/lustre/lustre/obdclass/obd_mount.c b/drivers/staging/lustre/lustre/obdclass/obd_mount.c index 59fbc29aae94..5473615cd338 100644 --- a/drivers/staging/lustre/lustre/obdclass/obd_mount.c +++ b/drivers/staging/lustre/lustre/obdclass/obd_mount.c @@ -89,11 +89,14 @@ int lustre_process_log(struct super_block *sb, char *logname, lustre_cfg_bufs_set(bufs, 2, cfg, sizeof(*cfg)); lustre_cfg_bufs_set(bufs, 3, &sb, sizeof(sb)); lcfg = lustre_cfg_new(LCFG_LOG_START, bufs); + if (IS_ERR(lcfg)) { + rc = PTR_ERR(lcfg); + goto out_free; + } + rc = obd_process_config(mgc, sizeof(*lcfg), lcfg); lustre_cfg_free(lcfg); - kfree(bufs); - if (rc == -EINVAL) LCONSOLE_ERROR_MSG(0x15b, "%s: The configuration from log '%s' failed from the MGS (%d). Make sure this client and the MGS are running compatible versions of Lustre.\n", mgc->obd_name, logname, rc); @@ -104,6 +107,9 @@ int lustre_process_log(struct super_block *sb, char *logname, rc); /* class_obd_list(); */ + +out_free: + kfree(bufs); return rc; } EXPORT_SYMBOL(lustre_process_log); @@ -127,6 +133,9 @@ int lustre_end_log(struct super_block *sb, char *logname, if (cfg) lustre_cfg_bufs_set(&bufs, 2, cfg, sizeof(*cfg)); lcfg = lustre_cfg_new(LCFG_LOG_END, &bufs); + if (IS_ERR(lcfg)) + return PTR_ERR(lcfg); + rc = obd_process_config(mgc, sizeof(*lcfg), lcfg); lustre_cfg_free(lcfg); return rc; @@ -159,6 +168,9 @@ static int do_lcfg(char *cfgname, lnet_nid_t nid, int cmd, lustre_cfg_bufs_set_string(&bufs, 4, s4); lcfg = lustre_cfg_new(cmd, &bufs); + if (IS_ERR(lcfg)) + return PTR_ERR(lcfg); + lcfg->lcfg_nid = nid; rc = class_process_config(lcfg); lustre_cfg_free(lcfg); -- 2.9.3 From christophe.jaillet at wanadoo.fr Sun Nov 6 17:11:20 2016 From: christophe.jaillet at wanadoo.fr (Christophe JAILLET) Date: Sun, 06 Nov 2016 17:11:20 -0000 Subject: [lustre-devel] [PATCH 1/2] staging: lustre: obdclass: Fix error handling Message-ID: <20161106171110.24881-1-christophe.jaillet@wanadoo.fr> 'lustre_cfg_new()' does not return NULL but ERR_PTR(-ENOMEM) So IS_ERR should be used to check the return value. Signed-off-by: Christophe JAILLET --- drivers/staging/lustre/lustre/obdclass/obd_config.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/staging/lustre/lustre/obdclass/obd_config.c b/drivers/staging/lustre/lustre/obdclass/obd_config.c index bbed1b72d52e..0e922c5f8b33 100644 --- a/drivers/staging/lustre/lustre/obdclass/obd_config.c +++ b/drivers/staging/lustre/lustre/obdclass/obd_config.c @@ -1382,8 +1382,8 @@ int class_manual_cleanup(struct obd_device *obd) lustre_cfg_bufs_reset(&bufs, obd->obd_name); lustre_cfg_bufs_set_string(&bufs, 1, flags); lcfg = lustre_cfg_new(LCFG_CLEANUP, &bufs); - if (!lcfg) - return -ENOMEM; + if (IS_ERR(lcfg)) + return PTR_ERR(lcfg); rc = class_process_config(lcfg); if (rc) { -- 2.9.3 From nicholasjhanley at gmail.com Mon Nov 7 19:06:06 2016 From: nicholasjhanley at gmail.com (Nicholas Hanley) Date: Mon, 07 Nov 2016 19:06:06 -0000 Subject: [lustre-devel] [PATCH] staging: lustre: o2iblnd: use bool assignment to true/false Message-ID: <20161107190138.5730-1-nicholasjhanley@gmail.com> Replace 0 with false in tx_pages_mapped = 0 to be consistent with the rest of the lustre code. Signed-off-by: Nicholas Hanley --- drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c index 13235b0..e2fc65f 100644 --- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c +++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c @@ -1637,7 +1637,7 @@ int kiblnd_fmr_pool_map(struct kib_fmr_poolset *fps, struct kib_tx *tx, { __u64 *pages = tx->tx_pages; bool is_rx = (rd != tx->tx_rd); - bool tx_pages_mapped = 0; + bool tx_pages_mapped = false; struct kib_fmr_pool *fpo; int npages = 0; __u64 version; -- 2.10.2 From anand.jain at oracle.com Thu Nov 10 16:16:45 2016 From: anand.jain at oracle.com (Anand Jain) Date: Thu, 10 Nov 2016 16:16:45 -0000 Subject: [lustre-devel] [PATCH] f2fs: support multiple devices In-Reply-To: <749156c9-2b3e-4210-a89b-2d664f9d2fc2@gmail.com> References: <20161109205653.70061-1-jaegeuk@kernel.org> <0D1876A8-BB77-4C1A-BE4F-B4A0E81DD4EA@dilger.ca> <4ec4d8f2-da23-762d-ba81-12e76ed09793@cn.fujitsu.com> <749156c9-2b3e-4210-a89b-2d664f9d2fc2@gmail.com> Message-ID: (this is deviating from the subject, sorry about that) > Pretty much, if you're using just raid1 mode, without > compression, on reasonable storage devices, things are rock-solid > relative to the rest of BTRFS. IMO, BTRFS volume manger feature is incomplete and there is RAID1 critical bug which affects availability, so its not suitable for enterprise solutions yet. However it should be fine in a setup where dedicated sysadmin and maintenance downtime is a choice. From andrew.kanner at gmail.com Thu Nov 3 21:57:24 2016 From: andrew.kanner at gmail.com (Andrew Kanner) Date: Thu, 03 Nov 2016 21:57:24 -0000 Subject: [lustre-devel] [PATCH] staging: lustre: fixed shadowed variable in socklnd_cb.c Message-ID: <1478210059-6091-1-git-send-email-andrew.kanner@gmail.com> Changed variable 'tx' name in local scope Fixed: sparse warning: socklnd_cb.c:2476:41: warning: symbol 'tx' shadows an earlier one socklnd_cb.c:2435:25: originally declared here Signed-off-by: Andrew Kanner --- drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c index c1c6f60..03fe4e5 100644 --- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c +++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd_cb.c @@ -2473,11 +2473,11 @@ ksocknal_check_peer_timeouts(int idx) * holding only shared lock */ if (!list_empty(&peer->ksnp_tx_queue)) { - struct ksock_tx *tx = list_entry(peer->ksnp_tx_queue.next, + struct ksock_tx *_tx = list_entry(peer->ksnp_tx_queue.next, struct ksock_tx, tx_list); if (cfs_time_aftereq(cfs_time_current(), - tx->tx_deadline)) { + _tx->tx_deadline)) { ksocknal_peer_addref(peer); read_unlock(&ksocknal_data.ksnd_global_lock); -- 2.1.4 From jaegeuk at kernel.org Thu Nov 10 01:12:08 2016 From: jaegeuk at kernel.org (Jaegeuk Kim) Date: Thu, 10 Nov 2016 01:12:08 -0000 Subject: [lustre-devel] [PATCH] f2fs: support multiple devices In-Reply-To: <0D1876A8-BB77-4C1A-BE4F-B4A0E81DD4EA@dilger.ca> References: <20161109205653.70061-1-jaegeuk@kernel.org> <0D1876A8-BB77-4C1A-BE4F-B4A0E81DD4EA@dilger.ca> Message-ID: <20161110011201.GA72134@jaegeuk> On Wed, Nov 09, 2016 at 03:57:53PM -0700, Andreas Dilger wrote: > On Nov 9, 2016, at 1:56 PM, Jaegeuk Kim wrote: > > > > This patch implements multiple devices support for f2fs. > > Given multiple devices by mkfs.f2fs, f2fs shows them entirely as one big > > volume under one f2fs instance. > > > > Internal block management is very simple, but we will modify block > > allocation and background GC policy to boost IO speed by exploiting them > > accoording to each device speed. > > How will you integrate this into FIEMAP, since it is now possible if a > file is split across multiple devices then it will return ambiguous block > numbers for a file. I've been meaning to merge the FIEMAP handling in > Lustre to support multiple devices in a single filesystem, so that this > can be detected in userspace. > > struct ll_fiemap_extent { > __u64 fe_logical; /* logical offset in bytes for the start of > * the extent from the beginning of the file > */ > __u64 fe_physical; /* physical offset in bytes for the start > * of the extent from the beginning of the disk > */ > __u64 fe_length; /* length in bytes for this extent */ > __u64 fe_reserved64[2]; > __u32 fe_flags; /* FIEMAP_EXTENT_* flags for this extent */ > __u32 fe_device; /* device number for this extent */ > __u32 fe_reserved[2]; > }; > > This adds the 32-bit "fe_device" field, which would optionally be filled > in by the filesystem (zero otherwise). It would return the kernel device > number (i.e. st_dev), or for network filesystem (with FIEMAP_EXTENT_NET > set) this could just return an integer device number since the device > number is meaningless (and may conflict) on a remote system. Thank you for pointing this out. Indeed, I missed this case in this patch. The fe_device would be good to handle this. Is there a plan to merge that change? BTW, how about using __u64 given huge_encode_dev()? > Since AFAIK Btrfs also has multiple device support there are an increasing > number of places where this would be useful. That's cool! Actually, I think this'd be very versatile for different types of storages such as open-channel SSD and SMR; this patch already handles HM-SMR. In F2FS especially, background cleaner can migrate blocks seamlessly across the devices later. Thanks, > > Cheers, Andreas > > > > > Signed-off-by: Jaegeuk Kim > > --- > > fs/f2fs/data.c | 55 ++++++++++++++++--- > > fs/f2fs/f2fs.h | 29 ++++++++-- > > fs/f2fs/segment.c | 119 +++++++++++++++++++++++++++++------------ > > fs/f2fs/super.c | 138 ++++++++++++++++++++++++++++++++++++++---------- > > include/linux/f2fs_fs.h | 10 +++- > > 5 files changed, 277 insertions(+), 74 deletions(-) > > > > diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c > > index 47ded0c..e2be24e 100644 > > --- a/fs/f2fs/data.c > > +++ b/fs/f2fs/data.c > > @@ -88,6 +88,46 @@ static void f2fs_write_end_io(struct bio *bio) > > } > > > > /* > > + * Return true, if pre_bio's bdev is same as its target device. > > + */ > > +struct block_device *f2fs_target_device(struct f2fs_sb_info *sbi, > > + block_t blk_addr, struct bio *bio) > > +{ > > + struct block_device *bdev = sbi->sb->s_bdev; > > + int i; > > + > > + for (i = 0; i < sbi->s_ndevs; i++) { > > + if (FDEV(i).start_blk <= blk_addr && > > + FDEV(i).end_blk >= blk_addr) { > > + blk_addr -= FDEV(i).start_blk; > > + bdev = FDEV(i).bdev; > > + break; > > + } > > + } > > + if (bio) { > > + bio->bi_bdev = bdev; > > + bio->bi_iter.bi_sector = SECTOR_FROM_BLOCK(blk_addr); > > + } > > + return bdev; > > +} > > + > > +int f2fs_target_device_index(struct f2fs_sb_info *sbi, block_t blkaddr) > > +{ > > + int i; > > + > > + for (i = 0; i < sbi->s_ndevs; i++) > > + if (FDEV(i).start_blk <= blkaddr && FDEV(i).end_blk >= blkaddr) > > + return i; > > + return 0; > > +} > > + > > +static bool __same_bdev(struct f2fs_sb_info *sbi, > > + block_t blk_addr, struct bio *bio) > > +{ > > + return f2fs_target_device(sbi, blk_addr, NULL) == bio->bi_bdev; > > +} > > + > > +/* > > * Low-level block read/write IO operations. > > */ > > static struct bio *__bio_alloc(struct f2fs_sb_info *sbi, block_t blk_addr, > > @@ -97,8 +137,7 @@ static struct bio *__bio_alloc(struct f2fs_sb_info *sbi, block_t blk_addr, > > > > bio = f2fs_bio_alloc(npages); > > > > - bio->bi_bdev = sbi->sb->s_bdev; > > - bio->bi_iter.bi_sector = SECTOR_FROM_BLOCK(blk_addr); > > + f2fs_target_device(sbi, blk_addr, bio); > > bio->bi_end_io = is_read ? f2fs_read_end_io : f2fs_write_end_io; > > bio->bi_private = is_read ? NULL : sbi; > > > > @@ -273,7 +312,8 @@ void f2fs_submit_page_mbio(struct f2fs_io_info *fio) > > down_write(&io->io_rwsem); > > > > if (io->bio && (io->last_block_in_bio != fio->new_blkaddr - 1 || > > - (io->fio.op != fio->op || io->fio.op_flags != fio->op_flags))) > > + (io->fio.op != fio->op || io->fio.op_flags != fio->op_flags) || > > + !__same_bdev(sbi, fio->new_blkaddr, io->bio))) > > __submit_merged_bio(io); > > alloc_new: > > if (io->bio == NULL) { > > @@ -965,7 +1005,6 @@ static struct bio *f2fs_grab_bio(struct inode *inode, block_t blkaddr, > > { > > struct f2fs_sb_info *sbi = F2FS_I_SB(inode); > > struct fscrypt_ctx *ctx = NULL; > > - struct block_device *bdev = sbi->sb->s_bdev; > > struct bio *bio; > > > > if (f2fs_encrypted_inode(inode) && S_ISREG(inode->i_mode)) { > > @@ -983,8 +1022,7 @@ static struct bio *f2fs_grab_bio(struct inode *inode, block_t blkaddr, > > fscrypt_release_ctx(ctx); > > return ERR_PTR(-ENOMEM); > > } > > - bio->bi_bdev = bdev; > > - bio->bi_iter.bi_sector = SECTOR_FROM_BLOCK(blkaddr); > > + f2fs_target_device(sbi, blkaddr, bio); > > bio->bi_end_io = f2fs_read_end_io; > > bio->bi_private = ctx; > > > > @@ -1079,7 +1117,8 @@ static int f2fs_mpage_readpages(struct address_space *mapping, > > * This page will go to BIO. Do we need to send this > > * BIO off first? > > */ > > - if (bio && (last_block_in_bio != block_nr - 1)) { > > + if (bio && (last_block_in_bio != block_nr - 1 || > > + !__same_bdev(F2FS_I_SB(inode), block_nr, bio))) { > > submit_and_realloc: > > __submit_bio(F2FS_I_SB(inode), bio, DATA); > > bio = NULL; > > @@ -1738,6 +1777,8 @@ static ssize_t f2fs_direct_IO(struct kiocb *iocb, struct iov_iter *iter) > > return 0; > > if (test_opt(F2FS_I_SB(inode), LFS)) > > return 0; > > + if (F2FS_I_SB(inode)->s_ndevs) > > + return 0; > > > > trace_f2fs_direct_IO_enter(inode, offset, count, rw); > > > > diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h > > index 9650514..1737c45 100644 > > --- a/fs/f2fs/f2fs.h > > +++ b/fs/f2fs/f2fs.h > > @@ -730,6 +730,20 @@ struct f2fs_bio_info { > > struct rw_semaphore io_rwsem; /* blocking op for bio */ > > }; > > > > +#define FDEV(i) (sbi->devs[i]) > > +#define RDEV(i) (raw_super->devs[i]) > > +struct f2fs_dev_info { > > + struct block_device *bdev; > > + char path[MAX_PATH_LEN]; > > + unsigned int total_segments; > > + block_t start_blk; > > + block_t end_blk; > > +#ifdef CONFIG_BLK_DEV_ZONED > > + unsigned int nr_blkz; /* Total number of zones */ > > + u8 *blkz_type; /* Array of zones type */ > > +#endif > > +}; > > + > > enum inode_type { > > DIR_INODE, /* for dirty dir inode */ > > FILE_INODE, /* for dirty regular/symlink inode */ > > @@ -778,10 +792,8 @@ struct f2fs_sb_info { > > #endif > > > > #ifdef CONFIG_BLK_DEV_ZONED > > - unsigned int nr_blkz; /* Total number of zones */ > > unsigned int blocks_per_blkz; /* F2FS blocks per zone */ > > unsigned int log_blocks_per_blkz; /* log2 F2FS blocks per zone */ > > - u8 *blkz_type; /* Array of zones type */ > > #endif > > > > /* for node-related operations */ > > @@ -897,6 +909,8 @@ struct f2fs_sb_info { > > > > /* For shrinker support */ > > struct list_head s_list; > > + int s_ndevs; /* number of devices */ > > + struct f2fs_dev_info *devs; /* for device list */ > > struct mutex umount_mutex; > > unsigned int shrinker_run_no; > > > > @@ -2159,6 +2173,9 @@ void f2fs_submit_merged_bio_cond(struct f2fs_sb_info *, struct inode *, > > void f2fs_flush_merged_bios(struct f2fs_sb_info *); > > int f2fs_submit_page_bio(struct f2fs_io_info *); > > void f2fs_submit_page_mbio(struct f2fs_io_info *); > > +struct block_device *f2fs_target_device(struct f2fs_sb_info *, > > + block_t, struct bio *); > > +int f2fs_target_device_index(struct f2fs_sb_info *, block_t); > > void set_data_blkaddr(struct dnode_of_data *); > > void f2fs_update_data_blkaddr(struct dnode_of_data *, block_t); > > int reserve_new_blocks(struct dnode_of_data *, blkcnt_t); > > @@ -2446,11 +2463,15 @@ static inline int f2fs_sb_mounted_blkzoned(struct super_block *sb) > > > > #ifdef CONFIG_BLK_DEV_ZONED > > static inline int get_blkz_type(struct f2fs_sb_info *sbi, > > - block_t blkaddr) > > + struct block_device *bdev, block_t blkaddr) > > { > > unsigned int zno = blkaddr >> sbi->log_blocks_per_blkz; > > + int i; > > > > - return sbi->blkz_type[zno]; > > + for (i = 0; i < sbi->s_ndevs; i++) > > + if (FDEV(i).bdev == bdev) > > + return FDEV(i).blkz_type[zno]; > > + return -EINVAL; > > } > > #endif > > > > diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c > > index 7fb7dd3..ef727d1 100644 > > --- a/fs/f2fs/segment.c > > +++ b/fs/f2fs/segment.c > > @@ -403,6 +403,33 @@ void f2fs_balance_fs_bg(struct f2fs_sb_info *sbi) > > } > > } > > > > +static int __submit_flush_wait(struct block_device *bdev) > > +{ > > + struct bio *bio = f2fs_bio_alloc(0); > > + int ret; > > + > > + bio_set_op_attrs(bio, REQ_OP_WRITE, WRITE_FLUSH); > > + bio->bi_bdev = bdev; > > + ret = submit_bio_wait(bio); > > + bio_put(bio); > > + return ret; > > +} > > + > > +static int submit_flush_wait(struct f2fs_sb_info *sbi) > > +{ > > + int ret = __submit_flush_wait(sbi->sb->s_bdev); > > + int i; > > + > > + if (sbi->s_ndevs && !ret) { > > + for (i = 1; i < sbi->s_ndevs; i++) { > > + ret = __submit_flush_wait(FDEV(i).bdev); > > + if (ret) > > + break; > > + } > > + } > > + return ret; > > +} > > + > > static int issue_flush_thread(void *data) > > { > > struct f2fs_sb_info *sbi = data; > > @@ -413,25 +440,18 @@ static int issue_flush_thread(void *data) > > return 0; > > > > if (!llist_empty(&fcc->issue_list)) { > > - struct bio *bio; > > struct flush_cmd *cmd, *next; > > int ret; > > > > - bio = f2fs_bio_alloc(0); > > - > > fcc->dispatch_list = llist_del_all(&fcc->issue_list); > > fcc->dispatch_list = llist_reverse_order(fcc->dispatch_list); > > > > - bio->bi_bdev = sbi->sb->s_bdev; > > - bio_set_op_attrs(bio, REQ_OP_WRITE, WRITE_FLUSH); > > - ret = submit_bio_wait(bio); > > - > > + ret = submit_flush_wait(sbi); > > llist_for_each_entry_safe(cmd, next, > > fcc->dispatch_list, llnode) { > > cmd->ret = ret; > > complete(&cmd->wait); > > } > > - bio_put(bio); > > fcc->dispatch_list = NULL; > > } > > > > @@ -452,15 +472,11 @@ int f2fs_issue_flush(struct f2fs_sb_info *sbi) > > return 0; > > > > if (!test_opt(sbi, FLUSH_MERGE) || !atomic_read(&fcc->submit_flush)) { > > - struct bio *bio = f2fs_bio_alloc(0); > > int ret; > > > > atomic_inc(&fcc->submit_flush); > > - bio->bi_bdev = sbi->sb->s_bdev; > > - bio_set_op_attrs(bio, REQ_OP_WRITE, WRITE_FLUSH); > > - ret = submit_bio_wait(bio); > > + ret = submit_flush_wait(sbi); > > atomic_dec(&fcc->submit_flush); > > - bio_put(bio); > > return ret; > > } > > > > @@ -637,14 +653,18 @@ static void f2fs_submit_bio_wait_endio(struct bio *bio) > > > > /* this function is copied from blkdev_issue_discard from block/blk-lib.c */ > > static int __f2fs_issue_discard_async(struct f2fs_sb_info *sbi, > > - block_t blkstart, block_t blklen) > > + struct block_device *bdev, block_t blkstart, block_t blklen) > > { > > - struct block_device *bdev = sbi->sb->s_bdev; > > struct bio *bio = NULL; > > int err; > > > > trace_f2fs_issue_discard(sbi->sb, blkstart, blklen); > > > > + if (sbi->s_ndevs) { > > + int devi = f2fs_target_device_index(sbi, blkstart); > > + > > + blkstart -= FDEV(devi).start_blk; > > + } > > err = __blkdev_issue_discard(bdev, > > SECTOR_FROM_BLOCK(blkstart), > > SECTOR_FROM_BLOCK(blklen), > > @@ -662,18 +682,24 @@ static int __f2fs_issue_discard_async(struct f2fs_sb_info *sbi, > > } > > > > #ifdef CONFIG_BLK_DEV_ZONED > > -static int f2fs_issue_discard_zone(struct f2fs_sb_info *sbi, > > - block_t blkstart, block_t blklen) > > +static int __f2fs_issue_discard_zone(struct f2fs_sb_info *sbi, > > + struct block_device *bdev, block_t blkstart, block_t blklen) > > { > > - sector_t sector = SECTOR_FROM_BLOCK(blkstart); > > sector_t nr_sects = SECTOR_FROM_BLOCK(blklen); > > - struct block_device *bdev = sbi->sb->s_bdev; > > + sector_t sector; > > + int devi = 0; > > > > - if (nr_sects != bdev_zone_size(bdev)) { > > + if (sbi->s_ndevs) { > > + devi = f2fs_target_device_index(sbi, blkstart); > > + blkstart -= FDEV(devi).start_blk; > > + } > > + sector = SECTOR_FROM_BLOCK(blkstart); > > + > > + if (sector % bdev_zone_size(bdev) || nr_sects != bdev_zone_size(bdev)) { > > f2fs_msg(sbi->sb, KERN_INFO, > > - "Unaligned discard attempted (sector %llu + %llu)", > > - (unsigned long long)sector, > > - (unsigned long long)nr_sects); > > + "(%d) %s: Unaligned discard attempted (block %x + %x)", > > + devi, sbi->s_ndevs ? FDEV(devi).path: "", > > + blkstart, blklen); > > return -EIO; > > } > > > > @@ -682,14 +708,12 @@ static int f2fs_issue_discard_zone(struct f2fs_sb_info *sbi, > > * use regular discard if the drive supports it. For sequential > > * zones, reset the zone write pointer. > > */ > > - switch (get_blkz_type(sbi, blkstart)) { > > + switch (get_blkz_type(sbi, bdev, blkstart)) { > > > > case BLK_ZONE_TYPE_CONVENTIONAL: > > if (!blk_queue_discard(bdev_get_queue(bdev))) > > return 0; > > - return __f2fs_issue_discard_async(sbi, blkstart, > > - blklen); > > - > > + return __f2fs_issue_discard_async(sbi, bdev, blkstart, blklen); > > case BLK_ZONE_TYPE_SEQWRITE_REQ: > > case BLK_ZONE_TYPE_SEQWRITE_PREF: > > trace_f2fs_issue_reset_zone(sbi->sb, blkstart); > > @@ -702,14 +726,45 @@ static int f2fs_issue_discard_zone(struct f2fs_sb_info *sbi, > > } > > #endif > > > > +static int __issue_discard_async(struct f2fs_sb_info *sbi, > > + struct block_device *bdev, block_t blkstart, block_t blklen) > > +{ > > +#ifdef CONFIG_BLK_DEV_ZONED > > + if (f2fs_sb_mounted_blkzoned(sbi->sb) && > > + bdev_zoned_model(bdev) != BLK_ZONED_NONE) > > + return __f2fs_issue_discard_zone(sbi, bdev, blkstart, blklen); > > +#endif > > + return __f2fs_issue_discard_async(sbi, bdev, blkstart, blklen); > > +} > > + > > static int f2fs_issue_discard(struct f2fs_sb_info *sbi, > > block_t blkstart, block_t blklen) > > { > > + sector_t start = blkstart, len = 0; > > + struct block_device *bdev; > > struct seg_entry *se; > > unsigned int offset; > > block_t i; > > + int err = 0; > > + > > + bdev = f2fs_target_device(sbi, blkstart, NULL); > > + > > + for (i = blkstart; i < blkstart + blklen; i++, len++) { > > + if (i != start) { > > + struct block_device *bdev2 = > > + f2fs_target_device(sbi, i, NULL); > > + > > + if (bdev2 != bdev) { > > + err = __issue_discard_async(sbi, bdev, > > + start, len); > > + if (err) > > + return err; > > + bdev = bdev2; > > + start = i; > > + len = 0; > > + } > > + } > > > > - for (i = blkstart; i < blkstart + blklen; i++) { > > se = get_seg_entry(sbi, GET_SEGNO(sbi, i)); > > offset = GET_BLKOFF_FROM_SEG0(sbi, i); > > > > @@ -717,11 +772,9 @@ static int f2fs_issue_discard(struct f2fs_sb_info *sbi, > > sbi->discard_blks--; > > } > > > > -#ifdef CONFIG_BLK_DEV_ZONED > > - if (f2fs_sb_mounted_blkzoned(sbi->sb)) > > - return f2fs_issue_discard_zone(sbi, blkstart, blklen); > > -#endif > > - return __f2fs_issue_discard_async(sbi, blkstart, blklen); > > + if (len) > > + err = __issue_discard_async(sbi, bdev, start, len); > > + return err; > > } > > > > static void __add_discard_entry(struct f2fs_sb_info *sbi, > > diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c > > index eca9aea..4ccbb86 100644 > > --- a/fs/f2fs/super.c > > +++ b/fs/f2fs/super.c > > @@ -713,6 +713,19 @@ static void destroy_percpu_info(struct f2fs_sb_info *sbi) > > percpu_counter_destroy(&sbi->total_valid_inode_count); > > } > > > > +static void destroy_device_list(struct f2fs_sb_info *sbi) > > +{ > > + int i; > > + > > + for (i = 0; i < sbi->s_ndevs; i++) { > > + blkdev_put(FDEV(i).bdev, FMODE_EXCL); > > +#ifdef CONFIG_BLK_DEV_ZONED > > + kfree(FDEV(i).blkz_type); > > +#endif > > + } > > + kfree(sbi->devs); > > +} > > + > > static void f2fs_put_super(struct super_block *sb) > > { > > struct f2fs_sb_info *sbi = F2FS_SB(sb); > > @@ -773,6 +786,8 @@ static void f2fs_put_super(struct super_block *sb) > > crypto_free_shash(sbi->s_chksum_driver); > > kfree(sbi->raw_super); > > > > + destroy_device_list(sbi); > > + > > destroy_percpu_info(sbi); > > kfree(sbi); > > } > > @@ -1516,9 +1531,9 @@ static int init_percpu_info(struct f2fs_sb_info *sbi) > > } > > > > #ifdef CONFIG_BLK_DEV_ZONED > > -static int init_blkz_info(struct f2fs_sb_info *sbi) > > +static int init_blkz_info(struct f2fs_sb_info *sbi, int devi) > > { > > - struct block_device *bdev = sbi->sb->s_bdev; > > + struct block_device *bdev = FDEV(devi).bdev; > > sector_t nr_sectors = bdev->bd_part->nr_sects; > > sector_t sector = 0; > > struct blk_zone *zones; > > @@ -1529,15 +1544,21 @@ static int init_blkz_info(struct f2fs_sb_info *sbi) > > if (!f2fs_sb_mounted_blkzoned(sbi->sb)) > > return 0; > > > > + if (sbi->blocks_per_blkz && sbi->blocks_per_blkz != > > + SECTOR_TO_BLOCK(bdev_zone_size(bdev))) > > + return -EINVAL; > > sbi->blocks_per_blkz = SECTOR_TO_BLOCK(bdev_zone_size(bdev)); > > + if (sbi->log_blocks_per_blkz && sbi->log_blocks_per_blkz != > > + __ilog2_u32(sbi->blocks_per_blkz)) > > + return -EINVAL; > > sbi->log_blocks_per_blkz = __ilog2_u32(sbi->blocks_per_blkz); > > - sbi->nr_blkz = SECTOR_TO_BLOCK(nr_sectors) >> > > - sbi->log_blocks_per_blkz; > > + FDEV(devi).nr_blkz = SECTOR_TO_BLOCK(nr_sectors) >> > > + sbi->log_blocks_per_blkz; > > if (nr_sectors & (bdev_zone_size(bdev) - 1)) > > - sbi->nr_blkz++; > > + FDEV(devi).nr_blkz++; > > > > - sbi->blkz_type = kmalloc(sbi->nr_blkz, GFP_KERNEL); > > - if (!sbi->blkz_type) > > + FDEV(devi).blkz_type = kmalloc(FDEV(devi).nr_blkz, GFP_KERNEL); > > + if (!FDEV(devi).blkz_type) > > return -ENOMEM; > > > > #define F2FS_REPORT_NR_ZONES 4096 > > @@ -1562,7 +1583,7 @@ static int init_blkz_info(struct f2fs_sb_info *sbi) > > } > > > > for (i = 0; i < nr_zones; i++) { > > - sbi->blkz_type[n] = zones[i].type; > > + FDEV(devi).blkz_type[n] = zones[i].type; > > sector += zones[i].len; > > n++; > > } > > @@ -1666,6 +1687,77 @@ int f2fs_commit_super(struct f2fs_sb_info *sbi, bool recover) > > return err; > > } > > > > +static int f2fs_scan_devices(struct f2fs_sb_info *sbi) > > +{ > > + struct f2fs_super_block *raw_super = F2FS_RAW_SUPER(sbi); > > + int i; > > + > > + for (i = 0; i < MAX_DEVICES; i++) { > > + if (!RDEV(i).path[0]) > > + return 0; > > + > > + if (i == 0) { > > + sbi->devs = kzalloc(sizeof(struct f2fs_dev_info) * > > + MAX_DEVICES, GFP_KERNEL); > > + if (!sbi->devs) > > + return -ENOMEM; > > + } > > + > > + memcpy(FDEV(i).path, RDEV(i).path, MAX_PATH_LEN); > > + FDEV(i).total_segments = le32_to_cpu(RDEV(i).total_segments); > > + if (i == 0) { > > + FDEV(i).start_blk = 0; > > + FDEV(i).end_blk = FDEV(i).start_blk + > > + (FDEV(i).total_segments << > > + sbi->log_blocks_per_seg) - 1 + > > + le32_to_cpu(raw_super->segment0_blkaddr); > > + } else { > > + FDEV(i).start_blk = FDEV(i - 1).end_blk + 1; > > + FDEV(i).end_blk = FDEV(i).start_blk + > > + (FDEV(i).total_segments << > > + sbi->log_blocks_per_seg) - 1; > > + } > > + > > + FDEV(i).bdev = blkdev_get_by_path(FDEV(i).path, > > + sbi->sb->s_mode, sbi->sb->s_type); > > + if (IS_ERR(FDEV(i).bdev)) > > + return PTR_ERR(FDEV(i).bdev); > > + > > + /* to release errored devices */ > > + sbi->s_ndevs = i + 1; > > + > > +#ifdef CONFIG_BLK_DEV_ZONED > > + if (bdev_zoned_model(FDEV(i).bdev) == BLK_ZONED_HM && > > + !f2fs_sb_mounted_blkzoned(sbi->sb)) { > > + f2fs_msg(sbi->sb, KERN_ERR, > > + "Zoned block device feature not enabled\n"); > > + return -EINVAL; > > + } > > + if (bdev_zoned_model(FDEV(i).bdev) != BLK_ZONED_NONE) { > > + if (init_blkz_info(sbi, i)) { > > + f2fs_msg(sbi->sb, KERN_ERR, > > + "Failed to initialize F2FS blkzone information"); > > + return -EINVAL; > > + } > > + f2fs_msg(sbi->sb, KERN_INFO, > > + "Mount Device [%2d]: %20s, %8u, %8x - %8x (zone: %s)", > > + i, FDEV(i).path, > > + FDEV(i).total_segments, > > + FDEV(i).start_blk, FDEV(i).end_blk, > > + bdev_zoned_model(FDEV(i).bdev) == BLK_ZONED_HA ? > > + "Host-aware" : "Host-managed"); > > + continue; > > + } > > +#endif > > + f2fs_msg(sbi->sb, KERN_INFO, > > + "Mount Device [%2d]: %20s, %8u, %8x - %8x", > > + i, FDEV(i).path, > > + FDEV(i).total_segments, > > + FDEV(i).start_blk, FDEV(i).end_blk); > > + } > > + return 0; > > +} > > + > > static int f2fs_fill_super(struct super_block *sb, void *data, int silent) > > { > > struct f2fs_sb_info *sbi; > > @@ -1724,15 +1816,7 @@ static int f2fs_fill_super(struct super_block *sb, void *data, int silent) > > "Zoned block device support is not enabled\n"); > > goto free_sb_buf; > > } > > -#else > > - if (bdev_zoned_model(sb->s_bdev) == BLK_ZONED_HM && > > - !f2fs_sb_mounted_blkzoned(sb)) { > > - f2fs_msg(sb, KERN_ERR, > > - "Zoned block device feature not enabled\n"); > > - goto free_sb_buf; > > - } > > #endif > > - > > default_options(sbi); > > /* parse mount options */ > > options = kstrdup((const char *)data, GFP_KERNEL); > > @@ -1802,6 +1886,13 @@ static int f2fs_fill_super(struct super_block *sb, void *data, int silent) > > goto free_meta_inode; > > } > > > > + /* Initialize device list */ > > + err = f2fs_scan_devices(sbi); > > + if (err) { > > + f2fs_msg(sb, KERN_ERR, "Failed to find devices"); > > + goto free_devices; > > + } > > + > > sbi->total_valid_node_count = > > le32_to_cpu(sbi->ckpt->valid_node_count); > > percpu_counter_set(&sbi->total_valid_inode_count, > > @@ -1820,15 +1911,6 @@ static int f2fs_fill_super(struct super_block *sb, void *data, int silent) > > > > init_ino_entry_info(sbi); > > > > -#ifdef CONFIG_BLK_DEV_ZONED > > - err = init_blkz_info(sbi); > > - if (err) { > > - f2fs_msg(sb, KERN_ERR, > > - "Failed to initialize F2FS blkzone information"); > > - goto free_blkz; > > - } > > -#endif > > - > > /* setup f2fs internal modules */ > > err = build_segment_manager(sbi); > > if (err) { > > @@ -2007,10 +2089,8 @@ static int f2fs_fill_super(struct super_block *sb, void *data, int silent) > > destroy_node_manager(sbi); > > free_sm: > > destroy_segment_manager(sbi); > > -#ifdef CONFIG_BLK_DEV_ZONED > > -free_blkz: > > - kfree(sbi->blkz_type); > > -#endif > > +free_devices: > > + destroy_device_list(sbi); > > kfree(sbi->ckpt); > > free_meta_inode: > > make_bad_inode(sbi->meta_inode); > > diff --git a/include/linux/f2fs_fs.h b/include/linux/f2fs_fs.h > > index 422630b..cea41a1 100644 > > --- a/include/linux/f2fs_fs.h > > +++ b/include/linux/f2fs_fs.h > > @@ -52,10 +52,17 @@ > > > > #define VERSION_LEN 256 > > #define MAX_VOLUME_NAME 512 > > +#define MAX_PATH_LEN 64 > > +#define MAX_DEVICES 8 > > > > /* > > * For superblock > > */ > > +struct f2fs_device { > > + __u8 path[MAX_PATH_LEN]; > > + __le32 total_segments; > > +} __packed; > > + > > struct f2fs_super_block { > > __le32 magic; /* Magic Number */ > > __le16 major_ver; /* Major Version */ > > @@ -94,7 +101,8 @@ struct f2fs_super_block { > > __le32 feature; /* defined features */ > > __u8 encryption_level; /* versioning level for encryption */ > > __u8 encrypt_pw_salt[16]; /* Salt used for string2key algorithm */ > > - __u8 reserved[871]; /* valid reserved region */ > > + struct f2fs_device devs[MAX_DEVICES]; /* device list */ > > + __u8 reserved[327]; /* valid reserved region */ > > } __packed; > > > > /* > > -- > > 2.8.3 > > Cheers, Andreas From ahferroin7 at gmail.com Thu Nov 10 12:25:37 2016 From: ahferroin7 at gmail.com (Austin S. Hemmelgarn) Date: Thu, 10 Nov 2016 12:25:37 -0000 Subject: [lustre-devel] [PATCH] f2fs: support multiple devices In-Reply-To: <4ec4d8f2-da23-762d-ba81-12e76ed09793@cn.fujitsu.com> References: <20161109205653.70061-1-jaegeuk@kernel.org> <0D1876A8-BB77-4C1A-BE4F-B4A0E81DD4EA@dilger.ca> <4ec4d8f2-da23-762d-ba81-12e76ed09793@cn.fujitsu.com> Message-ID: <749156c9-2b3e-4210-a89b-2d664f9d2fc2@gmail.com> On 2016-11-09 21:29, Qu Wenruo wrote: > > > At 11/10/2016 06:57 AM, Andreas Dilger wrote: >> On Nov 9, 2016, at 1:56 PM, Jaegeuk Kim wrote: >>> >>> This patch implements multiple devices support for f2fs. >>> Given multiple devices by mkfs.f2fs, f2fs shows them entirely as one big >>> volume under one f2fs instance. >>> >>> Internal block management is very simple, but we will modify block >>> allocation and background GC policy to boost IO speed by exploiting them >>> accoording to each device speed. >> >> How will you integrate this into FIEMAP, since it is now possible if a >> file is split across multiple devices then it will return ambiguous block >> numbers for a file. I've been meaning to merge the FIEMAP handling in >> Lustre to support multiple devices in a single filesystem, so that this >> can be detected in userspace. >> >> struct ll_fiemap_extent { >> __u64 fe_logical; /* logical offset in bytes for the start of >> * the extent from the beginning of the file >> */ >> __u64 fe_physical; /* physical offset in bytes for the start >> * of the extent from the beginning of the >> disk >> */ >> __u64 fe_length; /* length in bytes for this extent */ >> __u64 fe_reserved64[2]; >> __u32 fe_flags; /* FIEMAP_EXTENT_* flags for this extent */ >> __u32 fe_device; /* device number for this extent */ >> __u32 fe_reserved[2]; >> }; > > Btrfs introduce a new layer for multi-device (even for single device). > > So fiemap returned by btrfs is never real device bytenr, but logical > address in btrfs logical address space. > Much like traditional soft RAID. This is a really important point. BTRFS does a good job of segregating the layers here, so the file-level allocator really has very limited knowledge of the underlying storage, which in turn means that adding this to BTRFS would likely be a pretty invasive change for the FIEMAP implementation. > >> >> This adds the 32-bit "fe_device" field, which would optionally be filled >> in by the filesystem (zero otherwise). It would return the kernel device >> number (i.e. st_dev), or for network filesystem (with FIEMAP_EXTENT_NET >> set) this could just return an integer device number since the device >> number is meaningless (and may conflict) on a remote system. >> >> Since AFAIK Btrfs also has multiple device support there are an >> increasing >> number of places where this would be useful. > > AFAIK, btrfs multi-device is here due to scrub with its data/meta csum. It's also here for an attempt at parity with ZFS. > > Unlike device-mapper based multi-device, btrfs has csum so it can detect > which mirror is correct. > This makes btrfs scrub a little better than soft raid. > For example, for RAID1 if two mirror differs from each other, btrfs can > find the correct one and rewrite it into the other mirror. > > And further more, btrfs supports snapshot and is faster than > device-mapper based snapshot(LVM). > This makes it a little more worthy to implement multi-device support in > btrfs. > > > But for f2fs, no data csum, no snapshot. > I don't really see the point to use so many codes to implement it, > especially we can use mdadm or LVM to implement it. I'd tend to agree on this, if it weren't for the fact that this looks to me like preparation for implementing storage tiering, which neither LVM nor MD have a good implementation of. Whether or not such functionality is worthwhile for the embedded systems that F2FS typically targets is another story of course. > > > Not to mention btrfs multi-device support still has quite a lot of bugs, > like scrub can corrupt correct data stripes. This sounds like you're lumping raid5/6 code in with the general multi-device code, which is not a good way of describing things for multiple reasons. Pretty much, if you're using just raid1 mode, without compression, on reasonable storage devices, things are rock-solid relative to the rest of BTRFS. Yes, there is a bug with compression and multiple copies of things, but that requires a pretty spectacular device failure to manifest, and it impacts single device mode too (it happens in dup profiles as well as raid1). As far as the raid5/6 stuff, that shouldn't have been merged in the state it was in when it got merged, and should probably just be rewritten from the ground up. > > Personally speaking, I am not a fan of btrfs multi-device management, > despite the above advantage. > As the complexity is really not worthy. > (So I think XFS with LVM is much better than Btrfs considering the > stability) From bigeasy at linutronix.de Sat Nov 26 23:14:54 2016 From: bigeasy at linutronix.de (Sebastian Andrzej Siewior) Date: Sat, 26 Nov 2016 23:14:54 -0000 Subject: [lustre-devel] [PATCH 21/22] staging/lustre/libcfs: Convert to hotplug state machine In-Reply-To: <20161126231350.10321-1-bigeasy@linutronix.de> References: <20161126231350.10321-1-bigeasy@linutronix.de> Message-ID: <20161126231350.10321-22-bigeasy@linutronix.de> From: Anna-Maria Gleixner Install the callbacks via the state machine. Cc: Oleg Drokin Cc: Andreas Dilger Cc: James Simmons Cc: Greg Kroah-Hartman Cc: lustre-devel at lists.lustre.org Cc: devel at driverdev.osuosl.org Signed-off-by: Anna-Maria Gleixner Signed-off-by: Sebastian Andrzej Siewior --- .../staging/lustre/lnet/libcfs/linux/linux-cpu.c | 89 ++++++++++++---------- include/linux/cpuhotplug.h | 1 + 2 files changed, 50 insertions(+), 40 deletions(-) diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c index e8b1a61420de..a75113ab2903 100644 --- a/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c +++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c @@ -73,6 +73,9 @@ struct cfs_cpt_data { }; static struct cfs_cpt_data cpt_data; +#ifdef CONFIG_HOTPLUG_CPU +static enum cpuhp_state lustre_cpu_online; +#endif static void cfs_node_to_cpumask(int node, cpumask_t *mask) @@ -942,48 +945,38 @@ cfs_cpt_table_create_pattern(char *pattern) } #ifdef CONFIG_HOTPLUG_CPU -static int -cfs_cpu_notify(struct notifier_block *self, unsigned long action, void *hcpu) + +static void cfs_cpu_incr_cpt_version(void) { - unsigned int cpu = (unsigned long)hcpu; - bool warn; - - switch (action) { - case CPU_DEAD: - case CPU_DEAD_FROZEN: - case CPU_ONLINE: - case CPU_ONLINE_FROZEN: - spin_lock(&cpt_data.cpt_lock); - cpt_data.cpt_version++; - spin_unlock(&cpt_data.cpt_lock); - /* Fall through */ - default: - if (action != CPU_DEAD && action != CPU_DEAD_FROZEN) { - CDEBUG(D_INFO, "CPU changed [cpu %u action %lx]\n", - cpu, action); - break; - } - - mutex_lock(&cpt_data.cpt_mutex); - /* if all HTs in a core are offline, it may break affinity */ - cpumask_copy(cpt_data.cpt_cpumask, - topology_sibling_cpumask(cpu)); - warn = cpumask_any_and(cpt_data.cpt_cpumask, - cpu_online_mask) >= nr_cpu_ids; - mutex_unlock(&cpt_data.cpt_mutex); - CDEBUG(warn ? D_WARNING : D_INFO, - "Lustre: can't support CPU plug-out well now, performance and stability could be impacted [CPU %u action: %lx]\n", - cpu, action); - } - - return NOTIFY_OK; + spin_lock(&cpt_data.cpt_lock); + cpt_data.cpt_version++; + spin_unlock(&cpt_data.cpt_lock); } -static struct notifier_block cfs_cpu_notifier = { - .notifier_call = cfs_cpu_notify, - .priority = 0 -}; +static int cfs_cpu_online(unsigned int cpu) +{ + cfs_cpu_incr_cpt_version(); + return 0; +} +static int cfs_cpu_dead(unsigned int cpu) +{ + bool warn; + int next; + + cfs_cpu_incr_cpt_version(); + + mutex_lock(&cpt_data.cpt_mutex); + /* if all HTs in a core are offline, it may break affinity */ + cpumask_copy(cpt_data.cpt_cpumask, topology_sibling_cpumask(cpu)); + next = cpumask_any_and(cpt_data.cpt_cpumask, cpu_online_mask); + warn = next >= nr_cpu_ids; + mutex_unlock(&cpt_data.cpt_mutex); + CDEBUG(warn ? D_WARNING : D_INFO, + "Lustre: can't support CPU plug-out well now, performance and stability could be impacted [CPU %u]\n", + cpu); + return 0; +} #endif void @@ -993,7 +986,9 @@ cfs_cpu_fini(void) cfs_cpt_table_free(cfs_cpt_table); #ifdef CONFIG_HOTPLUG_CPU - unregister_hotcpu_notifier(&cfs_cpu_notifier); + if (lustre_cpu_online) + cpuhp_remove_state_nocalls(lustre_cpu_online); + cpuhp_remove_state_nocalls(CPUHP_LUSTRE_CFS_DEAD); #endif if (cpt_data.cpt_cpumask) LIBCFS_FREE(cpt_data.cpt_cpumask, cpumask_size()); @@ -1002,6 +997,10 @@ cfs_cpu_fini(void) int cfs_cpu_init(void) { +#ifdef CONFIG_HOTPLUG_CPU + int ret; +#endif + LASSERT(!cfs_cpt_table); memset(&cpt_data, 0, sizeof(cpt_data)); @@ -1016,7 +1015,17 @@ cfs_cpu_init(void) mutex_init(&cpt_data.cpt_mutex); #ifdef CONFIG_HOTPLUG_CPU - register_hotcpu_notifier(&cfs_cpu_notifier); + ret = cpuhp_setup_state_nocalls(CPUHP_LUSTRE_CFS_DEAD, + "staging/lustre/cfe:dead", NULL, + cfs_cpu_dead); + if (ret < 0) + goto failed; + ret = cpuhp_setup_state_nocalls(CPUHP_AP_ONLINE_DYN, + "staging/lustre/cfe:online", + cfs_cpu_online, NULL); + if (ret < 0) + goto failed; + lustre_cpu_online = ret; #endif if (*cpu_pattern != 0) { diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h index 22acee76cf4c..141c3be242d1 100644 --- a/include/linux/cpuhotplug.h +++ b/include/linux/cpuhotplug.h @@ -41,6 +41,7 @@ enum cpuhp_state { CPUHP_NET_DEV_DEAD, CPUHP_PCI_XGENE_DEAD, CPUHP_IOMMU_INTEL_DEAD, + CPUHP_LUSTRE_CFS_DEAD, CPUHP_WORKQUEUE_PREP, CPUHP_POWER_NUMA_PREPARE, CPUHP_HRTIMERS_PREPARE, -- 2.10.2 From darrick.wong at oracle.com Wed Nov 9 23:05:20 2016 From: darrick.wong at oracle.com (Darrick J. Wong) Date: Wed, 09 Nov 2016 23:05:20 -0000 Subject: [lustre-devel] [PATCH] f2fs: support multiple devices In-Reply-To: <0D1876A8-BB77-4C1A-BE4F-B4A0E81DD4EA@dilger.ca> References: <20161109205653.70061-1-jaegeuk@kernel.org> <0D1876A8-BB77-4C1A-BE4F-B4A0E81DD4EA@dilger.ca> Message-ID: <20161109230511.GA16807@birch.djwong.org> On Wed, Nov 09, 2016 at 03:57:53PM -0700, Andreas Dilger wrote: > On Nov 9, 2016, at 1:56 PM, Jaegeuk Kim wrote: > > > > This patch implements multiple devices support for f2fs. > > Given multiple devices by mkfs.f2fs, f2fs shows them entirely as one big > > volume under one f2fs instance. > > > > Internal block management is very simple, but we will modify block > > allocation and background GC policy to boost IO speed by exploiting them > > accoording to each device speed. > > How will you integrate this into FIEMAP, since it is now possible if a > file is split across multiple devices then it will return ambiguous block > numbers for a file. I've been meaning to merge the FIEMAP handling in > Lustre to support multiple devices in a single filesystem, so that this > can be detected in userspace. > > struct ll_fiemap_extent { > __u64 fe_logical; /* logical offset in bytes for the start of > * the extent from the beginning of the file > */ > __u64 fe_physical; /* physical offset in bytes for the start > * of the extent from the beginning of the disk > */ > __u64 fe_length; /* length in bytes for this extent */ > __u64 fe_reserved64[2]; > __u32 fe_flags; /* FIEMAP_EXTENT_* flags for this extent */ > __u32 fe_device; /* device number for this extent */ > __u32 fe_reserved[2]; > }; > > This adds the 32-bit "fe_device" field, which would optionally be filled > in by the filesystem (zero otherwise). It would return the kernel device > number (i.e. st_dev), or for network filesystem (with FIEMAP_EXTENT_NET > set) this could just return an integer device number since the device > number is meaningless (and may conflict) on a remote system. Same field size and similar meaning as the the fmr_device field in GETFSMAP, at least for blockdev based filesystems. Intriguing.... ;) For GETFSMAP the number in fmr_device is either a device-unique 32-bit cookie or the implementer can set a magic FMH_OF_DEV_T flags bit marking it as an actual dev_t. What's the fe_device number mean for network filesystems? Magic unique number? > Since AFAIK Btrfs also has multiple device support there are an increasing > number of places where this would be useful. Ditto XFS. --D > > Cheers, Andreas > > > > > Signed-off-by: Jaegeuk Kim > > --- > > fs/f2fs/data.c | 55 ++++++++++++++++--- > > fs/f2fs/f2fs.h | 29 ++++++++-- > > fs/f2fs/segment.c | 119 +++++++++++++++++++++++++++++------------ > > fs/f2fs/super.c | 138 ++++++++++++++++++++++++++++++++++++++---------- > > include/linux/f2fs_fs.h | 10 +++- > > 5 files changed, 277 insertions(+), 74 deletions(-) > > > > diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c > > index 47ded0c..e2be24e 100644 > > --- a/fs/f2fs/data.c > > +++ b/fs/f2fs/data.c > > @@ -88,6 +88,46 @@ static void f2fs_write_end_io(struct bio *bio) > > } > > > > /* > > + * Return true, if pre_bio's bdev is same as its target device. > > + */ > > +struct block_device *f2fs_target_device(struct f2fs_sb_info *sbi, > > + block_t blk_addr, struct bio *bio) > > +{ > > + struct block_device *bdev = sbi->sb->s_bdev; > > + int i; > > + > > + for (i = 0; i < sbi->s_ndevs; i++) { > > + if (FDEV(i).start_blk <= blk_addr && > > + FDEV(i).end_blk >= blk_addr) { > > + blk_addr -= FDEV(i).start_blk; > > + bdev = FDEV(i).bdev; > > + break; > > + } > > + } > > + if (bio) { > > + bio->bi_bdev = bdev; > > + bio->bi_iter.bi_sector = SECTOR_FROM_BLOCK(blk_addr); > > + } > > + return bdev; > > +} > > + > > +int f2fs_target_device_index(struct f2fs_sb_info *sbi, block_t blkaddr) > > +{ > > + int i; > > + > > + for (i = 0; i < sbi->s_ndevs; i++) > > + if (FDEV(i).start_blk <= blkaddr && FDEV(i).end_blk >= blkaddr) > > + return i; > > + return 0; > > +} > > + > > +static bool __same_bdev(struct f2fs_sb_info *sbi, > > + block_t blk_addr, struct bio *bio) > > +{ > > + return f2fs_target_device(sbi, blk_addr, NULL) == bio->bi_bdev; > > +} > > + > > +/* > > * Low-level block read/write IO operations. > > */ > > static struct bio *__bio_alloc(struct f2fs_sb_info *sbi, block_t blk_addr, > > @@ -97,8 +137,7 @@ static struct bio *__bio_alloc(struct f2fs_sb_info *sbi, block_t blk_addr, > > > > bio = f2fs_bio_alloc(npages); > > > > - bio->bi_bdev = sbi->sb->s_bdev; > > - bio->bi_iter.bi_sector = SECTOR_FROM_BLOCK(blk_addr); > > + f2fs_target_device(sbi, blk_addr, bio); > > bio->bi_end_io = is_read ? f2fs_read_end_io : f2fs_write_end_io; > > bio->bi_private = is_read ? NULL : sbi; > > > > @@ -273,7 +312,8 @@ void f2fs_submit_page_mbio(struct f2fs_io_info *fio) > > down_write(&io->io_rwsem); > > > > if (io->bio && (io->last_block_in_bio != fio->new_blkaddr - 1 || > > - (io->fio.op != fio->op || io->fio.op_flags != fio->op_flags))) > > + (io->fio.op != fio->op || io->fio.op_flags != fio->op_flags) || > > + !__same_bdev(sbi, fio->new_blkaddr, io->bio))) > > __submit_merged_bio(io); > > alloc_new: > > if (io->bio == NULL) { > > @@ -965,7 +1005,6 @@ static struct bio *f2fs_grab_bio(struct inode *inode, block_t blkaddr, > > { > > struct f2fs_sb_info *sbi = F2FS_I_SB(inode); > > struct fscrypt_ctx *ctx = NULL; > > - struct block_device *bdev = sbi->sb->s_bdev; > > struct bio *bio; > > > > if (f2fs_encrypted_inode(inode) && S_ISREG(inode->i_mode)) { > > @@ -983,8 +1022,7 @@ static struct bio *f2fs_grab_bio(struct inode *inode, block_t blkaddr, > > fscrypt_release_ctx(ctx); > > return ERR_PTR(-ENOMEM); > > } > > - bio->bi_bdev = bdev; > > - bio->bi_iter.bi_sector = SECTOR_FROM_BLOCK(blkaddr); > > + f2fs_target_device(sbi, blkaddr, bio); > > bio->bi_end_io = f2fs_read_end_io; > > bio->bi_private = ctx; > > > > @@ -1079,7 +1117,8 @@ static int f2fs_mpage_readpages(struct address_space *mapping, > > * This page will go to BIO. Do we need to send this > > * BIO off first? > > */ > > - if (bio && (last_block_in_bio != block_nr - 1)) { > > + if (bio && (last_block_in_bio != block_nr - 1 || > > + !__same_bdev(F2FS_I_SB(inode), block_nr, bio))) { > > submit_and_realloc: > > __submit_bio(F2FS_I_SB(inode), bio, DATA); > > bio = NULL; > > @@ -1738,6 +1777,8 @@ static ssize_t f2fs_direct_IO(struct kiocb *iocb, struct iov_iter *iter) > > return 0; > > if (test_opt(F2FS_I_SB(inode), LFS)) > > return 0; > > + if (F2FS_I_SB(inode)->s_ndevs) > > + return 0; > > > > trace_f2fs_direct_IO_enter(inode, offset, count, rw); > > > > diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h > > index 9650514..1737c45 100644 > > --- a/fs/f2fs/f2fs.h > > +++ b/fs/f2fs/f2fs.h > > @@ -730,6 +730,20 @@ struct f2fs_bio_info { > > struct rw_semaphore io_rwsem; /* blocking op for bio */ > > }; > > > > +#define FDEV(i) (sbi->devs[i]) > > +#define RDEV(i) (raw_super->devs[i]) > > +struct f2fs_dev_info { > > + struct block_device *bdev; > > + char path[MAX_PATH_LEN]; > > + unsigned int total_segments; > > + block_t start_blk; > > + block_t end_blk; > > +#ifdef CONFIG_BLK_DEV_ZONED > > + unsigned int nr_blkz; /* Total number of zones */ > > + u8 *blkz_type; /* Array of zones type */ > > +#endif > > +}; > > + > > enum inode_type { > > DIR_INODE, /* for dirty dir inode */ > > FILE_INODE, /* for dirty regular/symlink inode */ > > @@ -778,10 +792,8 @@ struct f2fs_sb_info { > > #endif > > > > #ifdef CONFIG_BLK_DEV_ZONED > > - unsigned int nr_blkz; /* Total number of zones */ > > unsigned int blocks_per_blkz; /* F2FS blocks per zone */ > > unsigned int log_blocks_per_blkz; /* log2 F2FS blocks per zone */ > > - u8 *blkz_type; /* Array of zones type */ > > #endif > > > > /* for node-related operations */ > > @@ -897,6 +909,8 @@ struct f2fs_sb_info { > > > > /* For shrinker support */ > > struct list_head s_list; > > + int s_ndevs; /* number of devices */ > > + struct f2fs_dev_info *devs; /* for device list */ > > struct mutex umount_mutex; > > unsigned int shrinker_run_no; > > > > @@ -2159,6 +2173,9 @@ void f2fs_submit_merged_bio_cond(struct f2fs_sb_info *, struct inode *, > > void f2fs_flush_merged_bios(struct f2fs_sb_info *); > > int f2fs_submit_page_bio(struct f2fs_io_info *); > > void f2fs_submit_page_mbio(struct f2fs_io_info *); > > +struct block_device *f2fs_target_device(struct f2fs_sb_info *, > > + block_t, struct bio *); > > +int f2fs_target_device_index(struct f2fs_sb_info *, block_t); > > void set_data_blkaddr(struct dnode_of_data *); > > void f2fs_update_data_blkaddr(struct dnode_of_data *, block_t); > > int reserve_new_blocks(struct dnode_of_data *, blkcnt_t); > > @@ -2446,11 +2463,15 @@ static inline int f2fs_sb_mounted_blkzoned(struct super_block *sb) > > > > #ifdef CONFIG_BLK_DEV_ZONED > > static inline int get_blkz_type(struct f2fs_sb_info *sbi, > > - block_t blkaddr) > > + struct block_device *bdev, block_t blkaddr) > > { > > unsigned int zno = blkaddr >> sbi->log_blocks_per_blkz; > > + int i; > > > > - return sbi->blkz_type[zno]; > > + for (i = 0; i < sbi->s_ndevs; i++) > > + if (FDEV(i).bdev == bdev) > > + return FDEV(i).blkz_type[zno]; > > + return -EINVAL; > > } > > #endif > > > > diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c > > index 7fb7dd3..ef727d1 100644 > > --- a/fs/f2fs/segment.c > > +++ b/fs/f2fs/segment.c > > @@ -403,6 +403,33 @@ void f2fs_balance_fs_bg(struct f2fs_sb_info *sbi) > > } > > } > > > > +static int __submit_flush_wait(struct block_device *bdev) > > +{ > > + struct bio *bio = f2fs_bio_alloc(0); > > + int ret; > > + > > + bio_set_op_attrs(bio, REQ_OP_WRITE, WRITE_FLUSH); > > + bio->bi_bdev = bdev; > > + ret = submit_bio_wait(bio); > > + bio_put(bio); > > + return ret; > > +} > > + > > +static int submit_flush_wait(struct f2fs_sb_info *sbi) > > +{ > > + int ret = __submit_flush_wait(sbi->sb->s_bdev); > > + int i; > > + > > + if (sbi->s_ndevs && !ret) { > > + for (i = 1; i < sbi->s_ndevs; i++) { > > + ret = __submit_flush_wait(FDEV(i).bdev); > > + if (ret) > > + break; > > + } > > + } > > + return ret; > > +} > > + > > static int issue_flush_thread(void *data) > > { > > struct f2fs_sb_info *sbi = data; > > @@ -413,25 +440,18 @@ static int issue_flush_thread(void *data) > > return 0; > > > > if (!llist_empty(&fcc->issue_list)) { > > - struct bio *bio; > > struct flush_cmd *cmd, *next; > > int ret; > > > > - bio = f2fs_bio_alloc(0); > > - > > fcc->dispatch_list = llist_del_all(&fcc->issue_list); > > fcc->dispatch_list = llist_reverse_order(fcc->dispatch_list); > > > > - bio->bi_bdev = sbi->sb->s_bdev; > > - bio_set_op_attrs(bio, REQ_OP_WRITE, WRITE_FLUSH); > > - ret = submit_bio_wait(bio); > > - > > + ret = submit_flush_wait(sbi); > > llist_for_each_entry_safe(cmd, next, > > fcc->dispatch_list, llnode) { > > cmd->ret = ret; > > complete(&cmd->wait); > > } > > - bio_put(bio); > > fcc->dispatch_list = NULL; > > } > > > > @@ -452,15 +472,11 @@ int f2fs_issue_flush(struct f2fs_sb_info *sbi) > > return 0; > > > > if (!test_opt(sbi, FLUSH_MERGE) || !atomic_read(&fcc->submit_flush)) { > > - struct bio *bio = f2fs_bio_alloc(0); > > int ret; > > > > atomic_inc(&fcc->submit_flush); > > - bio->bi_bdev = sbi->sb->s_bdev; > > - bio_set_op_attrs(bio, REQ_OP_WRITE, WRITE_FLUSH); > > - ret = submit_bio_wait(bio); > > + ret = submit_flush_wait(sbi); > > atomic_dec(&fcc->submit_flush); > > - bio_put(bio); > > return ret; > > } > > > > @@ -637,14 +653,18 @@ static void f2fs_submit_bio_wait_endio(struct bio *bio) > > > > /* this function is copied from blkdev_issue_discard from block/blk-lib.c */ > > static int __f2fs_issue_discard_async(struct f2fs_sb_info *sbi, > > - block_t blkstart, block_t blklen) > > + struct block_device *bdev, block_t blkstart, block_t blklen) > > { > > - struct block_device *bdev = sbi->sb->s_bdev; > > struct bio *bio = NULL; > > int err; > > > > trace_f2fs_issue_discard(sbi->sb, blkstart, blklen); > > > > + if (sbi->s_ndevs) { > > + int devi = f2fs_target_device_index(sbi, blkstart); > > + > > + blkstart -= FDEV(devi).start_blk; > > + } > > err = __blkdev_issue_discard(bdev, > > SECTOR_FROM_BLOCK(blkstart), > > SECTOR_FROM_BLOCK(blklen), > > @@ -662,18 +682,24 @@ static int __f2fs_issue_discard_async(struct f2fs_sb_info *sbi, > > } > > > > #ifdef CONFIG_BLK_DEV_ZONED > > -static int f2fs_issue_discard_zone(struct f2fs_sb_info *sbi, > > - block_t blkstart, block_t blklen) > > +static int __f2fs_issue_discard_zone(struct f2fs_sb_info *sbi, > > + struct block_device *bdev, block_t blkstart, block_t blklen) > > { > > - sector_t sector = SECTOR_FROM_BLOCK(blkstart); > > sector_t nr_sects = SECTOR_FROM_BLOCK(blklen); > > - struct block_device *bdev = sbi->sb->s_bdev; > > + sector_t sector; > > + int devi = 0; > > > > - if (nr_sects != bdev_zone_size(bdev)) { > > + if (sbi->s_ndevs) { > > + devi = f2fs_target_device_index(sbi, blkstart); > > + blkstart -= FDEV(devi).start_blk; > > + } > > + sector = SECTOR_FROM_BLOCK(blkstart); > > + > > + if (sector % bdev_zone_size(bdev) || nr_sects != bdev_zone_size(bdev)) { > > f2fs_msg(sbi->sb, KERN_INFO, > > - "Unaligned discard attempted (sector %llu + %llu)", > > - (unsigned long long)sector, > > - (unsigned long long)nr_sects); > > + "(%d) %s: Unaligned discard attempted (block %x + %x)", > > + devi, sbi->s_ndevs ? FDEV(devi).path: "", > > + blkstart, blklen); > > return -EIO; > > } > > > > @@ -682,14 +708,12 @@ static int f2fs_issue_discard_zone(struct f2fs_sb_info *sbi, > > * use regular discard if the drive supports it. For sequential > > * zones, reset the zone write pointer. > > */ > > - switch (get_blkz_type(sbi, blkstart)) { > > + switch (get_blkz_type(sbi, bdev, blkstart)) { > > > > case BLK_ZONE_TYPE_CONVENTIONAL: > > if (!blk_queue_discard(bdev_get_queue(bdev))) > > return 0; > > - return __f2fs_issue_discard_async(sbi, blkstart, > > - blklen); > > - > > + return __f2fs_issue_discard_async(sbi, bdev, blkstart, blklen); > > case BLK_ZONE_TYPE_SEQWRITE_REQ: > > case BLK_ZONE_TYPE_SEQWRITE_PREF: > > trace_f2fs_issue_reset_zone(sbi->sb, blkstart); > > @@ -702,14 +726,45 @@ static int f2fs_issue_discard_zone(struct f2fs_sb_info *sbi, > > } > > #endif > > > > +static int __issue_discard_async(struct f2fs_sb_info *sbi, > > + struct block_device *bdev, block_t blkstart, block_t blklen) > > +{ > > +#ifdef CONFIG_BLK_DEV_ZONED > > + if (f2fs_sb_mounted_blkzoned(sbi->sb) && > > + bdev_zoned_model(bdev) != BLK_ZONED_NONE) > > + return __f2fs_issue_discard_zone(sbi, bdev, blkstart, blklen); > > +#endif > > + return __f2fs_issue_discard_async(sbi, bdev, blkstart, blklen); > > +} > > + > > static int f2fs_issue_discard(struct f2fs_sb_info *sbi, > > block_t blkstart, block_t blklen) > > { > > + sector_t start = blkstart, len = 0; > > + struct block_device *bdev; > > struct seg_entry *se; > > unsigned int offset; > > block_t i; > > + int err = 0; > > + > > + bdev = f2fs_target_device(sbi, blkstart, NULL); > > + > > + for (i = blkstart; i < blkstart + blklen; i++, len++) { > > + if (i != start) { > > + struct block_device *bdev2 = > > + f2fs_target_device(sbi, i, NULL); > > + > > + if (bdev2 != bdev) { > > + err = __issue_discard_async(sbi, bdev, > > + start, len); > > + if (err) > > + return err; > > + bdev = bdev2; > > + start = i; > > + len = 0; > > + } > > + } > > > > - for (i = blkstart; i < blkstart + blklen; i++) { > > se = get_seg_entry(sbi, GET_SEGNO(sbi, i)); > > offset = GET_BLKOFF_FROM_SEG0(sbi, i); > > > > @@ -717,11 +772,9 @@ static int f2fs_issue_discard(struct f2fs_sb_info *sbi, > > sbi->discard_blks--; > > } > > > > -#ifdef CONFIG_BLK_DEV_ZONED > > - if (f2fs_sb_mounted_blkzoned(sbi->sb)) > > - return f2fs_issue_discard_zone(sbi, blkstart, blklen); > > -#endif > > - return __f2fs_issue_discard_async(sbi, blkstart, blklen); > > + if (len) > > + err = __issue_discard_async(sbi, bdev, start, len); > > + return err; > > } > > > > static void __add_discard_entry(struct f2fs_sb_info *sbi, > > diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c > > index eca9aea..4ccbb86 100644 > > --- a/fs/f2fs/super.c > > +++ b/fs/f2fs/super.c > > @@ -713,6 +713,19 @@ static void destroy_percpu_info(struct f2fs_sb_info *sbi) > > percpu_counter_destroy(&sbi->total_valid_inode_count); > > } > > > > +static void destroy_device_list(struct f2fs_sb_info *sbi) > > +{ > > + int i; > > + > > + for (i = 0; i < sbi->s_ndevs; i++) { > > + blkdev_put(FDEV(i).bdev, FMODE_EXCL); > > +#ifdef CONFIG_BLK_DEV_ZONED > > + kfree(FDEV(i).blkz_type); > > +#endif > > + } > > + kfree(sbi->devs); > > +} > > + > > static void f2fs_put_super(struct super_block *sb) > > { > > struct f2fs_sb_info *sbi = F2FS_SB(sb); > > @@ -773,6 +786,8 @@ static void f2fs_put_super(struct super_block *sb) > > crypto_free_shash(sbi->s_chksum_driver); > > kfree(sbi->raw_super); > > > > + destroy_device_list(sbi); > > + > > destroy_percpu_info(sbi); > > kfree(sbi); > > } > > @@ -1516,9 +1531,9 @@ static int init_percpu_info(struct f2fs_sb_info *sbi) > > } > > > > #ifdef CONFIG_BLK_DEV_ZONED > > -static int init_blkz_info(struct f2fs_sb_info *sbi) > > +static int init_blkz_info(struct f2fs_sb_info *sbi, int devi) > > { > > - struct block_device *bdev = sbi->sb->s_bdev; > > + struct block_device *bdev = FDEV(devi).bdev; > > sector_t nr_sectors = bdev->bd_part->nr_sects; > > sector_t sector = 0; > > struct blk_zone *zones; > > @@ -1529,15 +1544,21 @@ static int init_blkz_info(struct f2fs_sb_info *sbi) > > if (!f2fs_sb_mounted_blkzoned(sbi->sb)) > > return 0; > > > > + if (sbi->blocks_per_blkz && sbi->blocks_per_blkz != > > + SECTOR_TO_BLOCK(bdev_zone_size(bdev))) > > + return -EINVAL; > > sbi->blocks_per_blkz = SECTOR_TO_BLOCK(bdev_zone_size(bdev)); > > + if (sbi->log_blocks_per_blkz && sbi->log_blocks_per_blkz != > > + __ilog2_u32(sbi->blocks_per_blkz)) > > + return -EINVAL; > > sbi->log_blocks_per_blkz = __ilog2_u32(sbi->blocks_per_blkz); > > - sbi->nr_blkz = SECTOR_TO_BLOCK(nr_sectors) >> > > - sbi->log_blocks_per_blkz; > > + FDEV(devi).nr_blkz = SECTOR_TO_BLOCK(nr_sectors) >> > > + sbi->log_blocks_per_blkz; > > if (nr_sectors & (bdev_zone_size(bdev) - 1)) > > - sbi->nr_blkz++; > > + FDEV(devi).nr_blkz++; > > > > - sbi->blkz_type = kmalloc(sbi->nr_blkz, GFP_KERNEL); > > - if (!sbi->blkz_type) > > + FDEV(devi).blkz_type = kmalloc(FDEV(devi).nr_blkz, GFP_KERNEL); > > + if (!FDEV(devi).blkz_type) > > return -ENOMEM; > > > > #define F2FS_REPORT_NR_ZONES 4096 > > @@ -1562,7 +1583,7 @@ static int init_blkz_info(struct f2fs_sb_info *sbi) > > } > > > > for (i = 0; i < nr_zones; i++) { > > - sbi->blkz_type[n] = zones[i].type; > > + FDEV(devi).blkz_type[n] = zones[i].type; > > sector += zones[i].len; > > n++; > > } > > @@ -1666,6 +1687,77 @@ int f2fs_commit_super(struct f2fs_sb_info *sbi, bool recover) > > return err; > > } > > > > +static int f2fs_scan_devices(struct f2fs_sb_info *sbi) > > +{ > > + struct f2fs_super_block *raw_super = F2FS_RAW_SUPER(sbi); > > + int i; > > + > > + for (i = 0; i < MAX_DEVICES; i++) { > > + if (!RDEV(i).path[0]) > > + return 0; > > + > > + if (i == 0) { > > + sbi->devs = kzalloc(sizeof(struct f2fs_dev_info) * > > + MAX_DEVICES, GFP_KERNEL); > > + if (!sbi->devs) > > + return -ENOMEM; > > + } > > + > > + memcpy(FDEV(i).path, RDEV(i).path, MAX_PATH_LEN); > > + FDEV(i).total_segments = le32_to_cpu(RDEV(i).total_segments); > > + if (i == 0) { > > + FDEV(i).start_blk = 0; > > + FDEV(i).end_blk = FDEV(i).start_blk + > > + (FDEV(i).total_segments << > > + sbi->log_blocks_per_seg) - 1 + > > + le32_to_cpu(raw_super->segment0_blkaddr); > > + } else { > > + FDEV(i).start_blk = FDEV(i - 1).end_blk + 1; > > + FDEV(i).end_blk = FDEV(i).start_blk + > > + (FDEV(i).total_segments << > > + sbi->log_blocks_per_seg) - 1; > > + } > > + > > + FDEV(i).bdev = blkdev_get_by_path(FDEV(i).path, > > + sbi->sb->s_mode, sbi->sb->s_type); > > + if (IS_ERR(FDEV(i).bdev)) > > + return PTR_ERR(FDEV(i).bdev); > > + > > + /* to release errored devices */ > > + sbi->s_ndevs = i + 1; > > + > > +#ifdef CONFIG_BLK_DEV_ZONED > > + if (bdev_zoned_model(FDEV(i).bdev) == BLK_ZONED_HM && > > + !f2fs_sb_mounted_blkzoned(sbi->sb)) { > > + f2fs_msg(sbi->sb, KERN_ERR, > > + "Zoned block device feature not enabled\n"); > > + return -EINVAL; > > + } > > + if (bdev_zoned_model(FDEV(i).bdev) != BLK_ZONED_NONE) { > > + if (init_blkz_info(sbi, i)) { > > + f2fs_msg(sbi->sb, KERN_ERR, > > + "Failed to initialize F2FS blkzone information"); > > + return -EINVAL; > > + } > > + f2fs_msg(sbi->sb, KERN_INFO, > > + "Mount Device [%2d]: %20s, %8u, %8x - %8x (zone: %s)", > > + i, FDEV(i).path, > > + FDEV(i).total_segments, > > + FDEV(i).start_blk, FDEV(i).end_blk, > > + bdev_zoned_model(FDEV(i).bdev) == BLK_ZONED_HA ? > > + "Host-aware" : "Host-managed"); > > + continue; > > + } > > +#endif > > + f2fs_msg(sbi->sb, KERN_INFO, > > + "Mount Device [%2d]: %20s, %8u, %8x - %8x", > > + i, FDEV(i).path, > > + FDEV(i).total_segments, > > + FDEV(i).start_blk, FDEV(i).end_blk); > > + } > > + return 0; > > +} > > + > > static int f2fs_fill_super(struct super_block *sb, void *data, int silent) > > { > > struct f2fs_sb_info *sbi; > > @@ -1724,15 +1816,7 @@ static int f2fs_fill_super(struct super_block *sb, void *data, int silent) > > "Zoned block device support is not enabled\n"); > > goto free_sb_buf; > > } > > -#else > > - if (bdev_zoned_model(sb->s_bdev) == BLK_ZONED_HM && > > - !f2fs_sb_mounted_blkzoned(sb)) { > > - f2fs_msg(sb, KERN_ERR, > > - "Zoned block device feature not enabled\n"); > > - goto free_sb_buf; > > - } > > #endif > > - > > default_options(sbi); > > /* parse mount options */ > > options = kstrdup((const char *)data, GFP_KERNEL); > > @@ -1802,6 +1886,13 @@ static int f2fs_fill_super(struct super_block *sb, void *data, int silent) > > goto free_meta_inode; > > } > > > > + /* Initialize device list */ > > + err = f2fs_scan_devices(sbi); > > + if (err) { > > + f2fs_msg(sb, KERN_ERR, "Failed to find devices"); > > + goto free_devices; > > + } > > + > > sbi->total_valid_node_count = > > le32_to_cpu(sbi->ckpt->valid_node_count); > > percpu_counter_set(&sbi->total_valid_inode_count, > > @@ -1820,15 +1911,6 @@ static int f2fs_fill_super(struct super_block *sb, void *data, int silent) > > > > init_ino_entry_info(sbi); > > > > -#ifdef CONFIG_BLK_DEV_ZONED > > - err = init_blkz_info(sbi); > > - if (err) { > > - f2fs_msg(sb, KERN_ERR, > > - "Failed to initialize F2FS blkzone information"); > > - goto free_blkz; > > - } > > -#endif > > - > > /* setup f2fs internal modules */ > > err = build_segment_manager(sbi); > > if (err) { > > @@ -2007,10 +2089,8 @@ static int f2fs_fill_super(struct super_block *sb, void *data, int silent) > > destroy_node_manager(sbi); > > free_sm: > > destroy_segment_manager(sbi); > > -#ifdef CONFIG_BLK_DEV_ZONED > > -free_blkz: > > - kfree(sbi->blkz_type); > > -#endif > > +free_devices: > > + destroy_device_list(sbi); > > kfree(sbi->ckpt); > > free_meta_inode: > > make_bad_inode(sbi->meta_inode); > > diff --git a/include/linux/f2fs_fs.h b/include/linux/f2fs_fs.h > > index 422630b..cea41a1 100644 > > --- a/include/linux/f2fs_fs.h > > +++ b/include/linux/f2fs_fs.h > > @@ -52,10 +52,17 @@ > > > > #define VERSION_LEN 256 > > #define MAX_VOLUME_NAME 512 > > +#define MAX_PATH_LEN 64 > > +#define MAX_DEVICES 8 > > > > /* > > * For superblock > > */ > > +struct f2fs_device { > > + __u8 path[MAX_PATH_LEN]; > > + __le32 total_segments; > > +} __packed; > > + > > struct f2fs_super_block { > > __le32 magic; /* Magic Number */ > > __le16 major_ver; /* Major Version */ > > @@ -94,7 +101,8 @@ struct f2fs_super_block { > > __le32 feature; /* defined features */ > > __u8 encryption_level; /* versioning level for encryption */ > > __u8 encrypt_pw_salt[16]; /* Salt used for string2key algorithm */ > > - __u8 reserved[871]; /* valid reserved region */ > > + struct f2fs_device devs[MAX_DEVICES]; /* device list */ > > + __u8 reserved[327]; /* valid reserved region */ > > } __packed; > > > > /* > > -- > > 2.8.3 > > Cheers, Andreas