<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
Reviewed-by: Doug Oucharek <<a href="mailto:dougso@me.com" class="">dougso@me.com</a>>
<div class=""><br class="">
</div>
<div class="">Doug</div>
<div class=""><br class="">
<div style="">
<blockquote type="cite" class="">
<div class="">On Sep 6, 2018, at 5:49 PM, NeilBrown <<a href="mailto:neilb@suse.com" class="">neilb@suse.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div class="">This will contain some fields from lnet_ni, to be shared<br class="">
between multiple ni on the one network.<br class="">
<br class="">
For now, only tunables are moved across, using<br class="">
struct lnet_ioctl_config_lnd_cmn_tunables<br class="">
which is changed to use signed values so -1 can be stored.<br class="">
-1 means "no value"<br class="">
If the tunables haven't been initialised, then net_tunables_set is<br class="">
false. Previously a NULL pointer had this meaning.<br class="">
<br class="">
A 'struct lnet_net' is allocated as part of lnet_ni_alloc(), and freed<br class="">
by lnet_ni_free().<br class="">
<br class="">
This is part of<br class="">
8cbb8cd3e771e7f7e0f99cafc19fad32770dc015<br class="">
LU-7734 lnet: Multi-Rail local NI split<br class="">
<br class="">
Signed-off-by: NeilBrown <<a href="mailto:neilb@suse.com" class="">neilb@suse.com</a>><br class="">
---<br class="">
.../staging/lustre/include/linux/lnet/lib-types.h | 25 ++++++--<br class="">
.../lustre/include/uapi/linux/lnet/lnet-dlc.h | 8 +--<br class="">
.../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c | 2 -<br class="">
.../lustre/lnet/klnds/o2iblnd/o2iblnd_modparams.c | 61 +++++++++++---------<br class="">
.../staging/lustre/lnet/klnds/socklnd/socklnd.c | 19 ++++--<br class="">
drivers/staging/lustre/lnet/lnet/api-ni.c | 45 +++++++++------<br class="">
drivers/staging/lustre/lnet/lnet/config.c | 24 ++++++--<br class="">
drivers/staging/lustre/lnet/lnet/lib-move.c | 5 +-<br class="">
drivers/staging/lustre/lnet/lnet/peer.c | 9 ++-<br class="">
drivers/staging/lustre/lnet/lnet/router.c | 8 ++-<br class="">
drivers/staging/lustre/lnet/lnet/router_proc.c | 6 +-<br class="">
11 files changed, 129 insertions(+), 83 deletions(-)<br class="">
<br class="">
diff --git a/drivers/staging/lustre/include/linux/lnet/lib-types.h b/drivers/staging/lustre/include/linux/lnet/lib-types.h<br class="">
index 078bc97a9ebf..ead8a4e1125a 100644<br class="">
--- a/drivers/staging/lustre/include/linux/lnet/lib-types.h<br class="">
+++ b/drivers/staging/lustre/include/linux/lnet/lib-types.h<br class="">
@@ -43,6 +43,7 @@<br class="">
<br class="">
#include <uapi/linux/lnet/lnet-types.h><br class="">
#include <uapi/linux/lnet/lnetctl.h><br class="">
+#include <uapi/linux/lnet/lnet-dlc.h><br class="">
<br class="">
/* Max payload size */<br class="">
#define LNET_MAX_PAYLOAD CONFIG_LNET_MAX_PAYLOAD<br class="">
@@ -252,17 +253,22 @@ struct lnet_tx_queue {<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>struct list_head<span class="Apple-tab-span" style="white-space:pre">
</span>tq_delayed;<span class="Apple-tab-span" style="white-space:pre"> </span>/* delayed TXs */<br class="">
};<br class="">
<br class="">
+struct lnet_net {<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>/* network tunables */<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>struct lnet_ioctl_config_lnd_cmn_tunables net_tunables;<br class="">
+<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>/*<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>* boolean to indicate that the tunables have been set and<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>* shouldn't be reset<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>*/<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>bool<span class="Apple-tab-span" style="white-space:pre">
</span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span> net_tunables_set;<br class="">
+};<br class="">
+<br class="">
struct lnet_ni {<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>spinlock_t<span class="Apple-tab-span" style="white-space:pre">
</span><span class="Apple-tab-span" style="white-space:pre"></span> ni_lock;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>struct list_head<span class="Apple-tab-span" style="white-space:pre">
</span> ni_list;<span class="Apple-tab-span" style="white-space:pre"> </span>/* chain on ln_nis */<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>struct list_head<span class="Apple-tab-span" style="white-space:pre">
</span> ni_cptlist;<span class="Apple-tab-span" style="white-space:pre"> </span>/* chain on ln_nis_cpt */<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>int<span class="Apple-tab-span" style="white-space:pre">
</span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span> ni_maxtxcredits; /* # tx credits */<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>/* # per-peer send credits */<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>int<span class="Apple-tab-span" style="white-space:pre">
</span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span> ni_peertxcredits;<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>/* # per-peer router buffer credits */<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>int<span class="Apple-tab-span" style="white-space:pre">
</span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span> ni_peerrtrcredits;<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>/* seconds to consider peer dead */<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>int<span class="Apple-tab-span" style="white-space:pre">
</span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span> ni_peertimeout;<br class="">
+<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>/* number of CPTs */<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>int<span class="Apple-tab-span" style="white-space:pre">
</span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>ni_ncpts;<br class="">
<br class="">
@@ -286,6 +292,9 @@ struct lnet_ni {<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>/* when I was last alive */<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>time64_t<span class="Apple-tab-span" style="white-space:pre">
</span><span class="Apple-tab-span" style="white-space:pre"></span>ni_last_alive;<br class="">
<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>/* pointer to parent network */<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>struct lnet_net<span class="Apple-tab-span" style="white-space:pre">
</span><span class="Apple-tab-span" style="white-space:pre"></span>*ni_net;<br class="">
+<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>/* my health status */<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>struct lnet_ni_status<span class="Apple-tab-span" style="white-space:pre">
</span>*ni_status;<br class="">
<br class="">
@@ -397,7 +406,7 @@ struct lnet_peer_table {<br class="">
* lnet_ni::ni_peertimeout has been set to a positive value<br class="">
*/<br class="">
#define lnet_peer_aliveness_enabled(lp) (the_lnet.ln_routing && \<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>(lp)->lp_ni->ni_peertimeout
> 0)<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>(lp)->lp_ni->ni_net->net_tunables.lct_peer_timeout
> 0)<br class="">
<br class="">
struct lnet_route {<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>struct list_head<span class="Apple-tab-span" style="white-space:pre">
</span>lr_list;<span class="Apple-tab-span" style="white-space:pre"> </span>/* chain on net */<br class="">
diff --git a/drivers/staging/lustre/include/uapi/linux/lnet/lnet-dlc.h b/drivers/staging/lustre/include/uapi/linux/lnet/lnet-dlc.h<br class="">
index c1619f411d81..a8eb3b8f9fd7 100644<br class="">
--- a/drivers/staging/lustre/include/uapi/linux/lnet/lnet-dlc.h<br class="">
+++ b/drivers/staging/lustre/include/uapi/linux/lnet/lnet-dlc.h<br class="">
@@ -39,10 +39,10 @@<br class="">
<br class="">
struct lnet_ioctl_config_lnd_cmn_tunables {<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>__u32 lct_version;<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>__u32 lct_peer_timeout;<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>__u32 lct_peer_tx_credits;<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>__u32 lct_peer_rtr_credits;<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>__u32 lct_max_tx_credits;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>__s32 lct_peer_timeout;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>__s32 lct_peer_tx_credits;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>__s32 lct_peer_rtr_credits;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>__s32 lct_max_tx_credits;<br class="">
};<br class="">
<br class="">
struct lnet_ioctl_config_o2iblnd_tunables {<br class="">
diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c<br class="">
index f496e6fcc416..0d17e22c4401 100644<br class="">
--- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c<br class="">
+++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c<br class="">
@@ -337,7 +337,7 @@ int kiblnd_create_peer(struct lnet_ni *ni, struct kib_peer **peerp,<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>peer->ibp_error = 0;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>peer->ibp_last_alive = 0;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>peer->ibp_max_frags = kiblnd_cfg_rdma_frags(peer->ibp_ni);<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>peer->ibp_queue_depth = ni->ni_peertxcredits;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>peer->ibp_queue_depth = ni->ni_net->net_tunables.lct_peer_tx_credits;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>atomic_set(&peer->ibp_refcount, 1); /* 1 ref for caller */<br class="">
<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>INIT_LIST_HEAD(&peer->ibp_list); /* not in the peer table yet */<br class="">
diff --git a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_modparams.c b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_modparams.c<br class="">
index 39d07926d603..a1aca4dda38f 100644<br class="">
--- a/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_modparams.c<br class="">
+++ b/drivers/staging/lustre/lnet/klnds/o2iblnd/o2iblnd_modparams.c<br class="">
@@ -171,7 +171,7 @@ int kiblnd_msg_queue_size(int version, struct lnet_ni *ni)<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>if (version == IBLND_MSG_VERSION_1)<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>return IBLND_MSG_QUEUE_SIZE_V1;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>else if (ni)<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>return ni->ni_peertxcredits;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>return ni->ni_net->net_tunables.lct_peer_tx_credits;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>else<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>return peer_credits;<br class="">
}<br class="">
@@ -179,6 +179,7 @@ int kiblnd_msg_queue_size(int version, struct lnet_ni *ni)<br class="">
int kiblnd_tunables_setup(struct lnet_ni *ni)<br class="">
{<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>struct lnet_ioctl_config_o2iblnd_tunables *tunables;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>struct lnet_ioctl_config_lnd_cmn_tunables *net_tunables;<br class="">
<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>/*<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>* if there was no tunables specified, setup the tunables to be<br class="">
@@ -204,35 +205,39 @@ int kiblnd_tunables_setup(struct lnet_ni *ni)<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>return -EINVAL;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>}<br class="">
<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>if (!ni->ni_peertimeout)<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>ni->ni_peertimeout = peer_timeout;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>net_tunables = &ni->ni_net->net_tunables;<br class="">
<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>if (!ni->ni_maxtxcredits)<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>ni->ni_maxtxcredits = credits;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>if (net_tunables->lct_peer_timeout == -1)<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>net_tunables->lct_peer_timeout = peer_timeout;<br class="">
<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>if (!ni->ni_peertxcredits)<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>ni->ni_peertxcredits = peer_credits;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>if (net_tunables->lct_max_tx_credits == -1)<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>net_tunables->lct_max_tx_credits = credits;<br class="">
<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>if (!ni->ni_peerrtrcredits)<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>ni->ni_peerrtrcredits = peer_buffer_credits;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>if (net_tunables->lct_peer_tx_credits == -1)<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>net_tunables->lct_peer_tx_credits = peer_credits;<br class="">
<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>if (ni->ni_peertxcredits < IBLND_CREDITS_DEFAULT)<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>ni->ni_peertxcredits = IBLND_CREDITS_DEFAULT;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>if (net_tunables->lct_peer_rtr_credits == -1)<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>net_tunables->lct_peer_rtr_credits = peer_buffer_credits;<br class="">
<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>if (ni->ni_peertxcredits > IBLND_CREDITS_MAX)<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>ni->ni_peertxcredits = IBLND_CREDITS_MAX;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>if (net_tunables->lct_peer_tx_credits < IBLND_CREDITS_DEFAULT)<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>net_tunables->lct_peer_tx_credits = IBLND_CREDITS_DEFAULT;<br class="">
<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>if (ni->ni_peertxcredits > credits)<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>ni->ni_peertxcredits = credits;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>if (net_tunables->lct_peer_tx_credits > IBLND_CREDITS_MAX)<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>net_tunables->lct_peer_tx_credits = IBLND_CREDITS_MAX;<br class="">
+<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>if (net_tunables->lct_peer_tx_credits ><br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span> net_tunables->lct_max_tx_credits)<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>net_tunables->lct_peer_tx_credits =<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>net_tunables->lct_max_tx_credits;<br class="">
<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>if (!tunables->lnd_peercredits_hiw)<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>tunables->lnd_peercredits_hiw = peer_credits_hiw;<br class="">
<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>if (tunables->lnd_peercredits_hiw < ni->ni_peertxcredits / 2)<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>tunables->lnd_peercredits_hiw = ni->ni_peertxcredits / 2;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>if (tunables->lnd_peercredits_hiw < net_tunables->lct_peer_tx_credits / 2)<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>tunables->lnd_peercredits_hiw = net_tunables->lct_peer_tx_credits / 2;<br class="">
<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>if (tunables->lnd_peercredits_hiw >= ni->ni_peertxcredits)<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>tunables->lnd_peercredits_hiw = ni->ni_peertxcredits - 1;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>if (tunables->lnd_peercredits_hiw >= net_tunables->lct_peer_tx_credits)<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>tunables->lnd_peercredits_hiw = net_tunables->lct_peer_tx_credits - 1;<br class="">
<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>if (tunables->lnd_map_on_demand <= 0 ||<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span> tunables->lnd_map_on_demand > IBLND_MAX_RDMA_FRAGS) {<br class="">
@@ -252,21 +257,23 @@ int kiblnd_tunables_setup(struct lnet_ni *ni)<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>if (tunables->lnd_map_on_demand > 0 &&<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span> tunables->lnd_map_on_demand <= IBLND_MAX_RDMA_FRAGS / 8) {<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>tunables->lnd_concurrent_sends =<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>ni->ni_peertxcredits
* 2;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>net_tunables->lct_peer_tx_credits
* 2;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>} else {<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>tunables->lnd_concurrent_sends = ni->ni_peertxcredits;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>tunables->lnd_concurrent_sends =<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>net_tunables->lct_peer_tx_credits;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>}<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>}<br class="">
<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>if (tunables->lnd_concurrent_sends > ni->ni_peertxcredits * 2)<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>tunables->lnd_concurrent_sends = ni->ni_peertxcredits * 2;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>if (tunables->lnd_concurrent_sends > net_tunables->lct_peer_tx_credits * 2)<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>tunables->lnd_concurrent_sends = net_tunables->lct_peer_tx_credits * 2;<br class="">
<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>if (tunables->lnd_concurrent_sends < ni->ni_peertxcredits / 2)<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>tunables->lnd_concurrent_sends = ni->ni_peertxcredits / 2;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>if (tunables->lnd_concurrent_sends < net_tunables->lct_peer_tx_credits / 2)<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>tunables->lnd_concurrent_sends = net_tunables->lct_peer_tx_credits / 2;<br class="">
<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>if (tunables->lnd_concurrent_sends < ni->ni_peertxcredits) {<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>if (tunables->lnd_concurrent_sends < net_tunables->lct_peer_tx_credits) {<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>CWARN("Concurrent sends %d is lower than message queue size: %d, performance may drop slightly.\n",<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span> tunables->lnd_concurrent_sends, ni->ni_peertxcredits);<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span> tunables->lnd_concurrent_sends,<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span> net_tunables->lct_peer_tx_credits);<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>}<br class="">
<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>if (!tunables->lnd_fmr_pool_size)<br class="">
diff --git a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c<br class="">
index 4dde158451ea..4ad885f10235 100644<br class="">
--- a/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c<br class="">
+++ b/drivers/staging/lustre/lnet/klnds/socklnd/socklnd.c<br class="">
@@ -2739,12 +2739,19 @@ ksocknal_startup(struct lnet_ni *ni)<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>goto fail_0;<br class="">
<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>spin_lock_init(&net->ksnn_lock);<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>net->ksnn_incarnation = ktime_get_real_ns();<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>ni->ni_data = net;<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>ni->ni_peertimeout = *ksocknal_tunables.ksnd_peertimeout;<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>ni->ni_maxtxcredits = *ksocknal_tunables.ksnd_credits;<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>ni->ni_peertxcredits = *ksocknal_tunables.ksnd_peertxcredits;<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>ni->ni_peerrtrcredits = *ksocknal_tunables.ksnd_peerrtrcredits;<br class="">
+ net->ksnn_incarnation = ktime_get_real_ns();<br class="">
+ ni->ni_data = net;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>if (!ni->ni_net->net_tunables_set) {<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>ni->ni_net->net_tunables.lct_peer_timeout =<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>*ksocknal_tunables.ksnd_peertimeout;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>ni->ni_net->net_tunables.lct_max_tx_credits =<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>*ksocknal_tunables.ksnd_credits;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>ni->ni_net->net_tunables.lct_peer_tx_credits =<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>*ksocknal_tunables.ksnd_peertxcredits;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>ni->ni_net->net_tunables.lct_peer_rtr_credits =<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>*ksocknal_tunables.ksnd_peerrtrcredits;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>ni->ni_net->net_tunables_set = true;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>}<br class="">
<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>net->ksnn_ninterfaces = 0;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>if (!ni->ni_interfaces[0]) {<br class="">
diff --git a/drivers/staging/lustre/lnet/lnet/api-ni.c b/drivers/staging/lustre/lnet/lnet/api-ni.c<br class="">
index f9fcce2a5643..cd4189fa7acb 100644<br class="">
--- a/drivers/staging/lustre/lnet/lnet/api-ni.c<br class="">
+++ b/drivers/staging/lustre/lnet/lnet/api-ni.c<br class="">
@@ -1036,11 +1036,11 @@ lnet_ni_tq_credits(struct lnet_ni *ni)<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>LASSERT(ni->ni_ncpts >= 1);<br class="">
<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>if (ni->ni_ncpts == 1)<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>return ni->ni_maxtxcredits;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>return ni->ni_net->net_tunables.lct_max_tx_credits;<br class="">
<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>credits = ni->ni_maxtxcredits / ni->ni_ncpts;<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>credits = max(credits, 8 * ni->ni_peertxcredits);<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>credits = min(credits, ni->ni_maxtxcredits);<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>credits = ni->ni_net->net_tunables.lct_max_tx_credits / ni->ni_ncpts;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>credits = max(credits, 8 * ni->ni_net->net_tunables.lct_peer_tx_credits);<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>credits = min(credits, ni->ni_net->net_tunables.lct_max_tx_credits);<br class="">
<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>return credits;<br class="">
}<br class="">
@@ -1271,16 +1271,16 @@ lnet_startup_lndni(struct lnet_ni *ni, struct lnet_ioctl_config_data *conf)<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>*/<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>if (conf) {<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>if (conf->cfg_config_u.cfg_net.net_peer_rtr_credits >= 0)<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>ni->ni_peerrtrcredits =<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>ni->ni_net->net_tunables.lct_peer_rtr_credits =<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>conf->cfg_config_u.cfg_net.net_peer_rtr_credits;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>if (conf->cfg_config_u.cfg_net.net_peer_timeout >= 0)<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>ni->ni_peertimeout =<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>ni->ni_net->net_tunables.lct_peer_timeout =<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>conf->cfg_config_u.cfg_net.net_peer_timeout;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>if (conf->cfg_config_u.cfg_net.net_peer_tx_credits != -1)<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>ni->ni_peertxcredits =<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>ni->ni_net->net_tunables.lct_peer_tx_credits =<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>conf->cfg_config_u.cfg_net.net_peer_tx_credits;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>if (conf->cfg_config_u.cfg_net.net_max_tx_credits >= 0)<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>ni->ni_maxtxcredits =<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>ni->ni_net->net_tunables.lct_max_tx_credits =<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>conf->cfg_config_u.cfg_net.net_max_tx_credits;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>}<br class="">
<br class="">
@@ -1297,8 +1297,6 @@ lnet_startup_lndni(struct lnet_ni *ni, struct lnet_ioctl_config_data *conf)<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>goto failed0;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>}<br class="">
<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>LASSERT(ni->ni_peertimeout <= 0 || lnd->lnd_query);<br class="">
-<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>lnet_net_lock(LNET_LOCK_EX);<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>/* refcount for ln_nis */<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>lnet_ni_addref_locked(ni, 0);<br class="">
@@ -1314,13 +1312,18 @@ lnet_startup_lndni(struct lnet_ni *ni, struct lnet_ioctl_config_data *conf)<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>lnet_ni_addref(ni);<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>LASSERT(!the_lnet.ln_loni);<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>the_lnet.ln_loni = ni;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>ni->ni_net->net_tunables.lct_peer_tx_credits = 0;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>ni->ni_net->net_tunables.lct_peer_rtr_credits = 0;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>ni->ni_net->net_tunables.lct_max_tx_credits = 0;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>ni->ni_net->net_tunables.lct_peer_timeout = 0;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>return 0;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>}<br class="">
<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>if (!ni->ni_peertxcredits || !ni->ni_maxtxcredits) {<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>if (!ni->ni_net->net_tunables.lct_peer_tx_credits ||<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span> !ni->ni_net->net_tunables.lct_max_tx_credits) {<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>LCONSOLE_ERROR_MSG(0x107, "LNI %s has no %scredits\n",<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span> libcfs_lnd2str(lnd->lnd_type),<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span> !ni->ni_peertxcredits
?<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span> !ni->ni_net->net_tunables.lct_peer_tx_credits
?<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span> "" : "per-peer
");<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>/*<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>* shutdown the NI since if we get here then it must've already<br class="">
@@ -1343,9 +1346,11 @@ lnet_startup_lndni(struct lnet_ni *ni, struct lnet_ioctl_config_data *conf)<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>add_device_randomness(&seed, sizeof(seed));<br class="">
<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>CDEBUG(D_LNI, "Added LNI %s [%d/%d/%d/%d]\n",<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span> libcfs_nid2str(ni->ni_nid), ni->ni_peertxcredits,<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span> libcfs_nid2str(ni->ni_nid),<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>ni->ni_net->net_tunables.lct_peer_tx_credits,<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span> lnet_ni_tq_credits(ni) * LNET_CPT_NUMBER,<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span> ni->ni_peerrtrcredits, ni->ni_peertimeout);<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span> ni->ni_net->net_tunables.lct_peer_rtr_credits,<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>ni->ni_net->net_tunables.lct_peer_timeout);<br class="">
<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>return 0;<br class="">
failed0:<br class="">
@@ -1667,10 +1672,14 @@ lnet_fill_ni_info(struct lnet_ni *ni, struct lnet_ioctl_config_data *config)<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>}<br class="">
<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>config->cfg_nid = ni->ni_nid;<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>config->cfg_config_u.cfg_net.net_peer_timeout = ni->ni_peertimeout;<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>config->cfg_config_u.cfg_net.net_max_tx_credits = ni->ni_maxtxcredits;<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>config->cfg_config_u.cfg_net.net_peer_tx_credits = ni->ni_peertxcredits;<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>config->cfg_config_u.cfg_net.net_peer_rtr_credits = ni->ni_peerrtrcredits;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>config->cfg_config_u.cfg_net.net_peer_timeout =<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>ni->ni_net->net_tunables.lct_peer_timeout;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>config->cfg_config_u.cfg_net.net_max_tx_credits =<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>ni->ni_net->net_tunables.lct_max_tx_credits;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>config->cfg_config_u.cfg_net.net_peer_tx_credits =<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>ni->ni_net->net_tunables.lct_peer_tx_credits;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>config->cfg_config_u.cfg_net.net_peer_rtr_credits =<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>ni->ni_net->net_tunables.lct_peer_rtr_credits;<br class="">
<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>net_config->ni_status = ni->ni_status->ns_status;<br class="">
<br class="">
diff --git a/drivers/staging/lustre/lnet/lnet/config.c b/drivers/staging/lustre/lnet/lnet/config.c<br class="">
index 091c4f714e84..86a53854e427 100644<br class="">
--- a/drivers/staging/lustre/lnet/lnet/config.c<br class="">
+++ b/drivers/staging/lustre/lnet/lnet/config.c<br class="">
@@ -114,29 +114,38 @@ lnet_ni_free(struct lnet_ni *ni)<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>if (ni->ni_net_ns)<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>put_net(ni->ni_net_ns);<br class="">
<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>kvfree(ni->ni_net);<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>kfree(ni);<br class="">
}<br class="">
<br class="">
struct lnet_ni *<br class="">
-lnet_ni_alloc(__u32 net, struct cfs_expr_list *el, struct list_head *nilist)<br class="">
+lnet_ni_alloc(__u32 net_id, struct cfs_expr_list *el, struct list_head *nilist)<br class="">
{<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>struct lnet_tx_queue *tq;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>struct lnet_ni *ni;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>int rc;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>int i;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>struct lnet_net<span class="Apple-tab-span" style="white-space:pre">
</span><span class="Apple-tab-span" style="white-space:pre"></span>*net;<br class="">
<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>if (!lnet_net_unique(net, nilist)) {<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>if (!lnet_net_unique(net_id, nilist)) {<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>LCONSOLE_ERROR_MSG(0x111, "Duplicate network specified: %s\n",<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span> libcfs_net2str(net));<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span> libcfs_net2str(net_id));<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>return NULL;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>}<br class="">
<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>ni = kzalloc(sizeof(*ni), GFP_NOFS);<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>if (!ni) {<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>net = kzalloc(sizeof(*net), GFP_NOFS);<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>if (!ni || !net) {<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>kfree(ni); kfree(net);<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>CERROR("Out of memory creating network %s\n",<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span> libcfs_net2str(net));<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span> libcfs_net2str(net_id));<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>return NULL;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>}<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>/* initialize global paramters to undefiend */<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>net->net_tunables.lct_peer_timeout = -1;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>net->net_tunables.lct_max_tx_credits = -1;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>net->net_tunables.lct_peer_tx_credits = -1;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>net->net_tunables.lct_peer_rtr_credits = -1;<br class="">
<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>spin_lock_init(&ni->ni_lock);<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>INIT_LIST_HEAD(&ni->ni_cptlist);<br class="">
@@ -160,7 +169,7 @@ lnet_ni_alloc(__u32 net, struct cfs_expr_list *el, struct list_head *nilist)<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>rc = cfs_expr_list_values(el, LNET_CPT_NUMBER, &ni->ni_cpts);<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>if (rc <= 0) {<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>CERROR("Failed to set CPTs for NI %s: %d\n",<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span> libcfs_net2str(net), rc);<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span> libcfs_net2str(net_id), rc);<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>goto failed;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>}<br class="">
<br class="">
@@ -173,8 +182,9 @@ lnet_ni_alloc(__u32 net, struct cfs_expr_list *el, struct list_head *nilist)<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>ni->ni_ncpts = rc;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>}<br class="">
<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>ni->ni_net = net;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>/* LND will fill in the address part of the NID */<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>ni->ni_nid = LNET_MKNID(net, 0);<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>ni->ni_nid = LNET_MKNID(net_id, 0);<br class="">
<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>/* Store net namespace in which current ni is being created */<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>if (current->nsproxy->net_ns)<br class="">
diff --git a/drivers/staging/lustre/lnet/lnet/lib-move.c b/drivers/staging/lustre/lnet/lnet/lib-move.c<br class="">
index edcafac055ed..f186e6a16d34 100644<br class="">
--- a/drivers/staging/lustre/lnet/lnet/lib-move.c<br class="">
+++ b/drivers/staging/lustre/lnet/lnet/lib-move.c<br class="">
@@ -524,7 +524,8 @@ lnet_peer_is_alive(struct lnet_peer *lp, unsigned long now)<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span> lp->lp_timestamp >= lp->lp_last_alive)<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>return 0;<br class="">
<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>deadline = lp->lp_last_alive + lp->lp_ni->ni_peertimeout;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>deadline = lp->lp_last_alive +<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>lp->lp_ni->ni_net->net_tunables.lct_peer_timeout;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>alive = deadline > now;<br class="">
<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>/* Update obsolete lp_alive except for routers assumed to be dead<br class="">
@@ -569,7 +570,7 @@ lnet_peer_alive_locked(struct lnet_peer *lp)<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span> libcfs_nid2str(lp->lp_nid),<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span> now, next_query,<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span> lnet_queryinterval,<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span> lp->lp_ni->ni_peertimeout);<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span> lp->lp_ni->ni_net->net_tunables.lct_peer_timeout);<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>return 0;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>}<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>}<br class="">
diff --git a/drivers/staging/lustre/lnet/lnet/peer.c b/drivers/staging/lustre/lnet/lnet/peer.c<br class="">
index d9452c322e4d..b76ac3e051d9 100644<br class="">
--- a/drivers/staging/lustre/lnet/lnet/peer.c<br class="">
+++ b/drivers/staging/lustre/lnet/lnet/peer.c<br class="">
@@ -342,8 +342,8 @@ lnet_nid2peer_locked(struct lnet_peer **lpp, lnet_nid_t nid, int cpt)<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>goto out;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>}<br class="">
<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>lp->lp_txcredits = lp->lp_ni->ni_peertxcredits;<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>lp->lp_mintxcredits = lp->lp_ni->ni_peertxcredits;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>lp->lp_txcredits = lp->lp_ni->ni_net->net_tunables.lct_peer_tx_credits;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>lp->lp_mintxcredits = lp->lp_ni->ni_net->net_tunables.lct_peer_tx_credits;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>lp->lp_rtrcredits = lnet_peer_buffer_credits(lp->lp_ni);<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>lp->lp_minrtrcredits = lnet_peer_buffer_credits(lp->lp_ni);<br class="">
<br class="">
@@ -383,7 +383,7 @@ lnet_debug_peer(lnet_nid_t nid)<br class="">
<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>CDEBUG(D_WARNING, "%-24s %4d %5s %5d %5d %5d %5d %5d %ld\n",<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span> libcfs_nid2str(lp->lp_nid), lp->lp_refcount,<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span> aliveness, lp->lp_ni->ni_peertxcredits,<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span> aliveness, lp->lp_ni->ni_net->net_tunables.lct_peer_tx_credits,<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span> lp->lp_rtrcredits, lp->lp_minrtrcredits,<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span> lp->lp_txcredits, lp->lp_mintxcredits, lp->lp_txqnob);<br class="">
<br class="">
@@ -438,7 +438,8 @@ lnet_get_peer_info(__u32 peer_index, __u64 *nid,<br class="">
<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>*nid = lp->lp_nid;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>*refcount = lp->lp_refcount;<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>*ni_peer_tx_credits = lp->lp_ni->ni_peertxcredits;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>*ni_peer_tx_credits =<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>lp->lp_ni->ni_net->net_tunables.lct_peer_tx_credits;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>*peer_tx_credits = lp->lp_txcredits;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>*peer_rtr_credits = lp->lp_rtrcredits;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>*peer_min_rtr_credits = lp->lp_mintxcredits;<br class="">
diff --git a/drivers/staging/lustre/lnet/lnet/router.c b/drivers/staging/lustre/lnet/lnet/router.c<br class="">
index 02241fbc9eaa..7d61c5d71426 100644<br class="">
--- a/drivers/staging/lustre/lnet/lnet/router.c<br class="">
+++ b/drivers/staging/lustre/lnet/lnet/router.c<br class="">
@@ -57,9 +57,11 @@ MODULE_PARM_DESC(auto_down, "Automatically mark peers down on comms error");<br class="">
int<br class="">
lnet_peer_buffer_credits(struct lnet_ni *ni)<br class="">
{<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>struct lnet_net *net = ni->ni_net;<br class="">
+<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>/* NI option overrides LNet default */<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>if (ni->ni_peerrtrcredits > 0)<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>return ni->ni_peerrtrcredits;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>if (net->net_tunables.lct_peer_rtr_credits > 0)<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span>return net->net_tunables.lct_peer_rtr_credits;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>if (peer_buffer_credits > 0)<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>return peer_buffer_credits;<br class="">
<br class="">
@@ -67,7 +69,7 @@ lnet_peer_buffer_credits(struct lnet_ni *ni)<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>* As an approximation, allow this peer the same number of router<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>* buffers as it is allowed outstanding sends<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span>*/<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span>return ni->ni_peertxcredits;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span>return net->net_tunables.lct_peer_tx_credits;<br class="">
}<br class="">
<br class="">
/* forward ref's */<br class="">
diff --git a/drivers/staging/lustre/lnet/lnet/router_proc.c b/drivers/staging/lustre/lnet/lnet/router_proc.c<br class="">
index 31f4982f7f17..19cea7076057 100644<br class="">
--- a/drivers/staging/lustre/lnet/lnet/router_proc.c<br class="">
+++ b/drivers/staging/lustre/lnet/lnet/router_proc.c<br class="">
@@ -489,7 +489,7 @@ static int proc_lnet_peers(struct ctl_table *table, int write,<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>int nrefs = peer->lp_refcount;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>time64_t lastalive = -1;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>char *aliveness = "NA";<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>int maxcr = peer->lp_ni->ni_peertxcredits;<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>int maxcr = peer->lp_ni->ni_net->net_tunables.lct_peer_tx_credits;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>int txcr = peer->lp_txcredits;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>int mintxcr = peer->lp_mintxcredits;<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span>int rtrcr = peer->lp_rtrcredits;<br class="">
@@ -704,8 +704,8 @@ static int proc_lnet_nis(struct ctl_table *table, int write,<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span> "%-24s
%6s %5lld %4d %4d %4d %5d %5d %5d\n",<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span> libcfs_nid2str(ni->ni_nid),
stat,<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span> last_alive,
*ni->ni_refs[i],<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span> ni->ni_peertxcredits,<br class="">
-<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span> ni->ni_peerrtrcredits,<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span> ni->ni_net->net_tunables.lct_peer_tx_credits,<br class="">
+<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span> ni->ni_net->net_tunables.lct_peer_rtr_credits,<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span> tq->tq_credits_max,<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span> tq->tq_credits,<br class="">
<span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span><span class="Apple-tab-span" style="white-space:pre"></span> tq->tq_credits_min);<br class="">
<br class="">
<br class="">
</div>
</div>
</blockquote>
</div>
<br class="">
</div>
</body>
</html>