[lustre-discuss] lustre client not able to lctl ping or mount

[Pak Lui]

Rick,

That ko2iblnd-opa was the cause, thanks.

In the default ko2iblnd.conf that was provided, there has already the
ko2iblnd-opa there appeared to be aliased to back to ko2iblnd, I didn't
think twice about changing the alias and the -opa lines initially which is
why it did not work by just modifying the map_on_demand parameter but not
the rest in that file.

Thank you all.

Regards,
- Pak

My current ko2iblnd.conf that is working:

options ko2iblnd peer_credits=128 peer_credits_hiw=64 credits=1024 ntx=2048
map_on_demand=16 fmr_pool_size=2048 fmr_flush_trigger=512 fmr_cache=1
conns_per_peer=4
install ko2iblnd /usr/sbin/ko2iblnd-probe


The default ko2iblnd.conf:

# Currently it isn't possible to auto-tune the o2iblnd parameters optimally
# inside the kernel since the OFED API hides the details from us.
# Unfortunately, there isn't a single set of parameters that provide optimal
# performance on different HCA/HFI types, so this file provides optimized
# tunables based on the installed IB interface cards.
#
# Card detection and tunable selection is handled via
/usr/sbin/ko2iblnd-probe
# at runtime when the ko2iblnd module is installed, either at boot or when
# Lustre is first mounted.

alias ko2iblnd-opa ko2iblnd
options ko2iblnd-opa peer_credits=128 peer_credits_hiw=64 credits=1024
ntx=2048 map_on_demand=256 fmr_pool_size=2048 fmr_flush_trigger=512
fmr_cache=1 conns_per_peer=4

install ko2iblnd /usr/sbin/ko2iblnd-probe



On Tue, Sep 4, 2018 at 9:40 AM, Mohr Jr, Richard Frank (Rick Mohr) <
rmohr at utk.edu> wrote:

>
> > On Sep 4, 2018, at 12:12 PM, Pak Lui <pak.lui at linaro.org> wrote:
> >
> > I have tried "map_on_demand=16" to the "/etc/modprobe.d/ko2iblnd.conf"
> that was suggested. Also tried "map_on_demand=0" as suggested here:
> http://wiki.lustre.org/Optimizing_o2iblnd_Performance
> >
> > /etc/modprobe.d/ko2iblnd.conf
> > alias ko2iblnd-opa ko2iblnd
> > # tried, as suggested in  http://wiki.lustre.org/
> Optimizing_o2iblnd_Performance
> > #options ko2iblnd-opa peer_credits=128 peer_credits_hiw=64 credits=1024
> ntx=2048 map_on_demand=0 fmr_pool_size=2048 fmr_flush_trigger=512
> fmr_cache=1 conns_per_peer=4
> > options ko2iblnd-opa peer_credits=128 peer_credits_hiw=64 credits=1024
> ntx=2048 map_on_demand=16 fmr_pool_size=2048 fmr_flush_trigger=512
> fmr_cache=1 conns_per_peer=4
> > install ko2iblnd /usr/sbin/ko2iblnd-probe
>
> The lines with ko2iblnd-opa are intended for OmniPath hardware.  Since you
> are using IB, you will want to just set your options like this:
>
> options ko2iblnd peer_credits=…, etc.
>
> Have you verified that the firewall is not running?  It’s possible a
> firewall might be allowing ping traffic but blocking the port needed by
> Lustre.
>
> --
> Rick Mohr
> Senior HPC System Administrator
> National Institute for Computational Sciences
> http://www.nics.tennessee.edu
>
>


-- 
Regards,
- Pak
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20180904/386cbfed/attachment.html>